bmj open · for peer review only abstract objective to compare consistency of adverse drug reaction...
TRANSCRIPT
For peer review only
Variation in brand drug harm reports between the United States and Europe: results from a matched-paired drug
document review
Journal: BMJ Open
Manuscript ID bmjopen-2015-010599
Article Type: Research
Date Submitted by the Author: 18-Nov-2015
Complete List of Authors: Cornelius, Victoria; Kings College London, Primary Care and Public Health Liu, Kun; Kings College London, Primary Care and Public Health Peacock, Janet; Kings College London, Primary Care and Public Health
Sauzet, Odile; Universität Bielefeld
<b>Primary Subject Heading</b>:
Pharmacology and therapeutics
Secondary Subject Heading: Public health
Keywords: adverse drug reactions, antidepressent, anticonvulsants, Summary of Product Characteristics, United States Product Inserts
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open on M
ay 22, 2021 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2015-010599 on 20 March 2016. D
ownloaded from
For peer review only
Variation in brand drug harm reports between the United States and Europe: results from
a matched-paired drug document review
Victoria R Cornelius PhD, Kun Liu PhD, Janet Peacock PhD, Odile Sauzet PhD
Word count = 2979
Corresponding author: Victoria R Cornelius [email protected] , Senior Lecturer
4th
Floor Addison House, Guy’s Campus, London SE1 1UL
Tel: 020 7848 8700/6604; Fax 020 7848 6620
Dr Kun Liu: [email protected], Research Fellow
4th
Floor Addison House, Guy’s Campus, London SE1 1UL
Professor Janet Peacock: [email protected], Professor
4th
Floor Addison House, Guy’s Campus, London SE1 1UL
Odile Sauzet: [email protected] , Senior Research Fellow
AG Epidemiology and International Public Health, University of Bielefeld, Bielefeld, Germany
Page 1 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Copyright: The Corresponding Author has the right to grant on behalf of all authors and does grant on
behalf of all authors, a worldwide licence
(http://www.bmj.com/sites/default/files/BMJ%20Author%20Licence%20March%202013.doc) to the
Publishers and its licensees in perpetuity, in all forms, formats and media (whether known now or
created in the future), to i) publish, reproduce, distribute, display and store the Contribution, ii)
translate the Contribution into other languages, create adaptations, reprints, include within collections
and create summaries, extracts and/or, abstracts of the Contribution and convert or allow conversion
into any format including without limitation audio, iii) create any other derivative work(s) based in whole
or part on the on the Contribution, iv) to exploit all subsidiary rights to exploit all subsidiary rights that
currently exist or as may exist in the future in the Contribution, v) the inclusion of electronic links from
the Contribution to third party material where-ever it may be located; and, vi) licence any third party to
do any or all of the above. All research articles will be made available on an Open Access basis (with
authors being asked to pay an open access fee—see http://www.bmj.com/about-bmj/resources-
authors/forms-policies-and-checklists/copyright-open-access-and-permission-reuse). The terms of such
Open Access shall be governed by a Creative Commons licence—details as to which Creative Commons
licence will apply to the research article are set out in our worldwide licence referred to above.
Competing interests: All authors have completed the Unified Competing Interest form at
www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no
support from any organisation for the submitted work; no financial relationships with any organisations
that might have an interest in the submitted work in the previous three years, no other relationships or
activities that could appear to have influenced the submitted work.
Transparency: VC affirms that the manuscript is an honest, accurate, and transparent account of the
study being reported; that no important aspects of the study have been omitted; and that any
discrepancies from the study as planned (and, if relevant, registered) have been explained.
Ethical approval: no ethical approval was required as this was an analysis of published data.
Funding/Support: The research was supported by the National Institute for Health Research (NIHR)
Biomedical Research Centre based at Guy's and St Thomas' NHS Foundation Trust and King's College London. The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or
the Department of Health. VRC, KL, JLP were supported by National Institute for Health Research
(NIHR) Biomedical Research Centre based at Guy's and St Thomas' NHS Foundation Trust and King's
College London. OS was supported by University of Bielefeld
Data sharing statement: All data in this study is available from the drug product documents available
from the FDA and EMA websites. The corresponding author will supply document level summary data on
request.
Contributors: Conception/design of the work: VC, OS. Acquisition of data: VC,KL, OS; Data analysis:
VC, KL. Data interpretation and drafting Manual: VC,OS. Revising it critically for important intellectual
content and final approval of the version to be published: all authors. VC is guarantor.
Page 2 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
ABSTRACT
Objective To compare consistency of adverse drug reaction (ADR) data in publicaly available
drug product information documents for brand drugs between the United States and
Europe. To assess the usefulness of information contained in documents for prescribers and
patients.
Design A matched-paired comparison review of antidepressants and anticonvulsants brand
drugs concurrently marketed in United States and Europe.
Setting United States and Europe. Data was obtained for each brand drug from the United
States Product Inserts (USPI) and the European Summary of Product Characteristics (SmPC)
document between 09/2013 – 01/2015.
Participants: Individuals contributing harm information to the product information
documents.
Main Outcomes measures All adverse reactions (ARs) reported in sections 5 & 6 (USPI) and
4·4 & 4·8 (SmPC).
Results Twelve brand drugs - 24 paired documents were included. On average there were
77 more ARs reported in the USPI compared to the matched SmPC with a median number of
201 ARs (range: 65-425) in USPI and 114 (range:56-265) in SmPC. More USPIs reported
information on the source of evidence (ten vs. five) and risk (nine vs. five) for greater than
80% of ARs reported in the document. There was negligible information included regarding
duration, severity, reversibility, or recurrence of ARs. On average only 29% of the total
number of ARs terms were reported in both paired documents.
Conclusion Drug product information documents contained a large number of ARs but
lacked contextual data and information important to patients and prescribers such as
Page 3 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
duration, severity and reversibility. The harm profile was found to be inconsistently
reported between the US and Europe for the same drug. Identifying, selecting, summarising,
and presenting multidimensional harm data should be underpinned by practical evidence-
based guidelines. This would enable meaningful comparisons across competing drug
therapies which could facilitate informed risk-benefit decisions and allow robust assessment
of cost-effectiveness.
Strengths and limitations of this study
• A matched-pair review allowed for a perfect comparison to examine adverse
reaction profiles between the United States and Europe
• Adverse reaction data and document level data were extracted in duplicate
• Adverse reaction terms were required to be identical to be counted as matched
• The degree of inconsistency between ADR profiles and lack of contextual
information found in antidepressant and anticonvulsant drug documents may not be
representative of other therapeutic areas
Page 4 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
INTRODUCTION
Prescribing choice between several available drug therapies is based on clinical efficacy with
a consideration to the ‘perceived risk’ of the drug. The perceived risk plays an increasingly
important role in treatment decisions when there is little difference in efficacy or when
comparative effectiveness amongst competing treatments is unknown. Robust information
on the drug harm profile is therefore vital to facilitate informed risk-benefit decisions made
by patients and prescribers and enable a robust assessment of cost-effectiveness.
Drug harm profiles are established from various sources. Spontaneous reports of suspected
adverse drug reactions (ADRs) rely on healthcare workers or patients recognising an
association between a drug and an adverse event. Observational studies and electronic
healthcare records can be useful for identifying rare ADRs due to large sample size but they
are subject to a number of biases and often have no robust comparator. Clinical trials have
the potential to provide unbiased estimates of harm outcomes but typical sample sizes
result in uncommon ADRs remaining undetected.
Comprehensive information on drug harm in clinical trials is now routinely collected as a
result of improved regulation since the early 1990s. Extensive research has demonstrated
that both the collection and reporting of harms data in peer-reviewed published clinical
trials is still inadequate and as a consequence the data are not adequate to inform risk-
benefit decisions or contribute to cost-effectiveness analysis.1-3
The main reasons for this
are differing approaches in data collection, inadequate reporting of methodological details
and selective harm outcome reporting. Poor quality harm data found in our systematic
review of trials for neuropathic pain led us to investigate other potential reliable sources of
harm information.4 The regulatory application for marketing authorisation requires drug
manufacturers to list all known drug harm in a ‘product information document’ and is
Page 5 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
primarily written for use by healthcare workers.5,6
The aim of this regularly updated
document is to present a comprehensive picture, combining data from all available sources,
of the harm profile gathered throughout the development, testing, and use of the drug. The
document should include any known harm discovered by external parties. Consequently this
publically available document is uniquely placed to be an invaluable source of information
for patients, healthcare workers, researchers and regulators. This document is called the
‘summary of product characteristics’ (SmPC) in Europe and United States product insert
(USPI) in the US.
The aim of this study was to review the usefulness and relevance of the information
contained in product information documents. We examined the content and assessed
differences in presentation of data between United States and Europe for brand drugs
marketed in both regions.
METHODS
This review involved antidepressants and anticonvulsants brand drugs identified in a
previous systematic review of randomised control trials evaluating treatments of post-
herpetic neuralgia (PHN) and painful diabetic neuropathy (PDN).4 The brand drugs included
were those concurrently marketed in the United States and Europe with available USPI and
SmPC documents.
USPI and SmPC documents were downloaded between September 2013 to January 2015
from the Food and Drug Agency (FDA) and European Medicines Agency (EMA)
respectively.7-9
The document matched pairs were chosen to ensure inclusion of the same
dose and formulation and downloaded at the same time .
Data were extracted from sections 5 & 6 in the USPI and sections 4·4 & 4·8 in the SmPC.
Page 6 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Regulatory guidelines request that the manufacturers should report all adverse drug
reactions (ADRs), which are adverse events (AEs) where there is a reasonable possibility the
drug has played a causal role, however documents sometimes included the term ‘adverse
event’ for which the accepted definition implies that the drug may not be causal. As it was
often not clear whether the events were truly ADRs or AEs a pragmatic decision to extract
all harm events reported was made and hereafter referred to as adverse reactions (ARs).
Justification for this decision was based on the guidance by the EMA which specifies that
‘adverse events, without at least a suspected causal relationship should not be reported’
and the FDA guidance which stated (reporting of) ‘an adverse reaction not plausibly related
to drug therapy should be avoided’.
Data extracted included: all adverse reactions (ARs); risk estimates; study design; sample
size; the criterion used to identify ARs from all AEs recorded in a clinical trial; dictionary
used for coding ARs, whether information on recurrent ARs were reported; information on
onset and duration of ARs. Document level data were extracted by three reviewers (KL, VC
and OS) in duplicate using a customised and piloted spreadsheet. Any differences in
interpretation were resolved through discussion. Information on individual AR terms were
extracted electronically into a spreadsheet by a single reviewer (KL) and then checked by a
second reviewer (VC). Duplicate ARs within a document were removed and the greater risk
estimate was kept. ARs due to rapid withdrawal or interactions with other medications were
excluded from the review.
As this is a descriptive study appropriate measures of central tendency (mean, median)
Page 7 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
and dispersion (SD, range) were calculated for quantitative data, frequency and percentages
for categorical variables. No significance tests were performed. The study protocol is
available on request from the corresponding author.
RESULTS
Nineteen anticonvulsant and antidepressant brand drugs were included as contenders for
this review. Twelve of these were found to concurrently market in Europe and the United
States with SmPC and USPI documentation available. The drugs are listed in Table 1 and
characteristics of the documents are presented in Supplementary Table A.
On average 77 more ARs were reported in the USPI compared to the SmPC document. The
median number of ARs were 114 (IQR 93,150) and 201 (IQR 114, 262) in the SmPC and USPI
documents respectively (Table 2). The majority of the ARs terms differed between
documents, on average only 29% of the total number of ARs were reported in both
documents (Table 3).
Source of evidence
Knowing the source of the evidence of the AR report (clinical trial, observational study,
spontaneous report) provides important contextual information as both the study design
and data collection method can directly impact on the estimated risk of the AR.10,11
The
source of the AR report was specified more frequently in USPI documents compared to
SmPC documents. Of the 24 documents ten USPIs and five SmPCs reported the source for
80% or more of all ARs listed in the document, and the majority were reported as originating
from clinical trial data (Table 2).
Risk of an adverse reaction
Page 8 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
It is not possible to accurately estimate the risk of ARs identified though spontaneous
reports as the number of people prescribed the drug is unknown. Crude estimate can
sometimes be sought from sales or electronic health record data and used to provide an
approximate risk. It was unclear for all documents how risks of ARs originating from
spontaneous reports were derived.12
Nine of twelve USPI and five of twelve SmPC
documents reported risks for 80% or more of all ARs originating from clinical trials (Table 2).
Observation time
The causal mechanism of an adverse reaction is often time dependent and the absolute risk
will depend upon the duration of exposure therefore information on the observation period
is necessary for interpretation. Knowledge of the observation period also reveals how much
is known about long term use of the drug. Five USPIs and two SmPCs contained information
on the duration of the observation period for a couple of ARs but only two USPIs reported
this information for the majority of the ARs arising from clinical trials (Table 2).
Duration, severity, recurrence, and reversibility of an adverse reaction
The anticipated duration, severity, and likelihood of recurrence of an AR is important to
patients and prescribers. Only one USPI contained any information about duration of an AR,
four USPIs presented some information on severity but the information was limited to only
a few ARs. None of the documents presented data regarding recurrence or reversibility
(Table 2).
Identifying adverse reactions from adverse events reports collected from clinical trials
In clinical trials many adverse events (AEs) will be recorded for each treatment and not all of
these will be causally linked to the drug under investigation. In order to identify ARs from all
AEs reported during the trial, a criterion or rule is often used. It is important to know what
Page 9 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
criterion has been used as this will dictate which ARs are flagged. Nine USPIs and three
SmPC reported the criterion used, but the criterion was not the same in three paired
documents and multiple criteria were used in seven USPIs (Table 4).
Dictionary for coding adverse events
Medical terminology dictionaries are used to code adverse event terms with an aim to
standardise reports. For only one drug the dictionary used was reported in both documents
and these dictionaries were not the same (Supplementary Table B). Six USPIs and three
SmPCs specified the dictionary used.
Page 10 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Discussion
One of the aims of FDA and EMA guidances for developing product information documents
is to promote the consistency of harm reporting across drugs. It would be expected that the
harm profiles of a brand drug should be similar in the USPI and SmPC with the only
differences due to variability in regulatory recommendations. This study has identified a lack
of consistency across paired documents in the number and type of harms reported,
approaches to code harm terms, and the availability of contextual information required for
interpretation. The lack of consistency was in part due to differing arbitrary criteria used to
identify adverse reactions from all adverse events reported in a trial and this even varied
within a document. It was not possible for us to assess whether the trials contributing to
both paired documents were identical, potentially adding further discrepancies. The lack of
consistency demonstrated in this study across matched drug documents raises questions as
to whether information across a class of drugs produced by different manufacturers could
ever be usefully compared. Despite this risk-benefit comparisons, in which these
documents play a fundamental role in informing healthcare workers’ knowledge of harm,
are undertaken daily.
The differences in the number of ARs found between paired documents of the same drug
may not represent real differences in the harm profile but may just be a direct result of
differences in AR coding and presentation. With differing dictionaries used to code harm,
data aggregated to different levels and lack of procedures reported, it was not possible for
us to draw further conclusions within this study.
In general the USPI contained more information on methodological aspects and contextual
information needed to assess and interpret harm events such as data source of AR and
information by indication. However the additional volume of material in the USPI made the
Page 11 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
overall harm profile harder to absorb than the SmPC. The guidance from the EMA and FDA
state that harm information should be ‘clear and readily accessible’.5,6
Better ways to prioritise and present harm data are required to enable patients and
healthcare workers to make informed risk-benefit decisions. Current clinical practice may
often be too reliant on prescribers drawing on personal experience, knowledge from a
colleague or abbreviated harm summaries available in prescribing handbooks. Most of the
product documents included in this review reported more than 100 ARs, with one document
listing over 400. With such a large number of ARs reported and no information provided on
severity, duration or recurrence it is questionable how useful the information in these
documents are. It was often unclear if harm events were truly ARs rather than AEs and a
sceptic may wonder if the decision to include many events is made to minimise the risk of
litigation. This is especially pertinent since the manufacturers of the brand drug are legally
responsible.13
More rigorous guidance underpinned by empirical evidence is needed to
reduce current uncertainty around what AR information should be included and how it
should be summarised and presented.
The wide risk categories predominantly used in the SmPC should be reviewed. Adverse
reactions are placed into categories such as ‘very common’ (> 1/10) or ‘common’ (> 1/100
and < 1/10). While the purpose of these risk categories may be to assist readers in
assimilating a large volume of harm information, they are too vague to be useful for serious
ARs. ARs such as angle-closure glaucoma, rhabdomyolysis, ataxia, and arthralgia are
categorised as ‘common’ implying the risk could be anywhere from 1 in 100 to 1 in 10. It is
difficult to see how such an estimate can be useful in informing a risk-benefit decision
especially in the absence of any information on severity, duration or reversibility. Instead it
Page 12 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
may be preferable, where possible, to provide point estimates of risk with an indication of
precision for life threating events or events which significantly impact a patient’s quality of
life. In contrast wide risk categories for ARs with minimal impact on quality of life or physical
health, such as ‘yawning’, may be sufficient.
All antidepressants documents included a warning for suicidal events with anticipated time
to onset. No other information regarding onset, duration or recurrence was included for any
other AR. A patient may be willing to risk the occurrence of tinnitus or peripheral vision loss
if the condition resolves but not if the duration is excessive and recurring.
Current guidance for clinical trials in the published literature has existed since 200414
but
has made little impact on the quality of harm reporting.1-3
This present review also found
that data from trials were inadequately summarised and reported and, notably, differed
across paired documents of the same drug. Bringing together many sources of disparate
information from various locations is technically problematic. An ambitious new initiative
with the aim of collating information from a variety of sources (spontaneous reports,
published literature, product documents, and observational data) has been launched but it
is still in its infancy and it effectiveness is as yet unknown.15
There are initiatives to rank
drugs for risk of a specific drug harm such as drug-induced arrhythmias
(www.CredibleMeds.org , www.BrugadaDrugs.org ) but are lacking an overall harm profile.
Whilst the product information document is written with a focus on providing information
to healthcare workers, we believe this publically available document is uniquely placed to be
an invaluable source of information for patients and researchers. It is the only source that
contains unpublished evidence held by the pharmaceutical company and includes all
available evidence from clinical trials, post-authorisation safety studies and spontaneous
reporting.
Page 13 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Several studies have reported that patients would like more information on ‘side effects’,
believing that full disclosure of side effect information would help them make more
informed treatment decisions.16,17
Many studies have shown that giving patients more
information does not lead to an increase in the reporting of ‘side effects’.18,19
A recent
review showed that the patients have a tendency to underestimate their risk of harm.20
but
there is evidence that when patients are presented with harm leaflets that lack numerical
information they were more likely to overestimate their risk.21,22
It is a statutory
requirement for all drug manufacturers to include Patient Information Leaflets (PILs) as an
insert in the medicine package. A review of the effectiveness of PILs found that patients
thought them to be poor quality both in terms of layout and content, and considered that
they did not help increase knowledge to allow informed risk-benefit decision making.23
Whilst it is clear that all serious and unexpected ADRs should be reported, it is also
important to know the true risk of common and anticipated ADRs as they have cost-benefit
consequences and can significantly impact on a patient’s quality of life. In a trial many
hundreds of adverse events may be recorded and a decision of what should be reported has
to be made. Currently common harm outcomes are usually selected by means of an
arbitrary rule-based criterion such as ‘an AE that occurs in at least 2% of patients and is
twice the placebo rate’. Using this type of criterion leads to flagging the most frequent but
not necessarily the most important ADRs from a patient, healthcare provider or cost-
consequence perspective. It has previously been proposed that consistency in harm
information from trials could be improved by developing a set of ‘core outcomes’ to be
reported by drug class based on the principle proposed for effectiveness studies
(www.comet-initiative.org).2 These ‘core outcome sets’ are the agreed minimum outcomes
Page 14 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
that should be reported and are developed through a consensus process which includes all
key stakeholders. A particularly common issue pertinent to harm profiles concerns the
absence of information for a particular harm. It can never be clear whether this is because it
was just not reported (below the threshold of a rule-based criterion) or no events were
observed. To reduce the inconsistency demonstrated in this review we endorse
development of core ‘harm’ outcome sets by drug class to be reported in addition to all
serious and unexpected ARs.
How harm data are collected has an important impact on the type and the number of
events recorded.11,14
While we have not specifically explored this issue in this study it is
clearly vital and more attention should be paid to methods used to collect and record AEs.
This review highlights a number of weakness in harm information contained in product
descriptions that are also relevant to harm reporting in primary studies. We propose a
conceptual framework to address these issues in Figure 1.
Conclusions
Publically available drug product documents have the potential to be a valuable source of
harm information but data included were found to be inconsistent and not usefully
presented. Identifying, selecting, summarising, and presenting multidimensional harm data
should be underpinned by practical evidence-based guidelines. This would enable
meaningful comparisons across competing drug therapies which could facilitate informed
risk-benefit decisions and allow robust assessment of cost-effectiveness. In order to expose
the true harm profile and allow direct comparison across drugs, more evidence-based
specific guidance is required.
Page 15 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
REFERENCES
1. Ioannidis JP. Adverse events in randomized trials: neglected, restricted, distorted, and
silenced. Arch Intern Med 2009; 169(19): 1737-9.
2. Cornelius VR, Sauzet O, Williams JE, et al. Adverse event reporting in randomised controlled
trials of neuropathic pain: considerations for future practice. Pain 2013; 154(2): 213-20.
3. Maggi CB, Griebeler IH, Dal Pizzol Tda S. Information on adverse events in randomised
clinical trials assessing drug interventions published in four medical journals with high impact
factors. Int J Risk Saf Med 2014; 26(1): 9-22.
4. Sauzet O, Williams JE, Ross J, et al. The characteristics and quality of randomized controlled
trials in neuropathic pain: a descriptive study based on a systematic review. Clin J Pain 2013; 29(7):
591-9.
5. Food and Drug Administration. Adverse Reactions Section of Labeling for Human
Prescription Drug and Biological Products.
http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/UCM
075057.pdf. (last accessed 07/2015)
6. European Commission. A guideline on summary of product characteristics (SmPC)
http://ec.europa.eu/health/files/eudralex/vol-2/c/smpc_guideline_rev2_en.pdf . (last accessed
07/2015)
7. Food and Drug Administration. USPI.
http://www.accessdata.fda.gov/scripts/cder/drugsatfda (last accessed 07/2015)
8. European Medicine Agency SmPC.
http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/landing/epar_search.jsp&mid=W
C0b01ac058001d124 (last accessed 07/2015)
9. Medicines & Healthcare products Regulatory Agency. SmPCs. http://www.mhra.gov.uk/spc-
pil/ (last accessed 07/2015)
10. Golder S, Loke YK, Bland M. Comparison of pooled risk estimates for adverse effects from
different observational study designs: methodological overview. PLoS One 2013; 8(8): e71813.
11. Ioannidis JP, Mulrow CD, Goodman SN. Adverse events: the more you search, the more you
find. Ann Intern Med 2006; 144(4): 298-300.
12. Strom BaV, G. Identification of Mechanisms and Risk Groups. Drug Epidemiology and Post
Marketing Surveillance: Springer; 2013: 14.
13. Kesselheim AS, Green MD, Avorn J. Who is now responsible for discovering and warning
about adverse effects of generic drugs? JAMA 2013; 310(10): 1023-4.
14. Ioannidis JP, Evans SJ, Gotzsche PC, et al. Better reporting of harms in randomized trials: an
extension of the CONSORT statement. Ann Intern Med 2004; 141(10): 781-8.
15. Boyce RD, Ryan PB, Noren GN, et al. Bridging islands of information to establish an
integrated knowledge base of drugs and health outcomes of interest. Drug Saf 2014; 37(8): 557-67.
Page 16 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
16. Ziegler DK, Mosier MC, Buenaver M, Okuyemi K. How much information about adverse
effects of medication do patients want from physicians? Arch Intern Med 2001; 161(5): 706-13.
17. Nair K ; Dolovich L; Cassels A, et al . What patients want to know about their medications.
Focus group study of patient and clinician perspectives. Can Fam Physician 2002; 48: 104 - 10.
18. Howland JS, Baker MG, Poe T. Does patient education cause side effects? A controlled trial. J
Fam Pract 1990; 31(1): 62-4.
19. Lamb GC, Green SS, Heron J. Can physicians warn patients of potential side effects without
fear of causing those side effects? Arch Intern Med 1994; 154(23): 2753-6.
20. Hoffmann TC, Del Mar C. Patients' expectations of the benefits and harms of treatments,
screening, and tests: a systematic review. JAMA Intern Med 2015; 175(2): 274-86.
21. Peters E, Hart PS, Tusler M, Fraenkel L. Numbers matter to informed patient choices: a
randomized design across age and numeracy levels. Med Decis Making 2014; 34(4): 430-42.
22. Sinayev A, Peters E, Tusler M, Fraenkel L. Presenting Numeric Information with Percentages
and Descriptive Risk Labels: A Randomized Trial. Med Decis Making 2015.
23. Raynor DK, Blenkinsopp A, Knapp P, et al. A systematic review of quantitative and qualitative
research on the role and effectiveness of written information available to patients about individual
medicines. Health Technol Assess 2007; 11(5): iii, 1-160.
Page 17 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Table 1: Availability of product information documentation for anticonvulsants and
antidepressants included into the study.a
Anticonvulsants Brand name SmPC USPI Included
Carbamazepine Tegretol yes yes yes
Gabapentin Neurontin yes yes yes
Lamotrigine Lamictal yes yes yes
Oxcarbazepine Trileptal yes yes yes
Phenytoin Epanutin/Dilantin yes yes yes
Pregabablin Lyrica yes yes yes
Sodium Valproate Epilim yes no no
Topiramate Topamax yes yes yes
Zonisamide Zonegran yes yes yes
Antidepressants Brand name SmPC USPI Included
Amitriptyline Elavil yes no no
Citalopram Cipramil yes no no
Clomipramine Anafranil yes yes yes
Desipramine Norpramin no yes no
Duloxetine Cymbalta yes yes yes
Fluoxetine Prozac yes yes yes
Imipramine Janimine no yes no
Mianserin Mianserin yes no no
Nortriptyline Allegron yes no no
Venlafaxine Effexor/Efexor yes yes yes a drugs included where those identified from a systematic review of randomised controlled trials evaluating
treatments for post-herpetic neuralgia (PHN) or painful diabetic neuropathy (PDN)
Page 18 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Drug A B
n(%)
C
n(%)
D
n(%)
E
n(%)
F
n(%)
SmPC USPI SmPC USPI SmPC USPI SmPC USPI SmPC USPI SmPC USPI
Adverse Reactions (AR), N 164 122 117 425 56 241 89 215 179 275 265 248
Data source of AR report
1) Clinical trial 2 (1) 2 (2) 88 (75) 412 (99) 55 (98) 224 (93)
79
(89) 201 (93) 158 (88) 267 (97) 256 (97) 244 (98)
2) Spontaneous
report/other 2 (1) 0 (0) 28 (24) 13 (3) 0 (0) 11 (5)
10
(11) 14 (7) 21 (12) 8 (3) 7 (3) 1 (<1)
3) Unspecified 160 (98) 120 (98) 1 (1) 0 (0) 1 (2) 6 (2) 0 (0) 0 (0) 0 (0) 0 (0) 2 (1) 3 (1)
For each AR identified
from a clinical trial is there
information on:
1) No. of participants 0(0) 2 (100) 0 (0) 412(100) 0 (0) 221 (99) 0 (0) 71 (35) 158 (100) 265 (99) 256 (100) 241 (98)
2) Risk estimates 0(0) 2 (100) 86 (98) 330 (80) 0 (0) 221 (99)
71
(80) 71 (35) 158 (100) 265 (99) 256 (100) 226 (92)
3) Risk estimates by
severity 0(0) 2 (100) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0)
4) length of study 0 (0) 2 (100) 0 (0) 3 (<1) 0 (0) 1 (<1) 0 (0) 0 (0) 0 (0) 265 (99) 0 (0) 1 (<1)
Is there any information
on AR risk by:
1) Indication no yes no yes yes yes no no yes no yes
2) Dose no no no no no yes no yes no yes no yes
Did the document contain
any information on:
1) Recurrent ARs no no no no no no no no no no no
2) Duration of AR no no no no no no no no no no no
Drug G H I J K L
SmPC USPI SmPC USPI SmPC USPI SmPC USPI SmPC USPI SmPC USPI
Adverse Reactions (AR), N 97 186 96 325 136 142 111 95 86 65 126 105
Data source of AR report
1) Clinical trial 0 (0) 143 (77) 1 (1) 319 (98) 0 (0) 107 (76) 1 (1) 79 (83) 0 (0) 2(3) 3 (2) 36 (34)
2) Spontaneous
report/other 0 (0) 8 (4) 5 (5) 5 (2) 2 (1) 23 (16) 3 (3) 2 (2) 0 (0) 0 (0) 6 (5) 34 (32)
3) Unspecified 97 (100) 35 (19) 90 (94) 1 (<1) 134 (98) 11 (8)
107
(96) 14 (15) 86 (100) 63 (97) 117 (93) 35 (33)
For each AR identified
from a clincal trial is there
information on:
1) No. of participants - 143 (100) 0 (0) 298 (93) - 107 (100) 0(0) 35(44) - 0(0) 0(0) 25 (69)
2) Risk estimates - 140 (98) 0 (0) 299 (93) - 99 (93) 0(0) 79 (100) - 0(0) 1 (33) 25 (69)
3) Risk estimates by
severity - 3 (2) 0 (0) 1 (<1) - 8 (7) 0(0) 79 (100) - 0(0) 0(0) 0 (0)
4) length of study - 2 (1) 0 (0) 0 (0) - 84 (79) 0 (0) 1 (100) - 0 (0) 1 (33) 0 (0)
Is there any information
on AR risk by:
1) Indication no yes no no yes yes no yes no no no no
2) Dose no no no no no no no no no no no no
Did the document contain
any information on:
1) Recurrent ARs no no no no no no no no no no no no
2) Duration of AR no no no no no no no no no yes no no
Page 19 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from
For peer review only
Table 2: Data reported by document grouped by brand drug
Page 20 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from
For peer review only
Page 21 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from
For peer review only
Table 3: Reporting of adverse reactions by SmPC and USPI for each drug
SmPC Proportion
with same
AR term Drug USPI reported not reported
A reported 75 89
0.36 not reported 47 -
B reported 89 28
0.20 not reported 336 -
C reported 36 20
0.14 not reported 205 -
D reported 47 42
0.18 not reported 168 -
E reported 88 91
0.24 not reported 187 -
F reported 84 181
0.20 not reported 164 -
G reported 39 58
0.16 not reported 147 -
H reported 53 43
0.14 not reported 272 -
I reported 104 32
0.60 not reported 37 -
J reported 50 61
0.32 not reported 45 -
K reported 50 36
0.50 not reported 15 -
L reported 68 58
0.42 not reported 37 -
Page 22 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Table 4: Rule based criteria listed in documents for selecting adverse reactions to report in
document from all adverse events collected during a clinical trial
Drug Document No. Criteria listeda
B USPI 1 All AEs except those too general to be informative OR not reasonably
associated with the use of drug
C USPI 1
AEs from RCT except AEs that are too general to be informative or not
reasonably attributed to the drug
C USPI 2 AEs from RCT that were >5% AND greater than placebo arm
C USPI 3 AEs from RCT with frequency between 2-5% AND greater than placebo
arm
D SmPC 1 AEs that were clinically meaningful post marketing reports'
D USPI 1 AEs from RCT that were ≥ 2% AND the incidence was greater than
placebo
E SmPC 1 All AEs which occurred at an incidence greater than placebo and in
more than one patient
E USPI 1 AEs from RCT that were ≥ 1% AND at least numerically more than in
the placebo group
E USPI 2 AEs from RCT that were ≥ 2% AND the AE in higher dose group was
≥2% the rate in both the placebo and low dose groups
F USPI 1 AEs from RCT that were ≥ 2% AND the incidence was greater than
placebo
F USPI 2 AEs from RCT that were ≥ 1% AND the incidence was greater than
placebo
F USPI 3 All AEs except those too general to be informative OR not reasonably
associated with the use of drug
G USPI 1
all events included except those too general to be informative, trivial
events or not reasonably thought to be associated with drug
G USPI 2 AEs from RCT that were ≥ 2% AND the incidence was greater than
placebo
H USPI 1 Commonly observed AEs associated with the drug and not seen at an
equivalent incidence among placebo treated patients
H USPI 2 AEs leading to discontinuation
I SmPC 1 Most common AEs reported
I USPI 1 AEs from RCT that were ≥ 5% AND twice the placebo rate
I USPI 2 AEs from RCT that were ≥ 2% in RCT
I USPI 3 AEs from RCT that were ≥ 5% in RCT
J USPI 1
AEs from RCT that were >5% AND at least 2 times greater than placebo
J USPI 2
AEs from RCT that were ≥ 2% AND the incidence was greater than
placebo a
Criteria are from 9/12 USPI and 3/12 SmPC, the other documents did not report any criteria by which the
harms were selected for reporting in the document
Page 23 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Online supplementary tables
Table A: Characteristics of drug product documents
Drug Brand name Document Formulation Dose
Date of
document
Carbamazepine Tegretol SmPC tablets 100mg 07/06/2013
USPI
tablets/chewable
tablets/suspension 100 - 400mg 15/02/2013
Gabapentin Neurontin SmPC hard capsules 300mg 03/04/2013
USPI capsules/ tablets/suspension 100 - 800mg 01/05/2013
Lamotrigine Lamictal SmPC dispersible/chewable 200mg 01/02/2013
USPI dispersible/chewable 25 - 200mg 07/23/2012
Oxcarbazepine Trileptal SmPC tablets 150mg 17/04/2013
USPI tablets/ oral suspension 150 - 600mg 02/07/2013
Pregabalin Lyrica SmPC hard capsules 25mg 04/12/2013
USPI capsules 25 - 300mg 01/06/2013
Topiramate Topamax SmPC tablet 100mg 03/06/2013
USPI tablet/sprinkle capsules 25 - 200mg 01/10/2012
Zonisamide Zonegran SmPC hard capsules 25 - 100mg 13/02/2014
USPI capsules 25 - 100mg 24/01/2012
Clomipramine Anafranil SmPC capsules 10mg 10/07/2012
USPI capsules 25 -75mg 26/10/2012
Duloxetine Cymbalta SmPC capsules 30 - 60mg 15/07/2013
USPI capsules 20- 60mg /10/2012
Fluoxetine Prozac SmPC hard capsules 20mg 01/03/2013
USPI capsules 10 - 90mg 01/07/2014
Venlafaxine Efexor/Effexor SmPC capsules 75-100mg 26/01/2015
USPI capsules 37.5-150mg 01/12/2014
Phenytoin Epanutin/Dilantin SmPC oral suspensions/tablets 30-100mg 04/01/2013
USPI capsules/tablets 30-100mg 07/03/2014
Page 24 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Table B: Dictionaries used to code adverse event terms by document
Dictionary used: matched pair
Drug SmPC USPI
A not reported WHO-ART
B not reported not reported
C not reported COSTART
D not reported not reported
E not reported COSTART
F not reported COSTART
G MedDRA not reported
H MedDRA not reported
I MedDRA COSTART
J not reported not reported
K not reported not reported
L not reported
WHO-ART & investigators
terminology
Page 25 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
How best to summarise information?
Descriptive
Tabulation
Graphical
How to combine evidence from
differing sources?
Randomised Trial
Observational studies:
- Electronic healthcare record
Spontaneous reporting database
Claims database
Single ‘between the eyes’ case report
What should contribute to the
harm profile?
SUSARs and SARs
CORE harm outcomes
ADRs identified in:
RCTs
Observational studies
Single ‘between the eyes’ case
report
What measures of impact should
be included?
Duration
Severity
Recurrence
Reversibility
Best methods for identification?
Identification:
Criterion rule
Statistical test
Signal detection method
Casualty assessment method
How should risk be presented?
Common & non-serious
Impact QoL or life threatening
--------------------------------------
Risk categories
Point estimate of risk & precision
Presentation of
full harm profile
Comprehensible
Comparable
Reliable
Page 26 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from
For peer review only
PRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 Checklist
Section/topic # Checklist item Reported on page #
TITLE
Title 1 Identify the report as a systematic review, meta-analysis, or both. P1
ABSTRACT
Structured summary 2 Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number.
P3
INTRODUCTION
Rationale 3 Describe the rationale for the review in the context of what is already known. P4
Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).
P6
METHODS
Protocol and registration 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number.
P8
Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered,
language, publication status) used as criteria for eligibility, giving rationale. P6
Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.
P6
Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.
NA- not a SR
Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis).
P6
Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.
P7
Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.
P7
Risk of bias in individual studies
12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.
NA- aim of review
Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means). NA- descriptive
Page 27 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from
For peer review only
PRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 Checklist
Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done, including measures of consistency
(e.g., I2) for each meta-analysis.
NA-descriptive
Page 1 of 2
Section/topic # Checklist item Reported on page #
Risk of bias across studies 15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).
NA- aim of review
Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating
which were pre-specified. NA
RESULTS
Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.
P8
Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations.
P19
Risk of bias within studies 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12). NA
Results of individual studies 20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot.
P19
Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency. NA
Risk of bias across studies 22 Present results of any assessment of risk of bias across studies (see Item 15). NA-aim of review
Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]). NA
DISCUSSION
Summary of evidence 24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers).
P11
Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias).
P11
Conclusions 26 Provide a general interpretation of the results in the context of other evidence, and implications for future research. P15
FUNDING
Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review.
P2
Page 28 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from
For peer review only
PRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 Checklist
From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097
For more information, visit: www.prisma-statement.org.
Page 2 of 2
Page 29 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from
For peer review only
Variation in adverse drug reactions listed in product information for antidepressants and anticonvulsants
between the USA and Europe: a comparison review of paired regulatory documents
Journal: BMJ Open
Manuscript ID bmjopen-2015-010599.R1
Article Type: Research
Date Submitted by the Author: 04-Jan-2016
Complete List of Authors: Cornelius, Victoria; Kings College London, Primary Care and Public Health Liu, Kun; Kings College London, Primary Care and Public Health Peacock, Janet; Kings College London, Primary Care and Public Health Sauzet, Odile; Universität Bielefeld
<b>Primary Subject Heading</b>:
Pharmacology and therapeutics
Secondary Subject Heading: Public health
Keywords: adverse drug reactions, antidepressent, anticonvulsants, Summary of Product Characteristics, United States Product Inserts, product infomation
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open on M
ay 22, 2021 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2015-010599 on 20 March 2016. D
ownloaded from
For peer review only
Variation in adverse drug reactions listed in product information for antidepressants and 1
anticonvulsants between the USA and Europe: a comparison review of paired regulatory 2
documents 3
Victoria R Cornelius PhD, Kun Liu PhD, Janet Peacock PhD, Odile Sauzet PhD
4
Word count = 3519 5
Corresponding author: Victoria R Cornelius [email protected] , Senior Lecturer 6
4th
Floor Addison House, Guy’s Campus, London SE1 1UL 7
Tel: 020 7848 8700/6604; Fax 020 7848 6620 8
9
Dr Kun Liu: [email protected], Research Fellow 10
4th
Floor Addison House, Guy’s Campus, London SE1 1UL 11
Professor Janet Peacock: [email protected], Professor 12
4th
Floor Addison House, Guy’s Campus, London SE1 1UL 13
Odile Sauzet: [email protected] , Senior Research Fellow 14
AG Epidemiology and International Public Health, University of Bielefeld, Bielefeld, Germany 15
16
Page 1 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
ABSTRACT 1
Objective To compare consistency of adverse drug reaction (ADR) data in publicly available 2
product information documents for brand drugs between the USA and Europe. To assess the 3
usefulness of information for prescribers and patients. 4
Design A comparison review of product information documents for antidepressants and 5
anticonvulsants concurrently marketed by the same pharmaceutical company in the USA and 6
Europe. 7
Setting For each drug data were extracted from the United States Product Inserts and the 8
European Summary of Product Characteristics documents between 09/2013 – 01/2015. 9
Participants: Individuals contributing ADR information to product information documents. 10
Main Outcomes measures All adverse drug reactions reported in product information sections 5 11
& 6 (USA) and 4·4 & 4·8 (Europe). 12
Results Twelve brand drugs - 24 paired documents were included. On average there were 77 13
more ADRs reported in the USA compared to the European product information document with 14
a median number of 201 ADRs (range: 65-425) and 114 (range:56-265) respectively. More 15
product information documents in the USA reported information on the source of evidence (ten 16
vs. five) and risk (nine vs. five) for greater than 80% of ADRs included in the document. There 17
was negligible information included regarding duration, severity, reversibility, or recurrence of 18
ADRs. On average only 29% of ADRs terms were reported in both paired documents. 19
Conclusion Product information documents contained a large number of ADRs but lacked 20
contextual data and information important to patients and prescribers such as duration, severity 21
and reversibility. The ADR profile was found to be inconsistently reported between the USA and 22
Europe for the same drug. Identifying, selecting, summarising, and presenting multidimensional 23
Page 2 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
harm data should be underpinned by practical evidence-based guidelines. In order for 1
prescribers to provide considered risk-benefit advice across competing drug therapies to 2
patients they need access to comprehensible and reliable ADR information. 3
Strengths and limitations of this study 4
• A paired review allowed for a perfect comparison to examine adverse drug reaction 5
profiles between the USA and Europe 6
• Adverse drug reaction data and document level data were extracted in duplicate 7
• The degree of inconsistency between ADR profiles and lack of contextual information 8
found in antidepressant and anticonvulsant drug documents may not be representative 9
of other therapeutic areas 10
11
12
13
14
Page 3 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
INTRODUCTION 1
Prescribing choice between several available drug therapies is based on clinical efficacy with a 2
consideration to the ‘perceived risk’ of the drug. The perceived risk plays an increasingly 3
important role in treatment decisions when there is little difference in efficacy or when 4
comparative effectiveness amongst competing treatments is unknown. Robust information on 5
the drug harm profile is therefore vital to facilitate informed risk-benefit decisions made by 6
patients and prescribers. 7
Drug harm profiles are established from various sources. Spontaneous reports of suspected 8
adverse drug reactions (ADRs) rely on healthcare workers or patients recognising an association 9
between a drug and an adverse event. Observational studies and electronic healthcare records 10
can be useful for identifying rare ADRs due to large sample sizes but they are subject to a 11
number of biases and often have no robust comparator. Clinical trials have the potential to 12
provide unbiased estimates of harm outcomes but typical sample sizes result in uncommon ADRs 13
remaining undetected. 14
Comprehensive information on drug harm in clinical trials is now routinely collected as a result of 15
improved regulation since the early 1990s. Extensive research has demonstrated that both the 16
collection and reporting of harms data in peer-reviewed published clinical trials is still 17
substandard and as a consequence the data are not adequate to inform risk-benefit decisions or 18
contribute to cost-effectiveness analysis.1-3
The main reasons for this are differing approaches in 19
data collection, inadequate reporting of methodological details and selective harm outcome 20
reporting. Poor quality harm data found in our systematic review of randomised trials for 21
neuropathic pain led us to investigate other potential reliable sources of harm information.4 The 22
regulatory application for marketing authorisation requires drug manufacturers to list all known 23
drug harm in a ‘product information document’ which is primarily written for use by healthcare 24
Page 4 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
workers.5,6
The aim of this regularly updated document is to present a comprehensive picture, 1
combining data from all available sources, of the harm profile gathered throughout the 2
development, testing, and use of the drug. The document should include any known harm 3
discovered by manufacturers, who are the marketing authorisation holders, as well as external 4
parties. Consequently this publically available document is uniquely placed to be an invaluable 5
source of information for patients, healthcare workers, researchers and regulators. This 6
document is called the ‘summary of product characteristics’ (SmPC) and is approved by the 7
European Medicines Agency (EMA) and the United States product insert (USPI) is approved by 8
the Food and Drug Administration (FDA).5,6
9
Previous research has demonstrated inconsistencies in information included in the product 10
documents. Studies have reported discrepancies across countries for drugs marketed by the 11
same pharmaceutical company regarding the number of ADRs, differing risk estimates, and 12
number of patients in the safety sample.7-9
Other studies have found inconsistencies for the 13
updating of contraindication labelling, pregnancy and lactation labelling, and advice on overdose 14
in SmPCs compared to clinical management.10-12
15
The aim of our review was to examine the usefulness and relevance of ADR information for 16
antidepressants and anticonvulsants contained in product information for prescribers and 17
patients. We compared the content between USA and Europe for drugs marketed in both 18
regions by the same market authorisation holder to assess consistency. While some variability 19
would be anticipated due to differences in regulatory cycles between regions and construction 20
of the documents by different company teams, we would expect that the ADR profile between 21
paired documents would be similar as it should be based on centralised safety data. 22
METHODS 23
Page 5 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
This review involved antidepressants and anticonvulsants brand drugs identified in a previous 1
systematic review of randomised control trials evaluating treatments of post-herpetic neuralgia 2
(PHN) and painful diabetic neuropathy (PDN).4 The brand drugs included were those 3
concurrently marketed in the United States and Europe with available product information 4
documents (USPI and SmPC). 5
Documents were downloaded between September 2013 to January 2015 from the Food and 6
Drug Agency (FDA) and European Medicines Agency (EMA) respectively.13-15
The paired 7
documents were chosen to ensure inclusion of the same dose and formulation and downloaded 8
at the same time. 9
10
Data were extracted from sections 5 & 6 in the USPI and sections 4·4 & 4·8 in the SmPC. 11
Regulatory guidelines from the FDA and EMA request that the manufacturers should report all 12
adverse drug reactions (ADRs), which are adverse events (AEs) where there is a reasonable 13
possibility the drug has played a causal role, however documents sometimes included the term 14
‘adverse event’ for which the accepted definition implies that the drug may not be causal.5,6
As it 15
was often not clear whether the events were truly ADRs or AEs a pragmatic decision to extract all 16
harm events reported was made and hereafter referred to as adverse drug reactions (ADRs). 17
Justification for this decision was based on the guidance by the EMA which specifies that 18
‘adverse events, without at least a suspected causal relationship should not be reported’ and the 19
FDA guidance which stated (reporting of) ‘an adverse reaction not plausibly related to drug 20
therapy should be avoided’. A summary of the reporting guidelines for FDA and European 21
Commission can be seen in online supplement Table A. 22
23
Data extracted included: all ADRs; risk estimates; study design; sample size; the criterion used to 24
identify ADRs from all AEs recorded in a clinical trial; dictionary used for coding ADRs, whether 25
Page 6 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
information on recurrent ADRs were reported; information on onset and duration of ADRs. 1
Document level data were extracted by three reviewers (KL, VC and OS) in duplicate using a 2
customised and piloted spreadsheet. Any differences in interpretation were resolved through 3
discussion. Information on individual ADR terms were extracted electronically into a spreadsheet 4
by a single reviewer (KL) and then checked by a second reviewer (VC). Duplicate ADRs within a 5
document were removed and the greater risk estimate was kept. ADRs due to rapid withdrawal 6
or interactions with other medications were excluded from the review. 7
As this is a descriptive study appropriate measures of central tendency (mean, median) 8
and dispersion (SD, range) were calculated for quantitative data, frequency and percentages for 9
categorical variables. No significance tests were performed. The study protocol is available on 10
request from the corresponding author. 11
RESULTS 12
Nineteen anticonvulsant and antidepressant brand drugs were included as contenders for this 13
review. Twelve of these were found to be concurrently market in Europe and the USA with SmPC 14
and USPI documentation available. The characteristics of the documents for the included drugs 15
are listed in Table 1 and all drugs identified from our previous review are listed in online 16
supplement Table B. 17
18
Page 7 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Table 1: Characteristics of drug product information documents 1
2
On average 77 more ADRs were reported in the USA compared to the European document. The 3
median number of ADRs were 114 (IQR 93,150) and 201 (IQR 114, 262) in the European and USA 4
documents respectively (Table 2). The majority of the ADRs terms differed between documents, 5
on average only 29% of the total number of ADRs were reported in both documents (Table 3). 6
7
Drug Brand name
Company Document Formulation Dose
Date of
document
Carbamazepine Tegretol Novartis Europe tablets 100mg 07/06/2013
USA
tablets/chewable
tablets/suspension 100 - 400mg 15/02/2013
Gabapentin Neurontin Pfizer Europe hard capsules 300mg 03/04/2013
USA
capsules/
tablets/suspension 100 - 800mg 01/05/2013
Lamotrigine Lamictal GlaxoSmithKline Europe dispersible/chewable 200mg 01/02/2013
USA dispersible/chewable 25 - 200mg 07/23/2012
Oxcarbazepine Trileptal Novartis Europe tablets 150mg 17/04/2013
USA tablets/ oral suspension 150 - 600mg 02/07/2013
Pregabalin Lyrica Pfizer Europe hard capsules 25mg 04/12/2013
USA capsules 25 - 300mg 01/06/2013
Topiramate Topamax Janssen Europe tablet 100mg 03/06/2013
USA tablet/sprinkle capsules 25 - 200mg 01/10/2012
Zonisamide Zonegran Eisai Europe hard capsules 25 - 100mg 13/02/2014
USA capsules 25 - 100mg 24/01/2012
Clomipramine Anafranil Novartis Europe capsules 50mg 10/07/2012
USA capsules 25 -75mg 26/10/2012
Duloxetine Cymbalta Eli Lilly Europe capsules 30 - 60mg 15/07/2013
USA capsules 20- 60mg /10/2012
Fluoxetine Prozac Eli Lilly Europe hard capsules 20mg 01/03/2013
USA capsules 10 - 90mg 01/07/2014
Venlafaxine Efexor/Effexor Pfizer Europe capsules 75-100mg 26/01/2015
USA capsules 37.5-150mg 01/12/2014
Phenytoin Epanutin/Dilantin Pfizer Europe oral suspensions/tablets 30-100mg 04/01/2013
USA capsules/tablets 30-100mg 07/03/2014
Page 8 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Table 2: Summary of data reported by product information document 1
Europe USA
Item median ( IQR) median ( IQR)
Adverse Drug Reaction Number ADRs 114 (93,150) 201 (114, 262)
Data source of AR report
1) Clinical trial 3 (1, 84) 172 (58, 256)
2) Spontaneous report/other 4 (1, 9) 8 (2, 14)
3) Unspecified 88 (1, 112) 9 (1, 35)
Document Level reporting n(%) n(%)
Documents reporting items
for >80% of clinical trial
ADRs
1) No. of participants 2 (17) 8 (66)
2) Risk estimates 5 (41) 9 (75)
3) Risk estimates by severity 0 (0) 1 (8)
4) length of study 0 (0) 3 (25)
Is any information on ADR
risk reported by:
1) Indication (yes) 2 (17) 9 (75)
2) Dose (yes) 0 (0) 4 (33)
Did the document contain
any information on:
1) Recurrent ARs (yes) 0 (0) 0 (0)
2) Duration of AR (yes) 0 (0) 1(8)
2
Table 3: Reporting of adverse reactions by product information document and drug 3
Europe Proportion
with same
AR term Drug USA reported not reported
Carbamazepine reported 75 89
0.36 not reported 47 -
Gabapentin reported 89 28
0.20 not reported 336 -
Lamotrigine reported 36 20
0.14 not reported 205 -
Oxcarbazepine reported 47 42
0.18 not reported 168 -
Pregabablin reported 88 91
0.24 not reported 187 -
Topiramate reported 84 181
0.20 not reported 164 -
Zonisamide reported 39 58
0.16 not reported 147 -
Clomipramine reported 53 43
0.14 not reported 272 -
Duloxetine reported 104 32
0.60 not reported 37 -
Fluoxetine reported 50 61
0.32 not reported 45 -
Epanutin reported 50 36
0.50 not reported 15 -
Efexor reported 68 58
0.42 not reported 37 -
Page 9 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
1
Source of evidence 2
Knowing the source of the evidence of the ADR report (clinical trial, observational study, 3
spontaneous report) provides important contextual information as both the study design and 4
data collection method will directly impact on the estimated risk of the ADR.16,17
The source of 5
the ADR report was specified more frequently in USA compared to European documents. Of the 6
24 documents ten USA and five European documents reported the source of the ADR report for 7
80% or more of all ADRs listed in the document, and the majority were reported as originating 8
from clinical trial data (Table 2 & Table C online supplement). 9
10
Risk of an adverse reaction 11
It is not possible to accurately estimate the risk of ADRs identified though spontaneous reports 12
as the number of people prescribed the drug is unknown. Crude estimates can sometimes be 13
sought from sales or electronic health record data and can be used to calculate an approximate 14
risk. 18
The European Commission guidance recommends that this estimate is based on 15
‘adequately designed study’ data and with zero events observed suggests estimating the upper 16
95% confidence interval limit using 3/X, where X representing the total sample size summed up 17
across all relevant clinical trials and studies (Table A). It was unclear for all documents how risks 18
of ADRs originating from spontaneous reports were derived. Nine of twelve USA and five of 19
twelve European documents reported risks for 80% or more of all ADRs originating from clinical 20
trials (Table 2). 21
Observation time 22
Page 10 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
The causal mechanism of an adverse drug reaction is often time dependent and the absolute risk 1
will depend upon the duration of exposure therefore information on the observation period is 2
necessary for interpretation. Knowledge of the observation period also reveals how much is 3
known about long term use of the drug. Eight USA and one European document contained 4
information on the duration of the observation period for a couple of ADRs (online supplement 5
Table C) but only three USA documents reported this information for the majority of the ADRs 6
arising from clinical trials (Table 2). 7
Duration, severity, recurrence, and reversibility of an adverse reaction 8
The anticipated duration, severity, and likelihood of recurrence of an ADR is important to 9
patients and prescribers. Only one USA document contained any information about duration of 10
an ADR, four USA documents presented some information on severity but the information was 11
limited to only a few ADRs. None of the documents presented data regarding recurrence or 12
reversibility (Table 2). 13
Identifying adverse reactions from adverse events reports collected from clinical trials 14
In clinical trials many adverse events (AEs) will be recorded for each treatment and not all of 15
these will be causally linked to the drug under investigation. Ideally a signal detection method 16
which is statistically underpinned should be used to flag signals of ADRs from all AEs reported in 17
a clinical trial. None of the documents used a statistical method to flag a signal but nine USA and 18
three European documents reported the rule-based approach used. It is important to know what 19
method has been applied as this will dictate the number and type of ADRs that are flagged. The 20
full list of differing rule-based approaches used are reported in Table 4. It can be seen that rule-21
based methods were not the same in three paired documents and that seven USA documents 22
used multiple criteria were used within the same document. (Table 4). 23
24
Page 11 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Table 4: Rule -based criteria listed in documents for selecting adverse reactions to report in 1
document from all adverse events collected during a clinical trial 2
Drug Document No. Criteria listeda
Gabapentin USA 1 All AEs except those too general to be informative OR not reasonably associated with the use of
drug
Lamotrigine USA 1
AEs from RCT except AEs that are too general to be informative or not reasonably attributed to
the drug
USA 2 AEs from RCT that were >5% AND greater than placebo arm
USA 3 AEs from RCT with frequency between 2-5% AND greater than placebo arm
Oxcarbazepine Europe 1 AEs that were clinically meaningful post marketing reports'
USA 1 AEs from RCT that were ≥ 2% AND the incidence was greater than placebo
Pregabablin Europe 1 All AEs which occurred at an incidence greater than placebo and in more than one patient
USA 1 AEs from RCT that were ≥ 1% AND at least numerically more than in the placebo group
USA 2
AEs from RCT that were ≥ 2% AND the AE in higher dose group was ≥2% the rate in both the
placebo and low dose groups
Topiramate USA 1 AEs from RCT that were ≥ 2% AND the incidence was greater than placebo
USA 2 AEs from RCT that were ≥ 1% AND the incidence was greater than placebo
USA 3
All AEs except those too general to be informative OR not reasonably associated with the use of
drug
Zonisamide USA 1
all events included except those too general to be informative, trivial events or not reasonably
thought to be associated with drug
USA 2 AEs from RCT that were ≥ 2% AND the incidence was greater than placebo
Clomipramine USA 1 Commonly observed AEs associated with the drug and not seen at an equivalent incidence
among placebo treated patients
USA 2 AEs leading to discontinuation
Duloxetine Europe 1 Most common AEs reported
USA 1 AEs from RCT that were ≥ 5% AND twice the placebo rate
USA 2 AEs from RCT that were ≥ 2% in RCT
USA 3 AEs from RCT that were ≥ 5% in RCT
Fluoxetine USA 1 AEs from RCT that were >5% AND at least 2 times greater than placebo
USA 2 AEs from RCT that were ≥ 2% AND the incidence was greater than placebo
a Criteria are from 9/12 documents from USA and 3/12 from Europe, the other documents did not report any 3
criteria by which the harms 4
were selected for reporting in the document 5
6
Dictionary for coding adverse events 7
Medical terminology dictionaries are used to code adverse event terms with an aim to 8
standardise reports. For only one drug the dictionary used was reported in both documents and 9
these dictionaries were not the same (online supplement Table D). Six USA and three European 10
documents specified the dictionary used. 11
12
13
Page 12 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Discussion 1
One of the aims of FDA and EMA guidances for developing product information documents is to 2
promote the consistency of harm reporting across drugs. It would be expected that the harm 3
profiles of a drug marketed by the same company should be similar in the USA and European 4
document with the only differences due to variability in regulatory recommendations. This study 5
has identified a lack of consistency across paired documents in the number and type of harms 6
reported, approaches to code harm terms, and the availability of contextual information 7
required for interpretation. Our results are in-line with other studies who found a high number 8
of discrepancies in ADRs between USA and Europe across a range of differing therapeutic areas.7-
9
9 10
The lack of consistency could be in part due to differing rule-based arbitrary criteria used to 11
identify ADRs from all AEs reported in a trial as these criteria even varied within a document. It 12
was not possible for us to assess whether the trials contributing to both paired documents were 13
identical, potentially adding further discrepancies. It is possible that the year of approval may be 14
associated to lack of consistency, with only a few drugs included and low consistency across all 15
drugs we did not explored this aspect but previous studies have examined this and found 16
conflicting results. Eriksson et al.7 reported that drugs approved after 2000 showed higher 17
consistency but Kesselheim et al.8 reported that length of time on the market was not 18
associated with consistency. 19
The lack of consistency demonstrated in this study across matched drug documents raises 20
questions as to whether information across a class of drugs produced by different manufacturers 21
could ever be usefully compared. Despite this risk-benefit comparisons, in which these 22
documents play a fundamental role in informing prescribers knowledge of harm, are undertaken 23
daily. 24
Page 13 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
The differences in the number of ADRs found between paired documents of the same drug may 1
not all be real differences in the harm profile but could just be differences in ADR coding and 2
presentation. It has been shown that ADR summaries are dependent on the dictionary used and 3
even when the same dictionary is used there is interobserver variation due to coding 4
conventions.19-21
With differing dictionaries used to code harm, data aggregated to different 5
levels and lack of procedures reported, it was not possible for us to draw further conclusions 6
within this study. 7
In general the USA documents contained more information on methodological aspects and 8
contextual information needed to assess and interpret harm events such as data source of ADR 9
and information by indication. However the additional volume of material made the overall 10
harm profile harder to absorb than information in European documents. This contrasting 11
approach of reporting ADRs in either insufficient or excessive detail has previously been raised 12
by Kesselheim et al.8 The guidance from the EMA and FDA state that harm information should 13
be ‘clear and readily accessible’.5,6
Better ways to prioritise and present harm data are required 14
to enable patients and prescribers to undertake informed risk-benefit decisions. 15
Current clinical practice may often be too reliant on prescribers drawing on personal experience, 16
knowledge from a colleague or abbreviated ADR summaries available in prescribing handbooks. 17
Most of the product documents included in this review reported more than 100 ADRs, with one 18
document listing over 400. Studies in different therapeutic areas have also reported similar high 19
numbers with Kesselheim et al. reporting a median of 105 ADRs for 20 top-selling drugs across 4 20
countries8, Eriksson et al. reporting 4003 ADRs in 40 drugs across two countries
7, and Duke et al. 21
reporting a median of 49 ADRs across 5602 drugs and 10% having greater than 150 ADRs 22
reported22
. With such large numbers of ADRs reported and no information provided on severity, 23
duration or recurrence it is questionable how useful the information in these documents are. In 24
general the USA documents contained a larger number of ADRs and it was often unclear if these 25
Page 14 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
harm events were truly ADRs rather than AEs. It is possible that the over-inclusion of many AEs is 1
made to minimise the risk of litigation since the manufacturers of the brand drug are held legally 2
responsible if not reported.23
However the over-inclusion of AEs will make the absorption of 3
pertinent harm information harder for prescribers and patients. More rigorous guidance 4
underpinned by empirical evidence is needed to reduce current uncertainty around what ADR 5
information should be included and how it should be summarised and presented. 6
7
The wide risk categories predominantly used in the European documents should be reviewed. 8
Adverse reactions are placed into categories such as ‘very common’ (> 1/10) or ‘common’ (> 9
1/100 and < 1/10). While the purpose of these risk categories may be to assist readers in 10
assimilating a large volume of harm information, they are too vague to be useful for serious 11
ADRs. ADRs such as angle-closure glaucoma, rhabdomyolysis, ataxia, and arthralgia are 12
categorised as ‘common’ implying the risk could be anywhere from 1 in 100 to 1 in 10. It is 13
difficult to see how such an estimate can be useful in informing a risk-benefit decision especially 14
in the absence of any information on severity, duration or reversibility. Instead it may be 15
preferable, where possible, to provide point estimates of risk with an indication of precision for 16
life threating events or events which significantly impact a patient’s quality of life. In contrast 17
wide risk categories for ADRs with minimal impact on quality of life or physical health, such as 18
‘yawning’, may be sufficient. 19
All antidepressant documents included a warning for suicidal events with anticipated time to 20
onset. No other information regarding onset, duration or recurrence was included for any other 21
ADR. A patient may be willing to risk the occurrence of tinnitus or peripheral vision loss if the 22
condition resolves but not if the duration is excessive and recurring. 23
Page 15 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Current guidance for clinical trials in the published literature has existed since 200424
but has 1
made little impact on the quality of harm reporting.1-3
This present review also found that data 2
from trials were inadequately summarised and reported, and notably, differed across paired 3
documents of the same drug. Aagaard et al. found that source of discrepancies across 4
documents were primarily from information based on clinical trials. 9
5
6
Bringing together many sources of disparate information from various locations is technically 7
problematic. An ambitious new initiative with the aim of collating information from a variety of 8
sources (spontaneous reports, published literature, product documents, and observational data) 9
has been launched but it is still in its infancy and it effectiveness is as yet unknown.25
There are 10
initiatives to rank drugs for risk of a specific drug harm such as drug-induced arrhythmias 11
(www.CredibleMeds.org , www.BrugadaDrugs.org ) but these are lacking an overall harm 12
profile. Whilst the product information document is written with a focus on providing 13
information to healthcare workers, we believe this publically available document is uniquely 14
placed to be an invaluable source of information for patients and researchers. It is the only 15
source that contains unpublished evidence held by the pharmaceutical company and includes all 16
available evidence from clinical trials, post-authorisation safety studies and spontaneous 17
reporting. What ADRs should be included and how to usefully present this multidimensional data 18
to be informative to patients and prescribers requires further consideration. 19
Several studies have reported that patients would like more information on ‘side effects’, 20
believing that full disclosure of side effect information would help them make more informed 21
treatment decisions.26,27
Many studies have shown that giving patients more information does 22
not lead to an increase in the reporting of ‘side effects’.28,29
A recent review showed that the 23
patients have a tendency to underestimate their risk of harm.30
but there is evidence that when 24
patients are presented with harm leaflets that lack numerical information they were more likely 25
Page 16 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
to overestimate their risk.31,32
It is a statutory requirement for all drug manufacturers to include 1
Patient Information Leaflets (PILs) as an insert in the medicine package. A review of the 2
effectiveness of PILs found that patients thought them to be poor quality both in terms of layout 3
and content, and considered that they did not help increase knowledge to allow informed risk-4
benefit decision making.33
5
6
Whilst it is clear that all serious and unexpected ADRs should be reported, it is also important to 7
know the true risk of common and anticipated ADRs as they have cost-benefit consequences and 8
can significantly impact on a patient’s quality of life. In a trial many hundreds of adverse events 9
may be recorded and a decision of what should be reported has to be made. Currently common 10
harm outcomes are usually selected by means of an arbitrary rule-based criterion such as ‘an AE 11
that occurs in at least 2% of patients and is twice the placebo rate’. Using this type of criterion 12
leads to flagging the most frequent but not necessarily the most important ADRs from a patient 13
or healthcare provider perspective. It has previously been proposed that consistency in harm 14
information from trials in the published literature could be improved by developing a set of ‘core 15
outcomes’ to be reported by drug class based on the principle proposed for effectiveness studies 16
(www.comet-initiative.org).2 These ‘core outcome sets’ are the agreed minimum outcomes that 17
should be reported and are developed through a consensus process which includes all key 18
stakeholders. A particularly common issue pertinent to harm profiles concerns the absence of 19
information for a particular harm. It can never be clear whether this is because it was just not 20
reported (below the threshold of a rule-based criterion) or no events were observed. To reduce 21
the inconsistency demonstrated in this review we would endorse development of core ‘harm’ 22
outcome sets by drug class to be reported in addition to all serious and unexpected ADRs. 23
Page 17 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
How harm data are collected has an important impact on the type and the number of events 1
recorded.17,24
While we have not specifically explored this issue in this study it is clearly vital and 2
more attention should be paid to methods used to collect and record AEs. 3
This review highlights a number of weakness in harm information contained in product 4
descriptions that are also relevant to harm reporting in primary studies. We propose a 5
conceptual framework to address these issues in Figure 1. 6
Conclusions 7
Publically available drug product documents have the potential to be a valuable and vital source 8
of harm information but data included were found to be inconsistent and not usefully presented. 9
Identifying, selecting, summarising, and presenting multidimensional harm data should be 10
underpinned by practical evidence-based guidelines. Prescribers and patients require reliable 11
reporting of pertinent ADRs to enable them to undertake meaningful informed risk-benefit 12
decisions across competing drug therapies. 13
Page 18 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Copyright: The Corresponding Author has the right to grant on behalf of all authors and does grant on behalf 1 of all authors, a worldwide licence 2 (http://www.bmj.com/sites/default/files/BMJ%20Author%20Licence%20March%202013.doc) to the 3 Publishers and its licensees in perpetuity, in all forms, formats and media (whether known now or created in 4 the future), to i) publish, reproduce, distribute, display and store the Contribution, ii) translate the 5 Contribution into other languages, create adaptations, reprints, include within collections and create 6 summaries, extracts and/or, abstracts of the Contribution and convert or allow conversion into any format 7 including without limitation audio, iii) create any other derivative work(s) based in whole or part on the on the 8 Contribution, iv) to exploit all subsidiary rights to exploit all subsidiary rights that currently exist or as may 9 exist in the future in the Contribution, v) the inclusion of electronic links from the Contribution to third party 10 material where-ever it may be located; and, vi) licence any third party to do any or all of the above. All 11 research articles will be made available on an Open Access basis (with authors being asked to pay an open 12 access fee—see http://www.bmj.com/about-bmj/resources-authors/forms-policies-and-checklists/copyright-13 open-access-and-permission-reuse). The terms of such Open Access shall be governed by a Creative 14 Commons licence—details as to which Creative Commons licence will apply to the research article are set out 15 in our worldwide licence referred to above. 16
Competing interests: All authors have completed the Unified Competing Interest form at 17 www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no 18 support from any organisation for the submitted work; no financial relationships with any organisations that 19 might have an interest in the submitted work in the previous three years, no other relationships or activities 20 that could appear to have influenced the submitted work. 21
Transparency: VC affirms that the manuscript is an honest, accurate, and transparent account of the study 22 being reported; that no important aspects of the study have been omitted; and that any discrepancies from 23 the study as planned (and, if relevant, registered) have been explained. 24 25
Ethical approval: no ethical approval was required as this was an analysis of published data. 26
Funding/Support: The research was supported by the National Institute for Health Research (NIHR) 27 Biomedical Research Centre based at Guy's and St Thomas' NHS Foundation Trust and King's College London. 28 The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the 29 Department of Health. VRC, KL, JLP were supported by National Institute for Health Research (NIHR) 30 Biomedical Research Centre based at Guy's and St Thomas' NHS Foundation Trust and King's College London. 31 OS was supported by University of Bielefeld 32 33 34 Data sharing statement: No additional data available. 35
Contributors: Conception/design of the work: VC, OS. Acquisition of data: VC,KL, OS; Data analysis: VC, KL. 36 Data interpretation and drafting Manual: VC,OS. Revising it critically for important intellectual content and 37 final approval of the version to be published: all authors. VC is guarantor. 38
Acknowledgments 39
In memory of Kun Liu. 40
41
Page 19 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
REFERENCES
1. Ioannidis JP. Adverse events in randomized trials: neglected, restricted, distorted, and
silenced. Arch Intern Med 2009; 169(19): 1737-9.
2. Cornelius VR, Sauzet O, Williams JE, et al. Adverse event reporting in randomised
controlled trials of neuropathic pain: considerations for future practice. Pain 2013; 154(2): 213-
20.
3. Maggi CB, Griebeler IH, Dal Pizzol Tda S. Information on adverse events in randomised
clinical trials assessing drug interventions published in four medical journals with high impact
factors. Int J Risk Saf Med 2014; 26(1): 9-22.
4. Sauzet O, Williams JE, Ross J, et al. The characteristics and quality of randomized
controlled trials in neuropathic pain: a descriptive study based on a systematic review. Clin J Pain
2013; 29(7): 591-9.
5. Food and Drug Administration. Guidance for Industry- Adverse Reactions Section of
Labeling for Human Prescription Drug and Biological Products — Content and Format
http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm075057
.pdf &
http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm075082
(last accessed 12/2015)
6. European Commission. A guideline on summary of product characteristics (SmPC)
http://ec.europa.eu/health/files/eudralex/vol-2/c/smpc_guideline_rev2_en.pdf . (last accessed 12/2015)
7. Eriksson R, Aagaard L, Jensen LJ, et al. Discrepancies in listed adverse drug reactions in
pharmaceutical product information supplied by the regulatory authorities in Denmark and the
USA. Pharmacol Res Perspect 2014; 2(3): e00038.
8. Kesselheim AS, Franklin JM, Avorn J, Duke JD. Speaking the same language? International
variations in the safety information accompanying top-selling prescription drugs. BMJ Qual Saf
2013; 22(9): 727-34.
9. Aagaard L, Hansen EH. Adverse Drug Reaction Labeling for Atomoxetine,
Methylphenidate and Modafinil: Comparison of Product Information for oral Formulations in
Australia, Denmark and the United States. Curr Drug Saf 2013.
10. Garbe E, Andersohn F. Contraindication labelling changes in the United States and
Germany. Eur J Clin Pharmacol 2007; 63(1): 87-93.
11. Warrer P, Aagaard L, Hansen EH. Comparison of pregnancy and lactation labeling for
attention-deficit hyperactivity disorder drugs marketed in Australia, the USA, Denmark, and the
UK. Drug Saf 2014; 37(10): 805-13.
12. Wall AJ, Bateman DN, Waring WS. Variability in the quality of overdose advice in
Summary of Product Characteristics (SPC) documents: gut decontamination recommendations
for CNS drugs. Br J Clin Pharmacol 2009; 67(1): 83-7.
Page 20 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
13. - Food and Drug Administration. USPI.
http://www.accessdata.fda.gov/scripts/cder/drugsatfda (last accessed 07/2015)
14. European Medicine Agency SmPC.
http://www.ema.europa.eu/ema/index.jsp?curl=pages/medicines/landing/epar_search.jsp&mid=WC0b0
1ac058001d124 (last accessed 07/2015)
15. Medicines & Healthcare products Regulatory Agency. SmPCs. http://www.mhra.gov.uk/spc-pil/
(last accessed 07/2015)
16. Golder S, Loke YK, Bland M. Comparison of pooled risk estimates for adverse effects from
different observational study designs: methodological overview. PLoS One 2013; 8(8): e71813.
17. Ioannidis JP, Mulrow CD, Goodman SN. Adverse events: the more you search, the more
you find. Ann Intern Med 2006; 144(4): 298-300.
18. Strom BaV, G. Identification of Mechanisms and Risk Groups. Drug Epidemiology and
Post Marketing Surveillance: Springer; 2013; Section A; 14-15.
19. Brown DR, Brown EG, Moulvad TB. A comparison of two medical terminologies in coding
and analysing clinical trial safety data. Int J Pharm Med 1997; 11: 85-9.
20. Schroll JB, Maund E, Gotzsche PC. Challenges in coding adverse events in clinical trials: a
systematic review. PLoS One 2012; 7(7): e41174.
21. Maund E, Tendal B, Hrobjartsson A, Lundh A, Gotzsche PC. Coding of adverse events of
suicidality in clinical study reports of duloxetine for the treatment of major depressive disorder:
descriptive study. BMJ 2014; 348: g3555.
22. Duke J, Friedlin J, Ryan P. A quantitative analysis of adverse events and "overwarning" in
drug labeling. Arch Intern Med 2011; 171(10): 944-6.
23. Kesselheim AS, Green MD, Avorn J. Who is now responsible for discovering and warning
about adverse effects of generic drugs? JAMA 2013; 310(10): 1023-4.
24. Ioannidis JP, Evans SJ, Gotzsche PC, et al. Better reporting of harms in randomized trials:
an extension of the CONSORT statement. Ann Intern Med 2004; 141(10): 781-8.
25. Boyce RD, Ryan PB, Noren GN, et al. Bridging islands of information to establish an
integrated knowledge base of drugs and health outcomes of interest. Drug Saf 2014; 37(8): 557-
67.
26. Ziegler DK, Mosier MC, Buenaver M, Okuyemi K. How much information about adverse
effects of medication do patients want from physicians? Arch Intern Med 2001; 161(5): 706-13.
27. Nair K ; Dolovich L; Cassels A, et al. What patients want to know about their medications.
Focus group study of patient and clinician perspectives. Can Fam Physician 2002; 48: 104 - 10.
28. Howland JS, Baker MG, Poe T. Does patient education cause side effects? A controlled
trial. J Fam Pract 1990; 31(1): 62-4.
29. Lamb GC, Green SS, Heron J. Can physicians warn patients of potential side effects
without fear of causing those side effects? Arch Intern Med 1994; 154(23): 2753-6.
Page 21 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
30. Hoffmann TC, Del Mar C. Patients' expectations of the benefits and harms of treatments,
screening, and tests: a systematic review. JAMA Intern Med 2015; 175(2): 274-86.
31. Peters E, Hart PS, Tusler M, Fraenkel L. Numbers matter to informed patient choices: a
randomized design across age and numeracy levels. Med Decis Making 2014; 34(4): 430-42.
32. Sinayev A, Peters E, Tusler M, Fraenkel L. Presenting Numeric Information with
Percentages and Descriptive Risk Labels: A Randomized Trial. Med Decis Making 2015 ; Nov;
35(8): 937-47
33. Raynor DK, Blenkinsopp A, Knapp P, et al. A systematic review of quantitative and
qualitative research on the role and effectiveness of written information available to patients
about individual medicines. Health Technol Assess 2007; 11(5): iii, 1-160.
Page 22 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
Figure 1: Overview of issues to address in selection and presentation of harm information from clinical studies
253x122mm (300 x 300 DPI)
Page 23 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
1
Table A: Summary of FDA and European Commission Guidance for Industry for Adverse Drug Reaction reporting in
product information documents
Food and Drug Administration. Guidance for Industry Adverse Reactions Section of Labeling for Human Prescription Drug and Biological Products — Content and Format
INTRODUCTION
Applicants should assess such factors as seriousness, severity, frequency, and strength of causal association in determining which adverse reactions to include
ADVERSE REACTIONS section
In general, the ADVERSE REACTIONS section includes only information that would be useful to health care practitioners making treatment decisions and monitoring and advising patients
Exhaustive lists of every reported adverse event, including those that are infrequent and minor, commonly observed in the absence of drug therapy or not plausibly related to drug therapy should be avoided
Separate lists are required for:
adverse reactions identified from clinical trials
spontaneous reports after a drug has been marketed
A. Making the Most Clinically Important Information Accessible
The ADVERSE REACTIONS section should make it easier for health care practitioners to recognize and retain the adverse reactions information that is most important to prescribing decisions
The beginning of the ADVERSE REACTIONS section should identify the most clinically significant adverse reactions
B. Adverse Reactions From Clinical Trials
1. Description of Data Sources
The presentation of adverse reactions information identified from clinical trials must be preceded by information necessary to interpret the adverse reactions
e.g. number of patients, dose, schedule, duration), demographics of the exposed population, designs of the trials
2. Statement on the Significance of Adverse Reaction Data Obtained From Clinical Trials
3. Presentation of Common Adverse Reactions (the Adverse Reactions Table)
List the adverse reactions identified from clinical trials that occurred at or above a specified rate appropriate to the database
The listing must include the rate of occurrence of an adverse reaction for the drug and any comparators
a. Use Best Available Data e.g. placebo-controlled
b. Description of Data Sources for the Table, includes:
the basis for including adverse reactions in the table (e.g., all ARs occurring at > n% in the treated group and for which the rate for drug exceeds the rate for placebo)
the way in which adverse reaction rates were derived e.g. was the rate derived from all reported adverse events of that type not present at baseline or from a subset of reported events deemed by investigators to be drug-related
indicate the types of studies from which the information in the table was derived and whether the study data were pooled
c. How Many Tables?
A single adverse reaction table will usually be adequate
However, it may be more informative to present data in more than one table when a drug’s adverse reaction profile differs substantially from one setting or population to another
4. Presentation of Less Common Adverse Reactions
Inclusion of less common adverse reactions, but for which there is some basis to believe there is a causal relationship between the drug and the event
Lengthy lists of adverse events unlikely to have been caused by the drug are of little or no value to prescribers, and are therefore inappropriate for inclusion in labeling
Serious, low-frequency adverse events generally will be listed when there is reason to suspect that the drug may have caused the event
Typical reasons to suspect causality include (1) timing of onset or termination with respect to drug use, (2) plausibility in light of the drug’s known pharmacology, (3) occurrence at a frequency above that expected (4) occurrence of an event typical of drug-induced adverse reactions e.g. liver necrosis.
Non-serious, low-frequency adverse events should be listed only when there is strong evidence that the drug caused the event. Such evidence may include, for example, positive challenge/dechallenge tests or rate of occurrence in a large controlled trial that, although low, is markedly imbalanced between drug and control arms.
Page 24 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
2
5. Commentary on Listings of Common and Less Common Adverse Reactions
For adverse reactions with significant clinical implications must be supplemented with additional details about the nature, frequency, severity, dose-response, and demographic characteristics of the adverse reaction, to the extent data are available and important
a. Information on Nature, Frequency, and Severity
b. Dose-Response Information
c. Demographic and Other Subgroups
d. Multiple Indications
e. Multiple Formulations
C. Presentation of Adverse Reaction Information From Spontaneous Reports
The ADVERSE REACTIONS section must list adverse reactions identified from domestic and foreign spontaneous reports
This listing must be separate from the listing of adverse reactions identified in clinical trials
and must also be preceded by information necessary to interpret the adverse reactions
Decisions about whether to include an adverse event from spontaneous reports in labeling are typically based on one or more of the following factors: (1) seriousness of the event, (2) number of reports, or (3) strength of causal relationship to the drug
NB: full guidance listed in reference below. This summary has been constructed with relevance to the research objectives in this study.
http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guidances/ucm075057.pdf
European Commission. A guideline on summary of product characteristics (SmPC)
INTRODUCTION
The use of MedDRA as described in the annex for section 4.8 should be applied though the SmPC, in particular for sections 4.3 and 4.4 and 4.8
4.8 Undesirable effects
This section should include all adverse reactions from clinical trials, post-authorisation safety studies and spontaneous reporting for which, after thorough assessment, a causal relationship between the medicinal product and the adverse event is at least a reasonable possibility
Adverse events, without at least a suspected causal relationship, should not be listed in the SmPC
It is important that the whole section is worded in concise and specific language
a. Summary of the safety profile
Provide information about the most serious and/or most frequently occurring adverse reactions
If known, it may be helpful to indicate the timing when adverse reactions occur
Inform about adverse reactions associated with long-term use
Frequencies of cited adverse reactions should be stated as accurately as possible
b. Tabulated list of adverse reactions
A single table (or structured listing) should list all adverse reactions with their respective frequency category
Separate tables are acceptable in exceptional cases where the adverse reaction profiles markedly differ depending on the use of the product
The table should be introduced stating the source of the safety database (e.g. from clinical trials, post-authorisation safety studies or spontaneous reporting
The table should be presented according to the MedDRA system organ classification
Adverse reactions descriptions should be based on the most suitable representation within the MedDRA terminology (usually Preferred Term (PT) level)
Within each system organ class, the adverse reactions should be ranked under headings of frequency, most frequent reactions first.
Within each frequency grouping, adverse reactions should be presented in the order of decreasing seriousness.
Use Very common (≥1/10); common (≥1/100 to <1/10); uncommon (≥1/1,000 to <1/100); rare (≥1/10,000 to <1/1,000); very rare (<1/10,000)
c. Description of selected adverse reactions
Include information characterising specific adverse reaction which may be useful to prevent, assess or manage the occurrence of an adverse reaction
Page 25 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
3
Information characterising individual serious and/or frequently occurring adverse reactions, or those where there have been reports of particularly severe cases
The information should provide frequency and may describe for example reversibility, time of onset, severity, duration, mechanism of the reaction (if of clinical relevance), dose relationship, relationship with duration of exposure or risk factors
d. Paediatric population
The extent and age characteristics of the safety database in children should be described (e.g. from clinical trials or pharmacovigilance data)
Uncertainties due to limited experience should be stated
Any clinically relevant differences (i.e. in nature, frequency, seriousness or reversibility of adverse reactions) between the safety profiles in adult and paediatric populations
e. Other special populations
Include information on any clinically relevant differences (i.e. in nature, frequency, seriousness or reversibility of adverse reactions, or need for monitoring) specifically observed in other special populations such as elderly, patients with renal impairment, patients with hepatic impairment, patients with other diseases or a specific genotype
Further guidance on the estimation of frequency of adverse reactions
If the choice of the frequency category is based on different sources, the category representing the highest frequency should be chosen unless a more specific method has been applied and thus resulted in an estimate of clearly higher validity
Reactions that are reported under different terms but represent the same phenomenon (e.g., sedation, somnolence, drowsiness) should ordinarily be grouped together as a single adverse reaction to avoid diluting or obscuring the true effect
Adverse reactions from clinical trials
The frequency of adverse reactions should be derived from pooled placebo-controlled studies if these data are available
When a common, very common or serious adverse reaction (e.g. suicide) also occurs in the placebo group with a relevant frequency, both incidence rates can be stated to put the risk into perspective
Adverse reactions from spontaneous reporting
In case of an unexpected adverse reaction detected from spontaneous reporting, each adequately designed study where this adverse reaction could have been detected should be reviewed to choose a frequency category If the adverse reaction has never been observed in clinical trials, then the upper limit of the 95% confidence interval is not higher than 3/X, with X representing the total sample size summed up across all relevant clinical trials and studies
NB: full guidance listed in reference below. This summary has been constructed with relevance to the research objectives in this study.
http://ec.europa.eu/health/files/eudralex/vol-2/c/smpc_guideline_rev2_en.pdf
Page 26 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
4
Online supplementary material
Table B: Availability of product information documentation for anticonvulsants and antidepressants included
into the study.a
Anticonvulsants Brand name Europe USA Included
Carbamazepine Tegretol yes yes yes
Gabapentin Neurontin yes yes yes
Lamotrigine Lamictal yes yes yes
Oxcarbazepine Trileptal yes yes yes
Phenytoin Epanutin/Dilantin yes yes yes
Pregabablin Lyrica yes yes yes
Sodium Valproate Epilim yes no no
Topiramate Topamax yes yes yes
Zonisamide Zonegran yes yes yes
Antidepressants Brand name Europe USA Included
Amitriptyline Elavil yes no no
Citalopram Cipramil yes no no
Clomipramine Anafranil yes yes yes
Desipramine Norpramin no yes no
Duloxetine Cymbalta yes yes yes
Fluoxetine Prozac yes yes yes
Imipramine Janimine no yes no
Mianserin Mianserin yes no no
Nortriptyline Allegron yes no no
Venlafaxine Effexor/Efexor yes yes yes a drugs included where those identified from a systematic review of randomised controlled trials evaluating treatments for post-
herpetic neuralgia (PHN) or painful diabetic neuropathy (PDN)
Page 27 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
5
Table C: Data reported by document grouped by drug
Drug Carbamazepine n(%)
Gabapentin n(%)
Lamotrigine n(%)
Oxcarbazepine n(%)
Pregabablin n(%)
Topiramate n(%)
European USA SmPC USPI SmPC USPI SmPC USPI SmPC USPI SmPC USPI
Adverse Reactions n 164 122 117 425 56 241 89 215 179 275 265 248
Data source of AR report
1) Clinical trial 2 (1) 2 (2) 88 (75) 412 (99) 55 (98) 224 (93) 79 (89) 201 (93) 158 (88) 267 (97) 256 (97) 244 (98)
2) Spontaneous report/other 2 (1) 0 (0) 28 (24) 13 (3) 0 (0) 11 (5) 10 (11) 14 (7) 21 (12) 8 (3) 7 (3) 1 (<1)
3) Unspecified 160 (98) 120 (98) 1 (1) 0 (0) 1 (2) 6 (2) 0 (0) 0 (0) 0 (0) 0 (0) 2 (1) 3 (1)
For each AR identified from a clinical trial is there information on:
1) No. of participants 0(0) 2 (100) 0 (0) 412(100) 0 (0) 221 (99) 0 (0) 71 (35) 158 (100) 265 (99) 256 (100) 241 (98)
2) Risk estimates 0(0) 2 (100) 86 (98) 330 (80) 0 (0) 221 (99) 71 (80) 71 (35) 158 (100) 265 (99) 256 (100) 226 (92)
3) Risk estimates by severity 0(0) 2 (100) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) 0 (0)
4) length of study 0 (0) 2 (100) 0 (0) 3 (<1) 0 (0) 1 (<1) 0 (0) 0 (0) 0 (0) 265 (99) 0 (0) 1 (<1)
Is there any information on AR risk by:
1) Indication no yes no yes yes yes no yes no yes no yes
2) Dose no no no no no yes no yes no yes no yes
Did the document contain any information on:
1) Recurrent ARs no no no no no no no no no no no no
2) Duration of AR no no no no no no no no no no no no
Drug Zonisamide Clomipramine Duloxetine Fluoxetine Epanutin Efexor
n
SmPC USPI SmPC USPI SmPC USPI SmPC USPI SmPC USPI SmPC USPI
Adverse Reactions 97 186 96 325 136 142 111 95 86 65 126 105
Data source of AR report
1) Clinical trial 0 (0) 143 (77) 1 (1) 319 (98) 0 (0) 107 (76) 1 (1) 79 (83) 0 (0) 2(3) 3 (2) 36 (34)
2) Spontaneous report/other 0 (0) 8 (4) 5 (5) 5 (2) 2 (1) 23 (16) 3 (3) 2 (2) 0 (0) 0 (0) 6 (5) 34 (32)
3) Unspecified 97 (100) 35 (19) 90 (94) 1 (<1) 134 (98) 11 (8) 107 (96) 14 (15) 86 (100) 63 (97) 117 (93) 35 (33)
For each AR identified from a clincal trial is there information on:
1) No. of participants - 143 (100) 0 (0) 298 (93) - 107 (100) 0(0) 35(44) - 0(0) 0(0) 25 (69)
2) Risk estimates - 140 (98) 0 (0) 299 (93) - 99 (93) 0(0) 79 (100) - 0(0) 1 (33) 25 (69)
3) Risk estimates by severity - 3 (2) 0 (0) 1 (<1) - 8 (7) 0(0) 79 (100) - 0(0) 0(0) 0 (0)
4) length of study - 2 (1) 0 (0) 0 (0) - 84 (79) 0 (0) 1 (100) - 0 (0) 1 (33) 0 (0)
Is there any information on AR risk by:
1) Indication no yes no no yes yes no yes no no no no
2) Dose no no no no no no no no no no no no
Did the document contain any information on:
1) Recurrent ARs no no no no no no no no no no no no
2) Duration of AR no no no no no no no no no yes no no
Page 28 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
6
Table D: Dictionaries used to code adverse event terms by origin of document
Dictionary used: matched pair
Drug Europe USA
Carbamazepine not reported WHO-ART
Gabapentin not reported not reported
Lamotrigine not reported COSTART
Oxcarbazepine not reported not reported
Pregabablin not reported COSTART
Topiramate not reported COSTART
Zonisamide MedDRA not reported
Clomipramine MedDRA not reported
Duloxetine MedDRA COSTART
Fluoxetine not reported not reported
Epanutin not reported not reported
Efexor not reported WHO-ART & investigators
terminology
Page 29 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2015-010599 on 20 M
arch 2016. Dow
nloaded from
For peer review only
PRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 Checklist
Section/topic # Checklist item Reported on page #
TITLE
Title 1 Identify the report as a systematic review, meta-analysis, or both. P1
ABSTRACT
Structured summary 2 Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions and implications of key findings; systematic review registration number.
P3
INTRODUCTION
Rationale 3 Describe the rationale for the review in the context of what is already known. P5
Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).
P6
METHODS
Protocol and registration 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide registration information including registration number.
P8
Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered,
language, publication status) used as criteria for eligibility, giving rationale. P7
Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.
P7
Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.
NA- not a SR
Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis).
P7
Data collection process 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.
P8
Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.
P8
Risk of bias in individual studies
12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.
NA- aim of review
Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means). NA- descriptive
Page 30 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from
For peer review only
PRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 Checklist
Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done, including measures of consistency
(e.g., I2) for each meta-analysis.
NA-descriptive
Page 1 of 2
Section/topic # Checklist item Reported on page #
Risk of bias across studies 15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).
NA- aim of review
Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done, indicating
which were pre-specified. NA
RESULTS
Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.
P8
Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations.
P19
Risk of bias within studies 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12). NA
Results of individual studies 20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each intervention group (b) effect estimates and confidence intervals, ideally with a forest plot.
Sup table B
Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency. NA
Risk of bias across studies 22 Present results of any assessment of risk of bias across studies (see Item 15). NA-aim of review
Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]). NA
DISCUSSION
Summary of evidence 24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy makers).
P14
Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of identified research, reporting bias).
P14
Conclusions 26 Provide a general interpretation of the results in the context of other evidence, and implications for future research. P15
FUNDING
Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review.
P2
Page 31 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from
For peer review only
PRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 ChecklistPRISMA 2009 Checklist
From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(6): e1000097. doi:10.1371/journal.pmed1000097
For more information, visit: www.prisma-statement.org.
Page 2 of 2
Page 32 of 32
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on May 22, 2021 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2015-010599 on 20 March 2016. Downloaded from