final research report: the benefits and challenges of ...final research report (me-1303-5785) 6 of...

Final research report (ME‐1303‐5785) 1 of 50

Final Research Report: The Benefits and Challenges of Using Multiple Sources

of Information about Clinical Trials

Kay Dickersin, PhD1; Evan Mayo‐Wilson, MPA, DPhil1; Tianjing Li1

1Johns Hopkins Bloomberg School of Public Health, Baltimore, MD

For the MUDS Investigators

Original Project Title: Integrating Multiple Data Sources for Meta-analysis to Improve Patient-Centered Outcomes ResearchPCORI Project ID: ME‐1303‐5785

HSRproj ID: 20143578

_______________________________ To cite this document, please use: Dickersin K, Mayo‐Wilson E, Li T.,et al. 2018. The Benefits and Challenges of Using Multiple Sources of Information about Clinical Trials. Washington, DC: Patient‐Centered Outcomes Research Institute (PCORI). https://doi.org/10.25302/3.2018.ME.13035785

Final research report (ME-1303-5785) 2 of 50

Table of Contents

THE MUDS INVESTIGATORS ........................................................................................................................ 4

ABSTRACT .......................................................................................................................................................... 6

1. BACKGROUND .............................................................................................................................................. 9

1.1. Objectives and overview ................................................................................................................................... 9 1.2. Patient-centered outcomes .............................................................................................................................. 9 1.3. Reporting biases ............................................................................................................................................... 10 1.4. Multiple data sources ...................................................................................................................................... 10

2. METHODS ................................................................................................................................................... 11

2.1. Selection of case studies ................................................................................................................................. 11 2.1.1. Gabapentin for neuropathic pain .......................................................................................................................... 11 2.1.2. Quetiapine for bipolar depression ....................................................................................................................... 11

2.2. Methods for identifying patient-centered outcomes ........................................................................... 12 2.3. The effect of multiple data sources on meta-analyses ........................................................................ 12

2.3.1. Eligibility criteria for a trial’s inclusion in our study .................................................................................... 12 2.3.2. Search methods for identification of trials ....................................................................................................... 13 2.3.3. Selection of trials ......................................................................................................................................................... 14 2.3.4. Data collection .............................................................................................................................................................. 14

2.4. Analysis ................................................................................................................................................................ 15 2.4.1. Assessment of trial quality ...................................................................................................................................... 15 2.4.2. Series of meta-analyses ............................................................................................................................................ 16

3. RESULTS ...................................................................................................................................................... 17

3.1. Patient-centered outcomes identified through existing sources and stakeholder

coinvestigators .......................................................................................................................................................... 17 3.1.1. Core outcome sets ....................................................................................................................................................... 17 3.1.2. Social media ................................................................................................................................................................ ... 17 3.1.3. DRUGDEX ........................................................................................................................................................................ 17 3.1.4. Food and Drug Administration prescribing information ........................................................................... 18 3.1.5. Patient and stakeholder investigators ................................................................................................................ 18

3.2. The effects of adding and replacing data from multiple data sources on the results of meta-

analyses ........................................................................................................................................................................ 18 3.2.1. Results of the search .................................................................................................................................................. 18 3.2.2. Completeness of information ................................................................................................................................. 22


3.2.3. Locating patient-centered outcomes in multiple data sources ................................................................ 25 3.2.4. Multiple outcome definitions ................................................................................................................................. 28 3.2.5. Impact of adding or replacing multiple data sources on pooled effect sizes and precision ......... 31 3.2.6. Resources required to use multiple data sources in systematic reviews and meta-analyses ..... 35

3.3. Guidance for using multiple data sources in systematic reviews ................................................... 37

4. DISCUSSION................................................................................................................................................ 39

4.1. Identifying patient-centered outcomes .................................................................................................... 39 4.2. The effects of adding and replacing data from multiple data sources on the results of meta-

analyses ........................................................................................................................................................................ 39 4.3 Implications for data sharing ........................................................................................................................ 40 4.4 Limitations ........................................................................................................................................................... 41

5. CONCLUSIONS ........................................................................................................................................... 41

6. REFERENCES .............................................................................................................................................. 41


THE MUDS INVESTIGATORS Lorenzo Bertizzolo Department of Epidemiology Johns Hopkins University Bloomberg School of Public Health Joseph K. Canner Department of Surgery Johns Hopkins University School of Medicine Terrie Cowley The TMJ Association, Ltd. Kay Dickersin, PhD (Principal Investigator) Department of Epidemiology Johns Hopkins Bloomberg School of Public Health Peter Doshi Pharmaceutical Health Services Research University of Maryland School of Pharmacy Jeffrey Ehmsen Department of Neurology-Neuromuscular Medicine Johns Hopkins University School of Medicine Nicole Fusco Department of Epidemiology Johns Hopkins University Bloomberg School of Public Health Gillian Gresham Department of Epidemiology Johns Hopkins University Bloomberg School of Public Health Nan Guo Department of Epidemiology Johns Hopkins University Bloomberg School of Public Health Jennifer A Haythornthwaite Department of Psychiatry & Behavioral Sciences Johns Hopkins University School of Medicine James Heyward Department of Epidemiology Johns Hopkins University Bloomberg School of Public Health Hwanhee Hong Department of Mental Health Johns Hopkins University Bloomberg School of Public Health Tianjing Li Department of Epidemiology


Johns Hopkins Bloomberg School of Public Health Diana Lock Department of Health Policy and Management Johns Hopkins Bloomberg School of Public Health Evan Mayo-Wilson, MPA, DPhil Department of Epidemiology Johns Hopkins Bloomberg School of Public Health Jennifer L. Payne Psychiatry and Behavioral Sciences The Johns Hopkins Hospital Lori Rosman Welch Medical Library Johns Hopkins University School of Medicine Elizabeth A. Stuart Department of Mental Health Johns Hopkins University Bloomberg School of Public Health Catalina Suarez-Cuervo Department of Health Policy & Management Johns Hopkins University Bloomberg School of Public Health Elizabeth Tolbert Johns Hopkins University Peabody Institute Claire Twose Welch Medical Library Johns Hopkins University School of Medicine Swaroop Vedula Malone Center for Engineering in Healthcare Johns Hopkins University Whiting School of Engineering


ABSTRACT Background: International standards include recommendations that systematic reviews be comprehensive,

but time and resources may render it impractical to search for and extract data from all possible sources of

information. For example, many data sources exist for randomized controlled trials (RCTs), a number of which

are not publicly available or are difficult or impossible to access. Searching nonpublic sources of RCTs may

improve the impact of systematic reviews by identifying information that was recorded but not included in

public sources, thus reducing the need for additional RCTs and improving research efficiency.

Objectives: To determine whether multiple data sources about RCTs affected systematic reviews and meta-

analyses of patient-centered outcomes (PCOs) research (e.g., trial quality assessment, pooled effect

estimates) and to produce open access guidance about using multiple data sources, for producers of

systematic reviews of PCOs research.

Methods: We conducted a methods study related to the conduct of systematic reviews using 2 case studies:

(1) gabapentin for neuropathic pain and (2) quetiapine for bipolar depression. Applying comprehensive

searching methods as if we were conducting 2 systematic reviews, we attempted to identify all data sources

for eligible RCTs. We extracted data and compared the information about trial design and trial quality (i.e., risk

of bias) using the multiple data sources. Using these multiple data sources, we extracted information about

prespecified outcome domains, including the definition of each outcome (i.e., the outcome domain, measure,

metric, method of aggregation, and time-point) and the associated results (i.e., numerical estimates of

treatment effectiveness). Then, we compared the results of meta-analyses using multiple data sources by

conducting multiple meta-analyses in which we systematically added and replaced data from various sources.

We compared the outcomes that matter most to patients with the outcomes that were examined in clinical

trials.

Results: Most clinical trials in our case studies were associated with multiple data sources, including public

sources (e.g., journal articles, conference abstracts, trial registrations, and Food and Drug Administration

(FDA) reviews) and nonpublic sources (e.g., clinical study reports and individual patient data). We found 21

gabapentin RCTs (74 reports, 6 individual participant data) and 7 quetiapine RCTs (50 reports, 1 individual

participant data); we found nonpublic sources for 6 of 21 (29%) gabapentin and 4 of 7 (57%) quetiapine RCTs.

We identified literally hundreds of PCOs reported in the sources we found that could be included in our meta-

analyses. We surmised that there were many opportunities for selective outcome reporting by trialists and

systematic reviewers. The process of identifying all sources of information—and extracting and analyzing

data—required considerable time and skill. Nonpublic sources were especially difficult to identify. Clinical


study reports were time consuming to extract, and individual participant data required time and skill to

prepare the information for analysis.

Data sources differed in completeness. Most RCTs (18/21 [86%] and 6/7 [86%], respectively) were reported in

journal articles, which often presented unclear information related to trial quality. When nonpublic sources

were available for RCTs, clinical study reports contained mostly information about trial design and trial quality,

and clinical study reports and individual participant data contained the most results. In these case studies,

individual participant data were not accompanied by any metadata (e.g., codebooks or description of trial

methods).

For a single RCT reported in a single source, 1 outcome domain (e.g., pain intensity) could be associated with

multiple outcome definitions (e.g., using multiple measures) and results. Thus, multiple results for an outcome

could be presented selectively, even when an outcome domain was prespecified. Multiple data sources

associated with a single RCT sometimes contained contradicting information about trial design.

In the series of meta-analyses using results for a single outcome domain, measure, and time-point for each

case study, the effect of gabapentin on pain intensity ranged from SMD = –0.46 (95% CI –0.64, –0.28) to SMD

= –0.31 (95% CI –0.47, –0.15), and the effect of quetiapine on depression ranged from SMD = –0.42 (95% CI –

0.62, –0.22) to SMD = –0.44 (95% CI –0.63, –0.26). The range of results might have been wider had we

included other outcome measures and time-points in our meta-analyses.

Clinical study reports and individual participant data contained hundreds of harms (adverse events) that were

not included in public sources. Further analyses would be required to determine if conclusions about the

relative benefits and harms of treatment would be affected by a systematic synthesis of harms included only

in nonpublic sources.

Discussion: There is tremendous variation in the information available across multiple data sources from

individual trials. Reported estimates from individual trials were vulnerable to selective reporting at both the

trial and the systematic review levels.

Although an “open science” environment is a positive move forward, it will not solve all the problems that we

identified with how trial information is currently made available. For example, an open trial data set for the

purpose of making research reproducible allows for the investigation of new research questions, systematic

reviews, and individual participant data meta-analysis; however, the steps required to share a data set and the

steps required to utilize an open data set are resource intensive and demand skills that are not taught

routinely to trialists or systematic reviewers, such as creating, storing, accessing, synthesizing, and analyzing

information from multiple data sources. Furthermore, patients and stakeholders are often unfamiliar with the


wide variety of existing data sources and what can be gained from each of them. Thus, current proposals to

expand data sharing require careful and community-wide decision making about implementation.

Although guidance for systematic reviewers suggests searching extensively for information sources and

including all data sources found, this approach is neither practical nor possible for many systematic reviews.

Our research provides practical insights into a complex area that needs further research and discussion.


1. BACKGROUND

1.1. Objectives and overview

Our objectives were to determine whether multiple data sources about randomized controlled trials (RCTs)

affected systematic reviews and meta-analyses of patient-centered outcomes (PCOs) research and to produce

open access guidance about using multiple data sources for producers of systematic reviews of PCOs research.

To meet these objectives, we conducted a methods study related to the conduct of systematic reviews using 2

case studies: (1) gabapentin for neuropathic pain and (2) quetiapine for bipolar depression. We did not

conduct systematic reviews; instead, we examined whether and how the results of systematic reviews might

be affected by how trial data are made available, and how meta-analyses might be affected by those who use

data from multiple sources for a single trial. Although we investigated methods used in conducting systematic

reviews and the effect of those methods on meta-analyses, we did not consider our work to comprise multiple

systematic reviews because we did not evaluate the benefits and harms of the interventions per se. Our

findings may be of greatest interest to trialists, systematic reviewers, journal editors, and guideline

developers.

1.2. Patient-centered outcomes

We aimed to examine whether we found additional PCOs (meaningful to patients) by looking through all

reporting sources for each trial, not just a few sources. Systematic reviewers may rely on easy-to-find sources

(e.g., public sources) for trial outcomes, and they may not list PCOs that could be available from other sources

(e.g., nonpublic sources). Searching nonpublic sources may improve the impact of systematic reviews by

identifying PCOs that were recorded but not included in public sources. Examining nonpublic data sources for

PCOs could reduce the need for additional RCTs and improve the efficiency of PCOs research. As far as we are

aware, this possibility has never been addressed in research.


1.3. Reporting biases

Reporting biases threaten the reliability of RCTs and systematic reviews. Failure to report trials at all and

selective reporting of design, methods, or results represents an ethical breach with research participants who

believed that by participating in a trial they were contributing to knowledge. Reporting bias means that

information from a trial is not transmitted to the public and thus also constitutes wasted information.1 When

failure to report trials (or failure to report trials completely) is not random, it biases the trial findings and what

is generally believed to be true.2

1.4. Multiple data sources

To decrease the threat of reporting biases, current best practices for the conduct of systematic reviews

include a comprehensive search for trial information.3,4 Standards for such a search have focused on public

sources of information about trials and include going beyond journal articles and searching the “gray”

literature such as conference abstracts, FDA reviews, and trial registries. This is mainly because studies of

reporting bias have shown that not all information about trial design and quality can be found in journals, and

publication of trial results in a journal is associated with positive trial results.5,6

In addition, studies have shown that additional information about a trial can be found in nonpublic sources.7-9

Although systematic reviewers may wish to search for and include all relevant and reliable data in systematic

reviews, comprehensive searching adds to the time and resources required to complete systematic reviews. In

addition, information from the various sources may not always agree,8 and systematic reviewers must decide

which source is correct.10 The value of searching for nonpublic information is unclear, partly because the best

methods for searching are unknown.

Assuming that comprehensive searching for trial information provides additional data, questions remain

about which public and nonpublic sources should be searched for and used in systematic reviews. Empirically

grounded guidance is needed to help systematic reviewers make choices about the use of data from multiple

sources. Increasingly, the move to open science is making nonpublic sources available.11 Even if all that is

currently hidden becomes public, we would still need guidance to address the gains and challenges from

accessing multiple data sources for a single trial.


2. METHODS We published a protocol describing detailed methods for our study.12 What follows is a summary of our

methods.

2.1. Selection of case studies

To evaluate the effect of using data from multiple sources in systematic reviews and meta-analyses, we

examined 2 case studies regarding the effectiveness and safety of (1) gabapentin (Neurontin®) for neuropathic

pain in adults and (2) quetiapine (Seroquel®) for the treatment of depression in adults with bipolar disorder

following a published protocol.12 We used methods consistent with the National Academy of Medicine

(formerly the Institute of Medicine) standards for conducting systematic reviews.4

We selected these cases for several reasons. First, gabapentin and quetiapine are used commonly for these

respective conditions, so the included studies are clinically important. Second, pain and depression are

associated with several patient-reported outcomes. Methodologically, these cases also allow us to consider

different, typical situations that systematic reviewers might face with respect to data from multiple sources.

2.1.1. Gabapentin for neuropathic pain

Neuropathic pain occurs when there is damage to the peripheral nervous system, which is the array of nerves

that transmit information from the central nervous system (brain and spinal cord) to other parts of the body.

Between 3% and 10% of the population may be living with neuropathic pain, which affects 30% to 50% of

people with diabetes in particular.13 Other consequences include loss of sleep, reduced quality of life, and high

health care costs.

The FDA approved gabapentin for the treatment of epilepsy in 1993, and it approved gabapentin for the

treatment of postherpetic neuralgia (i.e., residual pain in people who have had shingles) in 2002. Gabapentin

is used off label for a variety of symptoms, including for neuropathic and other types of pain.

2.1.2. Quetiapine for bipolar depression

Bipolar disorder is characterized by episodes of depression and at least 1 episode of mania or hypomania14

and has a lifetime prevalence of 1% to 4%.15,16 Work, family life, and social life are impaired significantly by

depressive episodes.17,18 People with bipolar disorder are at increased risk of suicide compared with the

general population and compared with people who have other mental health problems.19,20

Quetiapine is an antipsychotic medication originally used for the treatment of acute psychotic episodes.

Quetiapine is currently recommended as a first-line choice for treatment of acute bipolar depression,21,22


although it is associated with outcomes that are undesirable to patients, including daytime sleepiness,

cognitive impairment, loss of libido, and rapid weight gain. Quetiapine and other antipsychotics are also

associated with serious adverse events, including cardiac and metabolic effects, and extrapyramidal

symptoms.23,24

2.2. Methods for identifying patient-centered outcomes

A goal of this project was to examine what matters most to patients who take gabapentin for neuropathic

pain and to patients who take quetiapine for bipolar depression. We used existing resources and consulted

patient coinvestigators. We compared the list of outcomes that matter most to patients with the outcomes

that were examined in clinical trials.

We identified PCOs by reviewing existing information sources, and we worked with our patient coinvestigators

and clinical experts to identify those outcomes they consider patient centered. Specifically, we examined the

following:

1. Core outcome sets (COSs) using the Core Outcome Measures in Effectiveness Trials database25 (http://www.comet-initiative.org) and the James Lind Alliance (http://www.lindalliance.org) in April and May 2014, respectively. We asked coinvestigators and colleagues to identify COSs related to neuropathic pain and bipolar disorder.

2. A social media website (http://www.patientslikeme.com/) in April 2014. PatientsLikeMe allows patients to enter information about their medical history and interventions they have used.

3. DRUGDEX (http://micromedex.com/) in January 2015. DRUGDEX is a compendium health care providers use to make treatment decisions. The Centers for Medicare and Medicaid may use DRUGDEX ratings to make reimbursement decisions related to medically accepted treatments. DRUGDEX contains a list, by organ system, of each drug’s adverse effects.

4. FDA prescribing information (“package inserts”) accessed through the Drugs@FDA website in April 2014

After examining existing sources, we asked patient and stakeholder partners to identify outcomes they

thought should be included in each case study. We discussed the outcomes identified through all sources, and

we selected those outcomes that patient and stakeholder partners deemed most important. We then

generated a list of outcomes reported in clinical trials for each case study, and we examined whether PCOs

were included in sources of clinical trials.

2.3. The effect of multiple data sources on meta-analyses

2.3.1. Eligibility criteria for a trial’s inclusion in our study

Our published protocol includes detailed inclusion and exclusion criteria for each case study.12 Table 1

describes the eligibility criteria for participants, interventions, outcomes, comparisons, and time-points.

http://www.comet-initiative.org/

http://www.lindalliance.org/

http://www.patientslikeme.com/

http://micromedex.com/


Table 1: Eligibility criteria for the case studies Gabapentin Quetiapine Participants Included:

• Adults (≥ 18 years) • Neuropathic pain • Not hospitalized Excluded: • Fibromyalgia

Included: • Adults (≥ 18 years) • Bipolar disorder (type I or II) • Acute depressive episode Excluded: • Rapid cycling

Interventions Included: • Oral gabapentin • ≥ 300mg per day • Regular or extended release Excluded: • Gabapentin enacarbil (a prodrug of gabapentin)

Included: • Oral quetiapine • ≥ 100mg per day • Regular or extended release Excluded: • Norquetiapine (a metabolite of quetiapine)

Comparisons Included: • No intervention • Pill placebo • Treatment(s) common to both groups (i.e., gabapentin was the only difference between groups) Excluded: • Other doses of gabapentin • Other active interventions • Discontinuation of gabapentin

Included: • No intervention • Pill placebo • Treatment(s) common to both groups (i.e., quetiapine was the only difference between groups) Excluded: • Other doses of quetiapine • Other active interventions • Discontinuation of quetiapine

Outcomes Five prespecified outcome domains: • Mood • Pain intensity • Pain interference • Quality of life • Sleep disturbance

Eight prespecified outcome domains: • Anxiety • Depression • Functioning • Psychiatric hospitalization • Quality of life • Sleep • Suicide • Weight

Time-points • 8 weeks (4-13 weeks) • 18 weeks (14-22 weeks) • 27 weeks (23-31 weeks)

• 8 weeks (4-13 weeks) • 18 weeks (14-22 weeks) • 27 weeks (23-31 weeks)

2.3.2. Search methods for identification of trials

We conducted electronic and additional searches to identify trials as described in our study protocol.12 We

searched for the following types of sources, which are listed by their approximate level of expected detail (i.e.,

less detail to more detail):

1. Trial registrations without results posted on trial registries (e.g., www.ClinicalTrials.gov) 2. Short reports (e.g., conference abstracts, posters)


3. Trial registrations without with summary data posted on trial registries 4. Peer reviewed journal articles 5. Dissertations (e.g., master’s or doctoral theses) 6. Unpublished manuscripts and reports (e.g., reports to funders, clinical study reports) 7. Information from regulators (i.e., FDA medical and statistical reviews) 8. Individual patient data

2.3.3. Selection of trials

Two investigators independently screened titles and abstracts to determine which were eligible for inclusion

in the review, as described in our study protocol.12 We independently screened full text reports of all

potentially relevant trials and resolved disagreements through consensus and by asking a third investigator as

needed.

2.3.4. Data collection

We extracted information about trial design as described in our study protocol.12 We recorded all outcomes

within the prespecified outcome domains, including the 5 elements described in Table 2.26,27

2.3.4.1. For sources of aggregate data

For trials associated with multiple sources (e.g., journal articles, clinical study reports), we extracted data from

each source, including trial design, trial quality (i.e., risk of bias), outcomes, and results. We developed data

collection forms and entered data in the Systematic Review Data Repository (SRDR).28,29

From single sources about more than 1 trial, we extracted information about each individual trial; we did not

extract results from multiple trials when the data had already been combined (i.e., we were unable to

separate data for an individual trial from the combined data).

We extracted results from figures and tables that included numerical data (e.g., counts and means included in

graphs), including statistical significance (e.g., p-value above or below 0.05). We did not meta-analyze

information that was presented only graphically. We extracted text from sources describing results as

“statistically significant” or “not significant.”

Table 1. The 5 Elements of an Outcome Element Definition Domain Title27 or concept30 to describe 1 or more outcomes Specific measure Instrument used to assess the outcome domain, including (1) the name

of the instrument or questionnaire and (2) the total score or the subscales that will be analyzed

Specific metric Unit of measurement (e.g., change from baseline, value at a time) Method of aggregation The primary statistical procedure for estimating the treatment effect

(e.g., mean, median), including definition of cutoff or pooling for


categorical variables (e.g., proportion with 50% decrease, proportion much or very much improved) The procedure for estimating the treatment effect, including (1) if the outcome will be treated as a continuous, categorical, or time-to-event variable, and (2) for continuous variables, a measure of central tendency (e.g., mean value); for categorical and time-to-event data variables, proportion with an event and, if relevant, the specific cutoff or categories compared

Time-point The period of follow-up, including (1) when outcome measurements will be obtained (e.g., weeks or months since randomization) and (2) which of the outcome measurements will be analyzed

Two investigators extracted data independently and resolved discrepancies by consensus and through

discussion with a third investigator if necessary. We used Stata 1431 to reconcile the records and to produce a

final data set that we will share publicly through SRDR at the end of the study.

2.3.4.2. For individual patient data

We received individual participant data for gabapentin trials from Pfizer as Microsoft Access database files.

When codebooks were not available, we compared the individual participant data with case report forms

(which often showed how and when data were recorded) and statistical analysis plans (which often showed

how data were coded and analyzed) to identify the variables contained in the individual participant data.

We did not receive any data sets about quetiapine from AstraZeneca. In 1 quetiapine clinical study report, we

identified an appendix with individual participant data, which we extracted using ABBYY FineReader version

11.32

2.4. Analysis

2.4.1. Assessment of trial quality

To compare the completeness of sources with respect to trial quality, we assessed each source using the

Cochrane Collaboration Risk of Bias Tool3 as described in our study protocol.12 Two investigators

independently rated each source for risk of bias (low, unclear, or high risk of bias) related to sequence

generation, allocation concealment, masking of participants, masking of outcome assessors, masking of

providers, and missing data. Investigators resolved discrepancies by consensus and through discussion with a

third investigator if necessary.


2.4.2. Series of meta-analyses

For each case study, we conducted a series of meta-analyses to explore the impact of multiple data sources on

the overall results. We analyzed these results by adding or replacing data as follows:

1. Including only results from publications in peer reviewed journals 2. Adding results from trial registries to Step 1 3. Adding results from short reports to Step 2 4. Adding or replacing results (from Step 3) using results obtained from regulators (e.g., FDA reviews) 5. Adding or replacing results (from Step 4) using summary results obtained from the authors or

manufacturers (e.g., clinical study reports) 6. Replacing the best available aggregate results for all trials (Step 5) with individual patient data where

available

For example, if there was no source for a trial in Steps 1 to 4 (i.e., no journal article, trial registration,

conference abstract, or FDA review) but there was a clinical study report, we added the results from the

clinical study report in Step 5. If there was a journal article, a FDA review, and a clinical study report, we

replaced the results from the journal article with the results from the FDA review in Step 4, and we replaced

the results from the FDA review with data from the clinical study report in Step 5.

For each outcome in each source, we determined if results included enough information to include them in

meta-analyses. We considered results meta-analyzable if they included a point estimate and measure of

precision (e.g., standard error, confidence interval). We meta-analyzed only data for which there were

sufficient information in a source to calculate the treatment effect; for example, we did not impute precision

if it was not reported.

For each case study, we combined results for the most common measure using the standardized mean

difference (SMD) for the difference between groups. In meta-analyses that included both aggregate data and

individual participant data, we analyzed individual participant data to obtain an aggregate result for each trial

and then we combined the aggregate results across all trials.33-36

We conducted all meta-analyses using random effects, and we report all SMDs with 95% confidence intervals

(CIs).37-39 We prespecified that we would use random effects because we anticipated that eligible trials might

include differences (e.g., dosage, participant characteristics) that would violate the assumptions required for

the fixed effect model. We compared information using the Dersimonian-Laird method, a very common

method of synthesis, assuming the random effects model. We have no reason to think that differences in

meta-analytic effect estimates using multiple data sources would be limited to whether we used 1 statistical

model or another to perform the meta-analysis.


Additional analyses—including other outcome measures, metrics, and methods of aggregation—are included

in manuscripts submitted for publication.

3. RESULTS Manuscripts describing our analyses and findings are currently under review. What follows is a summary of

our findings.

3.1. Patient-centered outcomes identified through existing sources and stakeholder coinvestigators

As our protocol12 describes, we selected outcomes that matter most to patients, using existing sources and in

collaboration with patient and stakeholder partners.

3.1.1. Core outcome sets

We identified a core outcome set for pain: the Initiative on Methods, Measurement, and Pain Assessment in

Clinical Trials (IMMPACT) consensus recommendations.40 IMMPACT recommends that the following 6 core

domains be assessed in clinical trials of chronic pain: pain, physical functioning, emotional functioning,

participant ratings of global improvement, symptoms and adverse events, and participant disposition.

We did not identify a core outcome set for bipolar depression. From the James Lind Alliance, we learned that

a working group had been formed to identify priorities for bipolar disorder in conjunction with the Oxford

Patients Active in Research project; no relevant outputs had been produced at the time of our study

3.1.2. Social media

Information on social media could not be used to identify PCOs. PatientsLikeMe reported the percentage of

patients who rated each intervention’s subjective effectiveness and side effects. Although patients could list

specific outcomes related to effectiveness (e.g., quality of life) or side effects (e.g., dizziness), PatientsLikeMe

does not include a function that allows patients to rate those specific side effects. PatientsLikeMe reports

information about the 6 most commonly rated side effects for each medication.

3.1.3. DRUGDEX

DRUGDEX entries for gabapentin and quetiapine included information about the use of the drugs for multiple

conditions. We identified information about the outcome domains pain intensity and depression, respectively;

however, we found little information about other potential benefits of treatment for either on-label or off-


label uses of the drugs. DRUGDEX did not always state whether harms were associated with all conditions for

which a drug might be used or with only specific conditions.

3.1.4. Food and Drug Administration prescribing information

Prescribing information included the outcome domains pain intensity for trials of gabapentin and depression

for trials of quetiapine. Prescribing information contained little information about other potential benefits. For

gabapentin, prescribing information did not include any information about potential benefits for off-label uses

(i.e., types of neuropathic pain for which gabapentin is not approved by FDA).

Prescribing information included many potential harms, and it did not always state whether information

about harms was applicable to any condition or to only specific conditions.

3.1.5. Patient and stakeholder investigators

Patient and stakeholder partners identified outcomes they thought should be included in each review. We

created a questionnaire with a list of potential outcomes, and patients rated how much the outcomes

mattered to them. We discussed the outcomes identified through all sources and selected those outcomes

that patient and stakeholder partners thought were most patient centered.

3.2. The effects of adding and replacing data from multiple data sources on the results of meta-analyses

3.2.1. Results of the search

We identified 21 eligible trials of gabapentin (from 1997 to 2013) and 7 eligible trials of quetiapine (from 2003

to 2014). As Tables 3 and 4 show, most of these trials were associated with more than 1 source (71% and

100%, respectively). We identified 74 (gabapentin) and 50 (quetiapine) reports and extracted information

from them. Journal articles and conference abstracts were the most common reports. We identified 1

gabapentin trial only through a conference abstract, and 1 gabapentin trial only on ClinicalTrials.gov

(www.clinicaltrials.gov).

FDA reviews included information about 2 trials of gabapentin for postherpetic neuralgia. FDA reviews

available online did not include trials of quetiapine for bipolar depression, which was added to the label in a

supplemental new drug application.

We identified few registrations for trials of gabapentin on ClinicalTrials.gov and other registries, none of the

trials conducted by the manufacturer (Pfizer). In contrast, all eligible trials of quetiapine were registered.

http://www.clinicaltrials.gov/


We received clinical study reports and individual participant data for 6 gabapentin trials as a result of the

litigation we describe above7,41; we received no additional information from Pfizer as a result of our requests.

By searching online, we identified clinical study reports for 2 quetiapine trials that were released during

litigation (http://psychrights.org/); 1 of the clinical study reports included individual participant data in an

appendix. AstraZeneca declined our request for more information about quetiapine trials, and our

correspondence with AstraZenenca has been published.42 Thus, in our study, all nonpublic sources were

previously nonpublic, since they were available to us online from litigation sources (i.e., they were disclosed

during patient lawsuits), and we received no additional sources from the trials’ sponsors.

In summary, we found multiple sources for most trials, the largest contributors to multiple reports being

journal articles and conference abstracts.

http://psychrights.org/


Table 3. Multiple Sources of Trials of Gabapentin for Neuropathic Pain Public Sources Nonpublic Sources

Trial Name Reports

Journal (1 Trial)

Journal (> 1 Trial)

Conference Abstract

Other Source

Trial Registry

FDA Review

CSR IPD All Sources

(Total) Arai 201043,44 1 0 1 0 0 0 0 0 2

Backonja 199845-58 2 1 5 5 0 0 1 1 15 Caraceni 200459-61 1 0 0 2 0 0 0 0 3

Hahn 200462 1 0 0 0 0 0 0 0 1 Hui 201063,64 1 0 0 0 1 0 0 0 2

Irving 200965,66 2 0 0 0 0 0 0 0 2 Milenkovic 200967 0 0 1 0 0 0 0 0 1

Mishra 201268 1 0 0 0 0 0 0 0 1 NCT0047590469 0 0 0 0 1 0 0 0 1

Rice 200149-53,70-77 2 2 2 4 0 2 1 1 14 Rowbotham 199849-53,71,73,76-81 1 2 2 5 0 2 1 1 14

Sandercock 201282-84 1 0 0 1 1 0 0 0 3 Sang 201385-101 1 4 10 1 1 0 0 0 17

Serpell 200249-53,71,102-105 1 2 3 3 0 0 1 1 11 Simpson 2001106,107 1 0 1 0 0 0 0 0 2

Study 945-22449-53,108 0 1 2 2 0 0 1 1 7 Study A945-1008109 0 0 0 0 0 0 1 1 2

Tamez Pérez 2000110,111 1 0 0 1 0 0 0 0 2 Wallace 201088-100,112-114 1 4 10 0 1 0 0 0 16

Yildirim 2003115 1 0 0 0 0 0 0 0 1 Zepeda Vazquez 2001116 1 0 0 0 0 0 0 0 1

Trials (total) 17 7 10 9 5 2 6 6 21 Unique sources (total) 20 6 20 15 5 2 6 6 80

Note: Journal (1 Trial) = journal article about a single included trial; Journal (> 1 Trial) = journal article about 2 or more included trials; Other Source = letters, summaries of journal articles, press releases, brief reports in trade publications (e.g., not peer reviewed); Trial Registry = trial registration on ClinicalTrials.gov or another registry; FDA Review = US Food and Drug Administration medical or statistical review; CSR = clinical study report; IPD = individual patient data. Each cell represents the number of sources of a particularly type (e.g., journal articles) for a given trial.


Table 4. Multiple Sources of Trials of Quetiapine for Bipolar Depression

Public Sources Nonpublic Sources

Trial Name Reports

Journal (1 Trial)

Journal (> 1 Trial)

Conference Abstract

Other Source

CSR-S Trial Registry

CSR IPD All Sources

(Total) Calabrese 2004117-142 4 5 14 1 0 1 1 1 27

Gao 2014143,144 1 0 0 0 0 1 0 0 2 Li 2014145,146 0 0 1 0 0 1 0 0 2

McElroy 2010125-129,133,141,147-151 1 1 7 1 1 1 0 0 12 Suppes 2010152-154 1 0 1 0 0 1 0 0 3

Thase 2006125-133,138,141,155-160 2 5 6 2 0 1 1 0 17 Young 2008125-129,133,141,148,161-166 1 1 8 2 1 1 0 0 14

Trials (total) 6 4 6 4 2 7 2 1 7 Unique sources (total) 10 5 19 5 2 7 2 1 51

Note: Journal (1 Trial) = journal article about a single included trial; Journal (> 1 Trial) = journal article about 2 or more included trials; Other Source = letters, summaries of journal articles, press releases, brief reports in trade publications (e.g., not peer reviewed); CSR-S = clinical study report synopsis; Trial Registry = trial registration on ClinicalTrials.gov or another registry; CSR = clinical study report; IPD = Individual patient data. Each cell represents the number of sources of a particularly type (e.g., journal articles) for a given trial.


3.2.2. Completeness of information

Trial design (e.g., participant, intervention characteristics) was best described in clinical study

reports and journal articles. For example, conference abstracts were incomplete and sometimes

contradicted other sources. Five trial registrations submitted to ClinicalTrials.gov by AstraZeneca

described a single intervention, “quetiapine fumarate,” while other sources indicated that 2 groups

received quetiapine at different doses (i.e., 300mg and 600mg).

Trial quality (defined as our assessment of the risk of bias for each trial) was often unclear in the

sources we identified (see Tables 5 and 6). When we located enough information to assess risk of

bias (usually in a journal article or clinical study report), that information almost always led from an

unclear to a low risk of bias rating. The only exception was information about missing data, where

identification of additional information led from an unclear to a high risk of bias rating for trials with

large amounts of missing data.

Statistical information about many outcomes described in journal articles and short reports did not

include sufficient detail for results to be included in meta-analysis. In contrast, data in clinical study

reports were usually complete and could almost always be used for analysis.

For these reasons, clinical study reports—when they were available—were the most complete

sources of information about methods and results.


Table 5. Trial Quality (Risk of Bias) for Sources of Gabapentin for Neuropathic Pain Trials

Total

Sources Sequence

Generation Allocation

Concealment Masking

Participants Masking Providers

Masking Outcome Assessment

Missing Data for Primary Outcome

Trial Name Reports H U L H U L H U L H U L H U L NA H U L NA Arai 201043,44 2 0 1 1 0 2 0 0 2 0 0 2 0 0 2 0 0 0 2 0 0

Backonja 199845-58 14 0 13 1 0 13 1 0 12 2 0 14 0 0 14 0 0 0 12 2 0 Caraceni 200459-61 3 0 3 0 0 2 1 0 3 0 0 3 0 0 3 0 0 0 2 1 0

Hahn 200462 1 0 1 0 0 0 1 0 0 1 0 1 0 0 1 0 0 0 1 0 0 Hui 201063,64 2 0 1 1 0 2 0 0 1 1 0 2 0 0 0 2 0 0 2 0 0

Irving 200965,66 2 0 1 1 0 2 0 0 1 1 0 2 0 0 2 0 0 0 1 1 0 Milenkovic 200967 1 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 0 1 0 0

Mishra 201268 1 0 0 1 0 1 0 0 1 0 0 1 0 0 1 0 0 0 1 0 0 NCT0047590469 1 0 1 0 0 1 0 0 0 1 0 0 1 0 0 1 0 0 0 1 0

Rice 200149-53,70-77 13 0 11 2 0 11 2 0 12 1 0 13 0 0 13 0 0 0 7 5 1 Rowbotham 199849-53,71,73,76-81 13 0 12 1 0 13 0 0 12 1 0 13 0 0 13 0 0 0 11 1 1

Sandercock 201282-84 3 1 2 0 0 3 0 0 3 0 0 3 0 0 3 0 0 0 1 1 1 Sang 201385-101 17 0 15 2 0 17 0 0 16 1 0 17 0 0 17 0 0 0 13 2 2

Serpell 200249-53,71,102-105 10 0 9 1 0 9 1 0 10 0 0 10 0 0 10 0 0 0 7 2 1 Simpson 2001106,107 2 0 2 0 0 2 0 0 2 0 0 2 0 0 2 0 0 0 2 0 0

Study 945-22449-53,108 6 0 6 0 0 5 1 0 5 1 0 5 1 0 6 0 0 0 5 1 0 Study A945-1008109 1 0 0 1 0 1 0 0 0 1 0 1 0 0 1 0 0 0 0 1 0

Tamez Pérez 2000110,111 2 0 2 0 0 2 0 0 2 0 0 2 0 0 2 0 0 0 2 0 0 Wallace 201088-100,112-114 16 0 15 1 0 15 1 0 15 1 0 16 0 0 16 0 0 0 13 1 2

Yildirim 2003115 1 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 0 Zepeda Vazquez 2001116 1 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 0 1 0 0

Note: For each included trial, this table shows the number of sources (excluding individual participant data) at high (H), unclear (U), and low (L) risk of bias for each item. NA = not applicable because the source did not report outcomes.


Table 6. Trial Quality (Risk of Bias) by Source of Quetiapine for Bipolar Depression Trials

Total

Sources Sequence

Generation Allocation

Concealment Masking

Participants Masking Providers

Masking outcome Assessment

Missing Data for Primary Outcome

Trial Name Reports H U L H U L H U L H U L H U L NA H U L NA Calabrese 2004117-142 26 0 22 4 0 22 4 0 25 1 0 25 1 0 22 1 3 5 14 0 7

Gao 2014143,144 2 0 2 0 0 2 0 0 1 1 0 1 1 0 2 0 0 1 1 0 0 Li 2014145,146 2 0 2 0 0 2 0 0 1 1 0 2 0 0 2 0 0 0 2 0 0

McElroy 2010125-129,133,141,147-151 12 0 10 2 0 10 2 0 11 1 0 12 0 0 12 0 0 2 7 0 3 Suppes 2010152-154 3 0 3 0 0 3 0 0 3 0 0 3 0 0 3 0 0 1 2 0 0

Thase 2006125-133,138,141,155-160 17 0 14 3 0 14 3 0 17 0 0 17 0 0 17 0 0 4 10 0 3 Young 2008125-129,133,141,148,161-166 14 0 12 2 0 12 2 0 13 1 0 13 1 0 14 0 0 2 9 0 3

Note: For each included trial, this table shows the number of sources (excluding individual participant data) at high (H), unclear (U), and low (L) risk of bias for each item. NA = not applicable because the source did not report outcomes.


3.2.3. Locating patient-centered outcomes in multiple data sources

Across all sources for all gabapentin trials, we located 4 of 5 prespecified outcome domains as

described in Section “3.1. Patient-centered outcomes identified through existing sources and

stakeholder coinvestigators” (Tables 7 and 8). Across all sources for all quetiapine trials, we

identified 7 of 8 prespecified domains. Quetiapine sources included about 6 (median) outcome

domains, and they were also highly variable (range 2 to 6 domains per source). To our knowledge,

none of the trials measured the prespecified outcome domains for pain interference (gabapentin) or

psychiatric hospitalization (quetiapine).

Conference abstracts and other short reports included little useful information about PCOs. Journal

articles contained more PCOs than did other public sources, although clinical study reports were

consistently the best source of aggregate information about PCOs. No outcome measures appeared

in public sources that did not also appear in corresponding clinical study reports; however, we

identified methods of aggregation in public sources (i.e., journal articles and an FDA review) for

gabapentin trials that were not included in clinical study reports.

In addition to the prespecified outcomes related to treatment benefits, clinical study reports and

individual participant data described hundreds of harms that were not included in public sources.167


Table 7. Number of Prespecified Outcome Domains in Each Source of Gabapentin for Neuropathic Pain Trials Public Sources Nonpublic Sources

Trial Name Reports Journal (1 Trial)

Journal (> 1 Trial)

Conference Abstract

Other Source

Trial Registry

FDA Review

CSR IPD All Sources (Total)

Arai 201043,44 0 NA 1 NA NA NA NA NA 1 Backonja 199845-58 4 4 4 3 NA NA 4 4 4 Caraceni 200459-61 0 NA NA 0 NA NA NA NA 0

Hahn 200462 2 NA NA NA NA NA NA NA 2 Hui 201063,64 1 NA NA NA 1 NA NA NA 1

Irving 200965,66 2 NA NA NA NA NA NA NA 2 Milenkovic 200967 NA NA 1 NA NA NA NA NA 1

Mishra 201268 1 NA NA NA NA NA NA NA 1 NCT0047590469 NA NA NA NA 1 NA NA NA 1

Rice 200149-53,70-77 3 3 0 1 NA 3 3 3 3 Rowbotham 199849-53,71,73,76-81 4 3 0 4 NA 4 4 4 4

Sandercock 201282-84 2 NA NA 2 2 NA NA NA 2 Sang 201385-101 2 1 1 1 2 NA NA NA 2

Serpell 200249-53,71,102-105 2 2 2 2 NA NA 2 2 2 Simpson 2001106,107 4 NA 3 NA NA NA NA NA 4

Study 945-22449-53,108 NA 3 0 1 NA NA 3 3 3 Study A945-1008109 NA NA NA NA NA NA 4 3 4

Tamez Pérez 2000110,111 1 NA NA 1 NA NA NA NA 1 Wallace 201088-100,112-114 2 1 1 NA 2 NA NA NA 2

Yildirim 2003115 1 NA NA NA NA NA NA NA 1 Zepeda Vazquez 2001116 1 NA NA NA NA NA NA NA 1

Total 4 4 4 4 2 4 4 4 4 Note: Journal (1 Trial) = journal article about a single included trial; Journal (> 1 Trial) = journal article about 2 or more included trials; Other Source = letters, summaries of journal articles, press releases, brief reports in trade publications (e.g., not peer reviewed); Trial Registry = trial registration on ClinicalTrials.gov or another registry; FDA Review = US Food and Drug Administration medical or statistical review; CSR = clinical study report; IPD = individual patient data. For each trial, we recorded the number of prespecified domains (pain intensity, pain interference, sleep disturbance, mood, and quality of life) that could be extracted from each type of source and from all sources about that trial. None of the reports described pain interference.


Table 8. Number of Prespecified Outcome Domains in Each Source of Quetiapine for Bipolar Depression Pain Trials Public Sources Nonpublic Sources

Trial Name Reports Journal (1 Trial)

Journal (> 1 Trial)

Conference Abstract

Other Source

CSR-S Trial Registry

CSR IPD All Sources (Total)

Calabrese 2004117-142 6 3 4 2 NA 0 6 6 6 Gao 2014143,144 3 NA NA NA NA 3 NA NA 3

Li 2014145,146 NA NA 0 NA NA 2 NA NA 2 McElroy 2010125-129,133,141,147-151 6 0 3 4 6 1 NA NA 6

Suppes 2010152-154 3 NA 1 NA NA 1 NA NA 3 Thase 2006125-133,138,141,155-160 6 2 0 3 NA 1 6 NA 6

Young 2008125-129,133,141,148,161-166 6 0 3 4 4 1 NA NA 6 Total 7 4 5 6 6 4 7 6 7

Note: Journal (1 Trial) = journal article about a single included trial; Journal (> 1 Trial) = journal article about 2 or more included trials; Other Source = letters, summaries of journal articles, press releases, brief reports in trade publications (e.g., not peer reviewed); CSR-S = clinical study report synopsis; Trial Registry = trial registration on ClinicalTrials.gov or another registry; CSR = clinical study report; IPD = individual patient data. For each trial, we recorded the number of prespecified domains (depression, functioning, quality of life, anxiety, sleep, weight, psychiatric hospitalization, and suicide) that could be extracted from each type of source and from all sources about that trial. None of the reports described psychiatric hospitalization.


3.2.4. Multiple outcome definitions

As noted earlier, it is also possible for a single prespecified outcome domain to be assessed in

multiple ways (i.e., by varying the measure, metric, and method of aggregation). For this reason, an

outcome might appear to be reported as described in the protocol when in fact there were multiple

opportunities for changing the associated results (Figure 1).

Figure 1. The Number of Outcomes in a Trial Is a Function of the Number of Definitions of Each of the 5 Elements

Note: In this hypothetical example, the number of treatment effects for a single trial is the product of definitions for each element. In a trial with 4 outcome domains, introducing 2 definitions for each of the other elements will result in 64 different outcomes.

For each case study, we identified hundreds of outcomes for the prespecified domains (Figure 2). By

varying 4 elements, gabapentin RCTs reported 214 outcomes and quetiapine RCTs reported 81

outcomes. For pain intensity and depression, the most common outcome domains for gabapentin

and quetiapine RCTs, we found 119 and 44 outcomes, respectively. Multiple methods of analysis

also contributed to multiple results for the same defined outcome. For these reasons, many meta-

analyses were possible for each outcome domain using the same included trials.167


Figure 2. The Number of Outcomes Observed Increased as a Consequence of Multiple Definitions of the Elements

Note: The points nearest the center of the circle represent the prespecified outcome domains for which we found any information in eligible trials (i.e., 4 outcome domains for gabapentin and 7 outcome domains for quetiapine); these are represented by different colors. The branches represent variations in outcome definitions associated with those domains (i.e., different measures, metrics, methods of aggregation, or time-points). Each point around the outer edge of the circle thus represents a unique outcome described in 1 or more of the trial sources we identified. (A) Specific measure (name of instrument): BPI = Brief Pain Inventory; CGIC = Clinician Global Impression of Change; ECOG = Eastern Cooperative Oncology Group; GIC = Global Impression of Change; GSS = Global Symptom Score; HADS = Hospital Anxiety and Depression Scale; HRQoLQ = Health Related Quality of Life Questionnaire; NPS = Neuropathic Pain Scale; Pain (11 pt) = pain score (11-point scale); Pain (4 pt) = pain score (4-point scale); PGIC = Patient Global Impression of Change; POMS = Profile of Mood States; SF-36 = Short Form 36 item; SFMPQ = Short Form McGill Pain Questionnaire; Sleep score =

MUDS Paper 2: Multiple outcomes and analyses 30


sleep score (11-point scale); TOPS = Treatment Outcomes in Pain Survey. Specific measure (total or subscale): % pain relief = percentage pain relief; Ave. pain = average pain score; Care sat = health care satisfaction; Current pain = current pain score; Fear/Avoid = fear avoidance; Gen health = general health; Globe intensity = global intensity; Globe unpleas = global unpleasantness; Lower body = lower body functional limitations; Ment health = mental health; Obj. social = objective family/social disability; Obj. work = objective work disability; Pain sympt = pain symptom; Passive cope = passive coping; Patient sat = patient satisfaction with outcomes; Phys funct = physical functioning; PPI = present pain intensity; Role emot = role emotional; Role phys = role physical; Social disab = perceived family/social disability; Social funct = social functioning; Solicitous = solicitous responses; Total pain = total pain experience; Total score = total; Upper body = upper body functional limitations; VAS = visual analog scale; Work limit = work limitations. Metric: Change = change from baseline; Value = value at a time-point. Method of aggregation: % change mean = percentage change mean; % change median = percentage change median; 2 decrease = at least 2-point decrease; 3 decrease = at least 3-point decrease; 4 decrease = at least 4-point decrease; M or VM improved = much or very much improved; M or VM worse = much worse or very much worse; Min or No change = minimally improved or no change; Min improved = minimally improved; Min worse = minimally worse; Mod improved = moderately improved; Mod or much imp = moderately or much improved; Mod worse = moderately worse; No change = no change (0%); Pain free = totally pain free; Response (ND) = response (not defined); V much improved = very much improved; V much worse = very much worse. (B) Specific measure (name of instrument): BMI = Body Mass Index; CGI-I = Clinical Global Impression - Improvement; CGI-S = Clinical Global Impression - Severity; HAM-A = Hamilton Anxiety Rating Scale; HAM-D = Hamilton Depression Rating Scale; MADRS = Montgomery Asberg Depression Rating Scale; MADRS/YMRS = Montgomery Asberg Depression Rating Scale & Young Mania Rating Scale; MOS Cog = Medical Outcomes Study Cognitive Scale; Not defined = not defined; PSQI = Pittsburgh Sleep Quality Assessment; Q-LES-Q = Quality of Life Enjoyment and Satisfaction Questionnaire; QIDS-SR-16 = Quick Inventory of Depressive Symptomology Self-Report 16; SDS = Sheehan Disability Scale; Ideation = suicidal ideation (self-reported). Metric: Change = change from baseline; Value = value at a time-point. Method of aggregation: “= 10” = less than or equal to 10; “= 12” = less than or equal to 12; “= 7” = less than or equal to 7; “= 8” = less than or equal to 8; 15% increase = at least 15% increase; 25% increase = at least 25% increase; 30% decrease = at least 30% decrease; 40% decrease = at least 40% decrease; 50% decrease = at least 50% decrease; 60% decrease = at least 60% decrease; 7% increase = at least 7% increase; 70% decrease = at least 70% decrease; Extremely ill = among the most extremely ill; M or VM improved = much or very much Improved; MADRS = 12, YMRS = 12 = MADRS less than or equal to 12 and YMRS less than or equal to 12; MADRS = 12, YMRS = 8 = MADRS less than or equal to 12 and YMRS less than or equal to 8; Mean % Max = mean percentage of the maximum; Min improved = minimally improved; Min worse = minimally worse; Normal or Borderline = normal, not at all ill or borderline ill; Normal, not ill = normal, not at all ill; Proportion = proportion of participants who experienced the event; Remission (ND) = remission (not defined); Response (ND) = response (not defined); V Much improved = very much improved; V Much worse = very much worse.


3.2.5. Impact of adding or replacing multiple data sources on pooled effect sizes and precision

For each case study, we conducted a series of meta-analyses in which we added or replaced data

from each of the multiple sources for a single outcome domain, measure, and time-point to evaluate

the effect of adding or replacing data. In the presence of reporting bias, whereby negative or null

results are reported in nonpublic sources but not in public sources, we would expect the addition of

nonpublic sources to change the average effects estimated in meta-analyses. This change would lead

to a meta-analytic effect estimate closer to the null.

Not all trials could contribute data to a meta-analysis (Tables 9 and 10). Of the 21 gabapentin and 7

quetiapine trials, 9 (43%) and 4 (57%), respectively, could not be included in an analysis of pain

intensity or depression using continuous data (e.g., the mean) because no source for those trials

included meta-analyzable data (see “2.4.2. Series of meta-analyses”).

When restricted to the most common outcome defined as the same outcome domain, measure, and

time-point, results in both case studies were largely consistent across sources. For gabapentin,

including data from previously nonpublic sources (i.e., clinical study reports and individual

participant data) reduced the magnitude of the effect estimate; adding additional sources did not

affect the statistical significance of the treatment effect. In our series of meta-analyses, the effect of

gabapentin on pain intensity ranged from SMD = –0.46 (95% CI –0.64, –0.28) to SMD = –0.31 (95%

CI –0.47, –0.15). For quetiapine, there were no meaningful differences as a result of adding and

replacing data from multiple sources. The effect of quetiapine on depression ranged from SMD = –

0.42 (95% CI –0.62, –0.22) to SMD = –0.44 (95% CI –0.63, –0.26).


Table 9. Series of Meta-analyses and the Effect of Adding or Replacing Data Sources on the Pooled Estimate of Pain Intensity in Trials of Gabapentin for Neuropathic Pain (SMD and 95% CI)

Trial Name Reports Including Only Peer Reviewed Journals

Journal Articles Plus Registries

Journal Articles, Registries Plus Conference Abstracts

FDA Reports, Journal Articles, Registries, and Conference Abstracts

CSRs, FDA Reports, Journal Articles, Registries, and Conference Abstracts

IPD, CSRs, FDA Reports, Journal Articles, Registries, and Conference Abstracts

Backonja 199845-58 –0.57 (–0.88, –

0.25) –0.57 (–0.88, –

0.25) –0.57 (–0.88, –0.25) –0.57 (–0.88, –

0.25) –0.56 (–0.87, –

0.24) –0.53 (–0.84, –

0.22)

Irving 200965,66 –0.37 (–0.71, –

0.04) –0.37 (–0.71, –

0.04) –0.37 (–0.71, –0.04) –0.37 (–0.71, –

0.04) –0.37 (–0.71, –

0.04) –0.37 (–0.71, –

0.04) Mishra 201268 –0.44 (–0.95, 0.06) –0.44 (–0.95, 0.06) –0.44 (–0.95, 0.06) –0.44 (–0.95, 0.06) –0.44 (–0.95, 0.06) –0.44 (–0.95, 0.06) NCT0047590469 No sources 0.32 (0.04, 0.60) 0.32 (0.04, 0.60) 0.32 (0.04, 0.60) 0.32 (0.04, 0.60) 0.32 (0.04, 0.60)

Rice 200149-53,70-77 –0.64 (–0.92, –

0.36) –0.64 (–0.92, –

0.36) –0.64 (–0.92, –0.36) –0.64 (–0.92, –

0.36) –0.63 (–0.86, –

0.40) –0.56 (–0.80, –

0.33)

Rowbotham 199849-53,71,73,76-81 –0.86 (–1.13, –

0.59) –0.86 (–1.13, –

0.59) –0.86 (–1.13, –0.59) –0.86 (–1.13, –

0.59) –0.86 (–1.13, –

0.59) –0.83 (–1.11, –

0.56)

Sandercock 201282-84 –0.49 (–0.84, –

0.15) –0.49 (–0.84, –

0.15) –0.49 (–0.84, –0.15) –0.49 (–0.84, –

0.15) –0.49 (–0.84, –

0.15) –0.49 (–0.84, –

0.15)

Sang 201385-101 –0.22 (–0.40, –

0.03) –0.22 (–0.40, –

0.03) –0.22 (-0.40, –0.03) –0.22 (–0.40, –

0.03) –0.22 (–0.40, –

0.03) –0.22 (–0.40, –

0.03) Serpell 200249-53,71,102-105 No sources No sources No sources No sources –0.22 (–0.45, 0.01) –0.20 (–0.43, 0.03) Study 945-22449-53,108 No sources No sources No sources No sources –0.09 (–0.35, 0.17) –0.06 (–0.32, 0.19)

Study A945-1008109 No sources No sources No sources No sources –0.43 (–0.66, –

0.20) –0.26 (–0.46, –

0.06) Wallace 201088-100,112-114 –0.20 (–0.40, 0.01) –0.20 (–0.40, 0.01) –0.20 (–0.4, 0.01) –0.20 (–0.40, 0.01) –0.20 (–0.40, 0.01) –0.20 (–0.40, 0.01) Total

Number (No.) of trials 8 9 9 9 12 12 No. of participants 1805 2019 2019 2019 3074 3144

Meta-analytic effect size (SMD) –0.46 (–0.64, –0.28)

–0.38 (–0.60, –0.15) –0.38 (–0.60, –0.15)

–0.38 (–0.60, –0.15)

–0.34 (–0.51, –0.18)

–0.31 (–0.47, –0.15)

I2 68.75 82.56 82.56 82.56 79.00 76.94



Note: Data for individual trials are provided in each row. In the columns, we first estimated the effect sizes using only journal articles; we then estimated the effect by including other sources to the analysis (i.e., adding data from new sources or replacing the data from sources appearing in columns the left). When we used IPD, we first obtained aggregated data and then combined data from all sources, as described in the Methods. For each included study, we calculated the standardized mean difference (SMD) and 95% CI for the intervention compared with placebo. We combined the results (“Total”) by conducting a meta-analysis using random effects. Effect sizes are given for all trials that reported pain intensity (measured using the average daily pain score on an 11-point scale) in at least 1 source.



Table 10. Series of Meta-analyses and the Effect of Adding or Replacing Data Sources on the Pooled Estimate of Depression in Trials of Quetiapine Bipolar Depression (SMD and 95% CI)

Trial Name Reports Including Only Peer Reviewed Journals

Journal Articles Plus Registries

Journal Articles, Registries Plus Conference Abstracts

FDA Reports, Journal Articles, Registries, and Conference Abstracts

CSRs, FDA Reports, Journal Articles, Registries, and Conference Abstracts

IPD, CSRs, FDA Reports, Journal Articles, Registries, and Conference Abstracts

Calabrese 2004117-142 –0.63 (–0.84, –0.41)

–0.63 (–0.84, –0.41)

–0.63 (–0.84, –0.41) –0.63 (–0.84, –0.41)

–0.59 (–0.78, –0.41)

–0.59 (–0.78, –0.4)

Thase 2006125-133,138,141,155-160 –0.37 (–0.56, –0.17)

–0.37 (–0.56, –0.17)

–0.37 (–0.56, –0.17) –0.37 (–0.56, –0.17)

–0.46 (–0.66, –0.27)

–0.46 (–0.66, –0.27)

Young 2008125-129,133,141,148,161-166 –0.27 (–0.46, –0.08)

–0.27 (–0.46, –0.08)

–0.27 (–0.46, –0.08) –0.27 (–0.46, –0.08)

–0.27 (–0.46, –0.08)

–0.27 (–0.46, –0.08)

Total Number (No.) of trials 3 3 3 3 3 3

No. of participants 1453 1453 1453 1453 1625 1629 Meta-analytic effect size (SMD) –0.42 (–0.62, –

0.22) –0.42 (–0.62, –

0.22) –0.42 (–0.62, –0.22) –0.42 (–0.62, –

0.22) –0.44 (–0.63, –

0.26) –0.44 (–0.62, –

0.26) I2 66.45 66.45 66.45 66.45 63.70 63.15

Note: Data for individual trials are provided in each row. In the columns, we first estimated the effect sizes using only journal articles; we then estimated the effect by including other sources to the analysis (i.e., adding data from new sources or replacing the data from sources appearing in columns the left). When we used IPD, we first obtained aggregated data and then combined data from all sources, as described in the Methods. For each included trial, we calculated the standardized mean difference (SMD) and 95% CI for the intervention compared with placebo. We combined the results (“Total”) by conducting a meta-analysis using random effects. Effect sizes are given for all trials that reported depression measured using the Montgomery Åsberg Depression Rating in at least 1 source.


3.2.6. Resources required to use multiple data sources in systematic reviews and meta-analyses

Available guidance for performing systematic reviews (e.g., from the Cochrane Collaboration and the

National Academy of Medicine [formerly the Institute of Medicine]) suggests that reviewers should

use all data sources for clinical trials.3,4 Identifying and extracting data from multiple data sources

was resource intensive and required many skills, and our findings suggest that existing guidance

could be improved to help systematic reviewers use multiple data sources efficiently (Table 11).


Table 11: Main Findings From MUDS and our Recommendations for Application of the Findings Findings Recommendation Audience FINDING 1: Accessing, Obtaining, and Analyzing Multiple Sources Is Resource Intensive and Tedious, and Requires Many Skills. Most trials lead to multiple sources. • Users of research should be able to identify clinical trials.

• PCOs can be found in multiple sources about trials. • Clinicians, patients • Guideline developers,

patients, systematic reviewers

Multiple sources are needed to understand what was planned and what actually occurred in clinical trials.

• To prevent bias and reduce waste, trials should use common PCOs (core outcome sets).

• All information about a clinical trial should be stored in a single public source.

• Trialists and manufacturers should make information consistent across sources.

• Funders, trialists • Funders, trialists • Trialists

Some sources (e.g., trial protocols) are rarely available. • Protocols submitted to IRBs should be publicly available. • IRBs should monitor compliance with registration and reporting

requirements.

• Funders, trialists • Funders, trialists

Some trials are not reported in public sources (e.g., journal articles), and nonpublic sources are available for many trials that are reported.

• All trials should be registered in advance. • Results from all trials should be shared with sufficient information to

understand the variables and to reproduce the analyses. • To reduce waste, trials should use common data elements. • Individual patient data are useful for some applications but not all.

• Trialists • Funders, trialists

• Funders, trialists • Systematic reviewers

FINDING 2: Information Varies Tremendously Across Sources. Sources differ in completeness. • Public sources should be available for all trials with sufficient

information to understand their design, quality, and results. • Outcomes data should be reported with sufficient information to

include results in meta-analyses.

• Funders, guideline developers, trialists

• Trialists

Sources include different information. • All sources should include complete descriptions of all outcomes examined, including all elements of each outcome.

• Users should expect variation across sources and prioritize variables/sources in advance.

• Trialists

• Guideline developers, systematic reviewers

Trials and sources included results that could not be compared as well as incomplete information.

• To be usable, databases should employ common fields. • Shared data should be accompanied by codebooks and other metadata

that allow examination of the data.

• Trialists

Nonpublic data sources included additional information. • Trialists should anticipate requests for data and plan to share from the time a trial is initiated.

• Systems are needed to archive and share new research, and to share “legacy” trials related to drugs in current use.

• Funders, trialists

• Guideline developers, funders

June 20, 2017 Final research report (ME-1303-5785) 37 of 50

In general, we were able to extract information from conference abstracts quickly; journal articles

took several hours to extract.

Clinical study reports, which included thousands of pages of appendices and hundreds of outcomes

and analyses, required dozens of hours. Two extractors each spent at least 5 8-hour days extracting

data from each clinical study report.

Because we did not receive codebooks for individual participant data, each data set was a unique

puzzle we had to solve even before we could synthesize the data. Members of our team spent

months developing codebooks and creating a common data format to combine data across trials.

Even with this investment, there were some variables we could not identify with certainty.

3.3. Guidance for using multiple data sources in systematic reviews

As a final step in our investigation, we developed guidance for systematic reviewers (Table 12). For

the open science movement to succeed, systematic reviewers need to make efficient use of

evidence from clinical trials, including previously nonpublic information. Although it would have

been desirable to identify a best source of information for clinical trials, we did not find a single

source or combination of sources that provided complete information about trial design, trial

quality, outcomes, and results. Our recommendations incorporate the main findings for each step in

the review process.


38

Table 12: Recommendations for Systematic Reviews Using Multiple Data Sources Task Guidance for Systematic Reviewers Selecting trials • Determine in advance if individual participant data, aggregate data

(e.g., clinical study reports), or both are required to answer the research questions.

• Search electronic databases for all publicly available sources. • Use registries (e.g., ClinicalTrials.gov), conference abstracts, and

regulatory data (e.g., FDA, EMA) to identify unpublished trials. • Search repositories (e.g., www.clinicalstudydatarequest.com) for

clinical study reports and individual participant data. • For unpublished data, allow time to establish data use agreements as

required for access to trial data; make alternative plans in case requests are ignored or refused.

• Contact trial authors and sponsors to confirm which sources are related to each trial.

Assessing trial design and trial quality

• Use clinical study reports to extract information about trial design and trial quality.

• In addition to clinical study reports, use journal articles and FDA reviews because they sometimes include information not contained in clinical study reports.

• Check trial registrations for information about prespecified methods and outcomes.

• Consider excluding conference abstracts when other sources are available for data extraction.

Extracting results

• Anticipate multiple definitions of outcomes; prespecify outcomes for extraction within each outcome domain.

• When clinical study reports are available, check journal articles and FDA reviews for additional analyses.

• Where clinical study reports are not available, use journal articles to assess effects on primary outcomes.

Analyzing results

• Consider excluding trials reported only in conference abstracts from meta-analyses.

• Be cautious in interpreting trials with only public data sources (eg, conduct sensitivity analyses limited to trials with clinical study reports and individual participant data).

• Use individual participant data for investigating research questions not addressed in clinical study reports (e.g., subgroup analyses, potential mediators and moderators) and for older trials for which new statistical methods to account for missing data (e.g., multiple imputation) might affect the results.

• Allow time (12-18 months) to recode individual participant data for analysis and to merge databases.

Note: EMA = European Medicines Agency, FDA = US Food and Drug Administration.


39

4. DISCUSSION

4.1. Identifying patient-centered outcomes

Our efforts to identify PCOs revealed little formal research about what patients with neuropathic

pain and bipolar disorder consider important outcomes. Better evidence is needed to determine

what should be measured by clinical trials in these areas and to determine how those outcomes

should be assessed (e.g., choice of specific measures and time-points).

4.2. The effects of adding and replacing data from multiple data sources on the results of meta-analyses

In both case studies, most trials were associated with multiple sources of information about their

methods and results (mostly reports about the trials and some individual participant data). The most

common sources, short reports (e.g., conference abstracts), were often the least informative about

trial design, trial quality, outcomes, and results. Journal articles contained information that was

consistent with nonpublic sources, but they did not include all of the information available from

nonpublic sources. FDA reviews contained less information than hoped, largely because the drugs

and indications we examined did not meet FDA’s criteria for inclusion in its public information. Trial

registries were also less useful than we had hoped because trials supporting current uses of the

drugs in our case studies were performed before legislation mandated trial registration and results

reporting. The nonpublic sources we obtained were made public through litigation, an approach that

has been used in previous studies to compare data sources.168

Because many sources were incomplete and inconsistent regarding trial design (e.g., number of

groups), sometimes we found it difficult to understand whether sources were about the same trial

or different trials.

Most sources for a single trial, viewed on their own, were unclear about whether there was risk of

bias in a trial’s design (also referred to as trial quality). An advantage of identifying multiple data

sources for each trial was that often information was provided in 1 source but not another; thus,

when all sources were taken together, we were able discern more about the trial quality.

The effects of multiple data sources on meta-analysis for gabapentin were consistent with evidence

that using FDA reviews and nonpublic sources may change the results of meta-analyses because

additional trials can be included using FDA reviews and nonpublic sources and because results in

these sources sometimes differ from results in journal articles and conference abstracts.169-172

Findings for quetiapine were consistent with evidence from other case studies that indicate the


40

results reported in multiple data sources sometimes agree.173-175 We are not aware of any other

cases in which researchers have had access to this range and number of sources.

Although they sometimes identified trials not reported in other sources (e.g., trials not published in

a journal article), short reports such as conference abstracts did not contribute new information

about methods and results when other data sources were available.

Most of the outcomes in each trial we examined were associated with a few domains (i.e., they

measured a few underlying concepts, such as pain intensity and depression), and these domains

were assessed using a variety of measures, metrics, methods of aggregation, and time-points both

within individual trials and across all eligible trials.167 For this reason, we believe it is possible for

investigators to prespecify an outcome domain, assess it in multiple ways, and choose the most

favorable result for public presentation.

In the 2 case studies we examined, inferences about effectiveness derived from meta-analyses were

only slightly different when we added or replaced data following the process described above.

However, some results associated with outcome definitions in each outcome domain were

statistically significant and some were not; such differences might lead trial authors to report

outcomes selectively or lead systematic reviewers to include outcomes selectively in their meta-

analyses. Reporting biases related to differences in outcome measures, metrics, and methods of

aggregation could lead decision makers to recommend or select 1 treatment over another.

Potential harms were viewed by our patient coinvestigators as among the most important

outcomes. There was limited information about harms in publicly available sources. Further

investigation of harmful outcomes as reported in public and nonpublic sources is warranted; findings

could have important implications for PCOs research.

4.3 Implications for data sharing

Discussions about open access have largely focused on developing systems for archiving and sharing

data from new trials. The drugs in our study are widely prescribed even though the trials were

completed before trial registration and reporting requirements came into effect.176 Open access to

trial data would make what are now nonpublic sources (e.g., clinical study reports, individual

participant data) more widely available. Our findings highlight the importance of open access to

legacy trial data (e.g., trials conducted before requirements to share data) because many older drugs

enjoy current widespread use. Before data similar to those we examined are lost forever, the

research community should develop a system for sharing legacy trials with future generations.


41

4.4 Limitations

We were not funded to analyze potential harms, so we do not know whether the relative benefits

and harms of these interventions differ between public and nonpublic data sources. Further

research is needed to explore the benefits of using clinical study reports and individual patient data

meta-analysis for identifying the frequency of specific harms and of groups of harms.

While many studies have shown that selective reporting is a problem, until now only a few studies

have explored the potential for systematic reviewers to select outcomes for meta-analysis,177-179 and

none have used both public and nonpublic sources. Although our results were consistent with

previous studies comparing various sources with trial registrations,180,181 conference abstracts,6,182

FDA reviews,174,183 journal articles,184 protocols,185-187 clinical study reports,8,188 and individual

participant data,9,189,190 we do not know the extent of applicability of our results to other CER

questions. These case studies included trials of relatively short duration, and there may be even

greater benefits to using nonpublic sources for trials with multiple assessments over long periods of

time.

5. CONCLUSIONS For systematic reviewers and meta-analysts, using multiple data sources for the same trial is time

consuming and comes with both rewards (e.g., more complete data) and challenges (e.g., the need

to resolve differences among contradictory sources). We found no advantage in using conference

abstracts except for identifying the existence of unregistered and otherwise unpublished trials. Trial

registrations in these case studies were relatively old (e.g., we only found information in

ClinicalTrials.gov, not other registries), and they provided less information than users might find in

more recent registrations. Systematic reviewers should anticipate the vast amount of information

that could be extracted from nonpublic sources and plan accordingly. Further research is needed to

determine if the results of these case studies are representative of all trials and to determine how

multiple data sources might affect meta-analyses of potential harms.

6. REFERENCES 1. Chan AW, Song F, Vickers A, et al. Increasing value and reducing waste: addressing inaccessible research. Lancet. 2014;383(9913):257-266. 2. Goodman S, Dickersin K. Metabias: a challenge for comparative effectiveness research. Ann Intern Med. 2011;155(1):61-62. 3. Higgins JP, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]: The Cochrane Collaboration; 2011. Available from http://handbook.cochrane.org.


42

4. Institute of Medicine. Finding What Works in Health Care: Standards for Systematic Reviews. Washington, DC: The National Academies Press; 2011. 5. Schmucker C, Schell LK, Portalupi S, et al. Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PLoS ONE. 2014;9(12):e114023. 6. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database of Systematic Reviews 2007, Issue 2. Art. No.: MR000005. DOI: 10.1002/14651858.MR000005.pub3 7. Doshi P, Dickersin K, Healy D, Vedula SS, Jefferson T. Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ. 2013;(346):f2865. 8. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361(20):1963-1971. 9. Stewart LA, Parmar MK. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet. 1993;341(8842):418-422. 10. Page MJ, Forbes A, Chau M, Green SE, McKenzie JE. Investigation of bias in meta-analyses due to selective inclusion of trial effect estimates: empirical study. BMJ Open. 2016;6(4):e011863. 11. Nosek BA, Alter G, Banks GC, et al. Scientific standards: promoting an open research culture. Science. 2015;348(6242):1422-1425. 12. Mayo-Wilson E, Hutfless S, Li T, et al. Integrating multiple data sources (MUDS) for meta-analysis to improve patient-centered outcomes research: a protocol for a systematic review. Syst Rev. 2015;(4):143. 13. Yawn BP, Wollan PC, Weingarten TN, Watson JC, Hooten WM, Melton LJ III. The prevalence of neuropathic pain: clinical evaluation compared with screening tools in a community population. Pain Med. 2009;10(3):586-593. 14. Harris RP, Helfand M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med. 2001;20(suppl 3):21-35. 15. Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):617-627. 16. Merikangas KR, Jin R, He JP, et al. Prevalence and correlates of bipolar spectrum disorder in the world mental health survey initiative. Arch Gen Psychiatry. 2011;68(3):241-251. 17. Calabrese JR, Hirschfeld RM, Frye MA, Reed ML. Impact of depressive symptoms compared with manic symptoms in bipolar disorder: results of a U.S. community-based sample. J Clin Psychiatry. 2004;65(11):1499-1504. 18. Hirschfeld RM, Vornik LA. Recognition and diagnosis of bipolar disorder. J Clin Psychiatry. 2004;65(suppl 15):5-9. 19. Ilgen MA, Bohnert AS, Ignacio RV, et al. Psychiatric diagnoses and risk of suicide in veterans. Arch Gen Psychiatry. 2010;67(11):1152-1158. 20. Holma KM, Haukka J, Suominen K, et al. Differences in incidence of suicide attempts between bipolar I and II disorders and major depressive disorder. Bipolar Disord. 2014: 16: 652–661. DOI: 10.1111/bdi.12195 21. Nivoli AM, Colom F, Murru A, et al. New treatment guidelines for acute bipolar depression: a systematic review. J Affect Disord. 2011;129(1-3):14-26. 22. NICE. Bipolar Disorder: The Management of Bipolar Disorder in Adults, Children and Adolescents, in Primary and Secondary Care. London: National Institute for Health and Care Excellence; 2014. 23. Newcomer JW. Second-generation (atypical) antipsychotics and metabolic effects: a comprehensive literature review. CNS Drugs. 2005;19(suppl 1):1-93. 24. Ray WA, Chung CP, Murray KT, Hall K, Stein CM. Atypical antipsychotic drugs and the risk of sudden cardiac death. N Engl J Med. 2009;360(3):225-235. 25. Gargon E, Williamson PR, Altman DG, Blazeby JM, Clarke M. The COMET initiative database: progress and activities update (2014). Trials. 2015;(16):515. 26. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database—update and key issues. N Engl J Med. 2011;364(9):852-860. 27. Saldanha IJ, Dickersin K, Wang X, Li T. Outcomes in Cochrane systematic reviews addressing four common eye conditions: an evaluation of completeness and comparability. PLoS ONE. 2014;9(10):e109400. 28. Ip S, Hadar N, Keefe S, et al. A Web-based archive of systematic review data. Syst Rev. 2012;(1):15. 29. Li T, Vedula SS, Hadar N, Parkin C, Lau J, Dickersin K. Innovations in data collection, management, and archiving for systematic reviews. Ann Intern Med. 2015;162(4):287-294. 30. Shamseer L, Moher D, Clarke M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.


43

31. Statistical Software: Release [computer program]. Version 14.0. College Station, TX: StataCorp Corporation; 2001. 32. FineReader [computer program]. Version 11. Milpitas, CA. ABBYY; 2013. 33. Stewart LA, Clarke MJ. Practical methodology of meta-analyses (overviews) using updated individual patient data: Cochrane Working Group. Stat Med. 1995;14(19):2057-2079. 34. Riley RD, Lambert PC, Abo-Zaid G. Meta-analysis of individual participant data: rationale, conduct, and reporting. BMJ. 2010;(340):c221. 35. Riley RD, Simmonds MC, Look MP. Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. J Clin Epidemiol. 2007;60(5):431-439. 36. Riley RD, Lambert PC, Staessen JA, et al. Meta-analysis of continuous outcomes combining individual patient data and aggregate dat. Stat Med. 2008;27(11):1870-1893. 37. Higgins JP, Whitehead A, Turner RM, Omar RZ, Thompson SG. Meta-analysis of continuous outcome data from individual patients. Stat Med. 2001;20(15):2219-2241. 38. Turner RM, Omar RZ, Yang M, Goldstein H, Thompson SG. A multilevel model framework for meta-analysis of clinical trials with binary outcomes. Stat Med. 2000;19(24):3417-3432. 39. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7(3):177-188. 40. Turk DC, Dworkin RH, Allen RR, et al. Core outcome domains for chronic pain clinical trials: IMMPACT recommendations. Pain. 2003;106(3):337-345. 41. Vedula SS, Bero L, Scherer RW, Dickersin K. Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med. 2009;361(20):1963-1971. 42. Mayo-Wilson E, Doshi P, Dickersin K. Are manufacturers sharing data as promised? BMJ. 2015;(351):h4169. 43. Arai YC, Matsubara T, Shimo K, et al. Low-dose gabapentin as useful adjuvant to opioids for neuropathic cancer pain when combined with low-dose imipramine. J Anesth. 2010;24(3):407-410. 44. Arai YP, Matsubara T, Shimo K, Ushida T, Osuga T, Nishihara M. Low dose gabapenttin as useful adjuvant to opioids for neuropathic cancer pain when combined with low dose imipramine. Anesth Analg. 2010;110(3):S365. 45. Backonja M, Beydoun A, Edwards KR, et al. Gabapentin for the symptomatic treatment of painful neuropathy in patients with diabetes mellitus: a randomized controlled trial. JAMA. 1998;280(21):1831-1836. 46. Backonja MM. Gabapentin monotherapy for the symptomatic treatment of painful neuropathy: a multicenter, double-blind, placebo-controlled trial in patients with diabetes mellitus. Epilepsia. 1999;40(Suppl 6):S57-S59; discussion S73-S74. 47. Backonja M. Gabapentin for painful diabetic neuropathy (in reply). JAMA. 1999;282(2):133-134. 48. Slawson D. How effective is gabapentin (Neurontin) in reducing the pain associated with diabetic peripheral neuropathy? Evidence-Based Practice. 1999;2(3):7. 49. Backonja MM, Mutisya E. Dose response to gabapentin across five multicenter trials for neuropathic pain. Ann Neurol. 2002(suppl 1):S81. 50. Backonja M, Glanzman RI. Gabapentin dosing for neuropathic pain: evidence from randomized, placebo-controlled clinical trials. Clin Ther. 2003;(25):81-104. 51. Backonja MM, Mutisva E. Gabapentin demonstrates a nonlinear dose-reponse across five multicenter trials for neuropathic pain. Poster presented at: EFNS Annual Congress; October 26-29, 2002; Vienna, Austria. 52. Rowbotham MC, Backonja MM, Knapp LE, Yan C, Purcell TJ, Koto E. Gabapentin in the management of neuropathic pain in patients with diabetic peripheral neuropathy, postherpetic neuralgia, and neuropathic pain of varying etiology: Overview of results from 5 placebo-controlled trials. Poster presented at: Annual Meeting of ICMTNP; 2002; San Francisco, CA. 53. Backonja M, Mutisya EM. Review of gabapentin dosing in five placebo-controlled clinical trials for neuropathic pain. Eur J Neurol. 2002;(suppl 2):191. 54. Anonymous. Gabapentin for diabetic neuropathy. Nurses' Drug Alert. 1999;23(2):11. 55. Edwards KR, Bannington VT, Hes MS, LaMoreaux LK, Garofalo EA, Koto EM. Gabapentin (Neurontin) for pain associated with diabetic peripheral neuropathy: a double-blind, placebo-controlled study (945-210) [abstract]. Neurology. 1998;(50):A378-A379. 56. Backonja M, Hes MS, LaMoreaux LK, Garofalo EA, Koto EM, The US Gabapentin Study Group 210. Gabapentin reduces pain in diabetics with painful peripheral neuropathy: results of a double-blind, placebo controlled trial (945-210). Poster presented at: 16th Annual Scientific Meeting of the American Pain Society; 1997. New Orleans, LA. 57. Seidl JJ, Slawson JG. Gabapentin for painful diabetic neuropathy. J Fam Pract. 1999;48(3):173-174.


44

58. Parke-Davis. Research Report: 720-03908. A double-blind placebo-controlled trial of gabapentin for treatment of painful diabetic peripheral neuropathy. Protocol Number: 945-210 30 December 1998. 59. Caraceni A, Zecca E, Bonezzi C, Bennett MI. Gabapentin significantly improves analgesia in people receiving opioids for neuropathic cancer pain. Cancer Treat Rev. 2005;31(1):58-62. 60. Caraceni A, Zecca E, Bonezzi C, et al. Gabapentin for neuropathic cancer pain: a randomized controlled trial from the Gabapentin Cancer Pain Study Group. J Clin Oncol. 2004;22(14):2909-2917. 61. Caraceni A, Zecca E, Bonezzi C, Spigel DR. Improving neuropathic pain due to cancer using gabapentin. J Clin Outcomes Manag. 2004;11(10):620-621. 62. Hahn K, Arendt G, Braun JS, et al. A placebo-controlled trial of gabapentin for painful HIV-associated sensory neuropathies. J Neurol. 2004;251(10):1260-1266. 63. Hui AC, Wong SM, Leung HW, Man BL, Yu E, Wong LK. Gabapentin for the treatment of carpal tunnel syndrome: a randomized controlled trial. Eur J Neurol. 2011;18(5):726-730. 64. Chinese University of Hong Kong. Gabapentin for carpal tunnel syndrome: a randomised controlled trial. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US). 2000 [cited 26 May 2015]. https://clinicaltrials.gov/ct2/show/NCT00137735. NLM Identifier: NCT00137735. 65. Jensen MP, Chiang YK, Wu J. Assessment of pain quality in a clinical trial of gabapentin extended release for postherpetic neuralgia. Clin J Pain. 2009;25(4):286-292. 66. Irving G, Jensen M, Cramer M, et al. Efficacy and tolerability of gastric-retentive gabapentin for the treatment of postherpetic neuralgia: results of a double-blind, randomized, placebo-controlled clinical trial. Clin J Pain. 2009;25(3):185-192. 67. Milenkovic T, Percan V, Petrovski G, Ahmeti I, Mishevska Jovanovska S. Management of painful diabetic neuropathy with gabapenthin. J Diabetes Complicat. 2009;(1):A202-A203. 68. Mishra S, Bhatnagar S, Goyal GN, Rana SP, Upadhya SP. A comparative efficacy of amitriptyline, gabapentin, and pregabalin in neuropathic cancer pain: a prospective randomized double-blind placebo-controlled study. Am J Hosp Palliat Care. 2012;29(3):177-182. 69. EpiCept Corporation. A phase 3 multicenter, randomized, double-blind, placebo-controlled study of the safety and efficacy of Gabapentin Extended Release (G-ER) tablets in the treatment of patients with postherpetic neuralgia. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 26 May 2015]. https://clinicaltrials.gov/ct2/show/NCT00475904. NLM Identifier: NCT00475904. 70. Rice ASC, Maton S. Gabapentin in postherpetic neuralgia: a randomised, double blind and placebo controlled study. [Spanish]. Revista de la Sociedad Española del Dolor. 2002;9(2):61-73. 71. Parsons B, Tive L, Huang S. Gabapentin: a pooled analysis of adverse events from three clinical trials in patients with postherpetic neuralgia. Am J Geriatr Pharmacother. 2004;2(3):157-162. 72. Rice AS, Maton S. Gabapentin in postherpetic neuralgia: a randomised, double blind, placebo controlled study. Pain. 2001;94(2):215-224. 73. Anonymous. First approval of oral treatment for postherpetic neuralgia. Formulary. 2002;37(7):335. 74. Rice AS, Maton S. Reply to comment on "Rice ASC, Maton S, the Postherpetic Neuralgia Study Group (UK), gabapentin in postherpetic neuralgia: a randomized, double blind, placebo-controlled study." Pain. 96(3):411-412. 75. Parke-Davis. Research Report:430-00124. A double-blind placebo controlled trial of gabapentin for the treatment of post herpetic neuralgia. Protocol Number: 945-430-295 (NN 025) 3 April 2000. 76. Food and Drug Administration (Center for Drug Evaluation and Research). Statistical Review NDA 21-397. 2002. 77. Food and Drug Administration (Center for Drug Evaluation and Research). Medical Review NDA 21-397. 2002. 78. Rowbotham M, Harden N, Stacey B, Bernstein P, Magnus-Miller L. Gabapentin for the treatment of postherpetic neuralgia: a randomized controlled trial. JAMA. 1998;280(21):1837-1842. 79. Slawson D. How effective and safe is gabapentin (Neurontin) for reducing the pain of postherpetic neuralgia? Evidence-Based Practice. 1999;2(3):7-8. 80. Anonymous. Gabapentin for postherpetic neuralgia. Nurses' Drug Alert. 1999;(23):2. 81. Parke-Davis. Research Report: 995-00070. Double-blind, randomized, placebo-controlled, parallel groups, multi-center trial to determine the efficacy and safety of Neurontin (gabapentin) in subjects with peripheral neuropathy (post-herpetic neuralgia). Protocol Number: 945-211 29 December 1998. 82. Sandercock D, Cramer M, Biton V, Cowles VE. A gastroretentive gabapentin formulation for the treatment of painful diabetic peripheral neuropathy: efficacy and tolerability in a double-blind, randomized, controlled clinical trial. Diabetes Res Clin Pract. 2012;97(3):438-445.

https://clinicaltrials.gov/ct2/show/NCT00137735



45

83. Sandercock D, Cramer M, Wu J, Chiang YK, Biton V, Heritier M. Gabapentin extended release for the treatment of painful diabetic peripheral neuropathy: efficacy and tolerability in a double-blind, randomized, controlled clinical trial. Diabetes Care. 2009;32(2):e20. 84. Depomed. A multicenter, randomized, double-blind, placebo-controlled study of the safety and efficacy of gabapentin extended release (G-ER) tablets in the treatment of patients with painful diabetic peripheral neuropathy. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 26 May, 2015]. https://clinicaltrials.gov/ct2/show/NCT00712439. NLM Identifier: NCT00712439. 85. Sang CN, Sathyanarayana R, Sweeney M. Gastroretentive gabapentin (G-GR) formulation reduces intensity of pain associated with postherpetic neuralgia (PHN). Clin J Pain. 2013;29(4):281-288. 86. Anonymous. Once-daily therapy for postherpetic neuralgia. Clinical Advisor. 2011;14(11):31-32. 87. Sweeney M, Sathyanarayana R. Efficacy and tolerability of once-daily gabapentin extended-release (G-ER) for the treatment of postherpetic Neuralgia in patients at least 65 years old. J Gen Intern Med. 2011:26(S161). DOI: https://doi.org/10.1007/s11606-011-1730-9. 88. Irving GA, Sweeney M. Tolerability and safety of gastroretentive once-daily gabapentin tablets for the treatment of postherpetic neuralgia. J Pain Res. 2012;(5):203-208. 89. Irving G, Sondag E, Sweeney M. Tolerability and safety of gabapentin extended-release tablets in the treatment of postherpetic neuralgia. Pain Med. 2011;12(3):509. 90. Irving G, Carter F, Sondag E, Sweeney M. Tolerability and safety of gabapentin extended-release tablets in the treatment of postherpetic neuralgia. J Am Pharm Assoc. 2011;51(2):253. 91. Gupta A, Li S. Safety and efficacy of once-daily gastroretentive gabapentin in patients with postherpetic neuralgia aged 75 years and over. Drugs Aging. 2013;30(12):999-1008. 92. Jensen MP, Hsu PH, Vanhove GF. Early pain reduction can predict treatment response: results of integrated efficacy analyses of a once-daily gastroretentive formulation of gabapentin in patients with postherpetic neuralgia. Pain Med. 2012;13(8):1059-1066. 93. Rauck RL, Irving GA, Wallace MS, Vanhove GF, Sweeney M. Once-daily gastroretentive gabapentin for postherpetic neuralgia: integrated efficacy, time to onset of pain relief and safety analyses of data from two phase 3, multicenter, randomized, double-blind, placebo-controlled studies. J Pain Symptom Manage. 2013;46(2):219-228. 94. Backonja M, Wallace MS, Freeman R, Sweeney M. Effect of gabapentin once daily on Neuropathic Pain Scale (NPS) score in patients with postherpetic neuralgia (PHN). Pain Med. 2012;13(2):341. 95. Sweeney M, Misha B, Wallace M, Freeman R. Effect of a once-daily gastroretentive formulation of gabapentin on the Brief Pain Inventory Scale scores in patients with postherpetic neuralgia. J Pain. 2012;13(4) (suppl 1):S71. 96. Jensen M, Rauck R, Irving G, Sweeney M, Vanhove G. Onset of treatment response to a once-daily gastroretentive formulation of gabapentin in patients with postherpetic neuralgia. J Pain. 2012;13(4) (suppl 1):S70. 97. Rauck R, Irving G, Wallace M, Vanhove G, Sweeney M. Integrated analysis of efficacy and safety of a once-daily gastroretentive formulation of gabapentin in patients with postherpetic neuralgia. Neurology. 2012;78(1).P04.161. 98. Rauck R, Irving G, Wallace MS, Vanhove GF, Sweeney M. Integrated analysis of efficacy and safety of a once-daily gastroretentive formulation of gabapentin in patients with postherpetic neuralgia. Pain Pract. 2012;(12):119. 99. Smith W, Rauck R, Irving G, Wallace M, Vanhove G, Sweeney M. Integrated analysis of efficacy and safety of a once-daily gastroretentive formulation of gabapentin in patients with postherpetic neuralgia. J Am Pharm Assoc. 2012;52(2):263. 100. Kantor D, Mathis JT, Rauck RL, Irving G, Sweeney M, Vanhove GF. Integrated analysis of efficacy of a once-daily gastroretentive formulation of gabapentin in patients with postherpetic neuralgia who are at least 75 years old. J Gen Intern Med. 2012;(27):S231. 101. Depomed. A phase 3 multicenter, randomized, double-blind, placebo-controlled study of the safety and efficacy of Gabapentin Extended Release (G-ER) tablets in the treatment of patients with postherpetic neuralgia. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 6 November 2015]. https://clinicaltrials.gov/ct2/show/NCT00636636. NLM Identifier: NCT00636636. 102. Serpell MP, The Neuropathic Pain Study Group. Gabapentin in neuropathic pain syndromes: a randomised, double‐blind, placebo‐controlled trial. Poster presented at: Fifth International Conference on Mechanisms and Treatment of Neuropathic Pain Annual Meeting; November 21-23, 2002; Hamilton, Bermuda.




46

103. Serpell MG. Gabapentin in neuropathic pain syndromes: a randomised, double-blind, placebo-controlled trial. Pain. 2002;99(3):557-566. 104. Serpell MG, Neuropathic Pain Study Group. Reply to Comment on: Serpell et al., gabapentin in neuropathic pain syndromes: a randomised double-blind, placebo controlled trial. Pain. 2003;103(1-2):228. 105. Parke-Davis. Research Report: 430-00125. A double blind placebo controlled trial of gabapentin for the treatment of patients exhibiting symptoms of neuropathic pain. Protocol Number: 945-430-306 (NN 026). 5 May 2000. 106. Simpson DA. Gabapentin and venlafaxine for the treatment of painful diabetic neuropathy. J Clin Neuromuscul Dis. 2001;3(2):53-62. 107. Simpson DA. Gabapentin and venlafaxine in the treatment of painful diabetic neuropathy. Muscle Nerve. 2000;23(10):1627. 108. Parke-Davis. Research Report: 720-04130. A double-blind placebo-controlled trial with 3 doses of gabapentin for treatment of painful diabetic neuropathy. Protocol Number: 945-224. 7 February 2000. 109. Pfizer. Final study report: A 15 Week randomized, double-blind, placebo controlled, parallel-group, multicenter study of Neurontin (gabapentin) for eficacy and quality of life in patients with painful diabetic peripheral neuropahty. Protocol A945-1008. 24 March 2005. 110. Pérez HE, Sánchez GF. Gabapentin therapy for diabetic neuropathic pain. Am J Med. 2000;108(8):689. 111. Tamez Pérez HE, Rodríguez Ayala M, Gómez de Ossio MD. Uso de gabapentina en la neuropatía diabética dolorosa [Use of gabapentin on neuropathic diabetic pain]. Med Interna Méx. 1998;14(6):251-253. 112. Wallace MS, Irving G, Cowles VE. Gabapentin extended-release tablets for the treatment of patients with postherpetic neuralgia: a randomized, double-blind, placebo-controlled, multicentre study. Clin Drug Investig. 2010;30(11):765-776. 113. Rowbotham MC. Gabapentin Extended Release (GER) in the treatment of postherpetic neuralgia: a randomized, placebo-controlled clinical trial. Pain Pract. 2009;(9):59-60. 114. Depomed. A phase 3 multicenter, randomized, double-blind, placebo-controlled study of the safety and efficacy of gabapentin extended release (G-ER) tablets in the treatment of patients with postherpetic neuralgia. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 6 November 2015]. https://clinicaltrials.gov/ct2/show/NCT00335933. NLM Identifier: NCT00335933. 115. Yildirim K, Sisecioglu M, Karatay S, et al. The effectiveness of gabapentin in patients with chronic radiculopathy. Pain Clinic. 2003;15(3):213-218. 116. Zepeda-Vazquez TC, Hernandez-Santos JR, Tenopala-Villegas S, et al. Manejo del dolor neuropatico en el paciente diabetico con tramadol via oral comparado con la administracion del mismo asociado a la amitrptilina O Gabapentina. Revista Mexicana de Anestesiología. 2001;1(2):84-87. 117. Calabrese JR, Keck PE, Macfadden W, et al. A randomized, double-blind, placebo-controlled trial of quetiapine in the treatment of bipolar I or II depression. Am J Psychiatry. 2005;162(7):1351-1360. 118. Macfadden W, Calabrese JR, Suppes T, et al. Quetiapine in bipolar I depression: doubleblind, placebo-controlled study. Bipolar Disord. 2005;7(72)(suppl 2). 119. Endicott J, Rajagopalan K, Minkwitz M, Macfadden W. A randomized, double-blind, placebo-controlled study of quetiapine in the treatment of bipolar I and II depression: improvements in quality of life. Int Clin Psychopharm. 2007;22(1):29-37. 120. Cookson J, Keck PE, Ketter TA, Macfadden W, Minkwitz MC, Mullen J. Quetiapine in bipolar depression: NNT and time-to-event analyses. Poster presented at: 158th Annual Meeting of the American Psychiatric Association; May 21-26, 2005; Atlanta, GA. 121. Hirschfeld RM, Weisler RH, Raines SR, Macfadden W. Quetiapine in the treatment of anxiety in patients with bipolar I or II depression: a secondary analysis from a randomized, double-blind, placebo-controlled study. J Clin Psychiatry. 2006;67(3):355-362. 122. Endicott J, Rajagopalan K, MacFadden W, Minkwitz MC, Gaddy J. Efficacy of quetiapine in improving quality of life in bipolar depression. Poster presented at: 158th Annual Meeting of the American Psychiatric Association; May 21-26, 2005; Atlanta, GA. 123. Cookson J, Keck PE, Ketter TA, Macfadden W. Number needed to treat and time to response/remission for quetiapine monotherapy efficacy in acute bipolar depression: evidence from a large, randomized, placebo-controlled study. Int Clin Psychopharm. 2007;22(2):93-100. 124. Calabrese JR, Macfadden W, McCoy R, Minkwitz M, Wilson E, Mullen J. Double-blind, placebo-controlled study of quetiapine in bipolar depression. Poster presented at: 157th Annual Meeting of the American Psychiatric Association; May 1-6, 2004; New York, NY.



47

125. Fajutrao L, Gustafsson U, Paulsson B, Eriksson H. Quetiapine in the treatment of bipolar depression: Improvements in quality of life and functioning in four randomized, placebocontrolled trials. Int J Psychiatry Clin Pract. 2009;13(S1):28-29. 126. Eriksson H, Fajutrao L, Gustafsson U, Paulsson B. Quetiapine in the treatment of bipolar depression: improvements in quality of life and functioning in four randomized, placebocontrolled trials. Bipolar Disord. 2009;11(S1):37. 127. Young AH, Calabrese JR, Gustafsson U, et al. The efficacy of quetiapine monotherapy in bipolar II depression: combined data from the BOLDER and EMBOLDEN studies. Bipolar Disord. 2009;11(S1):90. 128. Young AH, Macritchie KAN, Calabrese J, et al. The efficacy of Quetiapine monotherapy in bipolar II depression: combined data from the BOLDER and EMBOLDEN studies. Int J Psychiatry Clin Pract. 2009;13(S1):37. 129. Calabrese J, Macritchie KAN, Young AH, Gustafsson U, Paulsson B. The efficacy of Quetiapine monotherapy in bipolar depression: combined data from the BOLDER and EMBOLDEN studies. Int J Psychiatry Clin Pract. 2009;13(suppl 1):36-37. 130. Weisler RH, Calabrese JR, Thase ME, et al. Efficacy of quetiapine monotherapy for the treatment of depressive episodes in bipolar I disorder: a post hoc analysis of combined results from 2 double-blind, randomized, placebo-controlled studies. J Clin Psychiatry. 2008;69(5):769-782. 131. Endicott J, Paulsson B, Gustafsson U, Schiöler H, Hassan M. Quetiapine monotherapy in the treatment of depressive episodes of bipolar I and II disorder: improvements in quality of life and quality of sleep. J Affect Disord. 2008;111(2-3):306-319. 132. Suppes T, Hirschfeld RM, Vieta E, Raines S, Paulsson B. Quetiapine for the treatment of bipolar II depression: analysis of data from two randomized, double-blind, placebo-controlled studies. World J Biol Psychiatry. 2008;9(3):198-211. 133. Young AH, Calabrese JR, Gustafsson U, et al. Quetiapine monotherapy in bipolar II depression: combined data from four large, randomized studies. Int J Bipolar Disord. 2013;1(10):1-12. 134. AstraZeneca. Safety and efficacy trial of the use of quetiapine fumarate (SEROQUEL®) in the treatment of patients with bipolar depression. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US). 2000 [cited 1 June 2015]. https://clinicaltrials.gov/ct2/show/NCT00060489. NLM Identifier: NCT00060489. 135. MacFadden W, Calabrese J, Suppes T, et al. Quetiapine in bipolar I depression: double-blind, placebo-controlled study. Poster presented at: 158th Annual Meeting of the American Psychiatric Association; May 21-26, 2005; Atlanta, GA. 136. Cookson J, Keck PE, Ketter TA, Macfadden W, Minkwitz MC, Mullen J. Quetiapine in bipolar depression: NNT and time-to-event analyses. Bipolar Disord. 2005;7(suppl 2):27-117. 137. Endicott J, Rajagopalan K, MacFadden W, Minkwitz M, Gaddy J. Efficacy of quetiapine in improving quality of life in patients with bipolar depression. Bipolar Disord. 2005;7(suppl 2):76, 50. 138. Thase ME. Quetiapine monotherapy for bipolar depression. Neuropsychiatr Dis Treat. 2008;4(1):11-21. 139. Anonymous. New data shows the efficacy and good tolerance of Seroquel(R) in bipolar depression. Médecine et Hygiène. 2004;62(2489):1450. 140. Macfadden W, Calabrese JR, McCoy R, Minkwitz M, Wilson E, Mullen J. Antianxiety effects analysis of quetiapine in bipolar depression. Poster presented at: 157th Annual Meeting of the American Psychiatric Association; May 1-6, 2004; New York, NY. 141. Gustafsson U, Fajutrao L. Effect of quetiapine on functioning and quality of life in bipolar depression. Eur Psychiat. 2011; 26(supp 1):1935. 142. AstraZeneca. A multicenter, double-blind, randomized, placebo-controlled, double- dummy trial of the use of quetiapine fumarate (SEROQUEL[Registered]) in the treatment of patients with bipolar depression. Study code: 5077US/0049. Clinical Study Report 28 November 2005. 143. Gao K, Wu R, Kemp DE, et al. Efficacy and safety of quetiapine-XR as monotherapy or adjunctive therapy to a mood stabilizer in acute bipolar depression with generalized anxiety disorder and other comorbidities: a randomized, placebo-controlled trial. J Clin Psychiatry. 2014;75(10):1062-1068. 144. Gao K. Quetiapine XR in the treatment of comorbid generalized anxiety disorder in bipolar depression with or without substance use disorder. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 6 November 2015]. https://clinicaltrials.gov/ct2/show/NCT00671853. NLM Identifier: NCT00335933.




48

145. AstraZeneca. A multicenter, double-blind, randomized, placebo-controlled study to evaluate the efficacy and safety of quetiapine fumarate (SEROQUEL) extended release as monotherapy in the treatment of patients with bipolar depression. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 1 June 2015]. https://clinicaltrials.gov/ct2/show/NCT01256177. NLM Identifier: NCT01256177. 146. Li HF, Gu NF, Zhang HY, et al. The efficacy and safety of quetiapine extended release (XR) as mono-therapy in the treatment of Chinese patients with bipolar I or II depression. Bipolar Disord. 2014;(16):65-66. 147. McElroy SL, Weisler RH, Chang W, et al. A double-blind, placebo-controlled study of quetiapine and paroxetine as monotherapy in adults with bipolar depression (EMBOLDEN II). J Clin Psychiatry. 2010;71(2):163-174. 148. Anonymous. EMBOLDEN I and II: quetiapine effective for treating bipolar depression. Brown University Psychopharmacology Update. 2010;21(5):1. 149. AstraZeneca. Multicentre, double-blind, randomised, parallel group, placebo controlled, phase 3 study of the efficacy & safety of quetiapine fumarate & paroxetine as monotherapy in adult patients with bipolar depression for 8 weeks & quetiapine in continuation (Abbreviated). In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 1 June 2015]. https://clinicaltrials.gov/ct2/show/NCT00119652. NLM Identifier: NCT00119652. 150. AstraZeneca. An international, multi-centre, double-blind, randomised, parallel-group, placebo-controlled, phase III study of the efficacy and safety of quetiapine fumarate (Seroquel [Registered], single oral 300 mg or 600 mg dose) and paroxetine as monotherapy in adult patients with bipolar depression for 8 weeks and quetiapine in continuation treatment for 26 up to 52 weeks. Study code: D1447C00134. Synopsis 5 October 2007. 151. Young A, McElroy S, Chang W, Olausson B, Paulsson B, Brecher M. Placebo-controlled study with acute and continuation phase of quetiapine in adults with bipolar depression (EMBOLDEN II). Eur Neuropsychopharmacol. 2008;18(S4):S371-S372. 152. Suppes T, Datto C, Minkwitz M, Nordenhem A, Walker C, Darko D. Effectiveness of the extended release formulation of quetiapine as monotherapy for the treatment of acute bipolar depression. J Affect Disord. 2010;121(1-2):106-115. 153. Datto C, Minkwitz M, Nordenhem A, Walker C, Darko D, Suppes T. Effectiveness of the new extended-release formulation of quetiapine as monotherapy for the treatment of acute bipolar depression (trial D144CC00002). Eur Psychiat. 2009;24:S574. 154. AstraZeneca. A multicenter, double-blind, randomized, parallel-group, placebo-controlled, phase III study of the efficacy and safety of quetiapine fumarate (Seroquel SR®) sustained-release as monotherapy in adult patients with acute bipolar depression. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 1 June 2015]. https://clinicaltrials.gov/ct2/show/NCT00422214. NLM Identifier: NCT00422214. 155. Thase ME. BOLDER II study of quetiapine therapy for bipolar depression. Future Neurol. 2007;2(4):373-377. 156. Thase ME, Macfadden W, Weisler RH, et al. Efficacy of quetiapine monotherapy in bipolar I and II depression: a double-blind, placebo-controlled study (the BOLDER II study). J Clin Psychopharmacol. 2006;26(6):600-609. 157. AstraZeneca. Controlled study of the use of quetiapine fumarate in the treatment of patients with bipolar depression. In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 1 June 2015]. https://clinicaltrials.gov/ct2/show/NCT00083954. NLM Identifier: NCT00083954. 158. BOLDER II study confirms therapeutic potential of SEROQUEL in bipolar depression. EurekAlert! [21 October 2005]. Available from http://www.eurekalert.org/pub_releases/2005-10/hak-bis102105.php. 159. AstraZeneca. A confirmatory multicenter, double-blind, randomized, placebo-controlled study of the use of quetiapine fumarate (SEROQUEL) in the treatment of patients with bipolar depression. Study code: D1447C00135. Clinical Study Report 1 December 2005. 160. Goodwin GM. Quetiapine more effective than placebo for depression in bipolar I and II disorder. Evid Based Ment Health. 2007;10(3):82. 161. Young AH, McElroy SL, Bauer M, et al. A double-blind, placebo-controlled study of quetiapine and lithium monotherapy in adults in the acute phase of bipolar depression (EMBOLDEN I). J Clin Psychiatry. 2010;71(2):150-162. 162. AstraZeneca. Multi-centre, double-blind, randomised, parallel-group, placebo-controlled, phase 3 study of the efficacy & safety of quetiapine fumarate & lithium as monotherapy in adult patients with bipolar





http://www.eurekalert.org/pub_releases/2005-10/hak-bis102105.php


49

depression for 8 weeks & quetiapine in continuation (Abbreviated). In: ClinicalTrials.gov [Internet]. Bethesda (MD): National Library of Medicine (US) 2000 [cited 1 June, 2015]. https://clinicaltrials.gov/ct2/show/NCT00206141. NLM Identifier: NCT00206141. 163. Young AH. Erratum: A double-blind, placebo-controlled study with acute and continuation phase of quetiapine and lithium in adults with bipolar depression (Embolden I). Bipolar Disord. 2008;10(3):451. 164. AstraZeneca. An international, multi-centre, double-blind, randomised, parallel-group, placebo-controlled, phase III study of the efficacy and safety of quetiapine fumarate (Seroquel, single oral 300 mg or 600 mg dose) and lithium as monotherapy in adult patients with bipolar depression for 8 weeks and quetiapine in continuation treatment for 26 up to 52 weeks. Study code: D1447C0001. Synopsis. 165. Grunze H. Quetiapine is effective in the treatment of adults in the acute phase of bipolar depression. Evid Based Ment Health. 2010;13(3):88. 166. Young A, McElroy S, Chang W, Olausson B, Paulsson B, Brecher M. Placebo-controlled study with acute and continuation phase of quetiapine in adults with bipolar depression (EMBOLDEN I). Eur Neuropsychopharmacol. 2008;18(S4):S371-S372. 167. Mayo-Wilson E, Fusco N, Li T, et al. Multiple outcomes and analyses in clinical trials create challenges for interpretation and research synthesis. J Clin Epidemiol. 2017;86:39-50. doi:10.1016/j.jclinepi.2017.05.007. 168. Wieland LS, Rutkow L, Vedula SS, et al. Who has used internal company documents for biomedical and public health research and where did they find them? PLoS ONE. 2014;9(5):e94709. 169. Le Noury J, Nardo JM, Healy D, et al. Restoring Study 329: efficacy and harms of paroxetine and imipramine in treatment of major depression in adolescence. BMJ. 2015;351:h4320. 170. Eyding D, Lelgemann M, Grouven U, et al. Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ. 2010;341:c4737. 171. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med. 2008;358(3):252-260. 172. Jefferson T, Jones M, Doshi P, Spencer EA, Onakpoya I, Heneghan CJ. Oseltamivir for influenza in adults and children: systematic review of clinical study reports and summary of regulatory comments. BMJ. 2014;(348):g2545. 173. Rodgers MA, Brown JV, Heirs MK, et al. Reporting of industry funded study outcome data: comparison of confidential and published data on the safety and effectiveness of rhBMP-2 for spinal fusion. BMJ. 2013;(346):f3981. 174. Turner EH, Knoepflmacher D, Shapley L. Publication bias in antipsychotic trials: an analysis of efficacy comparing the published literature to the US Food and Drug Administration database. PLoS Med. 2012;9(3):e1001189. 175. Hart B, Lundh A, Bero L. Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012;344:d7202. 176. Food and Drug Administration Amendments Act (FDAAA), 42 USC § 8012007. 2007. 177. Tendal B, Nuesch E, Higgins JP, Juni P, Gotzsche PC. Multiplicity of data in trial reports and the reliability of meta-analyses: empirical study. BMJ. 2011;343:d4829. 178. Page MJ, McKenzie JE, Chau M, Green SE, Forbes A. Methods to select results to include in meta-analyses deserve more consideration in systematic reviews. J Clin Epidemiol. 2015;68(11):1282-1291. 179. Page MJ, McKenzie JE, Kirkham J, et al. Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions. Cochrane Database Syst Rev. 2014;(10):MR000035. 180. Scherer RW, Huynh L, Ervin AM, Taylor J, Dickersin K. ClinicalTrials.gov registration can supplement information in abstracts for systematic reviews: a comparison study. BMC Med Res Methodol. 2013;(13):79. 181. Baudard M, Yavchitz A, Ravaud P, Perrodeau E, Boutron I. Impact of searching clinical trial registries in systematic reviews of pharmaceutical treatments: methodological systematic review and reanalysis of meta-analyses. BMJ. 2017;(356):j448. 182. Saldanha IJ, Scherer RW, Rodriguez-Barraquer I, Jampel HD, Dickersin K. Dependability of results in conference abstracts of randomized controlled trials in ophthalmology and author financial conflicts of interest as a factor associated with full publication. Trials. 2016;17(1):213. 183. MacLean CH, Morton SC, Ofman JJ, Roth EA, Shekelle PG, Southern California Evidence-Based Practice Center: How useful are unpublished data from the Food and Drug Administration in meta-analysis? J Clin Epidemiol. 2003;56(1):44-51. 184. Gotzsche PC. Multiple publication of reports of drug trials. Eur J Clin Pharmacol. 1989;36(5):429-432.



50

185. Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA. 1992;267(3):374-378. 186. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991;337(8746):867-872. 187. Dwan K, Gamble C, Williamson PR, Kirkham JJ, Reporting Bias Group. Systematic review of the empirical evidence of study publication bias and outcome reporting bias - an updated review. PLoS ONE. 2013;8(7):e66844. 188. Wieseler B, Kerekes MF, Vervoelgyi V, McGauran N, Kaiser T. Impact of document type on reporting quality of clinical drug trials: a comparison of registry reports, clinical study reports, and journal publications. BMJ. 2012;344:d8141. 189. Fu R, Selph S, McDonagh M, et al. Effectiveness and harms of recombinant human bone morphogenetic protein-2 in spine fusion: a systematic review and meta-analysis. Ann Intern Med. 2013;158(12):890-902. 190. Simmonds MC, Brown JV, Heirs MK, et al. Safety and effectiveness of recombinant human bone morphogenetic protein-2 for spinal fusion: a meta-analysis of individual-participant data. Ann Intern Med. 2013;158(12):877-889.

Copyright© 2018 Johns Hopkins University. All Rights Reserved

Disclaimer:

The [views, statements, opinions] presented in this report are solely the responsibility of the author(s) and do not necessarily represent the

views of the Patient-Centered Outcomes Research Institute® (PCORI®), its Board of Governors or Methodology Committee.

Acknowledgement:

Research reported in this report was [partially] funded through a Patient-Centered Outcomes Research Institute® (PCORI®) Award (#ME-1303-5785).

final research report: the benefits and challenges of ...final research report (me-1303-5785) 6 of...

Documents