reg-eaaci taskforce report

REG-EAACI TASKFORCE REPORT

REG SUMMIT 2016, LYON, FRANCE, 16 APRIL

SESSION: ‘INFLUENCING’ Guideline Development: the REG/EAACI Taskforce Reports

TIME: 11.45AM-12.45PM

Presenters: Nicolas Roche (Hopital de l’Hotel Dieu, Paris) &

Jon Campbell (Skaggs School of Pharmacy, Denver, Colorado)

On behalf of: Nikos Papadopoulos, Leif Bjermer, Guy Brusselle, Alison Chisholm, Jerry Krishnan, Zoe Mitchel, David Price,

Mike Thomas, Eric van Ganse, Maarten van den Berge helped by Sarah Acaster and Katy Gallop

—

Taskforce Members

Leads: Nicolas Roche & Jon Campbell

• David Price

• Mike Thomas

• Eric van Ganse

• George Christoff

• Guy Brusselle

• Jennifer Quint

• Jerry Krishnan

• Leif Bjermer

• Nikos Papadopoulos

• Maarten Van Den Berge

Background

• RCTs are not sufficient to provide holistic

evidence

• Real-life studies are subject to many sources of

biases

Where do observational studies fit in?

• (Almost) Everybody agrees on:

o Pitfalls of RCTs

o Need for real-life data

• Guideline developers are often reluctant to include real-life data

• Quality issues

• Need to help readers

• Quality assessment

o Tools required

o Remaining difficulties in quality assessment (need to help

reviewers)

o Need to improve reporting

SETTING AND DEFINING STANDARDSCREATING A LEVEL PLAYING FIELD TO HELP RAISE

(AND GUIDE ASSESSMENT OF)

THE QUALITY OF REAL-LIFE RESEARCH

ATS 2013

—

Goal

• Create a level playing field and solid foundations

for future research that will:

o Standardize the field

o Enable benchmarking

o Assist in assessing the quality of real-life data

(including their potential value to clinical

practice guidelines)

Conceptual framework of therapeutic research, linking the various types of

studies based on ecology of care and population characteristics. Typical

positions of the most common study designs have been positioned but can

be moved in any direction depending on their specificities

Roche Net al. Lancet Respir Med. 2013;1:e29-30.

Framework for integrating evidence

GRADE classifications:

observational studies vs RCTs

Source of

evidence

Initial

quality

rating

Factors

decreasing

quality

Factors

increasing

quality

Final rating

RCTs High Risk of bias

Inconsistency

Indirectness

Imprecision

Publication bias

Large effect

Dose-

response

Influential

residual

confounders

High

Moderate

Low

Very LowObservational

studiesLow

Guyatt et al. PATS 2012 (2007 ATS/ERS Workshop)

+

-

Stakeholders

• Researchers

• Database designers / promotors

• Guidelines developers

• Policy makers

• Reviewers (journals, projects)

• Readers

• …

Aims & Objectives

• Conduct a systematic critical review of the real-life

asthma literature published between 2004/13

o Restricted to comparative effectiveness research

• Describe the quality of currently available real-life

research in asthma

• Highlight studies worthy of possible integration into

asthma-related guidelines and policy decisions

• Recommend quality targets for the future

o And topics to address in future observational CER studies

Strategy

• Agree on / build a quality assessment tool

• Define a search strategy, perform a review of the

literature

• Apply the quality assessment tool on retrieved articles

• Synthesise, discuss and conclude on quality, and potential

influence of current comparative effectiveness

literature on future guideline and/or need for additional

studies

Taskforce Timelines

2014-2015

• Task 1: literature search to identify asthma real-life

research articles (chair: N Roche)

• Task 2: construction of a dedicated quality assessment

tool (chair : J Campbell)

2015-2016

• Task 3: quality assessment of identified asthma real-life

research articles

• Task 4: report

• Task 5: disseminate

2014-2015

Task 1 (literature search):

• Formal targeted literature search

• Poll among Taskforce members and REG members

• Limit retrieved papers to top priority (4) PICOT

questions

Task 2 (quality assessment tool):

• Combination of available tools

• Discussion and final version of the quality

assessment tool

2015-2016

Task 3 (quality assessment):

• Quality assessment of articles identified from the literature search

• Conducted by members of the Taskforce and REG-EAACI network

reviewers

• Results of quality assessment used to determine which papers could

be used to complement results of RCTs and inform guidelines.

Task 4 (dissemination):

• Results to be presented at EAACI meeting

• Publication (submission ~Q3 2016; journals: Allergy and CTA):

o A review article (detailed results of quality assessment)

o A position paper (information of guidelines)

QUALITY ASSESSMENT TOOL

DEVELOPMENT

JON CAMPBELL: Skaggs School of Pharmacy, Denver, Colorado

—

Development Phases

• Phase I: literature review

• Phase II: Initial tool creation

• Phase III: Taskforce review and pilot

• Phase IV: Larger Pilot & tool finalization/minor

modifications

• Phase V: tool finalization for use – development

of an online tool

Phase I: Literature Review

Many Study Assessment Tools Exist

• STROBE Statement: Checklist of items that should be included in reports of

observational studies

• Quality standards for real-world research: List of quality criteria for

observational database comparative studies (Roche et al)

• Report of the ISPOR Task Force on retrospective databases: A checklist for

retrospective database studies

• GRACE Checklist: Quality of observational cohort studies for decision making

support

• ENCePP Checklist: checklist for study protocols for pharmacoepidemiology

• Standards in the conduct of registry studies for patient-centered

outcomes research (PCORI): review of existing guidelines and literature to

develop methodological standards


• Purpose of most existing tools:

o Primarily to provide standardization of best practice and

reporting of observational / comparative effectiveness

research studies.

• Purpose of REG-EAACI Taskforce tool:

o Decision aid to assess whether or not a study provides

evidence that could inform future guidelines (yes or no). If

yes, the tool’s criteria can aid in describing any particular

strengths or limitations of the evidence.


• A synthesis of existing tools was visually

presented in tabular form.

o Overlap was assessed across tools

• Taskforce decided to focus on merging two

existing tools

o Roche and colleagues

o ISPOR task force

– ISPOR task force tool included many of the

existing tools into its development

Quality criteria for observational

database comparative studiesRoche et al Arch Conference Ann Am Thorac Soc Feb 2014

ISPOR Task Force. Berger et al Value in Health 2014.

4 relevance & 28 credibility (yes/no) questions (weaknesses and fatal flaws identified)

Phase 1: Roche (REG) and

Berger (ISPOR)

• Roche has 24 relevance and credibility questions

• ISPOR starts with 4 relevance questions and then

moves to 28 credibility questions

o Much overlap between two on domains and items

o Both are yes/no items; ISPOR has a “can’t answer”

option (NA, not reported, not enough info, not enough

training to answer).

o ISPOR has the concept of fatal flaws and weaknesses

Phase II: Initial Tool Creation

• Synthesis of pre-existing quality

recommendations to develop a first draft

Taskforce Quality Assessment Tool

• Recognition that asthma specificity is not

necessary and provides greater tool utility

TABLE Checklist combining ISPOR/Roche et al. assessments

Red= Roche (Emphasizes derived from Roche et al.) Green= ISPOR (Emphasizes derived from ISPOR)

Assessment using Roche/ISPOR Yes or No

Yes = 1 point No = 0

Background/ Relevance 4 Items

1. Clear underlying hypotheses and specific research questions Yes/No

3. Relevant population Yes/No

4. Relevant interventions and outcomes are included Yes/No

5. Applicable context (setting/practice pattern) Yes/No

Maximum Raw Score = 4 pts Adjusted Score = 4/4 (100%) #Yes’s/#Items

Design 8 Items

1. Observational comparative effectiveness database study with a priori hypotheses and goals? Yes/No

2. (Independent steering committee involved in) a priori definition of study methodology? Yes/No

3. Evidence of a priori protocol, review of analyses, statistical analysis plan, and interpretation of results? Yes/No

4. Comparison groups concurrent or justified? Yes/No

5. Was a study design used to minimize or account for confounding? Yes/No

6. Comparison groups selected to be sufficiently similar to each other (e.g. either by restriction or recruitment based on the same indications for

treatment?

Yes/No

7. Sources criteria and methods for selecting participants appropriate to address study questions/hypotheses? Yes/No

8. Registration in a public repository with commitment to publish results Yes/No


Data/Database 3 Items

1. High quality databases that are sufficient to support the study Yes/No

2. Was exposure defined and measured in a valid way? Yes/No

3. Primary outcomes defined and measured in a valid way? Yes/No


OUTCOMES 6 Items

1. A. Clearly defined primary and secondary outcomes chosen a priori Yes/No

2. B. The use of proxy and composite measures is justified and explained Yes/No

3. C. Validity of proxy measures has been checked Yes/No

4. Length of observation: Sufficient f/u duration to reliably assess outcomes of interest and long-term Tx effects? Yes/No

5. Patients: Well described inclusion and exclusion criteria, reflecting target patients’ characteristics in the real world. Yes/No

6. Sample size: calculated based on clear a priori hypotheses regarding the occurrence of outcomes of interest and target effect of studied Tx versus

comparator?

Yes/No


TABLE Checklist combining ISPOR/Roche et al. assessments (continued)

Analyses 4 Items (Categorized under OUTCOMES umbrella in Roche et al.)

1. Study groups are compared at baseline using univariate analyses and analyses of subgroups or interaction effects reported. Yes/No

2. Avoided biases related to baseline differences using matching and/or adjustments Yes/No

Continued below

Analyses continued Yes = 1 pt No = 0

3. Sensitivity analyses are performed to check the robustness of results and the effects of key assumptions on definitions or outcomes. Yes/No

4. (From ISPOR Analyses #1) Thorough assessment of potential measured and un-measured confounders Yes/No


Results/Reporting 10 Items

1. Flow chart explaining all exclusions and individuals screened or selected at each stage of defining the final sample Yes/No

2. Sensitivity analyses of several databases go in the same direction as primary analyses. Yes/No

3. Detailed description of patients’ characteristics/descriptive statistics (Demographics, characteristic of disease of interest, Co-morbidities and

concomitant treatments)

Yes/No

4. Extensive presentation of results/authors describe the key components of their statistical approaches Yes/No

5. Extensive presentation of unmatched and matched populations (if matching) using univariate and multivariate, unadjusted and adjusted analyses Yes/No

6. If patients’ are lost to follow-up then there characteristics are compared with patients remaining? +

Was follow-up similar or accounted for (ISPOR Data #4) between groups?

Yes/No

7. Were confounder-adjusted estimates of Tx effects reported? Yes/No

8. Did the authors describe the statistical uncertainty of their findings? Yes/No

9. Was the extent of missing data reported? Yes/No

10. Were the absolute and relative measures of Tx effects reported? Yes/No


Discussion/Interpretation 6 Items

1. Discussion of differences with results of efficacy randomized controlled trials Yes/No

2. Results consistent with prior known information or if not was an adequate explanation provided? Yes/No

3. Are the observed Tx effects considered clinically meaningful? Yes/No

4. Summary and interpretation of findings, focusing 1st on whether they confirm or contradict a priori hypotheses Yes/No

5. Discussion of possible biases and confounding factors, especially related to the observational nature of the study Yes/No

6. Suggestions for future research to challenge, strengthen, or extend the study results Yes/No


Conflict of Interest 2 Items

1. Were there any potential conflicts of interest Yes/No

2. If there were potential conflicts of interest, were steps taken to address this? Yes/No


Design (8 items)

Outcomes (6 items)

Results/Reporting (11 items)

Conflict of Interest (2 items)

Figure: Wheel of Domains for combined Roche/ISPOR checklist

(modeled after PRECIS wheel)

Phase II: Initial Tool Creation

• The initial tool included:

o 13 primary quality criteria

– All primary items must be satisfied to recommend a study that

could inform future guidelines

o 14 secondary criteria

o Across 8 domains:

– Background/relevance, design, data, measures, analyses,

results/reporting, discussion/interpretation, conflict of interest

• Tool refined, based on:

o Task Force Members votes for items to keep/remove/merge

Phase III: Within taskforce pilot

• Methods:

o 6 papers distributed to 9 taskforce members

o Two reviewer groups (1 x group of 5 and 1 x group of 4)

o Each group member scored / appraised 3 papers

o Calculation of an “agreement rate” (per item & per domain – split by primary and

supporting “secondary” domains)

o Global assessment = paper of sufficient quality to inform guideline development

(Y/N) required fulfilment of all primary criteria and agreement by ALL raters

• Results:

o Overall agreement at ~50% level; agreement appeared to be random

• Action:

o Revise tool based on taskforce feedback to

– Simplify it, and

– Remove areas of potential ambiguity/subjectivity.

o Conduct an extended pilot, using a wider rater group, to see whether the tool

revisions had improved agreement rates.

Phase IV: Larger Pilot & tool

finalization/minor modifications

• Methods:

o 22 x raters involved in total

o 3 x rater groups:

– A, B, C (2 x groups of 7; 1 x group of 8)

– Members of groups A and B received 2 papers on

intervention adherence / persistence

– Members of group C received 2 papers on ICS particle

size / formulation

for quality assessment using the revised tool

o Agreement was evaluated across each field for each

rater group, separately and in total

AVERAGE AGREEMENT ACROSS

FIELDS Adherence (Elkout & Barnes)

HRU & persistence (Tan & Lee)

Particle size & formulation summary

(Barnes & Terzano)

OVERALL SUMMARY

(weighted by #

group contributors)

PRIMARY DOMAINS

1. Background 1.1. Clearly stated research question 79% 100% 86% 89%

2. Design

2.1 Population defined and justified 64% 94% 71% 77% 2.2. Comparision groups defined and justified 93% 71% 79% 79%

2.3. Setting defined and justified 93% 100% 93% 95%

3. Measures

3.1. (If relevant), exposure is clearly defined 93% 71% 76% 78% 3.2. Primary outcomes clearly defined and measured 71% 89% 93% 85%

4. Analylsis

4.1. Potential confounders are considered and adjusted for in the analysis, and reported 64% 81% 71% 73% 4.2. Study groups are compared at baseline 79% 79% 79% 77%

5. Results

5.1. Results are clearly presented for all primary and secondary endpoints as well as confounders 79% 94% 71% 82%

6. Discussion / Interpretation

6.1. Results consistent with known information or if not, an explanation is provided 86% 100% 86% 91% 6.2 The clincial relevance of the results is discussed 85% 88% 93% 88%

7. Conflict of interests

7.1. Potential conflicts of interest, including study funding, are stated

79% 100% 93% 91%

A B C A, B,C

Fulfilment of 100% of primary

criteria or …?

RATER GROUP A B C

PAPER TOIC AdherenceHRU &

persistenceParticle size & formulation

PAPERSElkout et al

Voshaar et al

Tan et al

Lee et al

Barnes et al

Terzano et al

NUMBER OF RATERS 8 7 7

# primary criteria

fulfillment

%primary criteria fulfillment

Quality criteria attainment

Average

attainment

(across A, B, C)

12 of 12 correct 100% 64% 64% 57% 62%

≥11 of 12 correct 92% 57% 71% 57% 62%

≥10 of 12 correct 83%) 57% 71% 57% 62%

≥9 of 12 correct 75% 71% 79% 64% 71%

≥8 of 12 correct67%

(67% correct)79% 93% 79% 83%

More lenient quality criteria fulfilemnt (e.g. 10-of-12 of 11-of-12) had limited impact on

overall agreement – decisions to require 100% fulfilment (pending further refinement)

Phase IV: Larger Pilot & tool

finalization/minor modifications

• Remaining disagreement: Reviewer feedback suggested some of

the persisting disagreement was the result of:

o Multiple “sub-questions” within some of the appraisal criteria,

e.g. “Potential conflicts of interest, including study funding, are

stated”, which contains a question about author conflicts of

interest and also the specific study funding. Such questions lead

to greater potential for disagreement between raters.

o Poorly written papers (even though those selected for evaluation

were of above-average quality)

• Action: Each tool criteria was reviewed by the taskforce members

once more and any remaining ambiguities removed. This was

undertaken at the Amsterdam Taskforce Meeting.

Phase V: tool finalization for use,

development of an online tool

• Once the Excel version of the tool had been

approved by the taskforce, it was converted into

a Google form.

• An online tool offered:

o Smoother user/rate experience,

o Minimized opportunities for data mis-entry, AND

o Allowed automated collation into an online

spreadsheet and delivered an overview of responses

to each question.

THE TOOL – desktop version (I)

THE TOOL – desktop version (II)

The

TOOL:

Web

version

RATER ASSESSMENT

(for taskforce purposes /

participant profiling only)

The

TOOL:

Web

version

PRIMARY

CRITERIA (I)

The

TOOL:

Web

version

PRIMARY

CRITERIA (II)

The

TOOL:

Web

version

PRIMARY

CRITERIA

(III)

The

TOOL:

Web

version

SECONDARY CRITERIA

Acknowledgements

• EAACI Task force members

• REG Staff

o Alison Chisholm

o Zoe Mitchell

• Sarah Acaster and Katy Gallop

• Robert Perry

LITERATURE REVIEW

NICOLAS ROCHE: Hopital de l’Hotel Dieu, Paris

—

PICOT questions

• TF members + informal literature search: n=21

• Preselection: n=9

o Education

o Adherence/persistence

o Smoking asthmatics

o Devices

o Molecules (ICS)

o Biotherapies (omalizumab)

o Formulations (Extra-fine)

o Strategies (ICS vs FDC…)

o Antibiotics for exacerbations (macrolides vs others)

o ACOS vs asthma vs COPD, ICS vs LABDs vs both

P People of all ages prescribed regular

maintenance ICS

I Adherence to recommended therapy

C Comparison of outcomes between groups

of different levels of adherence (e.g. 0-

25%, 25-50%, 50-75%, over 75%)

O Exacerbations, admissions, symptoms,

QOL

T 12 months


maintenance ICS

I Effectiveness of different inhaler

devices/delivery systems in maintaining

asthma control


using the same molecule through different

inhaler systems (pMDI, Breath activated

MDI, DPI)


QOL

T 12 months

P Smokers and ex-smokers with asthma

receiving ICS and having exacerbations

I Effectiveness of different treatment

regimens

C Comparison of outcomes between those

going from low dose ICS to: HDICS, LDICS

+ LABA, HDICS + LABA. All regimens +/-

LTRA

O Exacerbations

T 12 months

P Adults presenting for care with asthma

exacerbation in ambulatory setting

(depending on dataset, ED setting and

discharged to home would be great as

well)

I Azithromycin or other macrolide

C Two comparators: Other class of antibiotic

(e.g., fluoroquinolone; beta lactam); no

antibiotic.

O Relapse (all-cause unscheduled office

visit, presenting to ED, or hospitalization)

T 30 days

24

0

7

3


maintenance ICS



of different levels of adherence (e.g. 0-

25%, 25-50%, 50-75%, over 75%)


QOL

T 12 months


maintenance ICS

I Effectiveness of different inhaler

devices/delivery systems in maintaining

asthma control


using the same molecule through different

inhaler systems (pMDI, Breath activated

MDI, DPI)


QOL

T 12 months

P Smokers and ex-smokers with asthma

receiving ICS and having exacerbations

I Effectiveness of different treatment

regimens

C Comparison of outcomes between those

going from low dose ICS to: HDICS, LDICS

+ LABA, HDICS + LABA. All regimens +/-

LTRA

O Exacerbations

T 12 months


maintenance ICS

I Effectiveness of small vs standard-size

ICS particles in maintaining asthma

control


using the same or different molecules

administered as extrafine of fine particles


QOL

T 12 months

24

12

7

3

Search and selection flow

MEDLINE (hits) Jan 2004-Dec 14: 1,347

Jan 2015-Dec 15: 52

Total: 1,399

EMBASE (hits) Jan 2004-Dec 14: 2,086 Jan 2015-Dec 15: 275

Total: 2,361

Meeting inclusion criteria Jan 2004-Dec 14: 42 Jan 2015-Dec 15: 4

Total: 46

PICOT 1 ADHERENCE

Jan 2004-Dec 14: 23

Jan 2015-Dec 15: 1

24

Exclusions

Jan 2004-Dec 14: 2,823 Jan 2015-Dec 15: 280

• Not an asthma study (e.g. COPD, Allergic rhinitis)

• Not an observational study, including:

– Literature review

– Clinical trial – Case study

– Cross-sectional survey • Not aligned to one of the 4

PICOT questions

Search period: January 2004 to

December 2015

Total Papers Jan 2004-Dec 14: 3,433

Jan 2015-Dec 15: 327

Total: 3,760

Duplicates removed Jan 2004-Dec 14: 568

Jan 2015-Dec 15: 43

Total: 611

Total Abstracts Reviewed Jan 2004-Dec 14: 2,865 Jan 2015-Dec 15: 284

Total: 3,149

PICOT 2 DEVICE TYPE

Jan 2004-Dec 14: 7

Jan 2015-Dec 15: 0

7

PICOT 3 SMOKING ASTHMA

Jan 2004-Dec 14: 2

Jan 2015-Dec 15: 1

3

PICOT 4 PARTICLE SIZE

Jan 2004-Dec 14: 10

Jan 2015-Dec 15: 2

12

Rating and final selection strategy

Permutation Rater 1 Rater 2 Rater 3 Assessment

YESAll primary

criteria fulfilled

YESAll primary criteria

fulfilled

–POSITIVE

Paper is of sufficient quality to be

considered by future guideline

bodies

2NO

≥1 primary

criteria not

fulfilled


fulfilled


fulfilled

POSITIVEPaper is of sufficient quality to be

considered by future guideline

bodies

4NO

≥1 primary

criteria not

fulfilled


fulfilled

NO≥1 primary criteria not

fulfilled

NEGATIVEPaper is NOT of sufficient quality to

be considered by future guideline

bodies

6NO

≥1 primary

criteria not

fulfilled

NO≥1 primary criteria not

fulfilled

–NEGATIVE

Paper is NOT of sufficient quality to

be considered by future guideline

bodies

Who participated? (I)

0%

20%

40%

60%

80%

Perc

en

tag

e o

f ra

ters

Who participated? (II)

12%

62%

2%

6%

12%

2%6%

Commercial Researcher

Faculty

Hospital Practitioner

Medical Writer

Non-Profit Researcher

Private PracticeAllergologist

Student / Fellow

Who participated? (III)

0%

5%

10%

15%

20%

25%

30%

35%

40%

0-10 11-24 25-49 50-100 ≥100

Perc

en

tag

e o

f ra

ters

Number of peer review papers

Summary of ratings

8

4

2

8

22

15

1 1

4

21

12

0 0

3

0

5

10

15

20

25

Adherence Devices SmokingAsthma

Particle Size All Papers

NU

MB

ER

OF

PA

PE

RS

PICOT QUESTION FOCUS

Sufficiently High Quality Insufficient Quality TBC

Summary of ratings

33%

57%

67% 67%

48%

63%

14%

33% 33%

46%

4%

29%

0% 0%

7%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Adherence Devices Smoking Asthma Particle Size All Papers

PE

RC

EN

TA

GE

OF

PA

PE

RS

PICOT QUESTION FOCUS

Sufficiently High Quality Insufficient Quality TBC

n=24

n=7 n=3 n=12

Summary of failed criteria

0

2%

17%

4%

6%

17%

20%

14%

1%

4%

15%

0%

5%

10%

15%

20%

25%

1.1 2.1 2.2 3.1 3.2 4.1 4.2 5.1 6.1 6.2 7.1

Perc

en

tag

e o

f to

tal

Cri

teri

a

Failu

res R

eco

rded

by R

ate

rs

Taskforce Quality Assessment Tool Primary Criteria

Summary of failed categories

0%

19%

10%

37%

14%

5%

15%

0%

5%

10%

15%

20%

25%

30%

35%

40%

Pe

rce

nta

ge

of

tota

l C

rite

ria

Fail

ure

s

Reco

rded

by R

ate

rs

Taskforce Quality Assessment Tool Primary Quality Categories

Reading grid

• Quality assessment

• Summary of methods

o Studied population, intervention, outcomes, setting

• Summary of results

o Magnitude of differences / robustness

• Possible remaining biases

• Final level of evidence (GRADE)

• Comparison with data from RCTs

o Explanations for differences

o Possible impact on guidelines

Key messages from papers of

sufficient quality: PICOT 1

PICOT 1: “ADHERENCE TO ICS THERAPY”

P People of all ages prescribed regular maintenance ICS


CComparison of outcomes between groups of different levels of

adherence (e.g. 0-25%, 25-50%, 50-75%, over 75%)

O Exacerbations, admissions, symptoms, QOL

T 12 months

PICOT 1 / Adherence papers rated as being of sufficiently high quality to

inform future guideline development by Taskforce Quality Raters

1

Sadatsafavi M, Lynd LD, Marra CA, FitzGerald JM. Dispensation of long-acting β agonists with or

without inhaled corticosteroids, and risk of asthma-related hospitalisation: a population-based study.

Thorax. 2014;69(4):328-34

2

Friedman HS, Navaratnam P, McLaughlin J. Adherence and asthma control with mometasone furoate

versus fluticasone propionate in adolescents and young adults with mild asthma. J Asthma.

2010;47(9):994-1000.

3

Campbell JD, Allen-Ramey F, Sajjan SG, Maiese EM, Sullivan SD. Increasing pharmaceutical

copayments: impact on asthma medication utilization and outcomes. Am J Manag Care.

2011;17(10):703-10.

4

Tan H, Sarawate C, Singer J, Elward K, Cohen RI, Smart BA, Busk MF, Lustig J, O'Brien JD, Schatz M.

Impact of asthma controller medications on clinical, economic, and patient-reported outcomes. Mayo

Clin Proc. 2009;84(8):675-84

5

Williams LK, Pladevall M, Xi H, Peterson EL, Joseph C, Lafata JE, Ownby DR, Johnson CC.

Relationship between adherence to inhaled corticosteroids and poor outcomes among adults with

asthma. J Allergy Clin Immunol. 2004;114(6):1288-93

6

Taegtmeyer AB, Steurer-Stey C, Price DB, Wildhaber JH, Spertini F, Leuppi JD. Predictors of asthma

control in everyday clinical practice in Switzerland.

Curr Med Res Opin. 2009;25(10):2549-55

7

Laforest L, Licaj I, Devouassoux G, Chatté G, Belhassen M, Van Ganse E, Chamba G. Relative

exposure to controller therapy and asthma exacerbations: a validation study in community pharmacies.

Pharmacoepidemiol Drug Saf. 2014;23(9):958-64.

8Laforest L, Licaj I, Devouassoux G, Chatte G, Martin J, Van Ganse E. Asthma drug ratios and

exacerbations: claims data from universal health coverage systems. Eur Respir J. 2014;43(5):1378-86

9Björnsdóttir US, Sigurðardóttir ST, Jonsson JS, Jonsson M, Telg G, Thuresson M, Naya I, Gizurarson

S. Impact of changes to reimbursement of fixed combinations of inhaled corticosteroids and long-acting

β2-agonists in obstructive lung diseases: a population-based, observational study. Int J Clin Pract.

2014;68(7):812-9. Awaiting second review

Risk of asthma-related hospitalization

and maintenance treatment

• Regular treatment has to be ICS-based

o No risk difference between ICS-LABA and ICS alone

o Greater risk with LABA alone vs ICS-LABA & ICS alone

• Regular > irregular treatment or none

o Lower risk with regular treatment

o Except with LABA

• Database. Well matched

• GRADE: low → moderate

o No clinical data on severity

• Clinical impact: yes

• Similar evidence from RCTs: noSadatsafavi Thorax 2014

Adherence to ICS

• Adherence MF-DPI>FP

• SABA use MF-DPI<FP

• No difference in other clinical outcomes

• Database. Well matched



• Clinical impact: limited/none

• Similar evidence from RCTs: no

Freidman Thorax 2014

Impact of shifting part of drug

costs to patients• Higher copayments

o Less adherence

o More adverse asthma outcomes (outpatient and ED

visits), higher costs

• Database





Campbell Am J Manag Care 2011

ICS vs LTRA by adherence

• Monotherapy

o Adherent patients: ICS>LTRA

o Non-adherent patients: ICS<LTRA

• Combination: Best = ICS-LABA

• Database + survey


o No clinical data on severity for the database part

o Responders’ bias for the survey


• Similar evidence from RCTs: partly (pragmatic)

Tan Mayo Clin Proc 2009

Adherence and asthma

exacerbations

• Low adherence increases the risk of ED visits

and oral steroid treatment

• Database, well matched





Williams JACI 2004


control

• Lower ACQ improvement associated with low adherence

o Other predictors: age, severity

• Prospective cohort study, multivariate analysis


o Selection bias?



Taegtmeyer Curr Med Res Opin 2009


outcomes

• Low adherence associated with poorer control and more

hospital contacts and oral steroid courses

• Prospective study in pharmacies , multivariate analysis,

validation of MPR data


o Selection bias & single pharmacy for each patient



Laforest 2014

Pharmacoepidemiol Druf Saf


exacerbations• Low MPR increases the risk of oral steroid treatment and

hospitalization

• Database, unmatched, multivariate adjustment




• Similar evidence from RCTs: noLaforest ERJ 2014

PICOT 1: Conclusions

• Several concordant observational studies showing an

association between low adherence and poor asthma

outcomes

o Each individually provides a moderate level of evidence

o Globally high level (concordance)?

• Adherence is influenced by copayment level

o With an impact on outcomes

• Maintenance therapy with LABA alone is dangerous

Key messages from papers

of sufficient quality:

PICOT 2

PICOT 2: “DEVICE TYPE”


I Effectiveness of different inhaler devices/delivery systems in

maintaining asthma control

CComparison of outcomes between groups using the same molecule

through different inhaler systems (pMDI, Breath activated MDI, DPI)


T 12 months

PICOT 2 / Devices papers rated as being of sufficiently high

quality to inform future guideline development by Taskforce

Quality Raters

1

Price D, Chrystyn H, Kaplan A, Haughney J, Román-Rodríguez M, Burden A,

Chisholm A, Hillyer EV, von Ziegenweidt J, Ali M, van der Molen T. Effectiveness

of same versus mixed asthma inhaler devices: a retrospective observational

study in primary care. Allergy Asthma Immunol Res. 2012;4(4):184-91

2

Price D, Roche N, Christian Virchow J, Burden A, Ali M, Chisholm A, Lee AJ,

Hillyer EV, von Ziegenweidt J. Device type and real-world effectiveness of

asthma combination therapy: an observational study. Respir Med.

2011;105(10):1457-66. doi: 10.1016/j.rmed.2011.04.010. Epub 2011 May 25.

3

Price D, Haughney J, Sims E, Ali M, von Ziegenweidt J, Hillyer EV, Lee AJ,

Chisholm A, Barnes N. Effectiveness of inhaler types for real-world asthma

management: retrospective observational study using the GPRD. J Asthma

Allergy. 2011;4:37-47.

4

Thomas M, Price D, Chrystyn H, Lloyd A, Williams AE, von Ziegenweidt J.

Inhaled corticosteroids for asthma: impact of practice level device switching on

asthma control. BMC Pulm Med. 2009;9:1. doi: 10.1186/1471-2466-9-1.

Mixed vs single device(s)• Single device = better control and less severe

exacerbations

• Database study, adjusted analyses


o No clinical data on severity and pts knowledge/skills



Price Allergy Asthma Immunol Res 2012

Impact of switching

• DPI to pMDI or BAI or other DPI

• BAI to pMDI or other BAI

• Poorer outcomes following the switch

• Database study, matched & adjusted analyses

• GRADE: low → moderateo No clinical data on severity and pts knowledge/skills

• Clinical impact: possible


Thomas BMC Pulm Med 2009

DPI vs pMDI for SFC

• pMDI better for several asthma outcomes

• Database study, matched


o No clinical data on severity and pts

knowledge/skills/adherence

• Clinical impact: uncertain (reinforce adherence, account

for preferences?)


DPI vs pMDI & BAI for ICS

administration

• BAI and DPI better than pMDI for several asthmaoutcomes

• Database study, unmatched, adjusted


o No clinical data on severity and pts knowledge/skills/adherence

• Clinical impact: uncertain (reinforce adherence, account for preferences? Contradictory data)


Price J Asthma Allergy 2011

Summary

• Using a single device might be better

• Switching without a consultation is followed by

poorer outcomes

• pMDI better for FDC?

• While BAI and DPI better for ICS?



PICOT 3PICOT 3: “SMOKING ASTHMATICS”

P Smokers and ex-smokers with asthma receiving ICS and having

exacerbations

I Effectiveness of different treatment regimens

C Comparison of outcomes between those going from low dose ICS to:

HDICS, LDICS + LABA, HDICS + LABA. All regimens +/- LTRA

O Exacerbations

T 12 months

Impact of smoking on control

• Lower control in smokers

• Same benefit for all (current, ex, never smokers)

• Prospective cohort study, unadjusted analyses

• GRADE: low → low

o High drop-out rate,

o No adjustment nor matching

o Few clinical data available besides control

• Clinical impact: no


Brusselle Respir Med 2012

Impact of smoking on control

• Better outcomes with extrafine vs standard size particles, larger differences in current and ex smokers

• Database matched study, adjusted analyses

• GRADE: low → moderateo No clinical / spirometry data

• Clinical impact: uncertain (exploratory)


Roche AJRCCM 2015



PICOT 4PICOT 4: “SMALL AIRWAYS MANAGEMENT; ICS PARTICLE SIZE”


I Effectiveness of small vs standard-size ICS particles in maintaining

asthma control

C Comparison of outcomes between groups using the same or different

molecules administered as extrafine of fine particles


T 12 months

PICOT 2 / Smoking Asthma papers rated as being of sufficiently high quality to

inform future guideline development by Taskforce Quality Raters

1

van Aalderen WM, Grigg J, Guilbert TW, Roche N, Israel E, Martin RJ, Colice G, Postma DS, Hillyer

EV, Burden A, Thomas V, von Ziegenweidt J, Price D. Small-particle Inhaled Corticosteroid as First-

line or Step-up Controller Therapy in Childhood Asthma. J Allergy Clin Immunol Pract.

2015;3(5):721-31

2

Martin RJ, Price D, Roche N, Israel E, van Aalderen WM, Grigg J, Postma DS, Guilbert TW, Hillyer

EV, Burden A, von Ziegenweidt J, Colice G. Cost-effectiveness of initiating extrafine- or standard

size-particle inhaled corticosteroid for asthma in two health-care systems: a retrospective matched

cohort study. NPJ Prim Care Respir Med. 2014;24:14081

3

Colice G, Martin RJ, Israel E, Roche N, Barnes N, Burden A, Polos P, Dorinsky P, Hillyer EV, Lee AJ,

Chisholm A, von Ziegenweidt J, Barion F, Price D. Asthma outcomes and costs of therapy with

extrafine beclomethasone and fluticasone. J Allergy Clin Immunol. 2013;132(1):45-54

4

Price D, Thomas M, Haughney J, Lewis RA, Burden A, von Ziegenweidt J, Chisholm A, Hillyer EV,

Corrigan CJ. Real-life comparison of beclometasone dipropionate as an extrafine- or larger-particle

formulation for asthma. Respir Med. 2013;107(7):987-1000

5

Price D, Martin RJ, Barnes N, Dorinsky P, Israel E, Roche N, Chisholm A, Hillyer EV, Kemp L, Lee

AJ, von Ziegenweidt J, Colice G. Prescribing practices and asthma control with hydrofluoroalkane-

beclomethasone and fluticasone: a real-world observational study. J Allergy Clin Immunol.

2010;126(3):511-8.e1-10

6

Allegra L, Cremonesi G, Girbino G, Ingrassia E, Marsico S, Nicolini G, Terzano C; PRISMA

(PRospectIve Study on asthMA control) Study Group. Real-life prospective study on asthma control

in Italy: cross-sectional phase results. Respir Med. 2012;106(2):205-14

7

Barnes N, Price D, Colice G, Chisholm A, Dorinsky P, Hillyer EV, Burden A, Lee AJ, Martin RJ,

Roche N, von Ziegenweidt J, Israel E. Asthma control with extrafine-particle hydrofluoroalkane-

beclometasone vs. large-particle chlorofluorocarbon-beclometasone: a real-world observational

study. Clin Exp Allergy. 2011;41(11):1521-32

8

Price D, Small I, Haughney J, Ryan D, Gruffydd-Jones K, Lavorini F, Harris T, Burden A, Brockman

J, King C, Papi A. Clinical and cost effectiveness of switching asthma patients from fluticasone-

salmeterol to extra-fine particle beclometasone-formoterol: a retrospective matched observational

study of real-world patients. Prim Care Respir J. 2013;22(4):439-48

RiRL/REG retrospective

matched cohort studiesTreatments Population Database Results

1 BDP pMDI

St vs EF

Initiation

Step-up

Vs LABA

Children 5-11 UK (CPRD)

US

(Optuminsight)

EF>St

EF=adding LABA

4 BDP pMDI

St vs EF

Initiation

Switch

12-80 UK GPRD CPRD EF > St

5 pMDI

St FP vs EF BDP

Initiation

Step-up

5-60 UK GPRD EF >/= St at lower

doses

7 BDP pMDI

St vs EF

Initiation

Step-up

5-60 UK GPRD EF > St


matched cohort studies

Treatments Population Database Results

2

C-E

BDP/FP pMDI

St vs EF

Initiation

12-60/

12-80

UK/US EF dominant

3

C-E

pMDI

St FP vs EF BDP

Initiation

Step-up

12-80 UK/US EF >/= St at lower

doses and costs

8

C-E

St FP-SAL vs EF

BDP-FOR

C-E

18-80 UK GPRD CPRD EF >/= St at lower

doses

EF dominant


matched cohort studies


o No clinical data on severity and pts

knowledge/skills/adherence

o Considering all studies together: high?



Prospective cohort study

• Unmatched, adjusted

• Comparison between treatments = secondary /

exploratory objective

• GRADE: low → low

o Selection bias

o Secondary objective

• Clinical impact: no


Allegra Respir Med 2012

REG-EAACI TASKFORCE REPORT

PANEL DISCUSSION

REG SUMMIT 2016, LYON, FRANCE, 16 APRIL

SESSION: ‘INFLUENCING’ Guideline Development: the REG/EAACI Taskforce Reports

TIME: 12.45-13.45pm

Chair / Moderator: Nikos Papadopoulos

Centre for Paediatrics and Child Health, Institute of Human Development, University of Manchester,

Manchester, UK; Department of Allergy, 2nd Pediatric Clinic, University of Athens, Athens, Greece

—

• Novelty: never been done before

• Expert taskforce

• Wide range of participants:

o Countries, Societies, Guideline groups

• Focussed work

o Important evidence gaps

• Current literature

• High-threshold set for quality

• Bespoke tool

o Developed for specific purpose it was used for

o Tested for inter-rate agreement

STRENGTHS

• Narrow literature focus

• Small number of papers for some PICOT questions

• Expertise of reviewers

• Best-of-3 assessment approach?

• Assessment of reporting rather than study quality

(same thing from a guidelines’ perspective)?

WEAKNESSES

Discussion

• Considering

o the discussions around the QA tool

o the frequent difficulty in finding required information in

reviewed papers

• Integration of comparative effectiveness

research in asthma guidelines

o is at risk of remaining a subject of reluctance from

guidelines and policy makers

o until reporting markedly improves

PRACTICAL APPLICATIONS

• How the literature findings and literature quality

assessment tool can best be used in the future, e.g.:

o By guideline groups:

– To revise guidelines

– To appraise evidence

o Journal editors/reviewers:

– To assess the quality of submission

o Education:

– To train / guide new researchers

o Advocacy

– To highlight limitations in the evidence and clear areas for

improvement in the reporting of, if not the design of

observational studies in the future

Importance of the results

• The tool

• The literature appraisal

Importance by target

Target Importance

Researchers

Editorial boards

Reviewers

Guideline developers, policymakers

Others

Importance by topic

Topic Importance

PICOT 1: adherence

LABA alone dangerous

LTRA better

MF-DPI>FP

Hight copayment dangerous

PICOT 2: device

Single

Switch

pMDI vs BAI vs DPI

Importance by topic

Topic Importance

PICOT 3: smoking

Lower control in smokers

More treatment effect with EF in

smokers

PICOT 4: extrafine particles

EF>/=St

More cost-effective

POTENTIAL TO EXTEND THE WORK

• Disease areas

• Study design (pragmatic trials)

• Wider collaborations – REG/EAACI ±:

o ATS

o ± ERS

o ± others

reg-eaaci taskforce report

Health & Medicine