Using network meta-analysis to identify effective components

of complex mental health interventions

José A López-López1,2, Nicky J Welton1, Sarah R Davies1, and Deborah M Caldwell1

1. University of Bristol (UK)

2. University of Murcia (Spain)

1

Research SynthesisDubrovnik, 30 May 2019

Network Meta-Analysis

• Network meta-analysis (NMA) allows pooling evidence on multiple interventions from a set of studies

• A “network” is constructed by displaying the interventions compared in each study

• Comparison of multiple interventions for a health condition enables to address relevant questions for practitioners and policy makers

Pair-wise vs. network meta-analysis• Corticosteroids in septic shock

• Annane and colleagues (2015) published a Cochrane review comparing the efficacy of corticosteroids vs. placebo

• Gibbison and colleagues (2017) reanalysed the data from the review and conducted a NMA to examine each corticosteroid separately

Complex interventions for mental health

• Systematic reviews of interventions in mental health and other areas often deal with complex interventions which include several active ingredients or “components”

• If each combination of components is considered as a separate intervention, then NMA could be used to simultaneously compare the different interventions

• However, this could lead to a very large number of interventions (and possibly to disconnected networks)

Component-Level NMA

• Component-level network meta-analysis methods have been developed within a Bayesian framework (Welton et al., 2009)

• Component-level NMA may be used to examine• Role of each individual component• Interactions between multiple components

Example 1: CBT for adult depression

• Depression represents a substantial public health concern worldwide• Cognitive-Behavioural Therapy (CBT) is an effective psychological intervention for

depression• CBT interventions are complex

• Multiple content components• Delivered in different ways

6

Example 1: Systematic review

• Inclusion criteria:• Randomised controlled trials• Examining CBT interventions• In depressed adults

• Primary outcome: Change in depressive symptoms at short term• Effect size index: standardised difference in mean change (sDIMC)

7

Example 1: Included studies

• 91 studies reported one or more relevant outcomes• Primary outcome: 76 studies, 6973 patients• Large variability in

• Publication year• Study size• Country

8

Example 1: Definition of comparators

• Treatment as Usual (TAU): 38 interventions• No treatment: 7 interventions• Wait list: 33 interventions• Psychological/attention placebo: 14 interventions

9

Example 1: defining components

• Cognitive Techniques• Behavioural Activation• Psychoeducation• Homework• Problem Solving• Social Skills Training

• Relaxation• Goal Setting• Final Session• Mindfulness• Acceptance & Commitment

Therapy

10

Example 1: delivery methods

• Face-to-Face (F2F) CBT: 100 interventions• Hybrid CBT: 7 interventions• Multimedia CBT: 33 interventions

11

Example 1: Network meta-analysis models

• Full Interaction Model• Main Effects Model• Therapy Effects Model

12

Example 1: Full Interaction Model

• Standard Bayesian NMA (Dias et al., 2013)• Each delivery format and combination of components

is a unique intervention

Example 1: Main Effects Model

For a continuous outcome, treatment effect dt is estimated from:

14

1 2 3 4 5 6 7

8 9 10

* * * * * * * * *t CBT t t t t t t t

t t t

d d Multi Cog BA PsEd Home Prob SocRelax Goal Final

β β β β β β ββ β β

= + + + + + + ++ + +

Example 1: Therapy Effects Model

• Standard Bayesian NMA (Dias et al., 2013)• Treatments included

• TAU• No treatment• Wait list• Psychological/Attention placebo• Face-to-face CBT• Hybrid CBT• Multimedia CBT

Example 1: Results for Full Interaction Model

-8 -4.75 -1.5 1.75 5

Standardised Difference in Mean Change (compared to TAU)

60. MUL: Cog+BA+PsEd+Home+Goal+Final59. MUL: Cog+BA+PsEd+Home+Goal58. MUL: Cog+BA+PsEd+Home+Final57. MUL: Cog+BA+PsEd+Prob+Soc+Relax+Goal+Final56. MUL: Cog+BA+PsEd+Soc+Relax+Mind+ACT55. MUL: Cog+BA+PsEd+Goal+Final54. MUL: Cog+BA+PsEd+Final53. MUL: Cog+BA+Home+Prob+Soc52. MUL: Cog+BA+Home+Prob51. MUL: Cog+BA+Home+Soc+Relax50. MUL: Cog+BA+Home49. MUL: Cog+BA+Prob+Soc48. MUL: Cog+BA+Prob+Relax47. MUL: Cog+BA+Soc+Relax46. MUL: Cog+PsEd+Relax45. MUL: Cog+Home+Soc+Final44. MUL: Cog41. MUL: BA+Mind+ACT40. MUL: BA39. MUL: PsEd+Prob38. MUL: Home+Final37. MUL: Prob36. F2F: Cog+BA+PsEd+Home+Soc35. F2F: Cog+BA+PsEd+Home+Goal34. F2F: Cog+BA+PsEd+Home+Final33. F2F: Cog+BA+PsEd+Prob32. F2F: Cog+BA+PsEd+Soc+Relax31. F2F: Cog+BA+PsEd+Relax30. F2F: Cog+BA+PsEd29. F2F: Cog+BA+Home28. F2F: Cog+BA+Prob+Relax27. F2F: Cog+BA+Soc+Relax26. F2F: Cog+BA+Soc25. F2F: Cog+BA+Relax24. F2F: Cog+BA23. F2F: Cog+PsEd+Relax22. F2F: Cog+PsEd21. F2F: Cog+Home+Prob20. F2F: Cog+Home19. F2F: Cog+Prob18. F2F: Cog+Mind+ACT17. F2F: Cog14. F2F: BA+PsEd+Prob+Soc+Relax13. F2F: BA+Home+Goal12. F2F: BA+Home+Mind11. F2F: BA+Prob10. F2F: BA+Goal9. F2F: BA8. F2F: PsEd+Prob7. F2F: Home6. F2F: Prob5. F2F: ACT4. Placebo3. Wait list2. No treatment

-1.08 [-3.43, 1.26]-0.81 [-3.13, 1.55]-1.44 [-4.09, 1.22]0.02 [-2.64, 2.71]

-0.10 [-2.49, 2.28]-0.52 [-2.91, 1.86]-1.67 [-4.38, 1.04]0.14 [-2.50, 2.79]0.15 [-2.18, 2.49]

-0.19 [-2.22, 1.82]-2.08 [-4.47, 0.34]-0.87 [-3.53, 1.78]-0.58 [-2.92, 1.78]-0.85 [-3.36, 1.67]0.00 [-2.33, 2.38]

-0.12 [-2.79, 2.54]-0.41 [-1.94, 1.13]0.05 [-2.29, 2.42]

-0.80 [-3.14, 1.56]-1.08 [-3.78, 1.63]-0.82 [-3.17, 1.55]0.13 [-2.52, 2.81]

-0.47 [-2.83, 1.87]-0.41 [-1.99, 1.18]-1.64 [-4.32, 0.99]-1.37 [-3.23, 0.48]0.50 [-1.42, 2.40]0.55 [-2.09, 3.19]

-1.34 [-3.39, 0.71]-0.27 [-2.62, 2.12]-0.50 [-2.84, 1.85]-0.66 [-2.93, 1.64]0.08 [-2.26, 2.42]

-1.48 [-4.19, 1.24]-0.70 [-1.87, 0.46]-2.68 [-5.50, 0.16]-0.52 [-2.07, 1.02]1.10 [-1.88, 4.14]

-2.29 [-4.25, -0.36]-1.79 [-3.38, -0.22]-0.75 [-3.46, 1.95]-1.42 [-2.83, -0.01]-0.62 [-2.97, 1.72]0.55 [-1.74, 2.84]0.88 [-1.47, 3.23]

-2.29 [-5.13, 0.53]-0.35 [-3.01, 2.30]-1.57 [-2.65, -0.50]-1.75 [-4.45, 0.94]-6.92 [-9.29, -4.58]-2.02 [-3.46, -0.58]-1.71 [-4.39, 1.01]-0.32 [-1.64, 1.01]0.93 [-0.29, 2.13]

-0.57 [-2.14, 1.01]

Intervention sDiMC [95% CrI]

Example 1: Results for Full Interaction Model

Example 1: Results for Main Effects Model

-3 -1.5 0 1.5 3Standardised Difference in Mean Change (compared to TAU)

Final SessionGoal SettingRelaxationSocial Skills TrainingProblem SolvingHomeworkPsychoeducationBehavioural ActivationCognitive TechniquesMultimediaEFFECT MODIFIERSCBTPlaceboWait listNo TreatmentMAIN EFFECTS

-0.55 [-1.66, 0.54]0.61 [-0.40, 1.62]0.03 [-0.81, 0.88]0.55 [-0.37, 1.45]

-0.04 [-0.67, 0.59]-0.21 [-0.90, 0.49]0.04 [-0.66, 0.75]0.54 [-0.09, 1.16]0.38 [-0.21, 0.97]0.36 [-0.24, 0.96]

-1.77 [-2.57, -1.01]-0.53 [-1.41, 0.35]0.75 [ 0.09, 1.41]0.08 [-1.04, 1.23]

sDiMC [95% CrI]

Example 1: Results for Therapy Effects Model

19

1. TAU

2. No treatment

3. Wait list

4. Placebo5. F2F CBT

6. Hybrid CBT

7. Multimedia CBT

-3 -1.5 0 1.5 3Standardised Difference in Mean Change (compared to TAU)

Multimedia CBT

Hybrid CBT

F2F CBT

Placebo

Wait list

No treatment

-0.59 [-1.20, 0.02]

-1.06 [-2.05, -0.08]

-1.11 [-1.62, -0.60]

-0.34 [-1.21, 0.52]

0.72 [ 0.09, 1.35]

0.20 [-0.91, 1.31]

Intervention sDiMC [95% CrI]

Example 2: School-based interventions to prevent mental-ill-health in children and young people• Aim: to identify the most effective and cost-effective intervention

component(s), or combination of components for prevention of common mental health problems in children and young people

• Protocol available at http://www.crd.york.ac.uk/prospero/display_record.php?RecordID=48184&VersionID=75497

http://www.crd.york.ac.uk/prospero/display_record.php?RecordID=48184&VersionID=75497

Example 2: School-based interventions to prevent mental-ill-health in children and young people• Main outcomes

• Self-reported anxiety• Self-reported depression• Conduct problems

• Effect size index: standardised difference in mean change (sDIMC)• To be considered separately:

• Population: universal, targeted (selective/indicated)• Context: primary, secondary, university• Time point

Example 2: results for therapy effects model• Change in depression scores at mid term, universal, primary

Usual curriculum

No intervention

Wait list

Attention control

CBT

-3 -1.5 0 1.5 3

Difference in Mean Change (compared to Usual c

CBT

Attention control

Wait list

No intervention

-0

0

0

0

Intervention D

Example 2: results for full interaction model• Change in depression scores at mid term, universal, primary

UC

WL

None

AtCnt

Cog Beh

Cog Beh Relax

PsEd Cog Beh

PsEd Cog Beh Relax

-3 -1.5 0 1.5 3

Difference in Mean Change (compared to UC)

PsEd Cog Beh Relax

PsEd Cog Beh

Cog Beh Relax

Cog Beh

AtCnt

None

WL

-0

-0

-0

-0

-0

0

0

Intervention D

UC: Usual Curriculum; WL: Wait List; AtCnt: Attention Control; Cog: Cognitive; Beh: Behavioural; Relax: Relaxation; PsEd: Psychoeducation

Conclusions

• Component-level NMA is a promising approach with the potential to address relevant policy questions

• Adequate reporting of interventions is essential, as we rely on primary studies to characterize each intervention

• Further steps• Use Main Effects model to identify effective components• Simulation work to determine sample size required to obtain precise results

24

Using network meta-analysis to identify effective components

of complex mental health interventions

José A López-López1,2, Nicky J Welton1, Sarah R Davies1, and Deborah M Caldwell1

1. University of Bristol (UK)

2. University of Murcia (Spain)

25

Research SynthesisDubrovnik, 30 May 2019

Using network meta-analysis �to identify effective components �of complex mental health interventionsNetwork Meta-Analysis Pair-wise vs. network meta-analysisComplex interventions for mental healthComponent-Level NMAExample 1: CBT for adult depressionExample 1: Systematic reviewExample 1: Included studiesExample 1: Definition of comparatorsExample 1: defining componentsExample 1: delivery methodsExample 1: Network meta-analysis modelsExample 1: Full Interaction ModelExample 1: Main Effects ModelExample 1: Therapy Effects ModelExample 1: Results for Full Interaction ModelExample 1: Results for Full Interaction ModelExample 1: Results for Main Effects ModelExample 1: Results for Therapy Effects ModelExample 2: School-based interventions to prevent mental-ill-health in children and young peopleExample 2: School-based interventions to prevent mental-ill-health in children and young peopleExample 2: results for therapy effects modelExample 2: results for full interaction modelConclusionsUsing network meta-analysis �to identify effective components �of complex mental health interventions