1

Reporting Quality in Acupuncture Trials: An Assessment of the Literature to Update the NIH

Consensus Conference

Reporting Quality in Acupuncture Trials: An Assessment of the Literature to Update the NIH

Consensus Conference

Symposium for Portland Area Research in CAM (SPARC)

April 16, 2011

Ryan J Milley, MAcOM, LAcResearch Associate

22

3

Conclusions from NIH Consensus Development Conference

“…promising results have emerged … showing efficacy of acupuncture in adult postoperative and chemotherapy related nausea and vomiting and in postoperative dental pain…”“...acupuncture may be useful as an adjunct treatment, or an acceptable alternative…in addiction, stroke rehabilitation, headache, menstrual cramps, tennis elbow, fibromyalgia, myofascial pain, osteoarthritis, low back pain, carpal tunnel syndrome, and asthma...”

3

44

55

66

This statement is more than five years old and is provided solely for historical purposes. Due to the cumulative nature of medical research, new knowledge has inevitably accumulated in this subject area in the time since the statement was initially prepared. Thus some of the material is likely to be out of date, and at worst simply wrong. For reliable, current information...we recommend consulting the National Institutes of Health's MedlinePlus.

77

8

Continues to be cited as a definitive source for conditions in which acupuncture is effective

Outdated statement 600+ RCTs have been conducted since 1997

Numerous RCTs on previously unstudied conditions have been conducted

8

Conclusions from NIH Consensus Development Conference

9

Aim: Update the NIH recommendations

Develop an objective & comprehensive quality of reporting instrument

Systematic reviews and meta-analyses use a heterogenous group of subjective instruments

Couple quality score findings with primary study outcome(s)

Consider the value of differing research trial designs, e.g. sham or usual care controls

9

10

Methods

Developed the Oregon CONSORT STRICTA Instrument (OCSI) CONSORT: Consolidated Standards of

Reporting Trials [22 item guideline]• Published 1996, updated 2001 & 2010

STRICTA: Standards for Reporting Interventions in Controlled Trials of Acupuncture [6 item guideline]

• Modifies CONSORT to reflect acupuncture interventions; published 2001, updated 2010

10

11

Methods: OCSI Development

Adapts CONSORT and STRICTA guidelines to a 27 question instrument

Comprehensive in nature Most items contain sub-questions - 63 total Items range from aspects pertaining to

• Bias (randomization) • Treatment relevance (Practitioner experience with

the condition)• External validity (generalizabilty of findings)

11

12

OCSI Development & Findings Scores questions as Yes=2, Partial=1,

No=0, or N/A 333 RCTs met the inclusion criteria From 27 countries and 141 journals

12

13

OCSI Development & Findings Scores questions as Yes=2, Partial=1,

No=0, or N/A 333 RCTs met the inclusion criteria From 27 countries and 141 journals

Evid Based Complement Alternat Med. 2011. doi:10.1155/2011/183910

13

14

30.9% (P<0.001)

15

30.9% (P<0.001)

Mean OCSI Score 63% SD 16.5

16

Methods

Assessed quality of reporting in the 333 acupuncture RCTs published in English from 1998-2007

Inclusion criteria: Treatment with acupuncture needles using

manual and/or electro-stimulation Control groups of no treatment, a sham

procedure, or usual biomedical care Clinical end points; not experimental

16

17

Methods

Quality threshold set at 70% OCSI score Indexed variables of each RCT related to

treatment methodologies and outcomes RCTs categorized as:

Efficacy Trials Comparative Effectiveness Research (CER)

Trials

17

18

Differing Lenses

18

Efficacy

Sham acupuncture

control. Better than placebo?

CER Usual care

control. At least as good as usual care?

19

Results 133 (40%) RCTs scored ≥70% on OCSI Represent 43 varied conditions Published in 76 differing journals Conducted in 21 countries

Germany & USA - 31 RCTs each UK - 16 RCTs Sweden - 12 RCTs Norway - 7 RCTs

19

20

Results

20

NIH Consensus Development Conference Category

Distribution of RCTs (n)

NIH “Showing efficacy” list 5% (7)

NIH “May be useful” list 50% (66)

NIH “Potential Future Research” list 45% (60)

21

Efficacy Trial Designs, i.e. sham acupuncture control 58% (70/133)

Comparative Effectiveness Research Trial Designs, i.e. Usual care control 32% (42/133)

Combined Efficacy & CER Trial Designs 10% (12/133)

21

Results

22

Results

Top 5 most investigated areas Pregnancy & labor related conditions (n=18) Chronic low back pain (n=13) Headache conditions(n= 11) Osteoarthritic conditions (n=10) Neck & shoulder pain conditions (n=10)

22

23

Top 5 Investigated Conditions

23

Pregnancy & Labor Related Condition Trials (n) Outcome

Pain During Labor 4 4+

Emesis in Pregnancy 3 2+; 1-

Initiation of Labor 3 2+; 1-

Pain During IVF Procedures 3 3+

Pregnancy Rate with IVF Procedures 3 2+; 1-

Breech Presentation 1 1+

Pelvic Pain During Pregnancy 1 1+

TOTAL 18 15+; 3-

Primary outcome assessed as: Positive: + or Negative: -

2424

Top 5 Investigated Conditions

Condition Trials (n) Outcome

Chronic Low Back Pain 13 11+; 2-

Headache (Migraine & TTH) 11 4+; 7-

Osteoarthritis (Knee) 10 8+; 2-

Neck & Shoulder Pain 10 8+; 2-

Primary outcome assessed as: Positive: + or Negative: -

2525

Top 5 Investigated Conditions

Condition Trials (n) Outcome

Chronic Low Back Pain 13 11+; 2-

Headache (Migraine & TTH) 11 4+; 7-

Osteoarthritis (Knee) 10 8+; 2-

Neck & Shoulder Pain 10 8+; 2-

Primary outcome assessed as: Positive: + or Negative: -

2626

Conditions Effectiveness Efficacy+ - + -

Chronic Low Back Pain(n=13; #Combo n=2) 10*# 2 3 0

Headache Conditions(n=11; #Combo n=1) 4 1 1 6#

Osteoarthritic Conditions(n=10; #Combo n=2) 7 1 3 # 1#

Neck & Shoulder Conditions(n=10; #Combo n=1) 4 2 5# 0

*No Treatment Control, #Both CER & Efficacy Designs

Differing Lenses

2727

Conditions Effectiveness Efficacy+ - + -

Chronic Low Back Pain(n=13; #Combo n=2) 10*# 2 3 0

Headache Conditions(n=11; #Combo n=1) 4 1 1 6#

Osteoarthritic Conditions(n=10; #Combo n=2) 7 1 3 # 1#

Neck & Shoulder Conditions(n=10; #Combo n=1) 4 2 5# 0

Differing Lenses

*No Treatment Control, #Both CER & Efficacy Designs

2828

Conditions Effectiveness Efficacy+ - + -

Chronic Low Back Pain(n=13; #Combo n=2) 10*# 2 3 0

Headache Conditions(n=11; #Combo n=1) 4 1 1 6#

Osteoarthritic Conditions(n=10; #Combo n=2) 7 1 3 # 1#

Neck & Shoulder Conditions(n=10; #Combo n=1) 4 2 5# 0

Differing Lenses

*No Treatment Control, #Both CER & Efficacy Designs

2929

Conditions Effectiveness Efficacy+ - + -

Chronic Low Back Pain(n=13; #Combo n=2) 10*# 2 3 0

Headache Conditions(n=11; #Combo n=1) 4 1 1 6#

Osteoarthritic Conditions(n=10; #Combo n=2) 7 1 3 # 1#

Neck & Shoulder Conditions(n=10; #Combo n=1) 4 2 5# 0

Differing Lenses

*No Treatment Control, #Both CER & Efficacy Designs

3030

Pregnancy & Labor (n=18) Effectiveness Efficacy

+ - + -

Pain During Labor (n=4) 2 0 2 0

Emesis in Pregnancy (n=3) 1 0 0 2

Initiation of Labor (n=3) 2 1 0 0

Pain During IVF Procedures (n=3) 2 0 1 0

Pregnancy Rate with IVF Procedures (n=3) 1 0 1 1

Breech Presentation (n=1) 1* 0 0 0

Pelvic Pain During Pregnancy (n=1) 1 0 0 0

TOTAL 10 1 4 3*No Treatment Control

Differing Lenses

31

Conclusions

A significant focus of research (45%) has been on previously understudied conditions

The lens of comparative effectiveness research, lends clarity Acupuncture has demonstrated consistent

benefit for the treatment of each of the five most often investigated areas

31

32

Conclusions We suggest new recommendations

regarding the benefits of acupuncture “Showing efficacy” list

Chronic low back pain (11+/2-) Headache conditions (4+/7-, CER Lens 4+/1-) Osteoarthritic conditions (8+/2-) Neck & shoulder pain conditions (8+/2-) Pregnancy & labor related conditions

• Pain during labor (4+/0-)

32

33

Conclusions We suggest new recommendations

regarding the benefits of acupuncture “May be Useful” list

Peri-op/Post-op Pain (7+/1-) Anxiety (3+/1-) Pregnancy & labor related conditions

• Pain during IVF procedures Rhinitis (3+/0-)

33

34

Future Directions

Update currently underway of RCTs published from 2008-2010 ~150 RCTs

Consider ways to assess clinical relevance e.g. ecological validity

34

35

Many Thanks to the OCSI Team:

Richard Hammerschlag, PhDMikel Aikin, PhD

Agatha Colbert, MD, LAcScott Mist, PhD, LAc

Jeffrey Weih, LAcBeth Yohalem-Ilsley, LAc

Questions?RMilley@ocom.edu

Many Thanks to the OCSI Team:

Richard Hammerschlag, PhDMikel Aikin, PhD

Agatha Colbert, MD, LAcScott Mist, PhD, LAc

Jeffrey Weih, LAcBeth Yohalem-Ilsley, LAc

Questions?RMilley@ocom.edu

35

Many Thanks to the OCSI TEAM

Richard Hammerschag, PhDMikel Aikin, PhD

Agatha Colbert, MDScott Mist, PhD, Lac

Jeffrey Weih, LacBeth Yohalem-Ilsley, Lac

Questions?Rmilley@ocom.edu