Evidence Based Medicine and Level 1 Evidence Based Medicine and Level 1 Outcomes Research in Pediatric Outcomes Research in Pediatric

SurgerySurgery

George W. Holcomb, III, M.D., MBAChildren’s Mercy Hospital

Kansas City, Missouri

Center for Prospective Clinical Trialswww.cmhclinicaltrials.com

The Right People

• Integration of best research evidence with clinical expertise and patient values

• Treating patients based on data, not “feeling” (gestalt), or one’s own experience

Evidence Based Medicine

Levels of EvidenceQuestion

A clinical surgeon publishes a retrospective review of 350 patients over 20 years undergoing an endorectal pull-through (Soave procedure) for Hirschsprung’s Disease. This is felt to be a seminal paper on this disease in infants and children. What is the level of evidence for this paper?

Level 1 (A)Level 2 (B)Level 3 (C)Level 4 (D)Level 5 (E)

5 – Expert opinion, or applied principles from physiology, basic science, or other conditions

4 – Case series or poor quality case control and cohort studies

3 – Case control studies

2 – Review of case control or cohort studies with agreement or poor quality randomized trial

1 – Prospective, randomized controlled trials

Levels Of Evidence

Levels of Evidence

5. Expert opinion, or applied principles from physiology, basic science, or other conditions

Example:

• Leave patient intubated and paralyzed for 3-5 days following an esophageal resection to take tension off esophageal anastomosis

No data – no study to show intubated/paralyzed patient has less esophageal tension

• Transverse incision is used for abdominal exploration in baby b/c better exposure

No data to support this practice

Levels of Evidence

4. Case series or poor quality case control and cohort studies

Example:

1) Paper reviewing results from one approach to a disease

Large retrospective review of Soave operation for Hirschsprung’s Disease

Levels of Evidence

3. Case control studies

Example:

1) Paper showing different management strategies/operative technique for one disease process

Single center (or multicenter) retrospective review of Duhamel vs Soave operation for Hirschsprung’s Disease

Levels of Evidence

2. Review of case control or cohort (followed “long-term”) studies with agreement or a poorly performed prospective randomized trial

Example:

1) Review of two or three large series describing one management strategy (Soave procedure) compared to two or three large series describing another management strategy (Duhamel procedure)

Levels of Evidence

1. Prospective randomized trials

Flawed Prospective Studies Are Usually Better Than Retrospective

Data Collections

p - Value• Commonly accepted p-value is 0.05

• 5/100 (1/20) chance that if a difference is found b/w variables, there really is no difference b/w the variables

• If p=0.01 there is a 1/100 chance that if a difference is found b/w variables, there really is no difference

• P=0.07 7/100 – still pretty significant, but readers will think there really is no significance b/c >0.05 (depends on what variable one is looking at)

How To Set Up a Study

• Alpha –Usually 0.05 is used; establishes level of significance of study

Value is used to determine # of patients needed

Retrospective fundoplication review: 12% vs 5%• 360 patients needed to show a significant difference• If αα was 0.1, need fewer patients• If αα was 0.01, need more patients (because significance is

greater)

Set the αα low to avoid a Type I error

How To Set Up a Study

• Power – commonly accepted is 0.8

• 20% chance that the study will fail to detect a difference when there really is one

or

• 80% chance that the study will detect a difference when one does exist

• Underpowering a study risks a Type II error

Children’s Mercy Hospital

Focus on common conditions which are “controversial”:

• Pyloric stenosis

• Appendicitis

• Pectus excavatum

• Fundoplication for reflux

• Empyema

• Non-palpable intra-abdominal testis

Remember

• There is a lot more to an MIS operation than just technique

• Postoperative care is also important and open for study (antibiotics?, pain management?, etc.)

Landscape of Pediatric Surgery

• Young field, most practicing surgeons are 2-3 generations removed from the founders of the field

• Very few training centers, the opinions of a few affect the masses

• Wide variety of rare cases make acquisition of objective treatment data difficult

Open vs Lap Pyloromyotomy

• Lap vs Open – 2003 - controversial around the world and in our hospital

• Different feeding regimens used in our hospital (2 hours, 4 hours, 6 hours)

• Different postoperative pain management strategies utilized

• Differences between staff made it difficult for residents, NPs, floor nurses

• Benefits: single protocol for feeding, pain management, discharge used in study still used currently (6 years later)

• No level 1 data

ResultsOutcomesOutcomes

OPEN (n = 100)OPEN (n = 100) LAP (n = 100)LAP (n = 100) P ValueP Value

OR time OR time (mins)(mins) 19:28 +/- 0.60 19:28 +/- 0.60 19:34 +/- 0.7819:34 +/- 0.78

Emesis (#)Emesis (#) 2.61 +/- 0.27 1.84 +/- 0.23 2.61 +/- 0.27 1.84 +/- 0.23 Full Feeds (hrs)Full Feeds (hrs) 21:01 +/- 2.16 21:01 +/- 2.16 19:30 +/- 1.46 19:30 +/- 1.46

LOS (hrs)LOS (hrs) 33:10 +/- 1.63 33:10 +/- 1.63 29:38 +/- 1.69 29:38 +/- 1.69

Tylenol (doses)Tylenol (doses) 2.23 +/- 0.18 2.23 +/- 0.18 1.59 +/- 0.161.59 +/- 0.16

(Mean +/- S.E.) (Mean +/- S.E.)

0.93

0.05

0.43

0.12

0.01

Ann Surg 244:363-370, 2006 Ann Surg 244:363-370, 2006

ConclusionsConclusions

• Operative approach for pyloromyotomy has no significant influence on operating time or length of recovery

• Laparoscopic pyloromyotomy results in significantly less post-operative discomfort

• Fewer episodes of emesis and doses of Tylenol

• Laparoscopic pyloromyotomy results in obvious cosmetic benefits

ASA, 2006ASA, 2006

2. Thoracoscopy vs Fibrinolysis for Empyema

• Fibrinolysis had been shown to be better than chest tube drainage alone

• Primary thoracoscopic debridement had been shown to better than tube drainage alone in several retrospective studies

• At the initiation of this study, there were no comparative data between primary thoracoscopic debridement and fibrinolysis as initial treatment for empyema in children

Treatment Of Empyema

• Using our own institution’s retrospective data on length of hospitalization after intervention between thoracoscopic debridement and fibrinolysis with an alpha 0.05 and power of 0.8

• Sample size of 36 with 18 in each arm

Sample Size

Empyema Study ProtocolFibrinolysis

• 12 Fr tube placed by IR or surgery in procedure room

• 4mg tPA in 40ml NS given into tube on insertion and each day for 3 doses

ThoracoscopyThoracoscopy• Thoracoscopic debridement with chest tube left

behind on – 20 cm H20 suction

J Pediatr Surg 44:106-111, 2008J Pediatr Surg 44:106-111, 2008

• Length of hospitalization after intervention (tPA or thoracoscopic debridement) until discharge criteria met (chest tube removed, afebrile & oral analgesics)

Empyema Study ProtocolEmpyema Study Protocol

Primary Outcome Measure

J Pediatr Surg 44:106-111, 2008J Pediatr Surg 44:106-111, 2008

• Days of Tmax > 38CDays of tube drainage

• Doses of analgesia

• Days of oxygen requirement

• Hospital charges after intervention

• Procedure charges

Empyema Study ProtocolEmpyema Study ProtocolSecondary Outcome Measure

J Pediatr Surg 44:106-111, 2008J Pediatr Surg 44:106-111, 2008

Patient Variables at ConsultationPatient Variables at Consultation

WBC WBC 20.820.8 19.719.7 0.71 0.71

Weight (kg) Weight (kg) 24.624.6 20.720.7 0.52 0.52

Age (Years) Age (Years) 4.8 4.8 5.2 5.2 0.770.77

Days of SymptomsDays of Symptoms 9.0 9.0 10.610.6 0.320.32

VATSVATS tPAtPA P ValueP Value

O2 support (L/min)O2 support (L/min) 0.81 0.81 0.79 0.79 0.96 0.96

Study Results

ER/PCP visits ER/PCP visits 2.9 2.9 2.7 2.7 0.69 0.69

J Pediatr Surg 44:106-111, 2008J Pediatr Surg 44:106-111, 2008

OutcomesOutcomes

16.6% failure rate for fibrinolysis16.6% failure rate for fibrinolysis

VATSVATS tPAtPA P ValueP Value

PO Fever (Days)PO Fever (Days) 3.1 3.1 3.8 3.8 0.46 0.46

O2 tx (Days) O2 tx (Days) 2.25 2.25 2.33 2.33 0.89 0.89

LOS (Days)LOS (Days) 6.89 6.89 6.83 6.83 0.960.96

Patient ChargesPatient Charges $11,660 $11,660 $7,575$7,575 0.010.01

Analgesic dosesAnalgesic doses 22.322.3 21.421.4 0.90 0.90

Study Results

J Pediatr Surg 44:106-111, 2008J Pediatr Surg 44:106-111, 2008

London Prospective TrialVATS v Fibrinolysis w/UrokinaseVATS v Fibrinolysis w/Urokinase

• No difference in LOS (6 v 6 days)

• No difference in 6 month CXR

• VATS more expensive ($11.3K v $9.1K)

• 16 % failure rate for fibrinolysis

Am J Respir Crit Care Med 174:221-227, 2006Am J Respir Crit Care Med 174:221-227, 2006

• There appears to be no therapeutic or recovery advantages to thoracoscopic debridement compared to fibrinolysis as the primary treatment for empyema

• Thoracoscopy results in significantly higher patient charges

CONCLUSIONS

J Pediatr Surg 44:106-111, 2008J Pediatr Surg 44:106-111, 2008

PRCTs Now Enrolling• Burn study – SSD vs collagenase (100/150) – Stopped early at interim

analysis

• One stage vs 2 stage laparoscopic orchiopexy for intra-abdominal testis (28/30)

• Standardized feeding protocol vs ad lib feedings following laparoscopic pyloromyotomy (100/150)

• Esophago-crural sutures vs no sutures at laparoscopic fundoplication (both groups receive minimal esophageal dissection), APSA 2010

• Irrigation/suction vs suction alone in patients with perforated appendicitis American Surgical 2012?

• Use of US vs landmark guided CVL placement (with Stanford) (40/80)

• IR drainage of appendiceal abscess with vs w/o instillation of fibrinolytic agent (tPA) (39/62)

• SSULS cholecystectomy vs 4 port lap cholecystectomy, APSA 2012?

• SSULS splenectomy vs 4 port lap splenectomy (6/30)

SummarySummary

• Evidence based decision making will continue to have a stronger presence in medical training

• Patient management will continue to become more influenced by evidence over opinion

• Hurdles to performing prospective trials in surgery and medicine are mostly surmountable

QUESTIONS

www.cmhclinicaltrials.com