A Dedicated Evidence-Based Medicine Curriculum as part of Journal Club Improves

Resident Performance in Interpreting Medical Literature

Nicholas M. Mohr, Andrew J. Stoltze, Azeemuddin Ahmed, Jon N. Van Heukelom, Christopher P. Hogrefe, Karisa K. Harland.

Department of Emergency MedicineUniversity of Iowa

Disclosures

Nothing to Disclose

ACGME Emergency Medicine Program EBM Requirements Most recent requirements July 1, 2013. Residents are expected to develop skills and

habits to be able to meet the following goals: ▪ locate, appraise, and assimilate evidence from scientific

studies related to their patients’ health problems; (IV.A.5.d).(6)

▪ use information technology to optimize learning; (IV.A.5.d).(7)

▪ apply knowledge of study design and statistical methods to critically appraise the medical literature; (IV.A.5.d).(9)

▪ use information technology to improve patient care (IV.A.5.d).(10)

ACGME Program Requirements for Graduate Medical Education in Emergency Medicine. 7/1/2013.

Expectations of EBM Curricula: 2010 CORD Survey EBM Curricula:

75% reported no established EBM curriculum for journal club.

71% reported no critical appraisal instrument.

Carpenter CR et al. Academic Emergency Medicine 2010; 17:S54-S61.

Expectations of EBM Curricula: 2010 CORD Survey (continued)

27%

45%

28%

Most important outcome of GME EBM curricula - Residents

Become expert critical apprais-ers

Ability to find reliable materi-als

Differentiate flawed studies

Carpenter CR et al. Academic Emergency Medicine 2010; 17:S54-S61.

The Fresno Test

What it is: Series of open-ended questions. Standardized grading rubric. Measures knowledge and skills

necessary for EBM practice. Why use it:

Excellent inter-rater reliability, internal consistency, and item discrimination.

Construct validity established.Ramos KD et. al. BMJ. 2003; 326:319-21.

Objectives

Implement a novel dedicated curriculum in EBM.

Measure its effectiveness in improving EBM competence using a validated instrument (the Fresno test).

Methods: Curriculum

Dedicated Faculty Group

Methodology Lectures

Resident-Guided Critique

Article Selection to

Support Methodology

Topic

Methodology Lectures

Lecture Topics:- Introduction & PICO Question- Identifying an Article- Levels of Evidence- Gold Standard- Study Validity- CPC Competition- Bias- P-Values, Alpha- Sensitivity, Specificity, PPV, NPV- Number Needed to Treat- Absolute and Relative Risk Reduction- Meta-analysis and Cochrane

Methods: Measurement

Fresno test before and after curriculum.

Generalized Estimating Equations. Account for

repeated measures. Adjust for year in

residency.

Results

Majority of trainees completed the test in the pre-intervention (88%, n = 22) and post-intervention groups (92%, n = 23).

Results

Results

GEE ModelFactor Beta p

Year (pre-intervention referent)

14.55 (1.73 – 27.37) 0.026

R1 Referent 0.419

R2 11.16 (-7.04 – 29.36)

R3 10.58 (-10.40 – 31.56)

* note that advanced practice provider residents were excluded because no group existed after the intervention.

Summary

Competency in EBM improved significantly. (120.9 vs. 105.4, p = 0.026).

Improvement in evaluation of study validity was significant. (40.4 vs. 32.1, p = 0.034).

Not associated with performance: Attendance at journal club. Year in residency.

Limitations

No control group. Single program. Small number of participants. Multiple administrations of

instrument.

Conclusions

A novel, dedicated EBM curriculum implemented through resident journal club significantly improves EBM competency.

Acknowledgements Nicholas M. Mohr, MD Azeemuddin Ahmed, MD, MBA Jon N. Van Heukelom, MD Christopher P. Hogrefe, MD Karisa K. Harland, MPH, PhD

Department of Emergency Medicine, University of Iowa

Questions?

A Dedicated Evidence-Based Medicine Curriculum as part of Journal Club Improves

Resident Performance in Interpreting Medical Literature

Nicholas M. Mohr, Andrew J. Stoltze, Azeemuddin Ahmed, Jon N. Van Heukelom, Christopher P. Hogrefe, Karisa K. Harland.

Department of Emergency MedicineUniversity of Iowa