more on how usmle step 1 scores are challenging.4

7/26/2019 More on How USMLE Step 1 Scores Are Challenging.4

1/2

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.

Academic Medicine, Vol. 91, No. 5 / May 2016 609

Letters to the Editor

test preparation. There has been aproliferation of free and commercialeducational resources geared towardstandardized test preparation, whichprompts the question of whether thesethird parties are disrupting traditional

medical education and possibly evensupplanting elements of institution-derived and delivered curricula.Developing a universal preclinicalcurriculum to prepare students for Step1 could enable schools and students toface this challenge more effectively.

We surveyed 599 medical students acrossthe country to explore their perceptionsof Step 1 content coverage by the formalcurriculum at their medical schoolsand how those perceptions influenced

their methods and prioritization ofstudy. Students reported that Step 1content coverage in the medical schoolcurriculum was 17.6% less than theyexpected on average (a statisticallysignificant finding). As 40% of studentsbelieved their curriculum lackedoverlap with Step 1 content, studentsoverwhelmingly turned to review booksand question banks (98%), as well asto other online preparation products(79%). Reliance on these third-partyresources prompted a large proportionof students to shift their focus away fromtheir medical schools core curriculum(35%), and, by self-report, theirperformance in their schools curriculumsuffered (20%).

The medical student practice ofsupplementing and supplanting localinstitutional curricula with third-party materials may represent a trendtoward decentralization of curricularcontent. This observation offers further

justification for the academic medicinecommunity to reflect on the influence of

the USMLEs on medical school curriculaand student behavior. Does it stifleinnovation and individualization withinmedical schools and instead promote thegrowth of third-party learning productsthat are exempt from oversight andunregulated by the Liaison Committee onMedical Education?

When we presented our results atLearn Serve Lead 2014, attendees wereintrigued by a universal preclinicalcurriculum as an innovation (and an

To the Editor:United States

Medical Licensing Exam (USMLE)Step 1 scores are increasingly beingused to screen residency applicants.Recently, this misuse of the examwas highlighted by Prober andcolleagues1in Academic Medicine.The authors discussed unintendedconsequences of such behavior onmedical students, including greaterstress and anxiety, distortions ofmedical school curriculum, changesin specialty choice, and increasedfinancial and opportunity costs of

Academic Medicine, Vol. 91, No. 5 / May 2016


More on How USMLE Step1 Scores Are ChallengingAcademic Medicine

To the Editor:Prober andcolleagues1make a strong, rational

argument for not using United StatesMedical Licensing Examination(USMLE) Step 1 scores to screengraduate medical education (GME)applicants. The USMLE was designedto certify minimum competency formedical licensure. Unfortunately,residency program directors areinappropriately using USMLE scoresto screen applicants. The authorsrecommend development of systems tocollect competency-based evidence thatpredicts success in GME. We would liketo suggest a solution to institute sucha change.

First, the National Board of MedicalExaminers should stop reportingthree-digit scores for the USMLE Stepsand provide only pass/fail scores anda list of areas of strength and areas forimprovement based on performance.To promote habits of lifelonglearning, we should require studentsto address these recommended areasfor improvement and reflect on theefforts they make to improve. Program

directors would use this evidenceof efforts towards lifelong learninginstead of USMLE scores whenreviewing applications.

The mission of medical education isto improve the health of the nationspopulation. Failure to achievethis mission can jeopardize GMEfunding and the entire medicaleducation structure. It is acceptedthat assessment drives learning andshould inform curriculum design.

Thus our second suggestion is thatmajor consideration be given by GMEprograms to evidence of successfulparticipation in a population healthinitiative during medical school. Thiswill require evidence of working ininterprofessional teams, using data forquality improvement and knowledgeof population health. This requirementwill drive medical schools to helpstudents participate in innovativeprojects for population health anduse systems for 360-degree feedback


to assess students in a broad range ofcompetencies. This process will generatethe type of data needed by programdirectors and will complement theevidence for lifelong learning describedabove.

These goals are actually quite feasible. Atthe Cleveland Clinic Lerner College ofMedicine, we have had a No Tests, NoGrades philosophy since 2004. Studentslearn in collaborative small groups andreceive formative feedback from peersand faculty, which they use to strivefor constant improvement and write areflective essay on their progress each year.2A system such as this could be modified toimplement the proposals described above.

Disclosures:None reported.

Neil B. Mehta, MBBS, MS

Assistant dean of education informatics andtechnology, Cleveland Clinic Lerner College ofMedicine at Case Western Reserve University,Cleveland, Ohio; [email protected].

Alan Hull, MD, PhD

Associate dean of curricular affairs, Cleveland ClinicLerner College of Medicine at Case Western ReserveUniversity, Cleveland, Ohio.

James Young, MD

Executive dean, Cleveland Clinic Lerner Collegeof Medicine at Case Western Reserve University,

Cleveland, Ohio.

References

1 Prober CG, Kolars JC, First LR, Melnick DE.A plea to reassess the role of United StatesMedical Licensing Examination Step 1 scores inresidency selection. Acad Med. 2016;91:1215.

2 Dannefer EF, Henson LC. The portfolioapproach to competency-based assessmentat the Cleveland Clinic Lerner College ofMedicine. Acad Med. 2007;82:493502.
mailto:[email protected]:[email protected]

7/26/2019 More on How USMLE Step 1 Scores Are Challenging.4

2/2

Copyright by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.


Academic Medicine, Vol. 91, No. 5 / May 2016610

Reference

1 Prober CG, Kolars JC, First LR, MelnickDE. A plea to reassess the role of UnitedStates Medical Licensing Examination Step1 scores in residency selection. Acad Med.2016;91:1215.

Social Media and Storytellingin Medicine: Probing Deeper

To the Editor:We read with interestthe article Sounding Off on SocialMedia: The Ethics of Patient Storytellingin the Modern Era.1The authors addressa timely issuehealth professionalsincreasing use of social mediaandpresent three scenarios involving thesharing of patient stories on social mediato examine the considerable ethicalissues that arise from this modern formof digital storytelling. Like the authors,

for the USMLE are developed usingsuch a national consensus to whichfaculty from most U.S. medical schoolscontribute. Furthermore, in a priorCommentary one of us (C.G.P.)proposed the creation of a medicalschool collaborative, charged with theidentification of material that wouldrepresent a consensus opinion on

the core content of the curriculum.2We have initiated this process in onecontent area (microbiology) and a five-medical-school collaborative. If thiswere extended across the full curriculumand all medical schools, students wouldbe assured that they are being tested onwhat is being taught, and they wouldnot feel the need to rely on third-partysources to prepare for the USMLE.

Disclosures:None reported.

Charles G. Prober, MD

Senior associate dean for medical education andprofessor of pediatrics, Microbiology & Immunology,Stanford School of Medicine, Stanford, California;[email protected].

Joseph C. Kolars, MD

Senior associate dean for education and globalinitiatives, University of Michigan Medical School,Ann Arbor, Michigan.

Lewis R. First, MD

Professor and chair, Department of Pediatrics,University of Vermont College of Medicine,Burlington, Vermont, and past chair, National Boardof Medical Examiners, Philadelphia, Pennsylvania.

Donald E. Melnick, MD

President and chief executive officer, National Boardof Medical Examiners, Philadelphia, Pennsylvania.

References

1 Prober CG, Kolars JC, First LR, Melnick DE.A plea to reassess the role of United StatesMedical Licensing Examination Step 1 scoresin residency selection. Acad Med. 2016;91:14.

2 Prober CG, Khan S. Medical educationreimagined: A call to action. Acad Med.2013;88:14071410.

In Reply to Mehta et al and toLondon et al:We appreciate the letters

in response to our recent Commentary.1We hope that our Commentary and thisexchange continue to stimulate others toconsider strategies that enhance medicaleducation while clarifying the role ofstandardized testing in the holistic reviewof applicants for residency. A broadernational conversation that includes thoseresponsible for undergraduate medicaleducation, graduate medical education,and regulatory standards governingmedical education and licensure iscritical for meaningful and sustainable

education reform.

Mehta and colleagues suggest abandoningthe United States Medical LicensingExamination (USMLE) three-digit scoresin favor of a pass/fail score. We outlinedin our Commentary the limitations ofthis approach, but it should be recognizedthat individual medical schools canchoose to suppress the specific USMLEscores of their graduates in the ElectronicResidency Application Service. Mehtaand colleagues also suggest that evidence

of medical student engagement inpopulation health initiatives should be afactor considered in residency applicantselection. This is a good example ofprogram directors deciding what specificcharacteristics of applicants are mostcritical to success in their programs,guiding their selection accordingly.

London and colleagues provideinteresting medical student survey datathat underscore the preponderance ofstudents who rely upon third-partyresources to prepare for their USMLE

Step 1 examination. It has been suggestedthat medical students have threecurricula: one that we are teaching, onethat prepares them for their clinicalclerkships, and a third that is relevant toStep 1. On behalf of optimizing learning,we must assume responsibility forharmonizing these curricula.

London and colleagues suggestion of auniversal preclinical curriculum stronglyresonates. Relevant to this point, wenote that the content specifications

even more radical disruption) to wrestback control. Implementing a universalpreclinical curriculum could redefinethe requirements for entering medicalschool and could even call into questionthe necessity of delivering the preStep1 curriculum in a brick-and-mortarsetting. Students could decide whether totake courses offered by medical schools,

enroll in courses at other academicinstitutions, or study the material on theirown (as they, in part, do now). PerhapsStep 1 will eventually become theentrance examination for medical schooladmission.

It is essential that medical school curriculaprovide greater scope and nuance thanthat of the Step 1 blueprint. Considerationof a universal preclinical curriculum couldhelp us focus the medical schools primaryrole in establishing and implementing

the teaching and evaluation of essentialknowledge, attitudes, and skills requiredof students as they are trained to becomecompetent, entrustable physicians, notmaster test takers.

Disclosures:The data reported in this letter werepreviously presented at Learn Serve Lead 2014:The AAMC Annual Meeting, November 9, 2014,Chicago, Illinois.

Daniel A. London, MD, MS

Orthopaedic surgery resident, Icahn School ofMedicine at Mount Sinai, New York, New York.He was a medical student, Cleveland Clinic Lerner

College of Medicine of Case Western ReserveUniversity, Cleveland, Ohio, when this project wasundertaken; [email protected].

Regina Kwon

Medical student, University of Colorado School ofMedicine, Aurora, Colorado.

Anupama Atluru

Medical student, University of Texas SouthwesternMedical School, Dallas, Texas.

Katie Maurer, PhD

MD/PhD student, New York University School ofMedicine, New York, New York.

Ron Ben-Ari, MD

Vice chair for educational affairs, Department ofMedicine, associate dean for continuing medicaleducation and professional development, andassistant dean for curriculum, Keck School ofMedicine of University of Southern California, LosAngeles, California.

Pamela B. Schaff, MD

Associate professor of clinical pediatrics and familymedicine, associate dean for curriculum, anddirector, Program in Medical Humanities, Arts, andEthics, Keck School of Medicine of University ofSouthern California, Los Angeles, California.
mailto:[email protected]:[email protected]:[email protected]:[email protected]

more on how usmle step 1 scores are challenging.4

Documents