xerte standard setting undergraduate osce

of 24 /24
Xerte Xerte Standard Setting Standard Setting in in Psychiatry Undergraduate Psychiatry Undergraduate OSCE OSCE Dr Ben di Mambro Dr Ben di Mambro Clinical Lecturer in Clinical Lecturer in Psychiatry Psychiatry

Author: trinhhanh

Post on 15-Dec-2016




1 download

Embed Size (px)


  • XerteStandard SettinginPsychiatry Undergraduate OSCE

    Dr Ben di Mambro

    Clinical Lecturer in Psychiatry

  • Psychiatry Examination

    80 EMIs1 hour 15 minutesStandard set2 OSCE stationsEach station 10 minutes3 tasks based on competencies% given for each task

  • May OSCE

    StationMaxMinAllA60.1(14.3)46.4(5.6)54.5(10.3)B60.8 (6.6)48.2(4.4)55.5(9.0)C66.1(8.5)49.0(9.7)56.2(9.8)D67.7(12.8)47.2(4.1)54.3(10.8)

  • Psychiatry OSCE

    Changes to the OSCECriterion marking schemeGlobal clinical assessmentHonoursPassBorderlineFail2 examiners were placed in each roomStandard set pass mark for each task

  • Task

    Develop CAL package:

    Train the examiners in the new marking schemeStandardise marking schemeTo allow for multiple sites to accommodate increasing numbers of studentsImprove inter-rater reliability

  • Development of Package

    Scripted 10x 2-3 minute OSCE scenariosBorderline/PassPass/Honours2 hours filming in my office2 lecturers1 bullied student

  • Creation of Media

    wmv files converted to flv using Flash MacromediaImages created utilising Flash MacromediaMP3 creation using Audacity

  • Xerte

    Template ratings created by Julian TenneyMedia was imported into Xerte through the Wizard

  • The Package

    An introduction to the new marking schemeExample mark sheetsVideo of example studentsRated on criteriaGlobal impression givenUser then offered feedback

  • Using the Wizard

    Once Wizard create efficient method of presenting contentDefine scenarioDefine marking sheet, including number of criteriaDefine fail/borderline/pass/honoursImport media

  • Sample of Package

  • Distribution of Package

    Letters and emails were sent to all examining consultants with following link


    Package ran for ~20 minutes

  • Result of Package Introduction

    30% examiners accessed the package prior to the OSCERemainder underwent abridged version before the OSCE

  • Analysis

    ANOVAs undertaken on awarded scores by the different examiners on the same station

  • December OSCE


  • **p

  • Variation


  • Limitations

    Flash Player 7 not installed as standard on healthcare trusts computersMinor problems with Xerte - since resolvedDelay on slow broadband connections?Representative videosAny improvement likely to be due to the new marking scheme

  • Xerte

    Once familiar:

    Easy to useMultimedia packages can be create in minutesEasily distributedMuch more complicated to create bespoke packages

  • Future

    Improve IT access in the healthcare trustsReduce bitrateProduce improved examples of borderline studentsExtend the package to attempt to standardise the Formative assessments?Make available to students