XerteStandard SettinginPsychiatry Undergraduate OSCE

Dr Ben di Mambro

Clinical Lecturer in Psychiatry

Psychiatry Examination

80 EMIs1 hour 15 minutesStandard set2 OSCE stationsEach station 10 minutes3 tasks based on competencies% given for each task

May OSCE

StationMaxMinAllA60.1(14.3)46.4(5.6)54.5(10.3)B60.8 (6.6)48.2(4.4)55.5(9.0)C66.1(8.5)49.0(9.7)56.2(9.8)D67.7(12.8)47.2(4.1)54.3(10.8)

Psychiatry OSCE

Changes to the OSCECriterion marking schemeGlobal clinical assessmentHonoursPassBorderlineFail2 examiners were placed in each roomStandard set pass mark for each task

Task

Develop CAL package:

Train the examiners in the new marking schemeStandardise marking schemeTo allow for multiple sites to accommodate increasing numbers of studentsImprove inter-rater reliability

Development of Package

Scripted 10x 2-3 minute OSCE scenariosBorderline/PassPass/Honours2 hours filming in my office2 lecturers1 bullied student

Creation of Media

wmv files converted to flv using Flash MacromediaImages created utilising Flash MacromediaMP3 creation using Audacity

Xerte

Template ratings created by Julian TenneyMedia was imported into Xerte through the Wizard

The Package

An introduction to the new marking schemeExample mark sheetsVideo of example studentsRated on criteriaGlobal impression givenUser then offered feedback

Using the Wizard

Once Wizard create efficient method of presenting contentDefine scenarioDefine marking sheet, including number of criteriaDefine fail/borderline/pass/honoursImport media

Sample of Package

Distribution of Package

Letters and emails were sent to all examining consultants with following link

www.nottingham.ac.uk/~mczbjd/OSCE.htm

Package ran for ~20 minutes

Result of Package Introduction

30% examiners accessed the package prior to the OSCERemainder underwent abridged version before the OSCE

Analysis

ANOVAs undertaken on awarded scores by the different examiners on the same station

December OSCE

StationMaxMinAllA70.3(5.0)53.4(8.6)62.3(8.6)B72.6(9.1)52.1(8.85)63.7(11.5)C66.1(5.1)51.0(7.6)56.9(8.6)D66.1(56.7)56.6(5.0)60.7(7.4)

**p

Variation

StationMayDecA13.716.9B12.620.5C17.115.0D20.510.5

Limitations

Flash Player 7 not installed as standard on healthcare trusts computersMinor problems with Xerte - since resolvedDelay on slow broadband connections?Representative videosAny improvement likely to be due to the new marking scheme

Xerte

Once familiar:

Easy to useMultimedia packages can be create in minutesEasily distributedMuch more complicated to create bespoke packages

Future

Improve IT access in the healthcare trustsReduce bitrateProduce improved examples of borderline studentsExtend the package to attempt to standardise the Formative assessments?Make available to students