can simulation replace part of clinical time? two parallel randomised controlled trials

Can simulation replace part of clinical time? Twoparallel randomised controlled trialsKathryn Watson,1 Anthony Wright,2 Norman Morris,3 Joan McMeeken,4 Darren Rivett,5 Felicity Blackstock,6

Anne Jones,7 Terry Haines,8 Vivienne O’Connor,9 Geoffrey Watson,1 Raymond Peterson10 & Gwendolen Jull1

CONTEXT Education in simulated learningenvironments (SLEs) has grown rapidly across health careprofessions, yet no substantive randomised controlled trial(RCT) has investigated whether SLEs can, in part,substitute for traditional clinical education.

METHODS Participants were physiotherapy students(RCT 1, n = 192; RCT 2, n = 178) from six Australianuniversities undertaking clinical education in an ambula-tory care setting with patients with musculoskeletal disor-ders. A simulated learning programme was developed as areplica for clinical education in musculoskeletal practiceto replace 1 week of a 4-week clinical education place-ment. Two SLE models were designed. Model 1 provided1 week in the SLE, followed by 3 weeks in clinicalimmersion; Model 2 offered training in the SLE in parallelwith clinical immersion during the first 2 weeks of the4-week placement. Two single-blind, multicentre RCTs(RCT 1, Model 1; RCT 2, Model 2) were conducted usinga non-inferiority design to determine if the clinical com-petencies of students part-educated in SLEs would be anyworse than those of students educated fully in traditional

clinical immersion. The RCTs were conducted simulta-neously, but independently. Within each RCT, studentswere stratified on academic score and randomised toeither the SLE group or the control (‘Traditional’) group,which undertook 4 weeks of traditional clinical immer-sion. The primary outcome measure was a blindedassessment of student competency conducted over twoclinical examinations at week 4 using the Assessment ofPhysiotherapy Practice (APP) tool.

RESULTS Students’ achievement of clinical compe-tencies was no worse in the SLE groups than in the Tra-ditional groups in either RCT (Margin [D] ‡ 0.4 differenceon APP score; RCT 1: 95% CI ) 0.07 to 0.17; RCT 2: 95%CI ) 0.11 to 0.16).

CONCLUSIONS These RCTs provide evidence thatclinical education in an SLE can in part (25%) replaceclinical time with real patients without compromisingstudents’ attainment of the professional competenciesrequired to practise.

simulation

Medical Education 2012: 46: 657–667doi:10.1111/j.1365-2923.2012.04295.x

Discuss ideas arising from this article atwww.mededuc.com ‘discuss’

1Division of Physiotherapy, School of Health and RehabilitationSciences, University of Queensland, Brisbane, Queensland, Australia2School of Physiotherapy, Faculty of Health Sciences, CurtinUniversity of Technology, Perth, Western Australia, Australia3School of Physiotherapy and Exercise Science, Griffith HealthGriffith University, Gold Coast, Queensland, Australia4Division of Physiotherapy, School of Physiotherapy, Faculty ofMedicine, Dentistry and Health Sciences, University of Melbourne,Melbourne, Victoria, Australia5Discipline of Physiotherapy, School of Health Sciences, Universityof Newcastle, Newcastle, New South Wales, Australia6Department of Physiotherapy, School of Allied Health, La TrobeUniversity, Melbourne, Victoria, Australia7School of Public Health, Tropical Medicine and RehabilitationSciences, James Cook University, Townsville, Queensland, Australia

8Allied Health Research Unit, Southern Physiotherapy ClinicalSchool, Monash University, Cheltnam, Victoria, Australia9School of Medicine, University of Queensland, Brisbane,Queensland, Australia10Faculty of Health Sciences,The University of Adelaide, Adelaide,Australia

Correspondence: Gwendolen Jull, Division of Physiotherapy, Schoolof Health and Rehabilitation Sciences, University of Queensland,Brisbane St Lucia, Queensland 4072, Australia.Tel: 00 61 7 3365 1114; Fax: 00 61 7 3365 1622; E-mail:[email protected]

ª Blackwell Publishing Ltd 2012. MEDICAL EDUCATION 2012; 46: 657–667 657

INTRODUCTION

Simulated learning environments (SLEs) were firstemployed in medical education almost 50 yearsago.1 Their use has grown enormously over subse-quent decades.2–4 In emergency medicine pro-grammes, for example, the use of SLEs increasedfrom 29% to 85% within programme curriculawithin 5 years.5 The popularisation of the SLEreflects its many purported educational benefits,which include student-centred learning, safe skillacquisition, development of communication skillsand the provision of predetermined standardisedcase experiences,2,6–8 as well as its facility to addressgrowing ethical concerns about whether studentsshould ‘practise’ on patients, and issues involvingpotential litigation and patient safety.7,9–12 Thecurrent international shortage of suitable and avail-able student clinical education and specialty trainingplacements in many professions is concomitant withpressures to increase student numbers to meet risingdemand for health care provision.13–17 Simulation isnow being used to replace some of the time spent intraditional clinical learning environments withinmedicine, nursing and allied health training.3,18–22

In Australia, in an initiative of the Councils ofAustralian Governments, Health Workforce Austra-lia23 is working to implement the use of SLEs as ameans of increasing the capacity of the health caresystem to provide clinical training across the rangeof health disciplines.

There are concerns about introducing a clinicaleducation method that replaces time in a real clinicwithout evidence that it maintains educational out-comes.2,24 Enthusiasm for the use of simulationseems to far exceed evidence that students canachieve the range of competencies required in ahealth professional through practice in an SLE.Health education programmes should provideaccountable and beneficial results.25 There is evi-dence that certain technical skills can be developedwell.26–33 Likewise, there is evidence34,35 that com-munication skills can be taught using standardisedpatients (SPs), who may be actors who are trained tosimulate a patient’s illness in a standardised way oractual patients who are trained to present their ownillnesses in a standardised way.36 Nevertheless,although achieving isolated competencies is impor-tant, real-life practice with patients requires a rangeof professional competencies. It is unknown atpresent whether an SLE can be used for educationacross all key competencies and whether simulationcan in part replace traditional clinical education with

real patients sufficiently well to assist students toachieve the key professional competencies requiredfor total patient management.37 Despite the wide-spread incorporation of the SLE in health professionseducation programmes, no substantive randomisedcontrolled trial (RCT) has investigated its effective-ness as a partial substitute for traditional clinicaleducation, prompting growing calls for evidence-based research into the use of the SLE.12,33,38

To address this deficit, two RCTs were conducted todetermine the effects on student achievement ofclinical competencies of replacing 1 week of a 4-weektraditional clinical placement with learning in anSLE. A curriculum using SPs was developed in thecontext of entry-level physiotherapy student clinicaleducation in the management of patients with mus-culoskeletal disorders. It is unknown how best toconfigure a simulated learning experience withinclinical immersion. We developed two models tocombine an SLE with traditional clinical immersion.Each SLE model was tested in a separate RCT. Theprimary hypothesis for each trial was that the attain-ment of clinical competencies of students part-edu-cated in an SLE would be no worse than that ofstudents educated fully in the traditional clinicalimmersion model. Secondary outcomes referred tostudents’ self-reflection on their own growth acrosscompetencies. These trials were concerned with thefirst two levels of Kirkpatrick’s framework of trainingprogramme evaluation: the reaction of the student,and the learning outcome.39

METHODS

Trial design and participants

Two parallel-group, single-blind, multicentre RCTswere conducted simultaneously, but independently,using a non-inferiority design.40 Eligible participantswere physiotherapy students from six Australianuniversities, scheduled to undertake a clinical place-ment in musculoskeletal physiotherapy as part of theclinical education requirements for their entry-leveldegrees. Participants were drawn from studentcohorts at each university in 2009 and 2010. Studentsvolunteered to participate following a presentationabout SLEs and a full explanation of the study.

A sample size of 90 students per group in each RCTprovided power of 0.95 to detect a difference of 0.4 inmean clinical competence grading between groupsfor each trial. This analysis employed a two-tailed

658 ª Blackwell Publishing Ltd 2012. MEDICAL EDUCATION 2012; 46: 657–667

K Watson et al

alpha of 0.05 and assumed a standard deviation (SD)of 0.8. The SD estimate was drawn from clinicalevaluation data for 133 students at one participatinguniversity in 2005. A difference of 0.4 represented lessthan one half of one grading level on the clinicalassessment scale, which ranged from 1 to 5, and wasused to generate the SD data. Each RCT maintained apower of 0.93 to detect a difference of 0.4 betweengroup means even when incorporating a 10% drop-out rate. We planned to accept 92 students per group(n = 184 in each RCT) to permit an even division ofstudents across participating universities.

Ethical clearance was gained from the ethics com-mittees of all participating educational institutes andhospitals. All participants provided written informedconsent and understood that they would be allocatedto either trial and then randomly assigned to thesimulated or traditional education group within thattrial.

Interventions

An SLE programme, equivalent to one clinicaleducation week, was developed as a replica forclinical education in the musculoskeletal field. Itconsisted of nine patient cases to be portrayed bySPs. Cases represented a variety of conditions of thespine and extremities that are commonly encoun-tered in physiotherapy practice. Treatment time-lines, and ethics-related, safety and communicationlearning interactions were incorporated into thecase scenarios. This ensured that students wereexposed to elements that would assist them to attaina broad range of clinical competencies. In theprocess of preparing scenarios, real patients wereassessed and treated by experienced physiothera-pists, who were not members of the project team.The occasions were videotaped and scripted for SPtraining. Mock charts, X-rays, props, medical testresults and clinical reasoning and reflection taskswere built into the scenarios to replicate the clinicalsetting.

Two SLE models were designed. In Model 1 (RCT 1),full-time practice within an SLE replaced the firstweek of the 4-week clinical placement and studentsspent the remaining 3 weeks in clinical immersion. InModel 2 (RCT 2), the SLE was interspersed withclinical immersion over the first 2 weeks of the 4-weekplacement (i.e. students’ days in the first 2 weekswere divided equally between the SLE and clinicalimmersion so that the SLE training provided wasequivalent to that in one full-time week) and theremaining 2 weeks were spent entirely in clinical

immersion. Dedicated clinical educators were em-ployed by the universities to tutor students in the SLEprogrammes. The usual clinic or hospital educatorstutored students in the clinical venues. The educa-tor : student ratio was 1 : 4 in both the SLE andimmersion components.

Students in the control (‘Traditional’) groups in eachRCT undertook the traditional learning modelinvolving 4 weeks of full-time clinical immersion.They were tutored by usual clinic or hospital educa-tors at an educator : student ratio of 1 : 4. The SLEand Traditional groups spent equivalent time in thelearning environment.

Outcome measures

Primary outcome

The primary outcome in each RCT was studentcompetency to practise as an entry-level physio-therapist at the end of week 4 of the clinicalplacement. This was assessed in two clinical exam-inations, involving: (i) the evaluation and treatmentof a new patient, and (ii) a follow-up evaluationand treatment of a patient previously managed bythe student. The combined result of these exam-inations was used in analysis. Assessment was con-ducted by an independent examiner at eachuniversity who was blinded to the SLE or Tradi-tional group allocation of the student, and trainedin the use of the assessment tool. To ensureexaminer consistency across SLE and Traditionalgroup students at the various sites, each examinerassessed all eight of the students who comprised atrial unit (i.e. four SLE and four Traditional groupstudents).

Each examiner assessed students using the Assess-ment of Physiotherapy Practice (APP), a tooldeveloped in accordance with the competencystandards required for entry-level practice by theAustralian Physiotherapy Council.41,42 The toolcomprises 20 items and assesses performance inseven key areas, concerning: professional behaviour;communication; assessment; analysis and planning;intervention; evidence-based practice, and riskmanagement. Students are rated on a scale of 0–4,on which 0 = ‘infrequently ⁄ rarely demonstrates per-formance indicators’ and 4 = ‘demonstrates mostperformance indicators to an excellent standard’.The benchmark for comparison was a ‘new graduatephysiotherapist’. A category designated ‘N ⁄ A’ indi-cates an element was not assessed. An overallperformance score is calculated by summing scores


Can simulation replace part of clinical time?

for each item and dividing by the number of itemsscored. The tool has been shown to be a valid measureof student competency and to discriminate four levelsof competence.42 In a multicentre national studyusing 30 pairs of examiners, the APP proved to havehigh inter-examiner reliability (intraclass correlationcoefficient [ICC](2,1) = 0.92, 95% confidence interval[CI] 0.84–0.96). The standard error of measurementwas low (3.2 APP points on a scale width of 0–80) andthe minimal detectable change at 90% confidence was7.9 (9% of the scale width).43 The APP is currentlyused by physiotherapy schools in Australia and NewZealand.

Secondary outcome

The secondary outcome related to student reflectionand was measured with a questionnaire developed forthis research. It collected information about students’confidence with patients in relation to three areas ofcompetency concerning: communication; assess-ment, and management. Students responded to 13items pertinent to these three standards. Each itemwas introduced with a statement that began: ‘I feelconfident in my ability to [….]’. Responses were ratedon a 5-point Likert scale, on which 0 = ‘stronglydisagree’ and 5 = ‘strongly agree’. All students (SLEand Traditional groups) completed the questionnaireprior to and on exit from the 4-week clinicalplacement. The SLE groups also completed thequestionnaire immediately after the SLE module. Ineach RCT, the questionnaire was administered tostudents by a research assistant at each university.Once the RCTs commenced, there were no changesto the trial outcome measures.

Procedure

Standardised patient training

A training programme was conducted for the actorswho played the SP roles. It included instruction incommunication and the provision of feedback to thestudent on communication and the development ofrapport from a patient’s perspective. Each SP wasmatched as closely as possible physically and demo-graphically to the patient case he or she was toportray. Each SP was trained using the videotape ofthe real patient–physiotherapist interaction, as well asa script generated from the video. The SPs receivedpersonal instruction from a physiotherapist andsupervised practice in portraying the case. As far as itwas feasible over the 2-year period, the same SPsportrayed the same patients over the two RCTs ateach university.

Assignment and randomisation

All universities participating in the two RCTs con-tributed similar numbers of students to each trial forup to four trial units in each of RCT 1 and RCT 2 (togive a total of eight units per university) (Fig. 1). TheRCTs ran independently but in parallel. It was notfeasible to conduct a single three-arm RCT becausevarious constraints prevented the scheduling of 12students concurrently to accessible musculoskeletalclinical placements. Assignment to either RCT 1 or 2was not randomised for logistical reasons (e.g. SLEclinical educators needed to know in advancewhether they were to work 1 week full-time or 2 weekshalf-time). Randomisation occurred within eachRCT. The RCTs were conducted within the usualclinical settings, including hospital physiotherapy out-patient departments and university clinics and withineach university’s scheduled clinical education pro-gramme for the year. A manager, responsible fortimetabling clinical education placements at eachuniversity, was provided with the list of students ineach year who had volunteered for the study in orderto assign participants to RCTs 1 and 2. The clinicalmanager scheduled sets of eight students (a trialunit) to a musculoskeletal placement at times con-venient within a university’s overall programme. Theunits for each RCT were prescheduled in an arbitraryorder over the 2 years across all universities. Sets ofeight students entered the next scheduled time forone or other RCT.

The randomisation of individual students within eachRCT to the SLE and Traditional groups was con-ducted within each set of eight students at each

Trialn = 184

Volunteer students

Uni 1n = 32

Uni 2n = 32

Uni 3n = 32

Uni 4n = 32

Uni 5n =32

Uni 6n = 24

Stratified on GPA (sets of 8 students) Randomised on an individual basis

SLEn=4

Tradn=4

Trialn = 184

volunteer students

Uni 1n = 32

Uni 2n = 32

Uni 3n = 32

Uni 4n = 32

Uni 5n =32

Uni 6n = 24

Stratified on GPA (sets of 8 students) randomised on an individual basis

SLEn = 4

Tradn = 4

Figure 1 Schema of distribution of trial units between uni-versities (Uni) and the randomisation process used in eachrandomised controlled trial. GPA = grade point average;SLE = simulated learning environment; Trad = traditional


K Watson et al

university. Prior to randomisation, students were firststratified on current academic scores (grade pointaverage [GPA]) to ensure academic equivalencybetween the SLE and Traditional groups. Studentswere first ranked in grade bands. Those with similarGPAs were paired and each student’s name waswritten on a separate piece of paper, which was foldedand placed in respective grade band envelopes by aresearch assistant. A second, independent personthen drew each name from the envelopes. The firststudent to be drawn was allocated to the SLE and thesecond to the Traditional group, and so forth. Eachstudent received a participant number; all dataentered into the central database were de-indentified.Students were informed of their allocation on day 1of the clinical placement.

Clinical education in the SLE

Education with SPs took place in the usual clinicalenvironment at a university or hospital. All studentsparticipated in the assessment and management ofthe nine patient cases. Four cases were treated onthree occasions, four on two occasions, and one onone occasion, giving a total of 21 patient experiencesin the SLE period. Students worked once in a groupof four, eight times in pairs and on 11 occasions asthe sole practitioner. Learning strategies that arepossible only in an SLE were incorporated.44–46 Forinstance, many cases were timelined in order to allowstudents to gain experience with a total rehabilitationprogramme, rather than only for the window of timethey spent in the clinical placement. For example, thepatient presenting for rehabilitation after an anteriorcruciate ligament reconstruction was scripted topresent as if at 2 weeks, 8 weeks and 8 monthspostoperatively. Students were required to design andimplement management programmes suitable forthose time-points in the patient’s rehabilitation.Other learning strategies included ‘time outs’ and‘rewinds’. Time outs could be called by the clinicaleducator (for learning or teaching opportunities),the student (to seek assistance with clinical reasoningor an examination or treatment technique), peers (toprovide guidance or feedback) or the SP (in the eventof discomfort or concern). Rewinds provided stu-dents with the opportunity to practise putting aquestion to a patient in a different way or to repeat anexamination or treatment technique with guidancefrom the clinical educator. At all times, students,educators and SPs were required to act in theirprofessional capacities to ensure the fidelity of theexperience. Students were fully orientated to the SLEprior to commencing the experience. Sessions withSPs were followed by scheduled opportunities to

debrief and reflect on the experience with theeducator.

Ethics considerations

Procedures were put in place to protect studentsagainst any adverse effect on clinical performance orgrading for university purposes that might arise as aresult of their participation in the RCTs. It wasplanned a priori that if outcomes for students in theSLE groups were worse than those of students in theTraditional groups, the study would be terminatedimmediately. Evidence of worse outcomes was de-fined as a consistent reduction by one or more gradeson the APP scale. Results were monitored at eachuniversity over the trial period.

Statistical analysis

The primary endpoint for the RCTs was at week 4.A non-inferiority test for difference in means wasconducted for each trial. The null hypothesis was thatthe difference between the means of the SLE andTraditional groups would be > 0.4 in APP scores,a margin (D) that was regarded as the maximaldifference that was not significant in practice. Thusthe null and alternative hypotheses for each trialwere: null hypothesis: lTrad ) lSLE ‡ D; alternativehypothesis: lTrad ) lSLE < D. These hypotheses weretested by calculating the CIs for the difference inmeans to determine whether the value D was greaterthan the upper bound of the 95% CI.40 Independent-samples t-tests were used to investigate anybetween-group differences in scores on the sevencompetencies within the APP tool in each RCT.

For the secondary outcome, the scales for analysis ofstudent ratings for the three competencies werederived from the combined means of the 13 Likertitems. These scales were checked for reliability (Cron-bach’s a ranged from 0.77 to 0.9). The non-parametrictest suite in SPSS was used to investigate any groupdifferences in students’ confidence with patients overthree (SLE groups) or two (Traditional groups) time-points in each RCT. All analyses were conducted usingSPSS Version 19.0 (SPSS, Inc., Chicago, IL, USA). Thelevel of significance was set at p < 0.05.

RESULTS

The RCTs were completed without interruption inDecember 2010 once the sample size was reached.Figure 2 illustrates the flow of students through theRCTs. Although 184 students were assigned to each



RCT, on one occasion one unit assigned to Model 2 wasmistakenly given Model 1, resulting in 192 students inRCT 1. A replacement group of reserve students wasallocated to RCT 2. Three trial units in RCT 2 com-prised six rather than eight students as a result ofplacement and timetabling difficulties, resulting in 178students in RCT 2. Dropout rates amounted to only 1%in each RCT, although on nine occasions in each ofRCTs 1 and 2, students undertook only one of the twoclinical examinations for reasons of patient non-attendance. No harmful or unintended effects oc-curred in either RCT in any student group.

Primary outcomes

RCT 1: SLE Model 1

The mean APP score for the two clinical examina-tions was 2.73 (standard error [SE] 0.04) in the SLEgroup and 2.68 (SE 0.04) in the Traditional group.Analysis supported the alternative hypothesis. The

95% CI of the difference in means was ) 0.07 to 0.17and the upper bound of this CI is less than themargin of 0.4. The results for the seven key compe-tencies assessed within the APP tool are illustrated inFig. 3(a); analysis revealed no significant differences(at p < 0.05) in scores between the groups.

RCT 2: SLE Model 2

The mean overall APP score was 2.61 (SE 0.05) in theSLE group and 2.58 (SE 0.05) in the Traditionalgroup. Analysis again supported the alternativehypothesis. The 95% CI of the difference in meanswas ) 0.11 to 0.16; the upper bound of this CI wasalso < 0.4. There were no between-group differencesin scores for the seven competencies (Fig. 3b).

Secondary outcome

Table 1 presents SLE students’ self-assessment oftheir confidence with patients based on their ratings

1200 eligiblestudents 2009/10

studentvolunteersn = 410

RCT 1assignment

n = 192*

Randomisedto SLEn = 96

Loss to follow-upn = 1

student illness

Loss to follow-upfor one of two

vivas patient DNAn = 4

Completedexaminations

n = 186

Randomisedto Tradn = 96


student illness




n = 185

RCT 2assignmentn = 178**

Randomisedto SLEn = 89


student illness




n = 171

Randomisedto Tradn = 89


student illness




n = 172

40 volunteerstudents notallocated

370 studentsenrolled in trials

Figure 2 Participant flow and retention for each randomised controlled trial (RCT). SLE = simulated learning environment;Trad = traditional; DNA = did not attend; *An extra group undertook model 1; **3 trial units recruited 6 only of 8 students.


K Watson et al

for three areas of competency (communication,assessment, management) at three time-points(immediately before and after the simulation expe-rience, as well as at the end of the clinical placement)in both RCTs. To test the change in these measuresover time, Friedman’s two-way analysis of variance(ANOVA) by ranks test was used. In both RCTs, SLEstudents showed a significant change in all measuresover time (p < 0.01 in all cases). Inspection of thedata indicated that students in both SLE modelsdisplayed greatest improvement in self-rated confi-dence across the three areas immediately aftercompleting the SLE units and their confidencecontinued to improve over the remainder of theclinical immersion period, albeit at a slower rate.A similar analysis, but this time using Wilcoxon’s

matched-pair signed-rank test, was used for theTraditional groups, in which confidence had beenmeasured at two time-points (the start and end ofplacement). Again, students in both RCTs showedsignificant change in all measures over time(p < 0.01 in all cases), with confidence increasing atthe end of the placement. For both RCTs, the SLEand the matched Traditional groups were comparedusing the Mann–Whitney U-test for two samples. Inonly one case was there a significant differencebetween the two groups (RCT 1, communicationmeasure, at start of placement; p = 0.01). Overall,there was a significant effect of time in students’ self-rated confidence; all groups rated their confidencehigher at the end of the placement than at thebeginning.

Risk management

EBP

Intervention

Analysis and planning

Assessment

Communication

Professional behaviour

SLE studentsTrad students

0 1 2 3 4 0 1 2 3 4

Risk management

EBP

Intervention

Analysis and planning

Assessment

Communication

Professional behaviour

SLE studentsTrad students

Assessment of Physiotherapy Practice score Assessment of Physiotherapy Practice score(a) (b)

Figure 3 Scores (mean ± standard error) on the Assessment of Physiotherapy Practice tool for each domain for the simulatedlearning environment (SLE) and traditional (Trad) student groups in randomised controlled trial (RCT) (a) 1 and (b) 2

Table 1 Self-ratings by students in the simulated learning environment (SLE) and traditional learning (Trad) groups of their confidence withpatients in three domains of competency (communication, assessment, management) at three time-points in the SLE group (Time 1, prior tothe SLE unit; Time 2, immediately after the SLE unit; Time 3, on completion of the clinical placement in randomised controlled trials 1 and 2)and two time-points in the Traditional group (Time 1, at the start of the clinical placement; Time 3, at the end of the clinical placement)

Student group

Communication, mean ± SE Assessment, mean ± SE Management, mean ± SE

Time 1 Time 2 Time 3 Time 1 Time 2 Time 3 Time 1 Time 2 Time 3

SLE (RCT 1) 3.75 ± 0.05 4.24 ± 0.04 4.41 ± 0.04 3.26 ± 0.05 3.90 ± 0.05 4.06 ± 0.05 3.43 ± 0.05 3.90 ± 0.050 4.17 ± 0.05

SLE (RCT 2) 3.80 ± 0.05 4.31 ± 0.42 4.47 ± 0.05 3.24 ± 0.05 3.88 ± 0.05 4.07 ± 0.05 3.39 ± 0.05 3.95 ± 0.048 4.16 ± 0.05

Trad (RCT 1) 3.92 ± 0.05 – 4.42 ± 0.04 3.36 ± 0.05 – 4.07 ± 0.04 3.53 ± 0.57 – 4.14 ± 0.39

Trad (RCT 2) 3.94 ± 0.04 – 4.33 ± 0.05 3.39 ± 0.06 – 3.40 ± 0.05 3.52 ± 0.06 – 4.03 ± 0.05

SE = standard error; RCT = randomised controlled trial



DISCUSSION

Programmes for clinical education in SLEs areincreasing in popularity in several health profes-sions.3,47–50 In some jurisdictions, SLEs are beginningto be used to replace part of clinical time with patientsas a way of increasing the capacity of health systems toprovide clinical training across a range of healthdisciplines.23 Yet the evidence for their value as analternative to traditional real-life clinical practice issparse. The results of these RCTs indicate that clinicaleducation in an SLE can successfully replace a portionof clinical time (25%) with real patients withoutcompromising student learning outcomes. There wasno difference in mean clinical competency scores onthe final clinical examinations between the SLE andTraditional student groups in either RCT.

Previous simulation research has provided evidencefor effective learning of a particular competency orskill.26–33 This research provides evidence that learn-ing in an SLE can contribute to training in all of thekey competencies required of future health profes-sionals for total patient care. Notably, no differenceswere found between the SLE and Traditional groupsin either RCT in any competency (professionalbehaviour, communication, assessment, analysis andplanning, intervention, evidence-based practice, riskmanagement). The RCTs involved physiotherapystudents in a musculoskeletal ambulatory care setting.However, these core competencies are common tothe majority of health professionals and apply acrossvarious fields of practice.51–54 The evidence gainedfrom this research supports the growing trend forusing well-designed SLEs across the spectrum ofmedical, nursing and allied health professional edu-cation. The national multi-university context of theRCTs adds to the robustness of the results.

It is advocated that simulation should be used along-side clinical practice and linked closely with it in orderto allow its full potential as a learning aide to berealised.55,56 We developed and trialled two SLEmodels, in one of which the SLE component wasconducted immediately prior to students’ entry toclinical practice, whereas in the other the SLE expe-rience was interspersed with clinical practice. In thelatter, students spent 50% of the day in each environ-ment for the first 2 weeks of a 4-week placement. Theoutcomes of both RCTs were similar. Both models wereclosely linked to clinical practice and the choice ofmodel may ultimately relate to external factors such asthe timetabling preferences of the university, place-ment sites and convenience for students.

All students (in the Traditional and SLE groups) wereinitially reserved in their confidence about enteringclinical practice across the domains of communica-tion, assessment and management. They rated theiragreement with statements on confidence that began‘I feel confident in my ability to [….]’ as between‘undecided’ and ‘somewhat agree’. This is notunusual.57 Students’ confidence improved signifi-cantly and relatively equally in the SLE and Tradi-tional groups in both RCTs, with all ratings shifting to‘somewhat agree’ and ‘strongly agree’. Students inboth SLE models were also surveyed immediately afterthe SLE experience. The most dramatic improvementin their confidence levels occurred following the SLE,with confidence improving only marginally in theremaining weeks in the clinical environment. Thisimprovement in confidence to interact with patientsfollowing training in a ‘safe’ SLE is consistent withfindings by others.7,58,59 Students in the Traditionalgroups in both RCTs were only surveyed before andon completion of the clinical education placement.Therefore, rates of improvement in confidence can-not be compared between the SLE and Traditionalstudent groups. This limitation should be addressedin future research.

Potential limitations of the RCTs must be recognised.There is evidence that training in isolated skills in anSLE (e.g. communication, technical skills) can besuperior to that in conventional training,26,34 but thiswas not achieved in this study. The period of time inthe SLE (1 week) was relatively short, which may inpart explain why no differences were seen betweenthe Traditional and SLE groups. Alternatively, it isnot known whether the students in the SLE groupswould have gained competencies in the 3-weekclinical immersion alone, without the SLE week. Weare unable to resolve this issue based on these trials.Nevertheless, there is continuing debate about howmany clinical hours are required for students tobecome competent in a field of practice.60 Thelength of each clinical placement for specified fieldsof practice in physiotherapy in Australian universitieshas ranged from 4 weeks to 6 weeks.42 Currentpractice favours 5-week placements,43 a decisionbased on a consensus by clinical educators on thetime required by most students to become compe-tent. Therefore, we suggest that the 1-week period inthe SLE, which constituted 25% of the total clinicalplacement, positively contributed to students’achievement of key professional competencies.Building on these findings, future research mightexplore whether superior outcomes can be achievedby allotting more time to the SLE within a clinical


K Watson et al

placement. Research might then progress to deter-mine exactly how much learning can occur in an SLEto make a more significant contribution to increasingclinical capacity.

The fact that students were volunteers introduces arisk for selection bias. The GPA distributions ofstudents who did and did not volunteer at oneuniversity were analysed. No difference was evidentbetween the two cohorts (t(93) = 1.43, p = 0.16), whichgives us some confidence that our student cohortrepresented the whole student group. The APP toolused to assess clinical competencies has shown validityand reliability and can discriminate different levels ofcompetence.42,43 The use of clinical vivas in RCTsdemonstrates a student’s clinical competency at aparticular time-point and may not be sensitive enoughto provide a thorough investigation into the skillsacquired. Nevertheless, these conditions were consis-tent for both the SLE and Traditional groups andshould not compromise the primary findings. Thequestionnaire used for the secondary outcome ofstudent self-reflection on growth in competence wasdeveloped for this study by educators with expertise inthe relevant constructs of interest. We recognise that itmay contain potential sources of measurement error,but it will serve to inform further work.

The use of simulations is not without challenges in thefield of musculoskeletal physiotherapy. Standardisedpatients have limitations, notably in their physicalpresentations. The SP can be matched demographi-cally with the ‘real’ patient and trained to portray someaspects of the case well, but there are some physicalfeatures that SPs cannot simulate. This might beimproved by the use of hybrid simulations such as areemployed in other fields of practice.61,62 Currently,there are no commercially available hybrid simulationssuitable for training in the physical examination andmanagement of musculoskeletal disorders, other thanlow-fidelity static anatomical models that have no realvalue in this context. Future development of medium-and high-fidelity part-task trainers would undoubtedlyenrich the SLE experience in physiotherapy educa-tion. The costs of implementing an SLE need to betaken into account. A cost-effectiveness analysis is inprogress for this research.

CONCLUSIONS

Simulation will never totally replace real-patientlearning experiences in the clinical environment.However, the results of these novel RCTs provideinitial evidence that clinical education in an SLE can

replace 25% of clinical time with real patients withoutcompromising student learning outcomes, at least inthe field of musculoskeletal physiotherapy. Learningin an SLE need not be limited to a particularcompetency or skill, but can contribute successfully tostudents’ attainment of all key competencies requiredof future health professionals.

Contributors: KW and GJ made substantial contributions tothe conception and design of the study, the acquisition,analysis and interpretation of data, and the drafting of thearticle. AW, NM, JM, DR and AJ made substantialcontributions to the conception and design of the study,and the acquisition of data. FB, VO’C and RP madesubstantial contributions to the conception and design ofthe study. TH made a substantial contribution to theconception and design of the study, and the analysis andinterpretation of data. GW made a substantial contributionto analysis and interpretation of data and the drafting ofthe article and revising it critically for important intellectualcontent. All authors contributed to the critical revision ofthe article and approved the final manuscript forpublication.Acknowledgements: we wish to acknowledge thecontribution of the following Research Assistants: LucyThomas, Liz Paull, Debra Hedlefs, Penny Moss, Airi Repetti,Anneliese Synnot.Funding: funding was received from an AustralianResearch Council Linkage Grant (LP0776270). Partnershipfunding in this ARC Linkage Grant was received from:Physiotherapists Board of Queensland, AustralianPhysiotherapy Council, and Australian PhysiotherapyAssociation. In kind support was provided by: QueenslandHealth Clinical Skills Development Centre, QueenslandHealth, Department of Health Western Australia,Department of Health NSW, and Department of HumanServices Victoria.Conflicts of interest: none.

Ethical approval: this study was approved by the Universityof Queensland Medical Research Ethics Committee(MREC) (2007000750), the University of Melbourne MREC(0931120.1), the University of Newcastle Human ResearchEthics Committee (HREC) (H-2008-0166), CurtinUniversity HREC (HR39 ⁄ 2007), Griffith University HREC(PES ⁄ 06 ⁄ 08), James Cook University Human Ethics Sub-Committee (55 ⁄ 08), La Trobe University HREC (08 ⁄ 039),Alfred Hospital Ethics Committee (182 ⁄ 08), Austin HealthHREC (H2008 ⁄ 034250), Cairns Base Hospital EthicsCommittee (#518), Eastern Health HREC (E55 ⁄ 0809),Gold Coast Health Service District HREC (200858), HunterNew England HREC (09 ⁄ 04 ⁄ 15 ⁄ 12.01), Melbourne Health(2008.246), Princess Alexandra Hospital HREC (EC00167,2008 ⁄ 173), Royal Brisbane and Women’s Hospital HREC(EC00172, 2008 ⁄ 058), Sir Charles Gardner Hospital HREC(2009-102), St Vincent Hospital HREC (LRR 163 ⁄ 08) andthe Townsville Health Service District Institutional EthicsCommittee (55 ⁄ 08).



REFERENCES

1 Barrows H, Abrahamson S. The programmed patient: atechnique for appraising student performance in clin-ical neurology. J Med Educ 1964;39:802–5.

2 Bradley P. The history of simulation in medical edu-cation and possible future directions. Med Educ2006;40:254–62.

3 Gaba DM. The future vision of simulation in healthcare. Qual Saf Health Care 2004;13 (Suppl 1):2–10.

4 Scalese R, Obeso V, Issenberg S. Simulation technologyfor skills training and competency assessment in med-ical education. J Gen Intern Med 2008;23:46–9.

5 Okuda Y, Bond W, Bonfante G, McLaughlin S, SpillaneL, Wang E, Vozenilek J, Gordon J. National growth insimulation training within emergency medicine resi-dency programmes, 2003–2008. Acad Emerg Med2008;15:1113–6.

6 Berkenstadt H, Ziv A, Barsuk D, Levine I, Cohen A,Vardi A. The use of advanced simulation in the trainingof anaesthesiologists to treat chemical warfare casual-ties. Anesth Analg 2003;96:1739–42.

7 Gordon JA, Wilkerson WM, Shaffer DW, ArmstrongEG. ‘Practising’ medicine without risk: students’ andeducators’ responses to high-fidelity patient simulation.Acad Med 2001;76:469–72.

8 Hodder RV, Rivington RN, Calcutt LE, Hart IR. Theeffectiveness of immediate feedback during the objec-tive structured clinical examination. Med Educ1989;23:184–8.

9 Chikwe J, De Souza AC, Pepper JR. No time to train thesurgeons. BMJ 2004;328:418–9.

10 Kohn LT, Corrigan J, Donaldson MS. To Err is Human:Building a Safer Health System. Washington, DC: Instituteof Medicine, National Academy Press 1999.

11 Williams S, Dale J, Glucksman E, Wellesley A. Seniorhouse officers’ work-related stressors, psychologicaldistress, and confidence in performing clinical tasks inaccident and emergency: a questionnaire study. BMJ1997;314:713–8.

12 Ziv A, Wolpe P, Small S, Glick S. Simulation-basedmedical education: an ethical imperative. Acad Med2003;78:783–8.

13 Devey L. Will modernised medical careers produce abetter surgeon? BMJ 2005;331:1346.

14 Olson L, Hill S, Newby D. Barriers to student access topatients in a group of teaching hospitals. Med J Aust2005;183:461–3.

15 Payne SR, Shaw MK. What impact will shortenedtraining have on urological service delivery? Ann R CollSurg Engl 2005;87 (Suppl):373–8.

16 van der Weyden MB. The training tsunami. Med J Aust2008;189:353.

17 Young A, Redfern N, Sher L. Counting the cost ofsurgical training: perspective from a school of surgery.Ann R Coll Surg Engl 2010;92 (Suppl):46–50.

18 Hassam M, Williams M. Education via simulation:teaching safe chest percussion for preterm infants. HKPhysiother J 2003;21:22–8.

19 Hill A, Davidson B, Theodoros DG. A review of stan-dardised patients in clinical education: implications forspeech–language pathology programmes. Int J SpeechLang Pathol 2010;12:259–70.

20 Kneebone R. Simulation, safety and surgery. Qual SafHealth Care 2010;19 (Suppl 3):47–52.

21 Lindstrom-Hazel D, West-Frasier J. Preparing studentsto hit the ground running with problem-based learningstandardised simulations. Am J Occup Ther 2004;58:236–9.

22 Valler-Jones T, Meechan R, Jones H. Simulatedpractice – a panacea for health education? Br J Nurs2011;20:628–31.

23 Health Workforce Australia. Simulated learning envi-ronments (SLEs). 2012. http://www.hwa.gov.au/work-programs/clinical-training-reform/simulated-learning-environments-sles. [Accessed 20 February 2012.]

24 Harder B. Use of simulation in teaching and learningin health sciences: a systematic review. J Nurs Educ2010;49:23–8.

25 Murray E, Gruppen L, Catton P, Hays R, WoolliscroftJO. The accountability of clinical education: its defini-tion and assessment. Med Educ 2000;34:871–9.

26 Abrahamson S, Denson JS, Wolf RM. Effectiveness of asimulator in training anaesthesiology residents. QualSaf Health Care 2004;13:395–7.

27 Banks E, Chudnoff S, Karmin I, Wang C, Pardanani S.Does a surgical simulator improve resident operativeperformance of laparoscopic tubal ligation? Am J ObstetGynecol 2007;197:541.e1–e5.

28 Burchard E, Lockrow EG, Zahn CM, Dunlow SG, SatinAJ. Simulation training improves resident performancein operative hysteroscopic resection techniques. Am JObstet Gynecol 2007;197:542.e1–e4.

29 Fried MP, Satava R, Weghorst S et al. Identifying andreducing errors with surgical simulation. Qual SafHealth Care 2004;13 (Suppl 1):19–26.

30 Gaba DM, Howard SK, Fish KJ, Smith BE, Sowb YA.Simulation-based training in anaesthesia crisis resourcemanagement (ACRM): a decade of experience. SimulGaming 2001;32:175–93.

31 Good ML. Patient simulation for training basic andadvanced clinical skills. Med Educ 2003;37:14–21.

32 Hall RE, Plant JR, Bands CJ, Wall AR, Kang J, Hall CA.Human patient simulation is effective for teachingparamedic students endotracheal intubation. AcadEmerg Med 2005;12:850–5.

33 McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH,Wayne D. Does simulation-based medical educationwith deliberate practice yield better results than tradi-tional clinical education? A meta-analytic comparativereview of the evidence. Acad Med 2011;86:706–11.

34 Colletti L, Gruppen L, Barclay M, Stern D. Teachingstudents to break bad news. Am J Surg 2001;182:20–3.

35 Elman D, Hooks R, Tabak D, Regehr G, Freeman R.The effectiveness of unannounced standardised pa-tients in the clinical setting as a teaching intervention.Med Educ 2004;38:969–73.


K Watson et al

36 Barrows HS. Simulated (Standardized) Patients andOther Human Simulations: A Comprehensive Guide toTheir Training and Use in Teaching and Evaluation.Chapel Hill, NC: Health Sciences Consortium 1987;1–9.

37 Voelker R. Medical simulation gets real. JAMA2009;302:2190–2.

38 Colliver J. Educational theory and medical educationpractice: a cautionary note for medical school faculty.Acad Med 2002;77:1217–20.

39 Kirkpatrick D, Kirkpatrick J. Evaluating Training Pro-grams, 3rd edn. San Francisco, CA: Berrett-Koehler2006;21–6.

40 Rothmann M. Design and Analysis of Non-Inferiority Trials.Hoboken, NJ: Chapman & Hall ⁄ CRC Press 2011;401–5.

41 Dalton M, Keating J, Davidson M. Development of theAPP (Assessment of Physiotherapy Practice) Instru-ment: A Standardised and Valid Approach to Assess-ment of Clinical Competence in Physiotherapy.Brisbane, Qld: Griffith University 2009. http://www.olt.gov.au/resources?text=assessment+of+physiotherapy+practice. [Accessed 25 April 2012.]

42 Dalton M, Davidson M, Keating J. The assessment ofphysiotherapy practice (APP) is a valid measure ofprofessional competence of physiotherapy students: across-sectional study with Rasch analysis. J Physiother2011;57:239–46.

43 Dalton M, Davidson M, Keating J. The assessment ofphysiotherapy practice (APP) is a reliable measure ofprofessional competence of physiotherapy students: areliability study. J Physiother 2012;58:49–56.

44 Issenberg SB, McGaghie WC, Hart IR et al. Simulationtechnology for health care professional skills trainingand assessment. JAMA 1999;282:861–6.

45 Maran NJ, Glavin RJ. Low- to high-fidelity simulation –a continuum of medical education? Med Educ2003;37:22–8.

46 Collins JP, Harden RM. AMEE medical educationguide no. 13: real patients, simulated patients andsimulators in clinical examinations. Med Teach1998;20:508–21.

47 Gropper R, Harnett N, Parker K, Pearce S, MacIver D,Murray L, Ramsay L, Ripley A, Sands B, Zychla L. Thepath to simulated learning: developing a valid andreliable tool to evaluate performance of radiologicaltechnology students in patient interactions. J AlliedHealth 2010;39:28–33.

48 McLaughlin SA, Doezema D, Sklar DP. Human simu-lation in emergency medicine training: a model cur-riculum. Acad Emerg Med 2002;9:1310–8.

49 Nehring WM. US Boards of Nursing and the use ofhigh-fidelity patient simulators in nursing education.J Prof Nurs 2008;24:109–17.

50 Zraick R, Allen R, Johnson S. The use of standar-dised patients to teach and test interpersonal and

communication skills with students in speech–language pathology. Adv Health Sci Educ 2003;8:237–48.

51 Australian Physiotherapy Council. Australian Standardsfor Physiotherapy. Canberra, ACT: Australia Physiother-apy Council 2006. http://www.physiocouncil.com.au/standards. [Accessed 1 December 2011.]

52 Gibson T, Heartfield M. Competency standards fornurses in general practice: glossary. In: An AustralianNursing Federation Project Funded by the Australian Gov-ernment. Melbourne, Vic: Department of Health andAging 2011. http://www.anf.org.au/nurses_gp/[Accessed 25 April 2012.]

53 Hill L, Delafuente MS, Sicat B, Kirkwood CK. Devel-opment of a competency-based assessment process foradvanced pharmacy practice experiences. Am J PharmEduc 2006;70:1.

54 Taradejna C. ACGME Outcome Project. AccreditationCouncil for Graduate Medical Education 2007. http://www.acgme.org/acWebsite/irc/irc_compIntro.asp.[Accessed 1 December 2011.]

55 Kneebone RL, Scott W, Darzi A, Horrocks M. Simula-tion and clinical practice: strengthening the relation-ship. Med Educ 2004;38:1095–102.

56 McGaghie WC, Siddall VJ, Mazmanian PE, Myers J.Lessons for continuing medical education from simu-lation research in undergraduate and graduate medicaleducation. Chest 2009;135 (3 Suppl):62–8.

57 Cupit R. Student stress: an approach to coping at theinterface between pre-clinical and clinical education.Aust J Physiother 1988;34:215–9.

58 Bremner M, Aduddell K, Bennett DN, VanGeest J.The use of human patient simulators: best practiceswith novice nursing students. Nurse Educ 2006;31:170–4.

59 Corbridge S, McLaughlin R, Tiffen J, Wade L, TemplinR, Thomas C. Using simulation to enhance knowledgeand confidence. Nurse Pract 2008;33:12–3.

60 Crosbie J, Gass E, Jull G et al. Sustainable undergrad-uate education and professional competency. Aust JPhysiother 2002;46:5–7.

61 Crofts JF, Bartlett C, Ellis D, Winter C, Donald F, HuntLP, Draycott T. Patient-actor perception of care: acomparison of obstetric emergency training usingmanikins and patient-actors. Qual Saf Health Care2008;17:20–4.

62 Siassakos D, Draycott T, O’Brien K, Kenyon C, BartlettC, Fox R. Exploratory randomised controlled trial ofhybrid obstetric simulation training for undergraduatestudents. Simul Healthc 2010;5:193–8.

Received 19 December 2011; editorial comments to authors25 January 2012, 14 March 2012; accepted for publication21 March 2012



can simulation replace part of clinical time? two parallel randomised controlled trials

Documents