JOURNAL OF ENDOUROLOGYVolume 20, Number 4, April 2006© Mary Ann Liebert, Inc.

Comparison of Results of Virtual-Reality Simulator andTraining Model for Basic Ureteroscopy Training

DAVID S. CHOU, COROLLOS ABDELSHEHID, RALPH V. CLAYMAN, M.D., and ELSPETH M. MCDOUGALL, M.D., FRCSC

ABSTRACT

Background and Purpose: The traditional method of acquiring surgical skills is by apprenticeship and involvesan extensive period of training with patients. Model-based and virtual reality simulation is gaining interestas alternative training, allowing repetitive practice in a low-risk environment. The objective of this study wasto determine if a materials, model-based training format and an interactive virtual-reality simulator couldprovide equivalent teaching of basic ureteroscopy skills to the inexperienced medical student.

Subjects and Methods: Sixteen first-year medical students received the same didactic session and video view-ing on cystoscopy, guidewire access to the upper urinary tract, and ureteroscopy with intracorporeal laserlithotripsy and stone extraction by the same instructor. The participants were then randomized into two studygroups: Group 1 was trained on the ureteroscopy training model (TMU) from Limbs & Things and Group 2on the Simbionix UROMentor virtual-reality simulator (VRS) until the participants could perform the pro-cedure independently. Two months later, the participants independently performed a ureteroscopic proce-dure on a pig kidney/ureter model and were graded from 1 to 5 on their ability to complete the steps of theprocedure and the quality of their performance (handling of tissue, efficiency, instrument handling, knowl-edge of instruments, flow of operation, use of assistants, and knowledge of the specific procedure) for a pos-sible total of 35 points.

Results: All participants were able to perform the steps of the procedure correctly. The TMU group andthe VRS group received a mean of 22.9 � 4.8 and 23.6 � 5.4 points, respectively (P � 0.38) for their qualita-tive assessment.

Conclusion: The medical students’ skills and ability to perform a basic ureteroscopic stone-managementprocedure was independent of the training modality (VRS or TMU). Incorporating either of these devices intothe preliminary training of urology residents may improve their initial clinical performance of these skills.

INTRODUCTION

THE TRADITIONAL METHOD of acquiring surgical skillsis by the Halstedian apprenticeship and involves an exten-

sive period of hands-on-training with patients. Initially, theseprocedures are executed under close supervision by an expertclinician. Even so, there are certain inherent risks for the pa-tient, especially during the initial stages of the learning curve,associated with the more complex or challenging skills requiredfor minimally invasive surgery. In an effort to address this as-pect of learning for minimally invasive surgery techniques suchas endourology and laparoscopy, alternative methods of train-

ing have been developed. While the animal or cadaver labora-tories can provide comparable and realistic learning environ-ments, both are associated with high cost, require highly skilledsupport personnel, and usually provide only a one-time learn-ing experience. The teaching of the highly technical skills re-quired for minimally invasive surgery has recently generatedconsiderable debate. More extensive clinical experience withthese challenging procedures has been associated with reducedcomplications and better clinical outcomes.1–4

Minimally invasive surgery is usually performed in a videoenvironment with fixed instrument-access sites and with a fi-nite number of possible approaches and techniques that can be

Department of Urology, University of California, Irvine, Orange, California.

266

recreated realistically with bench models and simulators. Thisis in direct contrast to open surgery, which has a large numberof approaches and movements that can be utilized and evalu-ated. The training models and simulators offer trainees the op-portunity to practice and fine tune their surgical skills repeti-tively prior to performing the technique on an actual patient.Thus, there is growing interest in supplemental training mo-dalities such as the bench training model and the virtual-real-ity simulator (VRS).

The VRS involves complex electronics and computerized hu-man interfaces that seek to mimic an authentic surgical experi-ence visually. This requires expensive computer graphics andcostly equipment to provide both the visual and the tactile feed-back for the trainee. Several investigators have demonstratedthat the VRS can teach basic ureteroscopy skills effectively inthe mentor-directed environment.5,6 However, this form ofteaching is limited because of the simulator cost of between$60,000 and $85,000. The advantages of the VRS include theability to provide computer-generated basic instructions, im-mediate virtual instructor feedback, and a variety of clinicallyrelevant practice cases.

The available ureteroscopy training models, designed toteach the same surgical skills as the ureteroscopy VRS, are sig-nificantly less expensive, with a cost ranging from $3000 to$5000. Furthermore, there is some evidence that the lower-fi-delity “home-made” training models may be just as effective asthe more expensive VRS.7 The objective of this study was tocompare the ureteroscopy training model and the ureteroscopyVRS as training devices for first-year medical students who hadno previous exposure to endourology.

SUBJECTS AND METHODS

Following approval by the University of California Irvine(UCI) Institution Review Board, a recruitment e-mail for studyvolunteers was sent to all 100 first-year medical students at UCIMedical School. Sixteen students responded and were acceptedinto the study. None of the participants had prior endourologyor VRS experience. Fifteen of the subjects were male; the sin-gle female participant was randomized to the TMU group.

The students were paid $100 for completing the entire studyprotocol.

A survey was administered to each of the participating stu-dents regarding their experience with endourology, the VRS,and the ureteroscopy training model. These participants werethen randomized to one of two training modalities. Group 1trained on the URO-Scopic™ trainer (Limbs & Things, Bris-tol, UK), which is a static materials model for ureteroscopy us-ing the Storz flexible cystoscope and the Storz 7.5F flexibleureteroscope, Cook Urological guidewires and catheters, andthe Convergent holmium laser device with the 200-�m probe(TMU) (Fig. 1A). Group 2 trained on the UROMentor™ (Sim-bionix USA, Cleveland, OH), which is a dynamic, interactiveVRS for ureteroscopy (Fig. 1B). There was no difference in theage of the two groups (group A 23.4 � 1.7 years v group B23.9 � 2.2 years).

Both study groups received the same initial explanatory lec-ture on the medical indications for ureteroscopic managementof stones, access to the upper urinary-collecting system, and the

technique of flexible ureteroscopy and possible complicationsof the ureteroscopic management of stone disease. They ob-served a videotape of a ureteroscopy procedure for a midureteralcalculus utilizing holmium laser lithotripsy and basket extrac-tion of the stone. The participants then attended a training ses-sion for the specific device to which they had been random-ized. All participants were allowed to train on their respectivedevice for as long as 2 hours under close supervision of theminimally invasive surgery education fellow (DSC). The URO-Mentor group trained on the Practice Hall Level 1 and 2 train-ing games and then practiced on Tasks 3 and 4. The URO-Mentor group practiced with the Storz flexible cystoscope andureteroscope. Both groups were trained in basic flexible cysto-

VIRTUAL REALITY SIMULAR VERSUS TRAINING MODEL 267

A

B

FIG. 1. Models used for training. (A) URO-Scopic™ modeltrainer from Limbs & Things being used with Storz Endoscopyflexible cystoscope (www.limbsandthings.com). (B) URO-Mentor™ virtual-reality ureteroscopy simulator (www.sim-bionix.com).

scope and ureteroscope handling skills, intubation of theureteral orifice with a 0.035-inch floppy-tip guidewire and ac-cess to the upper collecting system, insertion of the flexibleureteroscope into the ureter, identification of the landmarks ofthe ureter and the ureteral calculus, use of the holmium laserfor lithotripsy, and use of a 1.9F basket for extraction of thefragments.

Two months after the initial training period, all of the par-ticipants returned and were evaluated on a midureteral-stoneprocedure in a porcine kidney model developed at UCI. Themodel consists of a round Tupperware container as the “blad-der” with three holes drilled into the sides (Fig. 2). Rubber tub-ing was cemented to one hole to act as the urethra, and intra-venous tubing was cemented to the other two holes to provide

access to the ureters. The intact pig kidney/ureter was connectedto the model by suturing the ureter to the intravenous tubing.The stones were created using plaster of Paris (DAP Products,Baltimore, MD) and ranged in size from 0.5 to 1 cm � 0.5–1cm � 0.2 to 0.5 cm. The stone was inserted into the mid to up-per level of the ureter by first passing a guidewire retrograde,with the rigid end first, up the ureter and through the kidney.The wire was retrieved and used to pass a 10/12F ureteral ac-cess sheath antegrade to the midportion of the ureter. The stonewas then passed through the access sheath and positioned at theappropriate level of the ureter. The access sheath was removed,and the nephrostomy site was sutured closed.

The Storz flexible cystoscope and ureteroscope were utilizedfor all the procedures, and the Convergent holmium laserlithotripter with a 200-�m laser probe was used for lithotripsy.The participants were evaluated by a single trained observer(DSC) during their performance of the flexible cystoscopy, ac-cess to the upper collecting system, flexible ureteroscopy,holmium laser lithotripsy, and stone extraction in this model.They were given no direction or assistance by the observer dur-ing the evaluation session. When the Tupperware containerbladder became full, it could be drained easily by removing thelid and dumping out the contents.

Using an objective structured assessment of technical skills(OSATS) form (Table 1), participants were scored on theirability to perform cystoscopy, access the ureter with aguidewire, insert the flexible ureteroscope, identify the ureteralstone, perform laser lithotripsy, basket extract the stone, andremove the ureteroscope. One point was awarded for each ofthese six steps as they were completed. In addition, the par-ticipants were evaluated on the quality of their performance,scored from 1 to 5, for their handling of tissue, efficiency, in-strument handling, knowledge of instruments, flow of the op-eration, use of assistants, and knowledge of the specific pro-cedure (Table 2). The qualitative assessment had a potentialmaximum score of 35 points. The results were analyzed usingindependent-samples t-tests (SPSS 12.0 for Windows; SPSS,Chicago, IL).

RESULTS

During the evaluation session, all participants performed allsix steps of the procedure correctly, and each received six points(Table 3). In the qualitative assessment, out of a potential max-

CHOU ET AL.268

FIG. 2. Porcine kidney model used for testing both groupson basic ureteroscopy skills.

TABLE 1. OBJECTIVE STRUCTURED ASSESSMENT OF TECHNICAL SKILLS (OSATS) FOR

URETEROSCOPY AND STONE MANAGEMENT IN PORCINE MODEL

QualityNot done Done (see attached

or incorrect correctly guide)

Flexible cystoscopy 0 1Guidewire access to upper collecting system 0 1Insertion of flexible ureteroscope 0 1Identification of ureteral stone 0 1Completion of holmium laser lithotripsy of stone 0 1Basket extraction of stone fragments and removal 0 1

of ureteroscopeTotal scores

imum score of 35 points, the VRS group achieved a mean scoreof 23.6 � 5.4 and the participants in the TMU group a mean of22.9 � 4.8 (P � 0.38). None of the participants achieved aquality score of 90% or more. Three of the VRS group achieveda quality score of 80% or more, and two in the TMU group

achieved this level of quality. Only one participant in eachgroup scored �50% (�18) on the test ureteroscopy procedure.A qualitative assessment score �70% (�25/35) was observedin four participants who trained on the VRS and in five whotrained on the TMU.

VIRTUAL REALITY SIMULAR VERSUS TRAINING MODEL 269

Respect fortissue

Time & motion

Instrumenthandling

Knowledge ofinstruments

Flow ofoperation

Use of assistants

Knowledge ofspecificprocedure

Frequently usedunnecessary

force or causeddamage by

inappropriateinstruments

Manyunnecessary

moves

Repeatedtentative or

awkward movesor inappropriate

use ofinstruments

Frequently askedfor wrong

instruments orused

inappropriateinstruments

Frequentlystopped

operating &seemed unsure

Poorly placed orfailed to use

assistants

Needed specificinstruction at all

steps

Occasionalunnecessary

force used orinappropriate

instrument used

Occasionalunnecessary

moves

Occasionaltentative or

awkward movesor inappropriate

use ofinstruments

Occasionallyasked for wrong

instrument ornot used

appropriately

Occasionallystopped

operating &seemed unsure

Failed to useassistants half

the time

Needed specificinstruction at

most steps

Careful handlingof tissues butoccasionally

causedinadvertent

tissue damage

Efficienttime/motion but

someunnecessary

moves

Competent useof instrumentsbut occasional

stiff or awkwardmoves

Knew names ofmost instruments

& usedappropriateinstruments

Had forwardplanning with

reasonableprogression

Used assistantswell most of the

time

Knew steps butneeded several

instructions

Careful handlingof tissues but on

one occasioncaused

inadvertenttissue damage

Efficienttime/motion butone unnecessary

move

Used appropriateinstruments but

made oneawkward move

Knew names ofthe instrumentsbut used one

inappropriately

Had forwardplanning with

only one unsureepisode

Only once failedto use assistants

Knew importantsteps but neededone instruction

Consistentlyhandled tissuesappropriatelywith minimaltissue damage

Clear economyof movement &

maximumefficiency

Fluid moveswith instruments

and noawkwardness

Obviouslyfamiliar with

instruments andtheir names

Well plannedoperation witheffortless flow

of moves

Used assistantsto best

advantage all thetime

Familiar with allaspects ofoperation

TABLE 2. QUALITY ASSESSMENT SCORE OF OPERATIVE TECHNIQUE FOR URETERORENOSCOPY

Circle number in each row that is appropriate for the corresponding technique

1 2 3 4 5

Add circled numbers for all rows for TOTAL: (max score 7 � 5 � 35)

TABLE 3. OBJECTIVE STRUCTURED ASSESSMENT OF TECHNICAL SKILLS

(OSATS) RESULTS FOR URETEROSCOPY PROCEDURE

VRS-trained TMU-trainedgroup 1 group 2 P value

Steps of procedure completed 6 6Mean OSATS Quality 23.6 � 5.4 22.9 � 4.8 0.38Evaluation (range) (16–31) (16–30)

DISCUSSION

There is increasing interest in training surgeons in a mannersimilar to the training of airline pilots, to wit, repetitive practiceon a validated model with the exploration of all possible out-comes in a risk-free environment. Intuitively, this approachwould seem to shorten the difficult learning curve of many chal-lenging minimally invasive surgery procedures. As such, model-based and virtual-reality-based simulations are being explored astechniques to incorporate into surgical training programs.

As the validity of the ureteroscopic VRS and the TMU areconfirmed, their utilization in endourologic training will havean ever-increasing role.5–9 Experienced urologists who haveperformed ureteroscopic procedures on a TMU have consideredthis training format to be equivalent to a real-patient environ-ment.9 Our results indicate that the VRS and TMU modalitiesare equally effective in teaching basic ureteroscopy skills. Al-though the ureteroscopic VRS utilizes a cartoon-like image andinstrumentation that is similar, but not identical, to the actualoperating-room devices, this did not impact the student’s abil-ity to perform the evaluation ureteroscopy procedure with theclinical instruments.

There remain individual differences in the level of skill acqui-sition despite the training modality. Whereas the models provideda similar training experience for both groups, it should be notedthat half of the trainees still performed at only a 70% quality level.Therefore, some individuals may require more training time oradditional expert supervision during training before reaching acompetency level acceptable for the clinical environment.

It must be realized that in this study, both the ureteroscopicVRS and the TMU groups were closely guided and supervisedby an expert endourologic surgeon during their initial training.This is important, even with the ureteroscopic VRS, to ensurethat the completely inexperienced trainee understands the con-cepts and acquires the correct clinical skills. Practice without abasic grounding in the principles and practices of surgical skillsmay only foster the learning of poor technique or acquisition ofincorrect skills. The educator remains a key element of the train-ing process, initially as an instructor and later in developing judg-ment and strengthening the knowledge and interpretation of whatis observed in order to create a truly competent surgeon.

The final evaluation revealed that the two groups acquired sim-ilar knowledge and skills to enable good performance of theureteroscopic stone-management procedure. However, the mainstrength of the VRS is its versatility in providing a large numberof training and clinical scenarios, along with its ability to providesimultaneous tutoring and feedback without human mentoring.This is particularly useful for the surgeon in training, as it allowsrepetitive practice of techniques and skills while simultaneouslyexploring possible risks and complications. The virtual instructorof the computer program provides the trainee with immediatefeedback and evaluation of his or her performance without theneed for participation of an expert clinician. However, this aspectof ongoing learning was not assessed in the present study.

The material, non-interactive ureteroscopy training modelssuch as the TMU are less contrived or artificial than the vir-tual-reality trainers in that the visual and tactile feedback arenot simulated by a computer. Instead, the trainee works withthe actual instruments used for patient care within an artificialscaffold that mimics the in-vivo situation. Hence, although less

complicated with regard to design, the ureteroscopy trainingmodel requires the availability of all ancillary equipment in ad-dition to the endoscopes, such as the laser device, the guidewiresand catheters, and the stone basket and grasping forceps. In ad-dition, the expert instructor must be in attendance during theentire learning experience.

Although there is an obvious advantage in the reduced pur-chase cost of the TMU ($3135.80) compared with the uretero-scopic VRS ($80,000), its use for training requires an actualflexible cystoscope ($7500), guidewires and catheters ($300),flexible ureteroscope ($17,500), holmium laser lithotripsy unitwith fibers ($55,000 for the generator; $50 for each fiber), andstone basket extractors ($200), the total of which is $80,500.Also, all of the equipment is rather fragile, and one must ac-count for the need to replace the laser fiber and guidewires pe-riodically, as well as the all-too-short lifespan of the flexibleureteroscope.10 The cost of repairing the flexible cystoscopeand ureteroscope may range from $1800 to $4500. In addition,an expert clinician must be in attendance throughout the train-ing to provide feedback guidance and assessment of thetrainee’s performance, whereas the ideal VRS would have acomprehensive curriculum and built-in educational program toprovide a virtual instructor and minimize the time an expertclinician is required. Thus, the required equipment and the timeof the expert surgeon must be factored into the overall cost ofthe TMU as a long-term training format. If an instructor’s timeis estimated to be $100 to $200 per hour and the average in-struction time is 2 hours, then after training 200 students, thecost ($400 per student) would equal the $80,000 purchase priceof the VRS machine. The sum total of these expenses wouldappear to more than justify the expense of continued develop-ment of the VRS machines over the TMU-type trainers forteaching minimally invasive surgical techniques. As such, theVRS, with no recurring costs and minimal instructor input, al-though more expensive to purchase, would appear to cost lessonce training has been achieved.

CONCLUSION

The TMU and the ureteroscopic VRS are equally effectivefor teaching basic ureteroscopy skills to inexperienced trainees.Dedicated teaching by the expert surgeon is a key element inthe success of the educational process despite the obvious dif-ference in the cost of the training devices. Incorporation of thesemodels and simulators into formalized urologic training pro-grams is now needed in order to test their effectiveness as aneducational format and to determine their ultimate value by as-sessing their concurrent and predictive validity.

REFERENCES

1. Perugini RA, Mason R, Czerniach DR, Novitsky YW, Baker S.Litwin DE, Kelly JJ. Predictors of complication and suboptimalweight loss after laparoscopic Roux-en-Y gastric bypass: A seriesof 188 patients. Arch Surg 2003;138:541–545.

2. Nguyen NT, Rivers R, Wolfe BM. Factors associated with opera-tive outcomes in laparoscopic gastric bypass. J Am Coll Surg2003;197:548–555.

CHOU ET AL.270

3. Sokol AI, Chuang K, Milad MP. Risk factors for conversion to lap-arotomy during gynecologic laparoscopy. J Am Assoc Gynecol Laparosc 2003;10:469–473.

4. Oliak D, Owens M, Schmidt HJ. Impact of fellowship training onthe learning curve for laparoscopic gastric bypass. Obes Surg2004;14:197–200.

5. Watterson JD, Beiko DT, Kuan JK, Denstedt JD. Randomizedprospective blinded study validating acquisition of ureteroscopyskills using computer based virtual reality endourological simula-tor. J Urol 2002;168:1928–1932.

6. Ogan K, Jacomides L, Shulman MJ, Roehrborn CG, Cadeddu JA,Pearle MS. Virtual ureteroscopy predicts ureteroscopic proficiencyof medical students on a cadaver. J Urol 2004;172:667–671.

7. Matsumoto ED, Hamstra SJ, Radomski SB, Cusimano MD. Theeffect of bench model fidelity on endourological skills: A ran-domized controlled study. J Urol 2002;167:1243–1247.

8. Brehmer M, Tolley D. Validation of a bench model for endo-scopic surgery in the upper urinary tract. Eur Urol 2002;42:175–179.

9. Matsumoto ED, Hamstra SJ, Radomski SB, Cusimano MD. Anovel approach to endourological training: Training at the Surgi-cal Skill Center. J Urol 2001;166:1261–1266.

10. Afane JS, Olweny EO, Bercowsky E, Sundaram CP, Dunn MD,Shalhav AL, McDougall EM, Clayman RV. Flexible ureteroscopes:A single center evaluation of the durability and function of the newendoscopes smaller than 9Fr. J Urol 2000;164:1164–1168.

Address reprint requests to:Elspeth M. McDougall, M.D., FRCSC

Dept. of UrologyUniversity of California Irvine

101 The City DriveBldg 55, Rm 304, Rt 81

Orange, CA 92868

E-mail: elspethm@uci.edu

VIRTUAL REALITY SIMULAR VERSUS TRAINING MODEL 271