neuro/orthopedic rehabilitation and disability …...neuro/orthopedic rehabilitation and disability...

CHAPTER 1

Neuro/Orthopedic Rehabilitation andDisability Solutions Using VirtualReality Technology

WALTER J. GREENLEAF, Ph.D.

Greenleaf Medical SystemsPalo Alto, California

1.1 VR Environments and Interfaces1.1.1 Head-Mounted Display1.1.2 Instrumented Clothing1.1.3 3-D Spatialized Sound1.1.4 Other VR Interfaces

1.2 Diversity of VR Applications

1.3 Current Status of VR Technology

1.4 VR-Based Medical Applications in Development1.4.1 Surgical Training and Planning1.4.2 Medical Education, Modeling, and Nonsurgical Training1.4.3 Anatomically Keyed Displays with Real-Time Data Fusion1.4.4 Telesurgery and Telemedicine

1.5 Neurologic Testing and Behavioral Intervention

1.6 Rehabilitation, Functional Movement Analysis, and Ergonomic Studies1.6.1 The Role of VR in Disability Solutions

1.7 Conclusion

References

Virtual reality (VR) is an emerging technology that allows individuals to ex-perience three-dimensional (3-D) visual, auditory, and tactile environments.Highly specialized sensors and interface devices allow the individual to become

Information Technologies in Medicine, Volume II: Rehabilitation and Treatment, Edited byMetin Akay and Andy Marsh.ISBN 0-471-41492-1 © 2001 John Wiley & Sons, Inc.

4 NEURO/ORTHOPEDIC REHABILITATION AND DISABILITY SOLUTIONS

immersed and to navigate and interact with objects in a computer-generatedenvironment. Most people associate VR with video games; however, researchersand clinicians in the medical community are becoming increasingly aware of itspotential benefits for people with disabilities and for individuals recoveringfrom injuries.

1.1 VR ENVIRONMENTS AND INTERFACES

The computer-generated environment, or virtual world, consists of a 3-Dgraphics program that relies on a spatially organized, object-oriented databasein which each object in the database represents an object in the virtual world(Fig. 1.1). A separate modeling program is used to create the individual objectsfor the virtual world. For greater realism, these modeling programs apply state-of-the-art computer-graphics techniques, such as texture mapping and shading,to all of the objects of the scene. The object database is manipulated using areal-time dynamics controller that specifies how objects behave within theworld according to user-specified constraints and according to natural laws,such as gravity, inertia, and material properties. These laws are applicationspecific. The dynamics controller also tracks the position and orientation of theuser's head and hand.

Modelingprogram

Application-specificprogramming

Content• 3-D graphics model• Texture maps

tTranslators• CAD satabase input

— *Dynamicsgenerator

Input peripherals• DataGlove• Head tracker• Microphone

L

Renderers• 3-D image• Spatialized sound

IOutput peripherals• HMD Video• HMD Audio• Motion platform

User actions•Turn• Grab• Speak

Figure 1.1. A Complete VR system HMD, head-mounted display.

1.1 VR ENVIRONMENTS AND INTERFACES 5

Common computer input devices, such as a mouse and a keyboard, do notprovide a sense of immersion in a virtual world. To create a VR experience, theconventional computer interface is replaced by one that is more natural andintuitive for interaction within complex 3-D environments. The need for im-proved human-computer interaction with virtual environments (VEs) has mo-tivated the development of a new generation of interface hardware. To date, themost common 3-D input devices used in VR applications are head-mounteddisplays (HMDs) and instrumented clothing (gloves and suits). VEs may alsobe created through circuambiant projections (1), 3-D spatialized sound (2),haptic feedback, and motion effectors.

1.1.1 Head-Mounted Display

The best-known tool for data output in VR is the head-mounted display. Itsupports first-person immersion by generating a wide field of view image foreach eye, often in true 3-D. Most lower-cost HMDs ($1000 range) use liquidcrystal displays (LCDs) others use small cathode ray tubes (CRTs). The moreexpensive HMDs ($60,000 and up) use optical fibers to pipe the images fromnon-HMDs. An HMD requires a position tracker in addition to the helmet.Alternatively, the binocular display can be mounted on an armature for sup-port and tracking (a Boom display) (3).

1.1.2 Instrumented Clothing

Among the most popular and widely available input devices for VR are hand-tracking technologies. Such glove-based input devices let VR users apply theirmanual dexterity to the VR activity. Hand-tracking gloves currently in useinclude Sayre Glove, MIT LED Glove, Digital Data-Entry Glove, DataGlove,Dexterous HandMaster, Power Glove, CyberGlove, and Space Glove (4). Thischapter describes two prototype clinical and rehabilitation applications usinginstrumented clothing technology (Fig. 1.2).

Originally developed by VPL Research, the DataGlove is a thin cloth glovewith engraved optical fibers running along the surface of each digit that loopback to a light-processing box. The optical fibers that cross each joint aretreated to increase the refractive surface area of that segment of the fiber overthe joint. Each optical fiber originates at, and returns to, a light-processing box.In the light-processing box, light-emitting diodes send photons along the fibersto the photo detector. When the joints of the hand bend, the optical fibers bendso that the photons refract out of the fiber, thus attenuating the signal thatpasses through the fibers. The transmitted signal is proportional to the amountof flexion of a single joint and is recorded as such.

Because the attenuation of light along each optical fiber is interpreted as ameasurement of joint flexion, the set of joint measurements can be thought of asa hand gesture. To provide feedback to the user, most VR applications render agraphic representation of the hand moving in real time; this representation

NEURO/ORTHOPEDIC REHABILITATION AND DISABILITY SOLUTIONS

Figure 1.2. The WristSystem, based on the VR DataGlove.

shadows the movements of the hand in the DataGlove and replicates even themost subtle actions.

To determine the orientation and the position of the hand in 3-D space, theglove relies on a spatial tracking system. Tracking systems usually rely onelectromagnetic, ultrasonic, or infrared sensors to determine the position andorientation of a the glove in relation to the signal source. Typically, the sourceis placed at a desired point of reference and the sensor is mounted on thedor sum of the glove.

The DataSuit is a custom-tailored body suit fitted with the same sophisti-cated fiberoptic sensors found in the DataGlove. The sensors are able to trackthe full range of motion of the person wearing the DataGlove or DataSuit as heor she bends, moves, grasps, or waves. Missing from the instrumented clothingis haptic feedback, which provides touch and force-feedback information to theVR participant.

1.1.3 3-D Spatialized Sound

The impression of immersion within a VE is greatly enhanced by inclusion of3-D spatialized sound (5). Stereo-pan effects alone are inadequate because theytend to sound as if they are originating inside the head. Research into 3-Daudio has shown the importance of modeling the head and pinna and using thismodel as part of the 3-D sound generation. A head-related transfer function

1.2 DIVERSITY OF VR APPLICATIONS 7

(HRTF) can be used to generate the proper acoustics. A number of problemsremain, such as the cone of confusion, wherein sounds behind the head areperceived to be in front of the head.

1.1.4 Other VR Interfaces

Senses of balance and motion can be generated in a VR system by a motionplatform. These have been used in flight simulators to provide motion cues thatthe mind integrates with other cues to perceive motion. Haptics is the gen-eration of touch and force-feedback information. Most systems to date havefocused on force feedback and kinesthetic senses, although some prototypesystems exist that generate tactile stimulation. Many of the haptic systems thusfar are exoskeletons used for position sensing as well as for providing resistanceto movement or active force application.

Some preliminary work has been conducted on generating the sense of tem-perature in VR. Small electrical heat pumps have been developed that producesensations of heat and cold as part of the simulated environment.

1.2 DIVERSITY OF VR APPLICATIONS

VR has been researched for decades in government laboratories and uni-versities, but because of the enormous computing power demands and asso-ciated high costs, applications have been slow to migrate from the researchworld to other areas. Recent improvements in the price:performance ratio ofgraphic computer systems have made VR technology more affordable andthus used more commonly in a wider range of applications. In fact, there is evena strong "garage VR" movement—groups of interested parties sharing infor-mation on how to build extremely low cost VR systems using inexpensive off-the-shelf components (6). These homemade systems are often inefficient, un-comfortable to use (sometimes painful), and slow; but they exist as a strongtestament to a fervent interest in VR technology.

Current VR applications are diverse and represent dramatic improvementsover conventional visualization and planning techniques:

• Public entertainment. VR is arguably the most important current trend inpublic entertainment, with ventures ranging from shopping mall gamesimulators to low-cost VR games for the home.

• Computer-aided design (CAD). Using VR to create virtual prototypes insoftware allows engineers to test potential products in the design phase,even collaboratively over computer networks, without investing time ormoney for conventional hard models.

• Military. Using VR, the military's solitary cab-based systems have evolvedinto extensive networked simulations involving a variety of equipmentand situations. Extensive battle simulations can now be created that net-


work tanks, ships, soldiers, and fighters all into the same shared trainingexperience.

• Architecture and construction. VR allows architects and engineers andtheir clients to walk through structural blueprints. Designs may be under-stood more clearly by clients who often have difficulty comprehending evenconventional cardboard models. The city of Atlanta credits its VR modelfor winning the site of the 1996 Olympics, and San Diego used a VR modelof a planned convention center addition to compete for (and obtain) the1996 Republican Party convention.

• Financial visualization. By allowing navigation through an abstract worldof data, VR helps users rapidly visualize large amounts of complex finan-cial market data and thus supports faster decision making.

VR is commonly associated with exotic fully immersive applications because ofthe overdramatized media coverage of helmets, body suits, entertainment sim-ulators, and the like. As important are the window-into-world applications bywhich the user or operator is allowed to interact effectively with virtual data,either locally or remotely.

1.3 CURRENT STATUS OF VR TECHNOLOGY

The commercial market for VR, although taking advantage of advances in VRtechnology at large, is nonetheless contending with the lack of integrated sys-tems and the lack of reliable equipment suppliers. Typically, researchers buyperipherals and software from separate companies and configure their ownsystems. Companies that can offer integrated systems for commercial applica-tions are expected to fill this gap over the next few years. Concurrently, thenature of the commercial VR medical market is expected to change as theprices of today's expensive, high-performance graphics systems decrease dra-matically. High-resolution display systems will also significantly drop in costas the VR display business can piggyback on HDTV projection and home-entertainment technologies.

Technical advances have occurred in networking applications, which includeimproved visual photo realism, decreased tracker latency through predictivealgorithms, and variable resolution image generators. Work to improve data-base access methods is under way. Important hardware advances include eyegear with an increased field of view, wireless communications, lighted andsmaller devices, and improved tracking systems.

1.4 VR-BASED MEDICAL APPLICATIONS IN DEVELOPMENT

The first wave of VR development efforts in the medical community addressedseven key categories:

1.4 VR-BASED MEDICAL APPLICATIONS IN DEVELOPMENT 9

Surgical training and surgical planning.

Medical education, modeling, and nonsurgical training.Anatomically keyed displays with real-time data fusion.Telesurgery and telemedicine.Patient testing and behavioral intervention.Rehabilitation, functional movement analysis, and motion/ergonomic

studies.Disability solutions.

The potential of VR through education and information dissemination indi-cates there will be few areas of medicine not taking advantage of this improvedcomputer interface. However, the latent potential of VR lies in its capacity tobe used to manipulate and combine heterogeneous datasets from many sources.This feature is most significant and likely to transform the traditional applica-tions environment in the near future.

1.4.1 Surgical Training and Planning

Various projects are under way to use VR and imaging technology to plan,simulate, and customize invasive (an minimally invasive) surgical procedures.Ranging from advanced imaging technologies for endoscopic surgery to routinehip replacements, these new developments will have a tremendous effect onimproving surgical morbidity and mortality. According to Merril (7), studiesshow that doctors are more likely to make errors when performing their firstfew to several dozen diagnostic and therapeutic surgical procedures then whenperforming later procedures. Merril claims that operative risk could be sub-stantially reduced by the development of a simulator that would allow trans-ference of skills from the simulation to the actual point of patient contact. Withsurgical modeling, we would generally expect a much higher degree of preci-sion, reliability, and safety, in addition to cost efficiency.

Several VR-based systems currently under development allow real-timetracking of surgical instrumentation and simultaneous display and manipulationof 3-D anatomy corresponding to the simulated procedure (8, 9). Using thisdesign, surgeons can practice procedures and experience the possible complica-tions and variations in anatomy encountered during surgery. Necessary soft-ware tools have been developed to enable the creation of virtual tissues thatreflect the physical characteristics of physiologic tissues. This technology oper-ates in real-time using 3-D graphics, on a high-speed computer platform.

1.4.2 Medical Education, Modeling, and Nonsurgical Training

Researchers at the University of California at San Diego are exploring thevalue of hybridizing elements of VR, multimedia (MM), and communicationstechnologies into a unified educational paradigm (10). The goal is to develop


powerful tools that extend the flexibility and effectiveness of medical teachingand promote lifelong learning. To this end, they have undertaken a multiyearinitiative, named the VR-MM Synthesis Project. Based on instructional designand user need (rather than technology per se), they plan to link the computersof the Data Communications Gateway, the Electronic Medical Record System,and the Simulation Environment. This system supports medical students, sur-gical residents, and clinical faculty and runs applications ranging from fullsurgical simulation to basic anatomic exploration and review, all via a commoninterface. The plan also supports integration of learning and telecommunica-tions resources (such as interactive MM libraries, on-line textbooks, databasesof medical literature, decision support systems, email, and access to electronicmedical records).

1.4.3 Anatomically Keyed Displays with Real-Time Data Fusion

An anatomically keyed display with real-time data fusion is currently in useat the New York University Medical Center's Department of Neurosurgery.The system allows both preoperative planning and real-time tumor visualiza-tion (11, 12). The technology offers the opportunity for a surgeon to plan andrehearse a surgical approach to a deep-seated, centrally located brain tumorbefore doing of the actual procedure. The imaging method (volumetric stereo-taxis) gathers, stores and reformats imaging-derived, 3-D volumetric informa-tion that defines an intracranial lesion (tumor) with respect to the surgical field.

Computer-generated information is displayed during the procedure on com-puter monitors in the operating room and on a heads-up display mounted onthe operating microscope. These images provide surgeons with CT- and MRI-defined maps of the surgical field scaled to actual size and location. This infor-mation guides the surgeon in finding and defining the boundaries of braintumors. The computer-generated images are indexed to the surgical field bymeans of a robotics-controlled stereotactic frame that positions the patient'stumor within a defined targeting area.

Simulated systems using VR models are also being advocated for other high-risk procedures, such as the alignment of radiation sources to treat canceroustumors.

1.4.4 Telesurgery and Telemedicine

Telepresence is the sister field of VR. Classically defined as the ability to act andinteract in an off-site environment by making use of VR technology, tele-presence is emerging as an area of development in its own right. Telemedicine(the telepresence of medical experts) is being explored as a way to reduce thecost of medical practice and to bring expertise into remote areas (13, 14).

Telesurgery is a fertile area for development. On the verge of realization,telesurgery (remote surgery) will help resolve issues that can complicate orcompromise surgery, including

1.5 NEUROLOGIC TESTING AND BEHAVIORAL INTERVENTION 11

• A patient that is too ill or injured to be moved for surgery.• A specialized surgeon located at some distance from the patient who

requires specialized intervention.

• Accident victims who need immediate, on-the-scene surgery.• Soldiers wounded in battle.

The surgeon really does operate—on flesh, not a computer animation. Althoughthe distance aspect of remote surgery is a provocative one, telepresence is provingto be an aid in nonremote surgery as well. It can help surgeons gain dexterityand improve their operative technique, which is expected to be particularlyimportant in endoscopic surgery. For example, suturing and knot tying will beas easy to see in endoscopic surgery as it is in open surgery, because telepres-ence offers the ability to emulate the look and feel of open surgery.

As initially developed at SRI International (15), telepresence not only offersa compelling sense of reality for the surgeon but also allows him or her to per-form the surgery according to the usual methods and procedures. There isnothing new to learn. Hand motions are quick and precise. The visual field, theinstrument motion, and the force feedback can all be scaled to make micro-surgery easier than it would be if the surgeon were at the patient's side. Whilecurrent technology has been implemented in several prototypes, SRI and Tele-surgical Corporation (Redwood City, CA) are collaborating to develop a fullcomercial system based on this novel concept.

1.5 NEUROLOGIC TESTING AND BEHAVIORAL INTERVENTION

For Parkinson disease victims, initiating and sustaining walking becomesprogressively difficult. The condition known as akinesia can be mitigated bytreatment with drugs such as L-dopa, a precursor of the natural neural trans-mitter dopamine, but usually not without unwanted side effects. Now, collabo-rators at the Human Interface Technology Laboratory and the Departmentof Rehabilitation Medicine at the University of Washington, along with theSan Francisco Parkinson's Institute are using virtual imagery to simulate aneffect called kinesia paradoxa, or the triggering of normal walking behavior inakinetic Parkinson patients (16).

Using a commercial, field-multiplexed, heads-up video display, the researchteam has developed an approach that elicits near-normal walking by presentingcollimated virtual images of objects and abstract visual cues moving throughthe patient's visual field at speeds that emulate normal walking. The combina-tion of image collimation and animation speed reinforces the illusion of space-stabilized visual cues at the patient's feet. This novel, VR-assisted technologymay also prove to be therapeutically useful for other movement disorders.

Lamson and Meisner (17) investigated the diagnostic and treatment possi-bilities of VR immersion on anxiety, panic and height phobias. By immersing


both patients and controls in computer-generated situations, the researcherswere able to expose the subjects to anxiety-provoking situations (such asjumping from a height) in a controlled manner. Experimental results indicate asignificant subject habituation and desensitization through this approach, andthe approach appears clinically useful.

Pugnetti (18) explored the potential of enhancing the clinical evaluation andmanagement of acquired cognitive impairments in adults. By using a YR-basednavigation paradigm, researchers were able to challenge both patients andnormal subjects with a complex cognitive activity and simultaneously generateperformance data. Behavioral data analysis was then carried out using estab-lished scoring criteria.

1.6 REHABILITATION, FUNCTIONAL MOVEMENT ANALYSIS, ANDERGONOMIC STUDIES

The field of VR is still at the proof-of-concept stage, yet there is a growingnumber of potential applications related to motion monitoring, rehabilitation,and ergonomic analysis. I (19) theorized that the rehabilitation process can beenhanced through the use of VR technology.

Evaluation of hand impairment involves detailed physical examination andfunctional testing of the afflicted person. The physical examination is designedto determine the presence of pain and any loss of strength, sensation, range ofmotion, and structure. The results are combined to produce a numerical as-sessment of impairment that is used to evaluate the patient's progress over time,yet examinations and calculations can be time-consuming, expensive, and sub-ject to observer error.

Functional evaluation is usually accomplished by subjective observation ofpatient performance on standardized tests for motor skills. However, repro-ducibility of measurements becomes an issue whenever different examinersevaluate the same patient, which makes it difficult to evaluate a patient's prog-ress over time. The more objective assessments of upper extremity motion fallinto two categories: visual and effective.

Visual methods involve digitizing and estimating a visual record of themotion: The patient is videotaped performing a task; then the individual framesof the video are digitized and evaluated to quantify the degree of motion ofthe joint under study. The main limitation of this technique is that the cameracan view motion in only two dimensions. To assess movement in the camera'svisual plane accurately, the third dimension must be held constant, i.e., theperson must move along a known line parallel to the plane of the film in thecamera. In most cases, the examiner cannot maintain the correct orientationeven for short periods, making this a difficult and cumbersome technique.

Effective methods measure the motion's effect rather than the motion itself.A work simulator is one example of an effective assessment tool. Work simu-lators measure the force exerted by a subject on a variety of attachments that

1.6 REHABILITATION, FUNCTIONAL MOVEMENT ANALYSIS, AND ERGONOMIC STUDIES 13

simulate tools used in the workplace. A major limitation of this approach isthat no data are collected on how the person effects the force.

Ideally one would like to collect and compare range of motion data for ajoint in several planes simultaneously while specific tasks were being performedby the patient, a measurement that is impossible using a standard goniometer.At one point my group considered using the DataGlove with its multiple sen-sors as a means of collecting dynamic functional movement data. However,migrating the DataGlove technology from the field of VR to clinical evaluationposed several problems. For example, during manufacture of the DataGlove,the treatment of individual fiberoptic cables was not identical, thus it wasimpossible to characterize and predict their behavior. Moreover, empirical ob-servations show hysteresis, making repeated measurements irreproducible andmaking it difficult to determine the accuracy of the measurements. For highlyaccurate measurements, it is important to have a perfect fit of the glove, be-cause poor placement of the sensitive areas of fibers yields incorrect measure-ments. Achieving a perfect fit of the DataGlove posed a serious challengebecause of the variability of hand shapes and sizes across a given patientpopulation. Moreover, the use of the fiberoptic DataGlove excluded the popu-lation of patients with anatomic deformities.

With the goal of obtaining accurate, dynamic range of motion data for thewrist joint, my group investigated other sensor materials and developed theglove-based WristSystem. Fiberoptic sensors were replaced by dual-axis elec-trogoniometric sensors that could be inserted into machine-washable Lycragloves that fit different sizes of hands.

WristSystem gloves are currently being used to track flexion, extension,radial, and ulnar deviations of the wrist while patients are performing routinetasks. A portable, lightweight DataRecorder worn in a waist pack permits thepatient to be ambulatory while data are being collected at the clinic or work site(Fig. 1.3); no visual observation or supervision is required beyond the initialcalibration of the glove sensors. Real-time visual and auditory feedback canalso be used to train the patient to avoid high-risk postures in the workplace orat home.

The WristSystem includes Motion Analysis System (MAS) software for theinterpretation of the dynamic-movement data collected by the DataRecorderover several minutes or hours. This software offers rapid, quantitative analysisof data that includes the total and percent time the wrist spends at critical an-gles (minimum, maximum, and mean wrist angles in four directions), the num-ber of repetitions, and the velocity and acceleration. Figure 1.4 shows a sampleplot of some of these data. In this example, it can be seen that the patient's righthand was ulnar-deviated >15° for 84% of the time he performed a certain task.

The WristSystem is currently being used by occupational and rehabilitationmedicine specialists (MDs, PTs, and OTs), ergonomists, industrial safety man-agers, biomechanical researchers, and risk management consultants. The ulti-mate extension of this project is to build an augmented-reality environment forquantitative evaluation of functional tasks. The system will link multiple input


Figure 1.3. The WristSystem being used to track functional movement at a job site.

devices or sensors worn by the patient and 3-D modeling software. Therapistswill be able to design a virtual world composed of objects traditionally used infunctional assessment tests, such as balls, cubes, keys, and pencils. The therapistwill be able to record the motion, location, and trajectory of the user's handthat is required to accomplish the motion and any associated hand tremors or

Greenleaf sample subject : Plot 1

ISession 2 [Sensors H RH R/U

Degrees

) 24.00 28.00 32.00 36.00 40.00 44.00 48.00 j Time/~**~*~~******'***--—•—*—-—————-—-=-—•—-——« Percent/|3O I seconds Across

Figure 1.4. MAS: WristSystem data plot.

1.6 REHABILITATION, FUNCTIONAL MOVEMENT ANALYSIS, AND ERGONOMIC STUDIES 15

spasms. Once the data are collected, the therapist can use statistical analysissoftware to interpret them. The clinician can also elect to review the motions byanimation of the data and to change the orientation to study the motion fromanother angle.

Other control devices originally developed for VR are being improved andapplied to the field of functional evaluation of movement in a variety of ways.Burdea (20) described a system that would couple a glove with force-feedbackdevices to rehabilitate a damaged hand or to diagnose a range of hand prob-lems. He describes another system under development that incorporates tactilefeedback in a glove to produce feeling in the fingers when virtual objects are"touched."

My research group previously theorized that the rehabilitation process couldbe enhanced through the use of VR technology (21). Perhaps the most signifi-cant advantage is that a single VR-based rehabilitation workstation can beeasily customized for individual patient needs. We are currently developing somebasic components for a VR-based workstation (Fig. 1.5) that will be used to

• Restructure the rehabilitation process into small, incremental functionalsteps.

Figure 1.5. Virtual reality technology for quanitative evaluation of functional tasks.


• Make the rehabilitation process more realistic and less boring, enhancingpatient motivation and recovery of function.

• Facilitate home rehabilitation.• Provide an objective index of function for activities of daily living and

work.

1.6.1 The Role of VR in Disability Solutions

One exciting aspect of VR technology is the inherent ability to enable individ-uals with physical disabilities to accomplish tasks and have experiences thatwould otherwise be denied them. There are approximately 35 million people inthe United States who have physical disabilities that affect their ability to workin certain types of jobs. Computers and VR software can provide increasedaccess to daily activities for people with disabilities. VR technology may pro-vide an adaptable mechanism for harnessing a person's strongest physicalability to operate an input device for a computer program.

A simple glove-based system allows users to record custom-tailored gesturesand to map these gestures to actions. These actions can range from a simplecommand, such as a mouse click on a computer screen, to more complex func-tions, such as controlling a robotic arm. An application programmer candefine the functional relationship between the sensor data and a task with real-time graphical representation on a computer screen. Simple gestures can betranslated to a preprogrammed set of instructions for speech or movement.

The prototype GloveTalker is an example of a one-to-one mapping of agesture to a computer-generated action to provide additional communica-tion skills to people with vocal impairment. The patient is able to speak bysignaling the computer with his or her personalized set of gestures while wear-ing the DataGlove, which recognizes hand positions (gestures) and passes theinformation to the computer's voice-synthesis system (Fig. 1.6). For example, apatient may map a specific gesture, such as a closed fist, for the phrase "Hello,my name is Susan." The computer has some freedom in interpreting the gestureso that people capable of only relatively gross muscle control may use thesystem. There is also the possibility of sending the voice output through a tele-phone system, enabling vocally impaired individuals to communicate verballyover distance.

1.7 CONCLUSION

VR tools and techniques are rapidly developing in the scientific, engineering,and medical areas. Although traditionally used as input devices for virtualworlds in the entertainment and defense industry, sensor-loaded gloves maybecome the clinical tools of choice for measuring, monitoring, and amplifyingupper-extremity motion. Although I have identified other potential clinical ap-plications, many technological challenges must be met before such devices canbe made available for patient care.

REFERENCES 17

Figure 1.6. The GloveTalker: speech communication for the vocally impaired.

Pioneers in the field of medical VR are being encouraged to design sophisti-cated devices that promote both physical and psychological gains for injuredand disabled patients while keeping costs for the devices within an acceptablerange for health-care providers and third-party payors. The mandate is com-plex, but like VR technology itself, the possibilities are promising and exciting.

REFERENCES

1. C. Cruz-Neira, D. J. Sandin, and T. A. DeFanti. Surround-screen projection-basedvirtual reality: the design and implementation of the CAVE. Paper presented atComputer Graphics. Anaheim, CA, 1993.

2. M. F. Deering. Explorations of display interfaces for virtual reality. Paper presentedat IEEE Virtual Reality Annual International Symposium. New York, 1993.

3. M. T. Bolas. Human factors in the design of an immersive display. IEEE ComputGraph Appl 1994;14:55-59.


4. D. J. Sturman and D. Zeltzer. A survey of glove-based input. IEEE Comput GraphAppl 1994; 14:30-39.

5. N. I. Durlach, B. G. Shinn-Cunningham, and R. M. Held. Supernormal auditorylocalization. I. General background. Presence 1993;2:89-103.

6. L. Jacobs. Garage virtual reality. Indianapolis: Sams, 1994.

7. J. R. Merril. Photorealistic interactive three-dimensional graphics in surgical simu-lation. Interactive technology and the new paradigm for healthcare. Burke, VA: IOSPress, 1995.

8. D. Hon. Ixion's laparoscopic surgical skills simulator. Paper presented at MedicineMeets Virtual Reality II. San Diego, CA, 1994.

9. K. T. McGovern and L. T. McGovern. Virtual clinic: a virtual reality surgical sim-ulator. Virtual Reality World 1994;2:1994.

10. H. M. Hoffman. Virtual reality and the medical curriculum: integrating extant andemerging technologies. Paper presented at Medicine Meets Virtual Reality II. SanDiego, CA, 1994.

11. P. J. Kelly. Quantitative virtual reality surgical simulation, minimally invasive ste-reotactic neurosurgery and frameless stereotactic technologies. Paper presented atMedicine Meets Virtual Reality II. San Diego, CA, 1994.

12. B. A. Kail, P. J. Kelly, S. O. Stiving, and S. J. Goerss. Integrated multimodalityvisualization in stereotactic neurologic surgery. Paper presented at Medicine MeetsVirtual Reality II. San Diego, CA, 1994.

13. M. Burrow. A telemedicine testbed for developing and evaluating telerobotic toolsfor rural health care. Paper presented at Medicine Meets Virtual Reality II. SanDiego, CA, 1994.

14. J. Rosen. The role of telemedicine and telepresence in reducing health care costs.Paper presented at Medicine Meets Virtual Reality II. San Diego, CA, 1994.

15. R. M. Satava. Robotics, telepresence and virtual reality: a critical analysis of thefuture of surgery. Minim Invasive Ther 1992; 1:357-363.

16. S. Weghorst, J. Prothero, and T. Furness. Virtual images in the treatment of par-kinson's disease akinesia. Paper presented at Medicine Meets Virtual Reality II. SanDiego, CA, 1994.

17. R. Lamson and M. Meisner. The effects of virtual reality immersion in the treatmentof anxiety, panic, & phobia of heights. Virtual reality and persons with disabilities.Paper presented at the 2nd Annual International Conference. San Francisco, CA,June 8-10, 1994.

18. D. V. Pugnetti. Recovery diagnostics and monitoring in virtual environments. Vir-tual reality in rehabilitation, research, experience and perspectives. Paper presentedat the 1st International Congress on Virtual Reality in Rehabilitation. Gubbio,Italy, June 13-18, 1994.

19. W. J. Greenleaf. DataGlove and DataSuit: virtual reality technology applied tothe measurement of human movement. Paper presented at Medicine Meets VirtualReality II. San Diego, CA, 1994.

20. G. Burdea, J. Zhuang, E. Roskos, et al. A portable dextrous master with forcefeedback. Presence 1992; 1:18-28.

21. W. J. Greenleaf and M. A. Tovar. Augmenting reality in rehabilitation medicine.Artif Intel Med 1994;6:289-299.

neuro/orthopedic rehabilitation and disability …...neuro/orthopedic rehabilitation and disability...

Documents