Games For Upper-limb

Stroke Rehabilitation

James Burke Michael McNeill

Philip Morrow Darryl Charles

School of Computing and Information Engineering

Suzanne McDonough

Jacqui Crosbie

School of Life and Health Sciences

School of Computing & Information Engineering,

University of Ulster, Northern Ireland

Contents

1. Research Overview

2. Background on Stroke & Traditional Therapy

3. Technology for Stroke Rehabilitation

4. Previous Studies

5. Game Design for Rehabilitation

6. Outcome Measures

7. Previous University of Ulster Work

8. Webcam Games (with evaluation)

9. Augmented Reality Games

10. Final Remarks

Research Overview

The Problem Area:

Stroke is a leading cause of severe physical disability.

1,100,000 people living in the UK with impairments as the

result of a stroke.

111,000 first time strokes occurring each year.1

Often difficult to provide appropriate level of therapy

to meet a patient‟s rehabilitation needs.

Difficult to maintain patient motivation.2

Research Overview

The Proposed Solution:

Video games may be effective in optimising

engagement & motivation.

Create a framework for the design of games for

upper-limb stroke rehabilitation.

Evaluation of appropriate technology for home-based

therapy.

Design, development and evaluation of rehabilitation

games using novel technology.

Research Overview

Mapping to rehabilitation

TechnologyGame

Design

Rehabilitation

Therapy

Tasks for rehabilitative systems Games

for Stroke

Rehabilitation

Stroke

Effects of stroke:

Attention and concentration deficiencies

Balance loss

Pain

Weakness and paralysis

Depression

Fatigue

Can make day-to-day activities difficult.

Upper limb remains weak in up to 66% cases.1

1. Van der Lee et al. (1999) “Forced use of the upper extremity in chronic stroke patients”.

Stroke Rehabilitation

Rehabilitation programmes usually devised with realistic goals for that particular patient.

Phases of rehabilitation:

In hospital

Travel to specialised units

Home-based rehabilitation

May involve a visiting professional

Patients often do not receive optimal level of therapy.1

1. Burdea (2002) "Key note address: Virtual rehabilitation – benefits and challenges."

Stroke Rehabilitation

Early and intensive practice of functional tasks show more positive outcomes for upper-limb rehabilitation.1

Typical movements practised:2

Reach towards object.

Grasp object.

Manipulate object.

Release object.

Single handed and bimanual tasks.

Tasks often perceived to be mundane and boring.

Therapy can be insufficient.

Technology for Stroke Rehab.

Benefits:

Rehabilitation systems can be novel, interesting, safe and customisable to a particular user‟s abilities and interests.

Motivate, involve and immerse the user.

Potential for home use.1

No need for therapist to be present?

Session data can be recorded at home & uploaded to remote site.2

Limitations:

Clinically valid?1,2

Cost of equipment? Logistics?

Expertise required to setup/operate?

1. Rizzo & Kim (2005) “A SWOT analysis of the field of virtual reality rehabilitation and therapy”.

2. Burdea (2002) "Key note address: Virtual rehabilitation – benefits and challenges."

Technology Interfaces

Input mechanisms

Electromagnetic sensors

Data-gloves

Mixed reality systems

Vision-based tracking (cameras)

Force feedback devices

Nintendo Wii remote

Output devices

Head mounted displays (HMDs)

Projectors

Monitors

Existing Studies

Bespoke:

Sensor-based systems (Crosbie 2004, Yeh 2005)

Xbox console with P5 glove (Morrow 2006)

Haptics & Robotics (Boian 2002, Guo 2007, Podobnik 2008)

Adapted:

Sony EyeToy (Rand 2004, Yavuzer 2008)

Some patients experienced difficulty playing games.

Lack of level grading and suitable challenge for games.

Clearly, rehab games need to be flexible to impairments.

Nintendo Wii (ongoing studies)

Game Design for Stroke Rehab

Existing systems use few or no game design principles.

“Designers of rehabilitation tasks can benefit from

examining the formulas that commercial game developers

use…” 1

Video games are often highly engaging.

Since rehabilitation should be intensive, games may therefore

offer high quality rehabilitation environment.

Commercial off-the-shelf games may already offer some

benefits to people with stroke, but this could be improved!

1. Rizzo & Kim (2005) “A SWOT analysis of the field of virtual reality rehabilitation and therapy”.

Game Design for Stroke Rehab

Play

Voluntary activity, rule-bound and meaningful.1

Manipulation that satisfies curiosity.3

Game

A system defined by rules, with a quantifiable outcome.2

A problem-solving activity, approached with a playful attitude.3

A series of meaningful choices.4

Game design

Determinant of game-play.

Aim to achieve meaningful play.2

1. Huizinga (1955) “Homo Ludens: A Study of the Play-Element in Culture”.

2. Salen & Zimmerman (2003) “Rules of Play: Game Design Fundamentals”.

3. Schell (2008) “The art of game design: a book of lenses”.

4. Sid Meier, game designer. Cited in 2.

Game Design for Stroke Rehab

Why play?

Challenge and competition

Dynamic interactive experience

Social interaction

Locally or remotely (online)

Escapism

Emotional experience

Fun

T-shirt from http://www.zazzle.co.uk/id_rather_be_playing_video_games_tshirt-235013304923558036

Game Design Principles

Three important principles

selected for rehabilitation:

Meaningful Play

Handling Failure

Appropriate level of challenge

Image from www.ubercool.com

Game Design Principles

Meaningful Play

Feedback allows the player to measure their progress.

Progress towards goals.

Upgrading equipment or gaining a level.

Goals can be short-term and long-term.

Motivate and increase longevity.

Inputs choices

Discernable and integrated outcome

Player GameMeaningful play

Feedback

Game Design Principles

Handling Failure

Failure is a prominent and expected

element of video games.

Should only occur due to player‟s incorrect choice.

Rehabilitation games should handle failure conservatively.

Reward all engagement.

Positive and encouraging feedback, even if performance is “poor”.

Scoring principles redesigned.

Game Design Principles

Maintaining Challenge

Appropriately

Match patient abilities.

How to maintain challenge?

Speed, position and size of game elements can set level of

challenge.

In-game calibration can determine suitable level of challenge

for player‟s abilities.

Outcome Measures

Usability

Playability

Motivation

Engagement / Immersion

Enjoyment

Functional outcome

Assessed by health professional.

Game Usability

In software engineering, usability refersto the ease of use of an application‟suser interface.1

Measured in terms of efficiency, effectivenessand satisfaction.

Differs in relation to video games:

A good tool should be both easy to learn and easy to master.

A good game should be easy to learn but difficult to master.2

Game satisfaction emerges from both UI and playability.

Both games & software require effective feedback mechanisms.

1. ISO 9241 Part 11 : Guidance on usability - http://tinyurl.com/ISO9241-11

2. Malone (1982) “Heuristics for designing enjoyable user interfaces: Lessons from computer games”

Image from http://iamthevan.wordpress.com/

Motivation

Goal-orientated behaviour

Intrinsic: Rewards inherent to the task itself.

The enjoyment of play, the resulting score, beating a previous high score, improving motor function.

Extrinsic: Rewards outside of the task.

Peer/therapist encouragement, competition, improving motor function.

Engagement / Immersion

Engagement

Entry level to immersion: Time, effort and attention need to be invested.

Player adequately in control of the game and gaining appropriate feedback.

Engrossment

The emotional investment the player puts into the game.

Player is less self-conscious and less aware of their surroundings.

Total immersion

Player no longer thinks about the fact that they are playing the game.

Brown & Cairns (2004) “A Grounded Investigation of Game Immersion”.

A Framework for Stroke Games

Design a framework as a guideline for designing stroke

rehabilitation games.

Map game design principles to stroke rehabilitation:

Suitable level of difficulty.

Smooth learning curve.

Exercise appropriate range of motions.

Locus of movement.

Competition.

Short-term & long-term goals.

A Framework for Stroke Games

Rehabilitation System

Mapping Game

Design to

Rehabilitation

Tailored Experience Meaningful Feedback

Engagement Enjoyment Motivation ChallengeLongevity

Improved Outcome?

Meaningful Play

Previous UU VR Stroke Projects

Bilateral catch task:

Uses an electromagnetic sensor

attached to a real physical basket.

User moves basket with both hands

to catch falling oranges.

Adaptive „Whack-a-Mouse‟ game:

Sensor attached to player‟s hand.

Encourages arm movement and

visual discrimination.

Not suitable for home use.

Webcam Games

Low cost game system.

Contains multiple games which share astored player profile.

Single and bimanual arm rehabilitation.

Tracks a coloured glove or mitt with anystandard webcam - intuitive controls.

Play standing or seated.

Optional adaptive dynamic difficulty.

Clear and consistent user interface.

No attaching of wires required.

Potential for home rehabilitation.

Webcam Games

Player

Movement

Wearing

Gloves

Image

Captured by

USB

Webcam

Image

Processed –

Pixels

identified

for each

glove

Output to

Monitor /

Projector

R

L

Webcam Games

Profile setup

Stores individual player

profile, per player.

Stores player information:

Player identifier.

Affected side.

Game speed settings.

Game session duration.

Webcam Games

Initial ability determination

test:

Player must roll each ball as

far as they possibly can.

Test done for each arm.

Allows system to determine

range of movement and

position game elements

accordingly.

Stored in player profile.

Can be used as baseline test.

Webcam Games

Single arm exercise:

“Rabbit Chase”

Player must catch a rabbit as it peers out of holes on the screen.

“Bubble Trouble”

Player must burst bubbles as they float around the screen.

Two arm exercise:

“Arrow Attack”

Player must touch corresponding arrows with both hands simultaneously as they enter boxes.

“Bubble Trouble” (two handed version)

Bubbles colour coded and show arrows.

Webcam Games

Effective feedback

Large, bold, easily identifiable graphics.

“Hit” and “Miss” colour coded messages.

Particle effects to show a hit.

Identifiable sound effects for hits and

misses.

Encouraging messages, regardless of

performance.

Score performance chart shows

progress over previous sessions.

Webcam Games - Tools

Camera Calibration tool

Allows refining of colourdetection settings.

Improve automation ofcolour detection in future.

Log Analyser

Shows graphical representationof movements.

Allows replay of game sessions.

View scores.

Potential for remote viewing.

Evaluation of Games

Initial phase – Evaluation with able-bodied users:

10 able bodied users playing webcam games.

Majority of participants enjoyed all games.

Players felt games were enjoyable, replayable and easy to play

due to intuitive controls.

Adaptivity approved by those who noticed it (>80%), indicating

the game would be less enjoyable without it.

One user expressed that the adaptivity was too aggressive.

> 80% agreed that the feedback was effective.

Evaluation of Games

Phase 2 – Single play evaluation with stroke users:

Single play sessions to gain feedback on games and determine

playability by people with stroke.

Eligibility determined by a process of screening for mental

confusion and visual impairment. Participants also required some

movement in the upper limb.

Three participants recruited with varying degrees of movement.

Played each game a number of times and filled out questionnaire

on usability (including playability).

Also rated level of exertion on the Borg-10 scale.

Evaluation of Games

Participant

ID

Gender Age Right/Left

Handed

Affected

side

Date of

Stroke

Movement

A Female 69 Right Right 2 years

ago

Severe

impairment,

wheelchair

B Male 65 Right Left 14 years

ago

Pain in affected

side

C Male 73 Right Left 7 years

ago

Unable to stand

for long, played

games seated

Evaluation of Games

All participants displayed enthusiasm and excitement during gameplay.

Some expressed interest in obtaining the games for home use.

All were able to play the games well despite varying levels of impairment.

Lowest score obtained in any game was a respectable 61%.

All rated games to be enjoyable and replayable.

Lowest score given for enjoyment of game was 7/10.

Feedback effective: all agree that they were able to identify when they had made a mistake and know how they made it.

All approved of adaptivity feature; however, felt that the games speeded up too quickly when performing well.

Evaluation of Games

Phase 3 – Three week single

user case studies:

Results not yet published.

Feedback from study very

positive.

Included motivation

questionnaire.

Performance results and

motor function outcome are

encouraging.

Augmented Reality Games

Marker-Based AR.

Markers tracked via image from standard webcam.

Real-world image augmented with virtual elements.

Allows use of physical objects of varying size, shape and weight.

Rehabilitate high quality motor skills which are more transferable to activities of daily living?

Various libraries: ARToolKit, ARToolKitPlus, ARTag.

Augmented Reality Games

“Brick „a‟ Break”

Clone of Atari‟s Breakout game.

Aim to encourage reach and grasp movement.

Player holds a small cube, representing a paddle.

No time limit for the player to clear a level –player‟s score determined by time taken.

Player does not lose a life for missing a ball –short pause while the ball is re-spawned.

Future versions will contain multiple levels.

Different level layouts.

Different ball speeds.

Different playfield size.

Augmented Reality Games

“Shelf Stack”

Designed to encourage reach, grasp, lift

and release motor function.

Player has several different real world

objects.

Prompted to pick up a particular object,

move it to a specific location on the shelf,

then return it to its position.

Points awarded for completion of selection-

placement tasks and speed.

Score as highly as possible within time limit.

Requires cognitive skills to discriminate the

correct object and placement of the object.

Final Remarks

Ongoing & Future Work:

Publication of three week trial of games with people with stroke.

Development of framework for designing stroke rehabilitation games.

Development and evaluation of Augmented Reality games.

Conclusion:

Games have potential to engage and motivate during rehabilitation.

No need for computer or video game experience.

Low-cost off-the-shelf hardware may offer an opportunity for home

rehabilitation in addition to traditional therapy.