special issue of media solutions that improve accessibility to disabled users

8/9/2019 Special Issue of Media Solutions That Improve Accessibility to Disabled Users

1/71

UUBBIICCCC JJoouurrnnaallUbiquitous Computing and Communication Journal

2010 Volume 5 . 2010-03-10 . ISSN 1992-8424

SSppeecciiaall IIssssuuee ooff MMeeddiiaa SSoolluuttiioonnss tthhaattIImmpprroovvee AAcccceessssiibbiilliittyy ttoo DDiissaabblleedd UUsseerrss

Unconstrained walking plan to virtual environment for spatiallearning by visually impaired

1

Application of virtual reality technologies in rapid developmentand assessment of ambient assisted living environments

8

PIXAR animation studios and disabled personages case study:Finding NEMO

16

Web accessible design centered on user experience 23

First steps towards determining the role of visual information inmusic communication

32

Examining the feasibility of face gesture detection for monitoringusers of autonomous wheel chairs

42

Personal localization in wearable camera platform towardsassistive technology for social interactions

58

UBICC Publishers 2010Ubiquitous Computing and Communication Journal


2/71

Managing Editor Dr. David Fonseca

Ubiquitous Computing and

Communication Journal

Book: 2010 Volume 5

Publishing Date: 2010-03-10

Proceedings

ISSN 1994-4608

This work is subjected to copyright. All rights are reserved whether the whole or part of the material

is concerned, specifically the rights of translation, reprinting, re-use of illusions, recitation,

broadcasting, reproduction on microfilms or in any other way, and storage in data banks.

Duplication of this publication of parts thereof is permitted only under the provision of the

copyright law 1965, in its current version, and permission of use must always be obtained from

UBICC Publishers. Violations are liable to prosecution under the copy right law.

UBICC Journal is a part of UBICC Publishers

www.ubicc.org

UBICC Journal

Printed in South Korea

Typesetting: Camera-ready by author, data conversation by UBICC Publishing Services, South

Korea

Working to grow libraries indeveloping countries

UbiCC Journal | www.ubicc.org


3/71

UNCONSTRAINED WALKING PLANE TO VIRTUAL ENVIRONMENT

FOR SPATIAL LEARNING BY VISUALLY IMPAIRED

Kanubhai K. Patel1, Dr. Sanjay Kumar Vij21School of ICT, Ahmedabad University, Ahmedabad, India, [email protected]

2Dept. of CE-IT-MCA, SVIT, Vasad, India, [email protected]

ABSTRACT

Treadmill-style locomotion interfaces for locomotion in virtual environmenttypically have two problems that impact their usability: bulky or complex drive

mechanism and stability problem. The bulky or complex drive mechanism

requirement restricts the practical use of this locomotion interface and stability

problem results in the induction of fear psychosis to the user. This paper describesa novel simple treadmill-style locomotion interface that uses manual treadmill with

handles to provide needbased support, thus allowing walking with assured stability.Its simplicity of design coupled with supervised multi-modal training facility

makes it an effective device for spatial learning and thereby enhancing the mobility

skills of visually impaired people. It facilitates visually impaired person in

developing cognitive maps of new and unfamiliar places through virtualenvironment exploration, so that they can navigate through such places with easeand confidence in real. In this paper, we describe the structure and control

mechanism of the device along with system architecture and experimental results

on general usability of the system.

Keywords: assistive technology, blindness, cognitive maps, locomotion interface,Virtual learning environment.

1 INTRODUCTIONUnlike in case of sighted people, spatial

information is not fully available to visually

impaired and blind people causing difficulties intheir mobility in new or unfamiliar locations. This

constraint can be overcome by providing mental

mapping of spaces, and of the possible paths for

navigating through these spaces which are essentialfor the development of efficient orientation and

mobility skills. Orientation refers to the ability tosituate oneself relative to a frame of reference, and

mobility is defined as the ability to travel safely,comfortably, gracefully, and independently [7, 18].

Most of the information required for mental mapping

is gathered through the visual channel [15]. Asvisually impaired people are handicapped to gather

this crucial information, they face great difficulties

in generating efficient mental maps of spaces and,therefore, in navigating efficiently within new or

unfamiliar spaces. Consequently, many visuallyimpaired people become passive, depending on

others for assistance. More than 30% of the blind do

not ambulate independently outdoors [2, 16]. Suchassistance might not be required after a reasonable

number of repeated visits to the new space as thesevisits enable formation of mental map of the new

space subconsciously. Thus, a good number ofresearchers focused on using technology to simulate

visits to a new space for building cognitive maps.Although isolated solutions have been attempted, nointegrated solution of spatial learning to visually

impaired people is available to the best of our

knowledge. Also most of the simulatedenvironments are far away from reality and the

challenge in this approach is to create a near real-life

experience.

Use of advanced computer technology offersnew possibilities for supporting visually impaired

people's acquisition of orientation and mobility skills,by compensating the deficiencies of the impaired

channel. The newer technologies including speechprocessing, computer haptics and virtual reality (VR)

provide us various options in design and

implementation of a wide variety of multimodalapplications. Even for sighted people, such

technologies can be used (a) to enhance the visual

information available to a person in such a way thatimportant features of a scene are presented visibly,

or (b) to train them through virtual environmentleading to create cognitive maps of unfamiliar areas

or (c) to get a feel of an object (using haptics) [16].

Virtual Reality provides for creation ofsimulated objects and events with which people can

interact. The definitions of Virtual Reality (VR),although wide and varied, include a common

statement that VR creates the illusion ofparticipation in a synthetic environment rather than

UbiCC Journal, Volume 5, March 2010 1


4/71

going through external observation of such an

environment [5]. Essentially, virtual reality allowsusers to interact with a simulated environment. Users

can interact with a virtual environment eitherthrough the use of standard input devices such as a

keyboard and mouse, or through multimodal devices

such as a wired glove, the Polhemus boom arm, or

else omni-directional treadmill.Even though in the use of virtual reality with thevisually impaired person, the visual channel is

missing, the other sensory channels can still lead tobenefits for visually impaired people as they engage

in a range of activities in a simulator relatively free

from the limitations imposed by their disability. In

our proposed design, they can do so in safe manner.

We describe the design of a locomotioninterface to the virtual environment to acquire spatial

knowledge and thereby to structure spatial cognitivemaps of an area. Virtual environment is used toprovide spatial information to the visually impaired

people and prepare them for independent travel. Thelocomotion interface is used to simulate walking

from one location to another location. The device isneeded to be of a limited size, allow a user to walk

on it and provide a sensation as if he is walking on

an unconstrained plane.The advantages of our proposed device are as

follows:

It solves instability problem during walking byproviding supporting rods. The limited width of

treadmill along with side supports gives a

feeling of safety and eliminates the possibility

of any fear of falling out of the device.

No special training is required to walk on it. The devices acceptability is expected to be high

due to the feeling of safety while walking on thedevice. This results in the formation of mental

maps without any hindrance.

It is simple to operate and maintain and it haslow weight.The remaining paper is structured as follows:

Section 2 presents the related work. Section 3

describes the structure of locomotion interface used

for virtual navigation of computer-simulated

environments for acquisition of spatial knowledge

and formation of cognitive maps; Section 4 describecontrol principle of locomotion device; Section 5

illustrates the system architecture; while Section 6describe the experiment for usability evaluation,

finally Section 7 concludes the paper and illustrates

future work.

2 RELATED WORKWe have categorized the most common virtualreality (VR) locomotion approaches as follow:

Omni-directional treadmills (ODT) [3, 8, 14, 4], The motion foot pad [10], Walking-in-place devices [19], actuated shoes [11], and

The string walker [12].The basic idea used in these approaches is that a

locomotion interface should cancel the users self

motion in a place to allow the user to move in a large

virtual space. For example, a treadmill can cancelthe users motion by moving its belt in the oppositedirection. Its main advantage is that it does not

require a user to wear any kind of devices asrequired in some other locomotion devices. However,

it is difficult to control the belt speed in order tokeep the user from falling off. Some treadmills can

adjust the belt speed based on the users motion.There are mainly two challenges in using the

treadmills. The first one is the users stability

problem while the second is to sense and change thedirection of walking. The belt in a passive treadmill

is driven by the backward push generated whilewalking. This process effectively balances the userand keeps him from falling off.

The problem of changing the walking direction isaddressed by [1, 6], who employed a handle to

change the walking direction. Iwata & Yoshida [13]developed a 2D infinite plate that can be driven in

any direction and Darken [3] proposed an Omnidirectional treadmill using mechanical belt. Noma &

Miyasato [17] used the treadmill which could turnon a platform to change the walking direction. Iwata

& Fujji [9] used a different approach by developing

a series of sliding interfaces. The user was requiredto wear special shoes and a low friction film was put

in the middle of shoes. Since the user was supported

by a harness or rounded handrail, the foot motion

was canceled passively when the user walked. The

method using active footpad could simulate variousterrains without requiring the user to wear any kind

of devices.

3 STRUCTURE OF LOCOMOTIONINTERFACE

Figure 1: Mechanical structure of locomotion

interface. There are three major parts in the figure:

(a) A motor-less treadmill, (b) mechanical rotating

base, and (c) block containing Servo motor andgearbox to rotate the mechanical base.



5/71

Figure 2: Locomotion interface.

As shown in Figure 1 and 2, our device consists

of a motor-less treadmill resting on a mechanicalrotating base. In terms of its physical characteristics,

our devices upper platform (treadmill) is 54 inlength and 30 wide with an active surface 48 X24. The belt of treadmill contains mat on which 24

stripes along the direction of motion, at a distance of

1 between two stripes. Below each stripe, there are

force sensors that sense the position of feet. Atypical manual treadmill passively rotates as the user

moves on its surface, causing belt to rotate backward

as the user moves forward. Advantages of thispassive (i.e. non-motorized) movement are: (a) to

achieve an almost silent device with negligible-noiseduring straight movement, and (b) the backward

movement of treadmill is synchronized with forwardmovement of user leading thereby jerk-free motion.

(c) Also in case of the trainee stopping to walk as

detected by non-movement of belt, our systemassists and guides the user for further movement.

The side handle support provides the feeling ofsafety and stability to the person which results in

efficient and effective formation of cognitive maps.

Human beings subconsciously place their feet at

angular direction whenever they intend to take a turn.

Therefore the angular positions of the feet on thetreadmill are monitored to determine not only usersintention to take a turn, but also the direction and

desired angle at granularity of 15o.

Rotation control system finds out angle through

which the platform should be turned, and turns the

whole treadmill with user standing on it, on

mechanical rotating base, so that the user can placenext footstep on the treadmills belt. The rotation of

platform is carried out using a servo motor. Servomotor and gearbox are placed in lower block which

is lying under the mechanical rotating base. Our

device also provides for safety mechanism through akill switch, which can be triggered to halt the device

immediately in case the user loses control or loses

his balance.

4 CONTROL PRINCIPLE OFLOCOMOTION DEVICE

Belt of treadmill of device rotates in backward

or forward direction as user moves in forward or

backward direction, respectively, on the treadmill.This is a passive, non-motorized, movement oftreadmill. The backward movement of belt of

treadmill is synchronized with forward movement ofuser leading thereby non-jerking motion. This solves

the problem of stability. For maneuvering, which

involves turning or side-stepping, our Rotation

control system rotates the whole treadmill in

particular direction on mechanical rotating base.In case of turning as shown in Figure 3, when

foot is on more than three strips then user wants toturn and we should rotate the treadmill. If middlestrip of new footstep is on left side of middle strip of

previous footstep then rotation is on left side and ifmiddle strip of new footstep is on right side of

middle strip of previous footstep then rotation is onright side.

Figure 3: Rotation of treadmill for veer left turn

(i.e. 45O) (a) Position of treadmill before turning (b)

after turning

Figure 4: Rotation of treadmill for side-stepping

(i.e. 15O) (a) Before side-stepping and (b) after side-

stepping

In case of side-stepping as shown in Figure 4,

When both feet are on three strips then compare



6/71

distance between current and the previous foot

positions to determine whether side-stepping hastaken placed or not. If it is more than a threshold

value, the side-stepping has taken placed otherwisethere is no side-stepping. If it is equal or less than

maximum gap distance then that is forward step, so

no rotation is performed.

After determining the direction and angle ofrotation, our software sends appropriate signals tothe servo motor to rotate in the desired direction by

given angle and, accordingly, the platform rotates.This process ensures that the user places the next

footstep on the treadmill itself, and do not go off the

belt.

The algorithm to find direction and angle of

turning is based on (a) number of strips pressed byleft foot (nl), (b) number of strips pressed by right

foot (nr), (c) distance between middle strips of twofeet (dist) and (d) threshold for the distance betweenmiddle strips of two feet. The outputs are direction

(Left Turn - lt, Right Turn - rt, Left Side stepping - ls,or Right Side stepping rs) and angle to turn.

Different possible cases of turning and sidesteppingare shown in Figure 5.

ALGORITHM

1: if (nl>3) && (dist>d) then //Case-1

2: find

3: left_turn = true //i.e. return lt

4: elseif (nl==3) && (dist>d) then //Case2

5: = 15o

6: left_side_stepping = true //i.e. return ls

7: elseif (nl>3) && (dist3) && (dist>d) then //Case411: find

12: right_turn = true //i.e. return rt

13: elseif (nr==3) && (dist>d) then //Case5

14: = 15o

15: right_side_stepping = true //i.e. return rs

16: elseif (nr>3) && (dist


7/71

Figure 6: Screen shot of Computer-simulatedenvironments

Additionally, occurrences of various events such

as (i) arrival of a junction, (ii) arrival of object(s) ofinterest, etc. are signaled by sound through speakers

or headphones. Whenever the cursor is moved nearan object, its sound features are activated, and acorresponding specific sound or a pre-recorded

message is heard by the participant. Participant canalso get information regarding orientation and

nearby objects, whenever needed, through help keys.The Simulator also generates audible alert when the

participant is approaching any obstacle. During

training, the Simulator continuously checks andrecords participants navigating style (i.e. normal

walk or drunkard/random walk) and the pathfollowed by the user when encountered with

obstacles.

Once the user gets confident and memorizes the

path and landmarks between source and destination,he navigates by using second mode of navigationthat is without systems help and tries to reach the

destination. The Simulator records participantsnavigation performance, such as path traversed, time

taken, distance traveled and number of steps taken to

complete this task. It also records the sequence of

objects encountered on the traversed path and the

positions where he seemed to have some confusion(and hence took relatively longer time). The DataCollection module keeps receiving the data from

Force Sensors, which is sent to VR system formonitoring and guiding the navigation. Feet position

data are also used for sensing the users intention to

take a turn, which is directed to the motor planning(rotation) module to rotate the treadmill.

6 EXPERIMENT FOR USABILITYEVALUATION

The evaluation consists of an analysis of timerequired and number of steps taken to train to

competence with our locomotion interface (LI), ascompared to other navigation method like keyboard

(KB), and comments from users that suggest areas

for improvement. The experimental tasks were to

travel two kinds of routes, one is easy path (with 2turns) and other is complex path (with 5 turns).

6.1 Participants16 blind male students, ranging from 17 to 21

years old and unknown about place equally divided

in to two groups, learned to form the cognitive mapsfrom a virtual environment exploration. Participantsin first group used our locomotion interface (LI) and

participants in second group used keyboard (KB) toexplore the virtual environment. Each repeated the

task 8 times, taking maximum 5 minutes for each

trial.

6.2 ApparatusUsing Virtual Environment Creator, we

designed virtual environment based on ground floorof our institute AESICS (as shown in Figure 6),which has three corridors and eight

landmarks/objects. It has one main entrance.Our system lets the participant to form cognitive

maps of unknown areas by exploring virtualenvironments. It can be considered an application of

learning-by-exploring principle for acquisition of

spatial knowledge and thereby formation ofcognitive maps using computer-simulated

environment. Computer-simulated virtualenvironment guides the blind through speech bydescribing surroundings, guiding directions, and

giving early information of a turning, crossings, etc.

Additionally, occurrences of various events (e.g.

arrival of a junction, arrival of object(s) of interest,etc.) are signaled by sound through speakers or

headphones.

6.3 MethodThe following two tasks were given to

participants:

Task 1: Go to the Faculty Room starting from ClassRoom G5.

Task 2: Go to the Computer Laboratory starting

from Main Entrance.

Task 1 is somewhat easier than Task 2. Onesimple path, with only two turns, and other little bit

more complex, with five turns.Before participants began their 8 trials, they

spent a few minutes using the system in a simplevirtual environment. The duration of the practicesession (determined by the participant) was typically

about 3 minutes. This gave the participants enough

training to familiarize themselves with the controls,

but not enough time to train to competence, before

the trials began.

6.4 ResultTable 1 and 2 show that participants performed



8/71

reasonably well while navigating using locomotion

interface in both the paths.

Table 1: Avg. Number of Steps Taken for Each

Trial

Trial 1 2 3 4 5 6 7 8

LI EP 54 52 51 48 45 43 42 41LI CP 90 86 83 76 72 70 70 65

KB EP 58 57 55 54 52 50 51 49

KB CP 93 91 90 88 85 83 82 80

Table 2: Avg. Time (in Minutes) Taken for Each

Trial

Trial 1 2 3 4 5 6 7 8

LI

EP

2.4 2.2 2.1 1.8 1.7 1.5 1.4 1.2

LICP

4.2 4.1 3.9 3.4 3.1 2.9 2.7 2.3

KBEP 2.8 2.7 2.5 2.5 2.4 2.2 2.1 2.1

KB

CP

4.6 4.5 4.3 4.3 4.1 3.9 3.8 3.6

On first path condition, task was completed on

average with fewer than 41 steps. While in complexpath condition, task was completed on average with

fewer than 65 steps. Average time was less than 1.2

minutes for easy path and 2.3 minutes for complexpath.

Participants performed relatively not good whilenavigating using keyboard in both the paths. On first

path condition, task was completed on average with49 steps. While in complex path condition, task was

completed on average with 80 steps. Average time

was less than 2.1 minutes for easy path and 3.6minutes for complex path.

Avg. Number of Steps taken

0

10

20

30

40

50

60

70

80

90

100

1 2 3 4 5 6 7 8

Trial Number

Avg.Num

berofSteps

LI EP

LI CP

KB EP

KB CP

Figure 7: Avg. Number of Steps taken for two

different paths using LI and KB

Avg. Time (Minutes) taken to complete tasks

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

1 2 3 4 5 6 7 8

Trial Number

A

vg.Tim

e

(in

M

inutes)

LI EP

LI CP

KB EP

KB CP

Figure 8: Avg. Time (in Minutes) for two differentpaths using LI and KB

Above figures show that locomotion interfaceusers reasonably improved their performances (time

and number of steps taken) over the course of the 8trials. However, time required during initial trials

would reduce significantly after 3 trials. To stabilize

the performance users may need 4 trials or more.User comments support this understanding:

The foot movements did not become natural until

4-5 trials with LI.The exploration got easier each time.I found it somewhat difficult to move with the LI.

As I explored, I got better.

Even after the 8 trials of practice, LI users still

reported some difficulty moving and maneuvering.These comments point us to elements of the

interface that still need improvement.

I had difficulty making immediate turns in the

virtual environment.

Walking on LI needs more efforts than realwalking.

7 CONCLUSION AND FUTURE WORKThis paper presents a new concept for a

locomotion interface that consists of a one-

dimensional passive treadmill mounted on a

mechanical rotating base. As a result the user canmove on an unconstrained plane. The novel aspect is

sensing of rotations by measuring the angle of foot

placement. Measured rotations are then convertedinto rotations of the entire treadmill on a rotary base.

The proposed device although is of limited size but itgives a user the sensation of walking on an

unconstrained plane. Its simplicity of design coupled

with supervised multi-modal training facility makesit an effective device for virtual walking simulation.

Experiment results indicate the pre-eminence oflocomotion interface over method of using keyboard

for virtual environment exploration. These results

have implications for using locomotion interface forthe visually impaired to structure the cognitive maps

of an unknown places and thereby to enhance themobility skills of them.



9/71

We tried to make a simple yet effective, loud-

less non-motorized locomotion device that helpsuser to hear the audio guidance and feedback

including contextual help of virtual environment. Infact, absence of mechanical noise reduces the

distraction during training thereby minimizing the

obstructions in the formation of mental maps. The

specifications and detailing of the design were basedon the series of interactions with selected blindpeople. Authors do not intend to claim that their

proposed device is the ultimate one. Howeverlocomotion interfaces have the advantage of

providing a physical component and stimulation of

the proprioceptive system that resembles the feeling

of real walking.

We do feel that the experimental results lead toimprovements in the device to become more

effective. One known limitation of our device is itsinability to simulate movements on slopes. We planto take up this enhancement in our future work.

ACKNOWLEDGMENT

We acknowledge Prof. H. B. Daves suggestions at

various stages during our studies and work leading

to this research paper.

8 REFERENCES[1] Brooks, F. P. Jr., (1986). Walk Through- a

Dynamic Graphics System for Simulating

Virtual Buildings. Proc. Of 1986 Workshop on

Interactive 3D Graphics, pp. 9-21.[2] Clark-Carter, D., Heyes, A. & Howarth, C.,

(1986). The effect of non-visual preview uponthe walking speed of visually impaired people.

Ergonomics, 29 (12), pp.157581.

[3] Darken, R. P., Cockayne, W.R., & Carmein, D.,(1997). The Omni-Directional Treadmill: A

Locomotion Device for Virtual Worlds. Proc. ofUIST97, pp. 213-221.

[4] De Luca A., Mattone, R., & Giordano, P.R.(2007). Acceleration-level control of theCyberCarpet. 2007 IEEE International

Conference on Robotics and Automation,Roma, I, pp. 2330-2335.

[5] Earnshaw, R. A., Gigante, M. A., & Jones, H.,editors (1993). Virtual Reality Systems.

Academic Press, 1993.[6] Hirose, M. & Yokoyama, K., (1997). Synthesis

and transmission of realistic sensation usingvirtual reality technology. Transactions of theSociety of Instrument and Control Engineers,

vol.33, no.7, pp. 716-722.

[7] Hollins, M. (1989). Understanding Blindness:An Integrative Approach, chapter Blindness and

Cognition. Lawrence Erlbaum Associates, 1989.

[8] Hollerbach, J. M., Xu, Y., Christensen, R., &Jacobsen, S.C., (2000). Design specifications forthe second generation Sarcos Treadport

locomotion interface. Haptics Symposium,Proc. ASME Dynamic Systems and Control

Division, DSC-Vol. 69-2, Orlando, Nov. 5-10,

2000, pp. 1293-1298.

[9]

Iwata, H. & Fujji, T., (1996). VirtualPreambulator: A Novel Interface Device forLocomotion in Virtual Environment. Proc. of

IEEE VRAIS96, pp. 60-65.[10]Iwata, H., Yano, H., Fukushima, H., & Noma,

H., (2005). CirculaFloor, IEEE Computer

Graphics and Applications, Vol.25, No.1. pp.

64-67.

[11]Iwata, H, Yano, H., & Tomioka, H., (2006).Powered Shoes, SIGGRAPH 2006 Conference

DVD (2006).[12]Iwata, H, Yano, H., & Tomiyoshi, M., (2007).

String walker. Paper presented at SIGGRAPH

2007.[13]Iwata, H. & Yoshida, Y., (1997). Virtual walk

through simulator with infinite plane. Proc. of2nd VRSJ Annual Conference, pp. 254-257.

[14]Iwata, H., & Yoshida, Y., (1999). PathReproduction Tests Using a Torus Treadmill.PRESENCE, 8(6), 587-597.

[15]Lynch, K. (1960). The image of the city.Cambridge, MA, MIT Press.

[16]Lahav, O. & Mioduser, D., (2003). A blindperson's cognitive mapping of new spaces using

a haptic virtual environment. Journal of

Research in Special Education Needs. v3 i3.172-177.

[17]Noma, H. & Miyasato, T., (1998). Design forLocomotion Interface in a Large Scale Virtual

Environment, ATLAS: ATR Locomotion

Interface for Active Self Motion. 7th Annual

Symposium on Haptic Interfaces for Virtual

Environment and Teleoperator Systems. TheWinter Annual Meeting of the ASME.Anaheim, USA.

[18]Shingledecker, C. A. & Foulke, E. (1978). Ahuman factors approach to the assessment of

mobility of blind Pedestrians. Human Factors,vol. 20, pp. 273-286.

[19]Whitton, M. C., Feasel, J., & Wendt, J. D.,(2008). LLCM-WIP: Low-latency, continuous-

motion walking-in-place. In Proceedings of the3D User Interfaces (3DUI 08), pp 97104.



10/71

APPLICATION OF VIRTUAL REALITY TECHNOLOGIES IN RAPID

DEVELOPMENT AND ASSESSMENT OF AMBIENT ASSISTED

LIVING ENVIRONMENTS

Viveca Jimenez-Mixco, Antonella Arca, Jose Antonio Diaz-Nicolas, Juan Luis Villalar , MariaFernanda Cabrera-Umpierrez, Maria Teresa Arredondo, Pablo Manchado, Maria Garcia-Robledo

Life Supporting Technologies, Technical University of Madrid, Spain

[email protected]

SIEMENS S.A., Spain

ABSTRACTIn the current society, where the group of elderly and people with disabilities is

constantly growing, especially due to the increase in life expectancy, it is

becoming a must for ICT developers to provide systems that meet the needs of this

community regarding accessibility and usability and enhance their quality of life

consequently. Ambient Assisted Living, intended to help people live

independently, with autonomy and security, is one of the most promising solutions

that are coming up to address this technological challenge. This paper presents theapproach proposed in the context of VAALID European funded project to make

possible real rapid prototyping of accessible and usable Ambient Intelligence

solutions, by integrating Virtual Reality simulation tools in the development cycle

as well as appropriate user interfaces. The first functional prototype has been

planned for March 2010 and will be evaluated during six months in three pilot sites

with up to 50 users, starting on May 2010.

Keywords: Virtual reality, ambient assisted living, rapid application development,

assessment, accessibility, usability.

1 INTRODUCTIONNowadays Society is facing a process where life

expectancy is gradually but constantly increasing. As

a result, the group of elderly people is growing to

become one of the most significant in the entire

population [1]. This also means that the prevalence of

physical and cognitive impairments is increasing in

proportion. Elderly people usually suffer from vision

deficiencies (yellowish and blurred image), hearing

limitations (especially at high frequencies) motor

impairments (for selection, execution and feedback)

and slight deterioration of their cognitive skills [2]. In

this context, providing the elderly and people with

disabilities with accessible systems and services that

could improve their level of independence, and thus

enhance their quality of life, has become a must for

ICT developers such as usability engineers and

interaction designers. Ambient Assisted Living (AAL)

is one of the solutions that are beginning to address

this technological challenge.

The concept of Ambient Assisted Living

represents a specific, user-oriented type of Ambient

Intelligence (AmI). It comprises technological and

organisational-institutional solutions that can help

people to live longer at the place they like most,

ensuring a high quality of life, autonomy and security

[3]. AAL solutions are sensitive and responsive to the

presence of people and provide assistive propositions

for maintaining an independent lifestyle [4].

Within this complex and continuously evolving

framework, it is very challenging to technologically

meet all users needs and requirements regarding

accessibility and usability along the development

process. Accessibility is a prerequisite for basic use of

products by as many users as possible, in particular

elderly persons and persons with sensory, physical or

cognitive disabilities. Usability denotes the ease with

which these products or services can be used to

achieve specified goals with effectiveness, efficiency

and satisfaction in a specified context of use [5].

These aspects should be taken into account during the

product design ideally from early stages, following a

more interactive and iterative design-development-

testing procedure. The major problem lies in the

global cost of the design and development process,

which can be critically increased, since AmI solutions

involve complex features such as ubiquity, context

awareness, smartness, adaptiveness and computing

embedded in daily life goods.

Life Supporting Technologies, the research group

responsible of this paper, has been addressing for

years the convergence of domotics and accessibility.

As a result of this process, the group is exploring the



11/71

application of Virtual Reality (VR) technologies in the

process of design and development of accessible

solutions for elderly and people with any kind of

disability. One of the achievements in this area was

the establishment of a living lab at the Technical

University of Madrid that allowed the assessment of

the user experience of people with disabilities in smart

homes using two key technologies: virtual reality and

domotics [6].

The living lab integrated a VR application into a

real smart home installation. It was configurable for

different settings and user profiles, and capable of

supporting multimodal interaction through a set of VR

and other commonly used devices and displays. The

design and implementation process ran under the

Design-For-All principles, taking into account

concepts such as usability, adaptability, multimodality

and standardisation. The living lab resulted in a useful

tool for interaction designers and usability engineers

to immerse users in a virtual environment and assess,

through the application, their experience in terms of

interaction devices, modalities and reactions within

smart home environments. Based on this assessment,

designers would be able to develop new concepts with

users, improve existing solutions, and explore, forinstance, the possibilities of innovative AAL products

and services.

The preliminary encouraging results allowed

envisioning multiple possibilities of VR on the

process of providing people with disabilities with

more adapted access to domotic-related applications.

However, this solution had important limitations,

especially as it required a significant amount of

implementation effort to finally address the

assessment of user experience in just one single

environment integrating a pre-defined set of products

and services.This paper presents an approach proposed in the

context of the European funded project VAALID that

extends the key concepts applied in this living lab,

providing an easier method to create virtual

environments and implement interactivity, enabling

dynamic changes of environment conditions and

characteristics, and allowing a thorough evaluation of

users and real-time interaction techniques. An

authoring tool will be developed in order to enable

real rapid prototyping and validation of accessible and

usable AmI solutions, by integrating Virtual Reality

(VR) tools and appropriate user interfaces. This

approach will bridge the gap between planning AmI

scenarios and their build-up and assessment in realityfrom the very beginning in the development process,

reducing the global design and development effort.

2 VAALID CONCEPTVAALID is a European research project that aims

to develop advanced computer-aid engineering tools

that will allow ICT developers, especially those ones

that design AAL products and services, to optimise

and make more efficient the whole process of user

interaction design and to validate usability and

accessibility at all development stages, following a

User Centred Design (UCD) process.

The VAALID platform will utilise VR

technologies to provide an immersive environment

with 3D virtual ambient, specifically created for each

possible use scenario, where AAL users can

experience new interaction concepts and techno-

elements, interactively. The usage of VAALID tools

will make feasible, both economically and technically,

the Universal Design of AAL solutions which have

the potential of being acceptable by most persons

since their needs are taken into account proactively

during the development phases.

The methodology proposed to address AAL

solutions is based on a UCD approach, drawing

together the practical, emotional and social aspects of

people's experience and bringing on the needed

innovation that delivers real user benefit. For that

reason, the UCD is particularly useful when a new

product or service is to be introduced, as it is the case

of AAL solutions. The methodology consists of four

iterative phases of design, development and evaluation,where both usability engineers and interaction

designers must participate, involving AAL users (i.e.

elderly and people with disabilities) all along the

process [7]:

Concept. First, AAL solution requirements mustbe extracted, including the functions that the

proposed solution provides and how it reacts and

behaves, as well as the constraints that should be

considered in the design process.

Design. Once the requirements are well identified,developers define the specifications of the AAL

solution, taking into account all significant facetsthat may have influence on the development

process. Low-fidelity virtual prototypes of the

AAL solution, including 3D virtual AAL-enabled

spaces, are built to reflect all aspects of the

conceptual design, and further evaluated by users.

Design iterations are driven by users feedback in

terms of acceptance and accessibility issues until

requirements are met.

Implementation. This phase involves the creationof real and fully functional high-fidelity AAL

solution prototypes, with the aim of transforming

the validated conceptual design into a concrete and

detailed solution. The components developed at

this stage must be tested against its accessibilityfeatures, and improvements or corrective actions

must be addressed accordingly.

Validation. Finally, the implementation of AALsolution prototypes is evaluated and assessed,

detecting usability issues both automatically and

with potential end users.

This methodology allows virtually simulating



12/71

each aspect of an AAL product/service and validating

it before the real implementation. The whole process

involves both virtual and mixed reality elements. The

simulation in the design phase requires mainly 3D

virtual environments to reproduce the conceptual

design of the solution; the implementation phase goes

a step further and adds the possibility to use mixed

reality elements, so that real functional prototypes can

be tested within virtual environments as well.

In order to permit developers to apply this

methodology across all the stages of the design cycle,

and thus make possible a rapid development of AAL

solutions and further assessment with users, the

VAALID platform will be structured in two parts: the

Authoring Framework and the Simulation Framework.

The Authoring Framework will provide the ICT

designer with the appropriate components to deal with

the three main pillars of an AAL solution, including

the creation of user profiles, the modification of AAL-

enabled 3D spaces (including sensors, communication

networks and interaction devices and functions), the

creation of virtual user-interaction devices (which may

be embedded in daily life objects) and new concepts

for devices and products. These individual

components will be afterward validated as anintegrated environment in the Simulation Framework.

The VAALID project started on May 2008 and

the first functional prototype of the VAALID platform

is planned for March 2010. This prototype will be

evaluated during six months in three pilot sites

(Germany, Italy, Spain) with up to 50 users, starting

on May 2010.

2.1 Target UsersVAALID target users can be divided into three

main groups: Primary users: Designers of AAL solutions that

will use VAALID as a professional instrument.

This group includes Interaction Engineers, who

design the structure of the simulation, building the

seniors profile and defining the interaction modes

with the environment, and Usability Engineers,

who plan the interface among AAL services and

senior citizens, through the study of their

interactions with the VAALID system.

Beneficiaries: The main target group of users whowill benefit from the results of using VAALID

tools. They will be:

Elderly people over 60 years old that mayhave light hearing/sight problems, mobilityimpairments, or the normal declined cognitive

and physical abilities related to age.

Young people with hearing/sight/mobilityproblems, or

Any other group of users that may profitfrom accessible AAL solutions.

Secondary users: All those users that may benefitindirectly from VAALID, using it as a consultancy

service. They are:

Architects, construction planners, carecentres, suppliers of interaction devices, public

administration, interior designers and other

stakeholders who work for companies that buy

and develop AAL services.

System designers, who implement AALsolutions validating usability and accessibility of

their products, like sensors, actuators or control

software.

2.2 Sample ScenarioThe potential use of VAALID can be illustrated

through the following simplified scenario: A small

company specialised in AAL wants to develop a

service for detecting fall of elderly people when they

are alone at home; if a fall is detected, an alarm is

generated and automatically sent to an emergency

centre.

Following the VAALID approach (see Fig. 1), an

interaction designer creates first a new project in the

Authoring Framework.

Figure 1: Development cycle proposed in VAALID.

He selects the user profile of a person over 80

with moderate hearing problems, and VAALID

automatically limits the possible elements and features

consistent with that profile. He imports an AutoCAD

model of a house, previously created in an architect

studio for the company, and adds to the 3D model all

the sensors and objects that will be involved. He also



13/71

selects from the libraries the service Fall down and

redesigns this model adding all the needed elements

for the service to work properly. In this case, he

decides to embed the sensors in a carpet in each room

of the house. By running the simulation in the

Simulation Framework he can check whether the

service has been correctly defined: the service

workflow is coherent, the sensors involved are placed

correctly around the house, all the features are defined

in accordance with the user profile, etc.

Now, the designer requires a real user to test the

service in a realistic environment to gather his opinion.

In the simulation room, which has been equipped with

specialised VR technologies, they use specific glasses

to get immersed in the virtual scene of the house.

Among the different options available in the

simulation room, the designer decides that the easiest

way for the user to simulate movements is body

gesture. After a short training, the user is capable of

moving around and interacts with the house. He lies

down in the floor of the simulation room to simulate a

fall, and therefore he can experience what would

happen in case he had really fallen down, and how the

alarm service would react. He asks the designer to

change the dimensions and the position of the carpet,and to reduce the time that the system should wait

before launching the alarm. The designer sets the new

preferences of the user in real-time.

At this point, the service is being simulated in a

3D environment with virtual elements; afterwards,

once the concept is fully defined and the prototype of

the smart carpet is created, it can be assessed in a

more realistic approach through a mixed reality

environment. This means that the carpet can be taken

out of the virtual scene, and instead, the real prototype

is tested by the user at the same simulation room. Thus,

enabling the simultaneous usage of virtual and realelements, the service can be validated before the

construction of a real living lab.

Several scenarios describing similar possible

situations were examined by experts from different

profiles, including interaction designers and usability

engineers, and their impressions and recommendations

regarding the main aspects of the VAALID concept

such as working with elderly, 3D and virtual reality

technologies have been taken into account for the final

definition of the characteristics and functionalities of

the Authoring and Simulation Frameworks.

2.3 Authoring FrameworkThe Authoring Framework [8] is a tool created for

interaction designers and usability engineers. Its main

objective is to support them to build the core element

that composes an AAL service simulation context.

The appearance of the Authoring Tool is based on the

look and feel of Eclipse (centre stage, properties tab,

project browser, etc.) so that an intuitive interface

helps the developer to rapidly create the virtual

environment where the user moves for tests. It can be

personalised and configured to fit the needs of each

designer, providing also a help section.

According to the RAD (Rapid Application

Development) methodology [9], this tool allows to

create a model containing all those templates that will

be integrated and then executed inside of the

Simulation Framework. The AAL simulation is

created from a conjunction of templates stored in a

project, the basic component of the Authoring

Framework. Every simulation is stored as a single

project that is composed of three elements: User

Model, Environment, and AAL Service. Each of these

elements is created by editing pre-existing

characteristics described as properties and behaviour.

Properties are defined through ontologies that

represent static features of a single model; behaviours

are described as workflows of the element in relation

with other elements by means of interaction. Through

this kind of information the designer can build models

in a rapid way following user needs.

2.3.1Authoring ToolkitThe Authoring Framework workspace is divided

in three editors, one for each model (Fig. 2):

User Model Builder. The term User here isreferred to the beneficiaries of VAALID, i.e.

elderly or people with disabilities people. This user

editor defines the user profile including physical,

sensory and cognitive abilities. This kind of

information is collected during the design and

testing phases when creating AAL services.

Functions implicated in this builder are: creating a

new User Model from scratch; importing or

exporting an existing User Model, by exchanging

profiles between the current Project and theLibrary (or Repository); and removing the User

Model associated to the current Project. The same

actions are available for the Behaviour of a User

Model, which can be imported, removed, exported

or associated to another User Model.

Environment Model Builder. The EnvironmentModel reproduces a standard real place with a

series of properties. This editor allows developing

the 3D simulation environment where users can be

immersed, like in a real assisted world, and try

new interaction modes and new (virtual)

interaction devices. Pre-existing 3D models can be

used to compose an Environment Model: common

objects (including rooms, furniture or in generalarchitectural elements), interaction devices (like

sensors and actuators) and complex devices (a

combination of the previous ones). Objects are

characterised by their properties; interaction

devices have also a behaviour. Complex devices

have the same characteristics of an interaction

device but are represented by a set of related

sensors and actuators, targeted to a unique and



14/71

specific function, as a single composite device. An

innovative feature is the possibility to browse

among existing objects within the VAALID

Library, allowing refining and reusing components,

starting from a CAD program or a 3D animation

tool which export objects as VRML or X3D files.

To make the simulation more realistic the designer

can make some minor modifications in dimensions

and positions of the objects inside a scene.

Similarly to the User Model, environments and

objects are composed by properties and behaviour,

and can be imported, exported, retrieved from the

Library and edited through graphical metaphors.

AAL Service Compositor. This tool is an editor forthe creation of an AAL Service Model, which is

mainly described as a workflow, providing links

between user and objects of the scene. It

essentially acts as a controller that processes

information coming from sensors, triggered by

explicit or implicit user actions, and consequently

activates relevant actuators (i.e. security systems,

lighting, heating/air conditioning), consistently

with the service specifications.

Figure 2: Authoring Framework scheme.

Three data layers are handled and exchanged for

most of the modelled elements:

Representation: graphical components that permitvisualisation of each element and interaction with

the designer.

Instance: structure of classes that holds the actualelement model and allows its management by Java

modules.

File: raw data that keep the element descriptionwhen stored in a drive or the library.

Once created, every model can be exported to the

VAALID Repository for reuse in further projects. Thisway the Authoring Framework gives the possibility to

have an increasing amount of models to use in

different simulations or execute many variants of the

same simulation. Finally, the Project Editor integrates

the three tools for editing models of user, environment

and AAL service in a common framework in order to

manage a single simulation.

2.3.2Authoring Implementation FactsAccording to the software architecture defined,

each tool works using collaborative modules,

managing and sharing pieces of software. The usage

of Eclipse RCP (Rich Client Platform) is a step

forward towards the implementation phase. This

particular distribution includes the subset of

components which are natively used to construct the

own Eclipse framework. In this sense, client

applications developed under Eclipse RCP share the

same software infrastructure of Eclipse, taking profit

from advanced built-in functionalities such as:

Native visual elements of the Eclipse deploymentplatform.

Perspective management, enabling differentsoftware views sharing the same data model.

Plugin-based architecture, facilitating versioncontrol and modular development.

Auto-update functionality that facilitates softwaremaintenance.

Integrated high-quality help files.Project Editor

User

Model

Builder

Environment

Model

Builder

AAL

Service

Compositor

3D Model

Manager

Ontology

Manager

Workflow

Manager

VRML

Parser

Ontology

Parser

Workflow

Parser

Representation

Instances

Files

Repository

File Manager

Figure 3: Authoring Framework modules diagram.

These capabilities enable certain advanced

capabilities of the VAALID user interface concept,

like the usage of perspectives to facilitate seamless

transition between Authoring and Simulation

Frameworks as well as to access content through

different views and levels of detail (e.g. object

browsers, flexible lists, 2D/3D floor plans), dependingon user preferences and expertise. Individualisation of

screen layout is also possible because RCP exploits

the native potential of the same visual components of

Eclipse.

Regarding the multi-developer condition of the

VAALID software, the RCP architecture based on

plugins allows modular independence among

implementation teams, considering each plugin as an



15/71

additional element of the final software framework.

The auto-update feature will help in this modular

approach, assisting in the adoption of updated plugin

versions as soon as they are released. The integrated

help infrastructure will make possible a low-effort

extra support for VAALID designers.

As shown in Fig. 3, each editor in the Authoring

Framework is composed of two main parts: Element

Manager (Ontology Manager, Workflow Manager and

3D Model Manager) and Element Parsers (Ontology

Parser, Workflow Parser and VRML Parser). The

Element Manager performs the translation between

instances and graphical representations, while keeping

in memory the actual model of the elements.

Particularly the Ontology Manager holds a more

relevant role since it acts as a kind of overall

controller, calling the other elements managers when

required. The Element Parser is responsible for

converting instances to files and vice versa, verifying

that each element maintains a convenient format. To

end with, it is remarkable that, according to the overall

VAALID architecture, the Authoring Framework

shares the same instance structure and memory with

the Simulation Environment, in particular with the

Simulation Control Panel. This assures seamlesstransition and permanent data consistency between

both frameworks.

2.3.3Viewing 2D/3D SpacesOne of the most innovative features of VAALID

is the integration of 3D technologies in the Authoring

Framework so as to dynamise and smooth the progress

of designing and evaluating AAL services. In addition

to the 3D view of the floor plan, the Authoring

Framework provides also a 2D view in which it is

possible to select objects and have a clearer idea ofdistances and orientation of all those elements that are

present in the scene. Selections are synchronised so as

the system automatically performs the changes in both

views.

The Eclipse RCP platform provides some

functionalities to facilitate 3D management. The use

of perspectives and views permits immediate changing

between 2D and 3D floorplans sharing the same data

model imported from the original VRML file. The

actions/views mechanisms enable direct manipulation

of objects from the environment taking into account

different selection sources (browser, flexible list,

floorplans, workflow editor, history lists, etc.). The 3D

Model Manager supports 3D rendering and navigation,allowing rotation, zoom and tilt within the user view,

while detecting object collision.

2.4 Simulation FrameworkOnce the individual elements are defined, the

process of creation of experimental AAL

environments needs a testing and assessment phase.

Thus, apart from a core set of technologies and

software building components, there is a need [10] of

appropriate facilities that offer the possibility of:

Testing different technical solutions from the

point of view of their overall usefulness to users.

Providing a common environment for testing

cooperative activities and virtual spaces.

Usually, testing ambient behaviour and interaction

is only possible in real laboratories. The innovation of

this approach is that it will be possible to test and

assess AAL scenarios, products and services across all

the development process in virtual environments,

before experimenting in real contexts.

The models (service, user and environment),

previously defined in the Authoring Framework, are

put together and run in the Simulation Framework

during the different stages of the development.

Simulations provide feedback to developers about the

accessibility, usability and user acceptance of the

human-environment interaction. The Simulation

Framework is composed of two main tools (Fig. 4):

the Simulation Control Panel, which allows

developers to configure and run the simulations, and

the 3D engine or AAL Services interaction simulator,which is a renderer for the 3D scenes, based on Instant

Reality system. Both of them communicate with a

workflow engine, which is in charge of executing all

the workflows related to a simulation.

Figure 4: Simulation Framework scheme.

There are two types of simulation-validation tests

that engineers can perform: A first type is done with virtual users. These are

models of users defined within the Authoring

Framework, and characterised by behaviour

models. This phase of assessment is important for

the integration of the different interactionmodalities, since it allows definition and

refinement of the behaviour model in any stage of

the design process. Engineers can check

constraints that state incompatible values for

specific properties of the different elements

defined in the AAL scenario.

The second type involves real users in animmersive environment (3D virtual environment).

Users will be allowed to experience real-time



16/71

interaction with an AAL environment using both

virtual and tangible interaction devices. A virtual

interaction device can be a sensor or an actuator

represented in the 3D virtual environment; a

tangible device (or simulation control) is physical

equipment that enables interaction between the

user and the virtual environment. The feedback

from real users to designers will be critical in the

process to meet their specific needs and

requirements. At the moment, the project is

exploring the feasibility of integrating several

simulation controls to the platform, such as

Nintendo Wii Remote, Intersense Head Tracking,

LED-based Gloves, Visual Hand Control or

Android Mobile Phone. These controls will be

extensively assessed during the pilot tests, with the

aim of finding the most adapted solution for each

user.

The possibility of performing these assessment

phases during the design process of AAL solutions,

before building up real living labs, has key benefits

such as saving of time and costs. In addition, users can

participate in a controlled environment, since VR

technologies assure safe and secure interaction. Thisdoes not mean that evaluation in a real living lab has

to be avoided, but that any further interaction

experiment will be enriched by the results obtained in

the preliminary design process.

2.4.1Study Case: Using an Android Mobile PhoneAs stated before, VAALID aims at providing VR-

founded tools that make easy the process of designing

accessible solutions for ambient intelligence

environments. The objective is to allow engineers to

pre-validate innovative services with final users in arealistic setting using virtual scenarios, as a first filterbefore the actual validation in living labs. One

important step in the investigation is the testing of

different interaction devices in order to test the

immersion feeling of users in joining the simulation.

Figure 5: Testing VR using a smart phone.

Taking advantage of the flexibility of Instant

Reality and the multimodal characteristics of the new

generation of smart phones, a special setting was

prepared to perform some technical and usability tests

[11]. Several engineers were told to explore and

interact with a 3D scene using an Android-based

mobile device (i.e. HTC Magic smart phone),

analysing the execution of some pre-defined tasks,

such as moving around, finding objects or grab a book.

After considering different approaches,

multimodal user interaction was defined using the

handheld device as follows, focusing on haptic

interfaces (Fig. 5):

Device rotation (i.e. forwards, backwards,clockwise and counter-clockwise): performs 3D

movements within the virtual environment

(respectively: advance, retreat, turn right and turn

left).

Finger dragging over touchscreen: performshorizontal movements of the virtual pointer.

Trackball rotation: performs vertical movements ofthe virtual pointer.

Trackball click: sequentially picks up/releases aparticular virtual object.

Vibrator: provides vibration feedback to the userwhen the virtual pointer collides with the virtual

object.

Considering the collected data, preliminary

results show that users feel comfortable in using the

device and defined the experience as realistic,

although there are valuable suggestions to improve the

interaction (e.g. allow sensitiveness calibration). From

a technical point of view, this can be taken as a good

starting point for future work with VR-based

applications, although further research is required

concerning its suitability for elderly users.

3 DISCUSSION AND CONCLUSIONAccessibility and usability concepts are currently

considered within a limited range of ICT applications

and services, mostly constraining its usage to research

and development activities and presenting significant

reservations when dealing with production and

deployment phases. Although the seven principles of

the universal design or Design for All [12] are well

known and applicable to a wide variety of domains,

business stakeholders are still highly reticent to apply

them in practice. This lack of commitment with the

elderly and disabled community, in particular whendesigning AAL solutions is mainly due to the high

costs involved in the iterative design-development-

testing procedure and the considerable time effort

needed to meet users needs.

On the other hand, the adoption of VR

technologies seems to confront with the purpose of

designing services for people with disabilities, as few

initiatives have been carried out in this field regarding



17/71

accessibility requirements. Most of them deal with

people with cognitive disabilities (dementia, autism,

schizophrenia, Down's syndrome, etc.), proposing

simple virtual worlds where users get immersed in

order to learn some tasks, acquire some habits or

recover some capabilities under a controlled scenario.

Nevertheless, VR has been proven to offer significant

advantages for persons with all kinds of disabilities. It

can present virtual worlds where users can be trained

or learn in a controlled environment, and then apply

the skills acquired to a real context. VR technologies

can be adapted to a wide range of users and needs, and

at the same time, users abilities and experience can be

assessed in order to reach an optimal adaptation

The work proposed in this paper brings together

all these issues into a technological approach that will

have a beneficial impact for all the involved parts: The

ICT designer will be able to evaluate the suitability of

the proposed solutions with a significant reduction of

the global design and development effort; business

stakeholders will have a cost-effective solution and

therefore new market opportunities, and finally, end-

users will be provided with new services to improve

their quality of life, and even better, they will be able

to active and critically participate in the process ofcreation of these services.

ACKNOWLEDGMENTS

This work has been partially funded by the

European Union in the context of the VAALID project

(ICT-2007-224309), coordinated by SIEMENS S.A.

The project started in 1st

May 2008, and will finish in

31st October 2010. The VAALID consortium is

composed of the following partners: SIEMENS S.A,

ITACA, Fh-IGD, UNIPR, VOLTA, UID, SPIRIT and

UPM.

4 REFERENCES[1] K. Giannakouris: Ageing characterises the

demographic perspectives of the European

societies. Eurostat, EUROPOP2008 convergence

scenario.

[2] Lillo, J.; Moreira, H. Envejecimiento y diseo

universal. Anuario de Psicologa, 35, 4 (Tema

monogrfico: Psicologa y ergonoma), (2004).

[3] H. Steg, H. Strese, C. Loroff, J. Hull, S. Schmidt:

Europe Is Facing a Demographic Challenge

Ambient Assisted Living Offers Solutions.

Ambient Assisted Living European Overview

Report. (2006).

[4] B. de Ruyter, E. Pelgrim: Ambient Assisted-

Living Research in CareLab. ACM-interactions.

SPECIAL ISSUE: Designing for seniors. New

York (2007)

[5] K. Wegge, D.Zimmermann: Accessibility,

Usability, Safety, Ergonomics:Concepts, Models,

and Differences. Universal Access in HCI, Part I,

HCII 2007, LNCS 4554, pp. 294301, 2007.

Springer-Verlag Berlin Heidelberg (2007)

[6] V. Jimenez-Mixco, R. de las Heras, J.L. Villalar,

M.T. Arredondo: A New Approach for

Accessible Interaction within Smart Homes

through Virtual Reality. Universal Access in HCI,

Part II, HCII 2009, LNCS 5615, pp. 7581.

Springer-Verlag Berlin Heidelberg (2009)

[7] J.C. Naranjo, C. Fernandez, P. Sala, M.

Hellenschmidt, F. Mercalli.: A modelling

framework for Ambient Assisted Living

validation. Universal Access in HCI, Part II,

HCII 2009, LNCS 5615, pp. 228237. Springer-

Verlag Berlin Heidelberg (2009)

[8] VAALID Deliverable 3.1.: Authoring

Environment Functional Specification. May2009. http://www.vaalid-project.org/ Contract

number: ICT-2007- 224309

[9] H. Mackay, C. Carne, P. Beynon-Davies and D.

Tudhope: Reconfiguring the User: Using Rapid

Application Development. Social Studies of

Science, Vol. 30, No. 5 (Oct., 2000), pp. 737-757

[10] P. L. Emiliani, C. Stephanidis: Universal access

to ambient intelligence environments:

Opportunities and challenges for people with

disabilities. IBM SYSTEMS JOURNAL, VOL

44, NO 3, (2005).

[11] Arca, J. Villal, J. Diaz, M. T. Arredondo. "HapticInteraction in Virtual Reality Environments

through Android-based Handheld Terminals",

3rd European Conference on Ambient

Intelligence, AmI09, pp259-263. M. Tscheligi et

al.(Eds.): AmI09 Salzburg, Austria, 2009,

ICT&S Center, University of Salzburg, ISBN:

978-3-902737-00-7

[12] M. F. Story: Maximizing Usability: The

Principles of Universal Design, Assistive

Technology 10, No. 1, 412 (1998).



18/71

PIXAR ANIMATION STUDIOS AND DISABLED PERSONAGES.

CASE STUDY: FINDING NEMO

Jaume DuranUNIVERSITAT DE BARCELONA, Barcelona, [email protected]

David FonsecaGTM / LA SALLE - UNIVERSITAT RAMON LLULL, Barcelona, Spain

[email protected]

ABSTRACT

Many of the different characters that appear in the computer animated movies ofPixar Animation Studios are personages. From a dramaturgical point of view, these

can be linked with the concept ofArchetype. The same types of personages appear inall times and in all cultures. The universal patterns make it possible for theexperience to be shared in different histories. These patterns do not identify concreteidiosyncrasies, but they function as a temporary development in a story for thepurpose of enriching of it. Another way of interpreting the personages of a narrativehistory is to consider them as complementary facets of the main character (the hero).As the history develops, the characteristics of these personages modify thepersonality of the future hero. The aim of this work is to analyze the influence ofthese complementary personages in the transformation of the main character andexamine whether the presence of a disability is used to obtain this transformation.As we will see, not only do we find personages whose disability affects thedevelopment of the protagonist, but others that simply fulfill other secondaryfunctions. The base of the presented study are the seven first full-length films

produced by Pixar, but we will center on the particular case ofFinding Nemo.

Keywords: Computer Animation, Pixar Animation Studios, Dramaturgy

1 A METHODE: THE TRIP OF THE HEROChristopher Vogler [33] related the mythical

structures and their mechanisms to the art of writingnarrative works and scripts, after studying theproposals of Joseph Campbell [2], and Carl GustavJung [13, 14, 15]. To do so, he divided thetheoretical trip of the fiction hero in twelve stagesand enumerated up to seven archetypes.

According to Vogler, most histories arecomposed of a few structural elements that we alsofind in universal myths, in stories, in movies, andeven in sleep. In them, the hero, generally theprotagonist, leaves their daily environment toembark on a journey that will lead them through aworld full of challenges. It can be a real trip, with aclear destination and definite purpose, or it can bean interior trip, which can take place in the mind,heart or spirit. In any case, the hero ends upsuffering changes, and growing throughout.

There are twelve stages that compose this trip:

The Ordinary World: the first stage whenthe hero appears in their daily environmentand their ordinary world.

The Call to Adventure: the second stagewhen the hero will generally face a

problem and an adventure will appearbefore them.

The Rejection of the Adventure: the thirdstage when frequently the hero refuses thecall to action.

The Meeting with the Mentor: the fourthstage when the personage of the mentorappears.

The Passage of the First Threshold: thefifth stage when the hero begins theadventure.

Tests, Allied Forces, Enemies: the sixthstage when new challenges are revealed


19/71

and at the same time the hero is presentedwith new allies and hostile enemies.

The Approach to the Deepest Cavern: theseventh stage when the hero prepares astrategy for the definitive moment and getsrid of the last impediments before

continuing.

The Odyssey or the Calvary: the eighthstage when the hero directly faces whatthey are most afraid of and begins a tough,battle that could result in their own death.

The Reward (Obtaining the Sword): theninth stage when, having survived battleagainst death, the hero takes possession ofthe reward. For example, the sought-aftersword or treasure.

The Return of Comeback: the tenth stagewhen the hero suffers the consequences oftheir clash with the forces of evil and forobtaining the reward.

The Resurrection: the eleventh stage whenthe hero is facing the second big momentof difficulty, where again they risk losingtheir life and must overcome once again.

The Comeback with the Elixir: the twelfthand last stage when the hero returns to theordinary world with the obtained treasure.This ends the trip of the hero.

All these stages are parts of a scheme thatmodifies particular details according to the historyand does not need to adhere to the order with rigor.It is possible that some stages can be suppressedwithout affecting the history. These stages can bedivided into three dramatic acts (so thedevelopment of the history occurs in three parts,where the first part occurs before the target of theprotagonist is known by the spectator):

First act: the first five stages (1 to 5). Second act: the next four stages (6 to 9).

Third act: the last three stages (10 to 12).

During the heros trip, different personages canbecome present. Their mission can link with theconcept ofArchetype, which Carl Gustav Jung [13]uses avoiding the models of personality that arerepeated from remote times and that suppose aheredity shared for every human being. The sameauthor sums this up under the concept ofUnconscious Group.

The universality of the patterns and personagesmakes it possible for the experience to be shared indifferent histories, but these are not necessarily

concrete idiosyncrasies that have to be supportedfrom beginning to end. Rather, they are functions

that develop temporarily inside a story for thepurpose of enriching the history. Also, we caninterpret these complementary patterns as facets ofthe heros personality, which may affect what he oshe learns and what their values are.

There are seven common archetypes:

The Hero is someone capable ofsacrificing their own needs for the sake ofothers. The word hero comes from theGreek root word that means to protect andserve. Generally, we tend to identify withthe hero because he or she tends to have acombination of qualities and skills. Thehero is framed within a history, and insidethis narrative is where the personage learnsand grows.

TheMentoris the personage who helps orinstructs the hero. Mentor comes to us

from Homer [12]. In the Odyssey, thepersonage called Mentorhelps Telemac inthe course of his trip. Joseph Campbell [2]defines it as the wise elderor wise oldsterin reference to the personage who teaches,protects and provides certain gifts to thehero. Vladimir Propp [29] defines this typeof personage as the donor, in relation ofthe act of providing a gift or of offeringsomething to the hero.

The ThresholdGuardian is one of the firstobstacles the hero finds in their adventure.

Generally, they are neither the antagonistof the history nor the principal malefactor,although they constitute a threat that thehero, if he or she interprets it well, canovercome.

TheHerald, in a strict sense, is the personwho has a message. In Greece and Rome,they were the manager of dispensing theorders of the ruling classes, of making theproclamations and of declaring the war.

The Changeable Figure is a personagedifficult to identify because they make a

show of their name. Their appearance andcharacteristics change when we examinethem closely. In fact, the hero may findthem a changeable and variable personagewho possesses two faces. The changeablefigure develops the function of introducingdoubt and the suspense in the history.Often, this figure is the love of the hero.

The Shade is the antagonist personage, theenemy, the malefactor. The shadechallenges the hero and is a worthyopponent to fight.

The Trickster is the personage whocaptures the energies of wickedness and


20/71

desire for change. A buffoon, a clown, or acomical follower are all clear examples,and develop the function of a comicalmitigation.

2

THE TRIP OF THE HERO IN THE FULL-LENGTH FILMS OF PIXAR (1995-2006)

Leaving aside shorts films or publicityproductions, there have been seven computer-animated full-length films produced by PixarAnimation Studios (an independent producer beforebeing acquired by Walt Disney Company in 2006[8, 27, 28]):

Toy Story (1995) of John Lasseter [9, 20,31].

A Bugs Life (1998) of John Lasseter andAndrew Stanton [1, 17].

Toy Story 2 (1999) of John Lasseter, AshBrannon and Lee Unkrich [32].

Monsters, Inc. (2001) of Pete Docter,David Silverman and Lee Unkrich [21,25].

Finding Nemo (2003) of Andrew Stantonand Lee Unkrich [5, 10].

The Incredibles (2004) of Brad Bird [4,30].

Cars (2006) of John Lasseter and JoeRanft [3, 34].

All these scripts continue, undoubtedly anddespite particular exceptions or absences of certainstages and archetypes, the closed method of the tripof the hero. To summarize:

In Toy Story, in the room of a child called Andy(the first stage, the ordinary world), a rag doll withrope, a cowboy called Woody (the hero), is thefavorite toy. The arrival of a new plastic spaceranger doll with many gadgets, Buzz Lightyear(theherald and the changeable figure at the same time),

causes Woody to mistrust him, although in thebeginning this is not of importance. After a fight,the two toys get lost in a petrol station and end upin the hands of Sid (the shade), Andys evilneighbor. The mutant toys of Sid help Woody and

Buzz Lightyearavoid a fatal ending (eighth stage,the odyssey), and both manage to return to theirowner (twelfth and last stage, the comeback withthe elixir) after having overcome new troubles(tenth and eleventh stages, the way of comebackand the resurrection).

InA Bugs Life, in an anthill (the first stage, theordinary world), the threat of a few grasshoppers

led by the perverse Hopper(the shade), forces theant Flik(the hero, but also the culprit of the above

mentioned threat after having lost the meal that theywere giving the grasshoppers) to go on a journey insearch of help (fifth stage, the passage of the firstthreshold). After finding a small metropolis createdout of human garbage (sixth stage, tests, alliedforces, enemies), the protagonist knows a few artist

insects (the slickers) which he confuses as potentialwarriors. They do not notice the confusion eitherand go to the anthill. Despite the misunderstanding,they help Flik, the princess Atta (the changeablefigure) and the other ants defeat the tyrants.

In Toy Story 2, Woody (the hero once again) iskidnapped by a collector calledAl (the herald) aftertrying to rescue to the doll penguin Wheezy from ahome-made flea market where Andys mother lefthim. AtAls home, he meets another dolls, Pete, thehorse Bullseye and Jessie (the changeable figure).

Buzz Lightyear, Mr. Potato Head, Slinky, Hammand Rex (the friends ofWoody) go out in search of

him, but once they find him Woody decides toremain with his new relatives, even though he issoon cheated and persuaded by one of them, Pete.

Al takes them to the airport to travel to Japan.Nevertheless, Woody, with the help of his friendsagain, manages to escape the plane (eighth stage,the odyssey) and they all return to Andys room,this time also in company ofBullseye andJessie.

In Monsters, Inc., the monster Sully (the hero)and his best friend, the monster Mike (the slicker),are employed at a factory that scares children in thereal world in order to gather their screams, whichare used for energy. One day, a girl called Boo (the

herald) crosses one of the many doors that serve toconnect these two realities and ends up inside themonsters world. The girl is discovered by Sully,who calls on Mike to help him return her to herhome. As both try to arrange the situation, themonster Randall (the shade) puts manyimpediments in their way. Finally, the girl isreturned to her world (ninth stage, the reward) andSully comes up with the idea of gathering theguffaws and laughter of the children for energyinstead of their screams of fear.

In Finding Nemo, Marlin (the hero) is a clownfish who lives with his sonNemo in a coral reef (the

first stage, the ordinary world). One day, Nemo iscaptured by a scuba-diving dentist, andMarlin mustgo on a long journey to find him and bring himhome. He is accompanied part of the way by a bluefish called Dory (the slicker). Meanwhile, Nemomeets a few new friends in the fishbowl where hehas been deposited. After finding Nemo andreturning home (tenth stage, the way of comeback),a fishing ship catches Dory along with other fishesin its nets (eleventh stage, the resurrection). Nemodecides to help them and is successful, despite hisfathers doubts.

In The Incredibles, Bob Parr/ Mr. Incredible

(the hero) is a superhero who does not adapthimself well to a new reality (the first stage, the


21/71

ordinary world) in which resolving the problems ofhumanity is completely prohibited. After receivinga request to fight a dangerous machine on a farawayisland, he goes to see Edna Moda (perhaps, thementor) who arranged his old supersuit and nowmakes a new one for him. On the island, it turns out

thatBuddy Pine/Syndrome (the shade) has set a trapfor Mr. Incredible. But with the help of his wife,Helen Parr/Elastigirl, and two of his sons,Dashielland Violet, who also have superpowers, they foilthe plan. Back at home, Mr. Incredible seesSyndrome kidnapping his smallest son Jack-Jack(eleventh stage, the resurrection), but Jack-Jackuses his superpowers to escape and the evilSyndrome is finally defeated.

In Cars,Lightning McQueen (the hero) is a veryambitious race car that tries to win the Piston Cup.After a draw with two of his opponents, Chick

Hicks and The King, a new race is necessary to find

the Piston Cup winner. But in the trip to the nextcircuit, the protagonist becomes lost in a villagecalled Radiator Springs where he is forced toremain. There, he meets a few very particular cars(sixth stage, tests, allied forces, enemies). Aftercoexisting with them and experiencing manyvicissitudes, he gains some new values and changeshis perception on the competition. Finally, in thetiebreak race, he allows Chick Hicks to win andhelps The King to the finish line.

3 DISABILITY IN THE FULL-LENGTHFILMS OF PIXAR (1995-2006)

Disability can be seen essentially as a limitationprovoked by a physical or mental impediment thatprevents certain activities being carried out.According to the World Health Organization [36],this concept can affect the functions of the body asfollows:

The physiological functions of the systemsof the body, including psychological.

The structures of the body, includinganatomical parts such as organs and othercomponents.

Damages or problems in the function orstructure of the body, such as significantdeviations or loss.

Activity, including the execution of a taskor an action on the part of an individual.

The participation in a certain situation. Limitations to activity. Restrictions of participation in an activity. Exogenous factors that constitute the

physical or social manner and the attitudewith which the people live their lives.

Departing from the WHO point of view andunderstanding that many of the personages in thecomputer animated full-length films of PixarAnimation Studios are animals or objects withhuman behaviors, attributions or qualities, here wefind characters with all kinds of disabilities. But

these do not always coincide with the personagesmodel archetypes (which we indicate with ),which are necessary for the quest of the hero.

In Toy Story, Woody (the hero) is a cowboy dollwith only a voice box and a missing gun, while

Buzz Lightyear(the herald and the changeablefigure at the same time) is a plastic space rangerwith a multiphrase voice simulator, a laser light,and wings with light indicators, as well as manyother gadgets. Although, Buzz eventually realizeshe is a toy and cannot fly, his characterization atfirst shows up what Woody lacks. WhenBuzz findsout about his real existence, he loses an arm after

rushing through a gap from the top of a few stairs.Th

special issue of media solutions that improve accessibility to disabled users

Documents