applied virtual environments to support learning of social interaction skills in users with...
TRANSCRIPT
This article was downloaded by: [Universitat Politècnica de València]On: 27 October 2014, At: 17:48Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registered office: MortimerHouse, 37-41 Mortimer Street, London W1T 3JH, UK
Digital CreativityPublication details, including instructions for authors and subscription information:http://www.tandfonline.com/loi/ndcr20
Applied virtual environments to support learningof social interaction skills in users with Asperger'sSyndromeSue Cobb , Luke Beardon , Richard Eastgate , Tony Glover , Steven Kerr , Helen Neale ,Sarah Parsons , Steve Benford , Eileen Hopkins , Peter Mitchell , Gail Reynard & JohnWilsonPublished online: 09 Aug 2010.
To cite this article: Sue Cobb , Luke Beardon , Richard Eastgate , Tony Glover , Steven Kerr , Helen Neale , SarahParsons , Steve Benford , Eileen Hopkins , Peter Mitchell , Gail Reynard & John Wilson (2002) Applied virtual environmentsto support learning of social interaction skills in users with Asperger's Syndrome, Digital Creativity, 13:1, 11-22
To link to this article: http://dx.doi.org/10.1076/digc.13.1.11.3208
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) containedin the publications on our platform. However, Taylor & Francis, our agents, and our licensors make norepresentations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose ofthe Content. Any opinions and views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be reliedupon and should be independently verified with primary sources of information. Taylor and Francis shallnot be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and otherliabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Any substantial or systematicreproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in anyform to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Digital Creativity2002, Vol. 13, No. 1, pp. 11–22
1462-6268/02/1301-0011$16.00© Swets & Zeitlinger
Applied virtual environments tosupport learning of social interactionskills in users with Asperger’sSyndrome
Sue Cobb1*, Luke Beardon4, Richard Eastgate1, Tony Glover2,Steven Kerr1, Helen Neale1, Sarah Parsons3, Steve Benford2,Eileen Hopkins4, Peter Mitchell3, Gail Reynard2 and John Wilson1
1 VIRART / 2 School of Computer Science and Information Technology /3 School of Psychology, University of Nottingham, UK
4 National Autistic Society, UK
1. Introduction
Asperger’s Syndrome (AS) falls at the highfunctioning end of the autistic spectrum and ischaracterised by average or above averageintelligence, obsessive interests in specific topicsand adherence to routines. There may also bemarked deficiencies in social skills, non-verbalcommunication, abstract thought processes,theory of mind, executive functioning abilities,and imaginative play. Individuals with ASexperience inherent difficulties in interactionwith their peers, despite an often strong desireto do so. Due to poor abilities in recognisingand interpreting social cues such as turn-takingand difficulties in non-verbal communication,they often display behaviour that is generallyconsidered to be socially and/or emotionallyinappropriate (Frith 1989). For example,interrupting conversations, talking at lengthabout a specific subject irrespective of either therelevance to the situation or the response of thelistener, or pointing out factually correct butsocially inappropriate information. Individualswith AS tend to have a restricted range ofinterests to the exclusion of other activities.This can result in detailed knowledge of specificsubjects but very limited awareness of others.There is often repetitive adherence to proce-dures and routines, some of which may becompulsive (Baron-Cohen and Bolton 1993).
Abstract
Asperger’s Syndrome (AS) is an autistic spectrumdisorder characterised by normal to high IQ but withmarked impairment in social skills. Successful socialskills training appears to be best achieved either insitu or in role-play situations where users canexplore different outcomes resulting from theirsocial behaviour. Single user virtual environments(SVEs) provide an opportunity for users with AS tolearn social interaction skills in a safe environmentwhich they can visit as many times as they like.The use of game-like tasks can provide an incentiveand can also be used to guide the user throughprogressive learning stages. Collaborative virtualenvironments (CVEs) allow several users to interactsimultaneously within the virtual environment, eachtaking different perspectives or role-play characters.Within the AS interactive project a series of SVEsand CVEs have been developed in collaborationwith users and professional groups with an overallaim of supporting social skills learning. Initial evalu-ation studies have been carried out which have beenused to both inform and refine the design of thesevirtual environments (VEs) as well as giving aninsight into the understanding and interpretation ofthese technologies by users with AS.
Keywords: Asperger’s Syndrome, collaborativevirtual environments, social skills learning, virtualenvironments
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
Dig
ital C
reat
ivity
, Vol
. 13,
No.
1Cobb et al.
1 2
For example, an individual may becomedistressed if they have a specific routine theyfollow which is then interrupted or changed.Social skills tend to be learnt by rote and there islimited evidence of sustained generalisation ofunderstanding or behaviour between learnedand novel situations/environments (e.g. Pollard1998). People with autistic spectrum disorders(ASDs) are often impaired in understandinghow other people’s mental states influence howthey behave and what they say (e.g. Baron-Cohen, Leslie and Frith 1985). This cantranslate into a difficulty in understanding socialsituations because behaviour and language canseem confusing and unpredictable. Thisproduces a limited understanding of socialnorms and expectations, particularly in novelsituations. A tendency to interpret speechliterally (Mitchell, Saltmarsh and Russell 1997)can lead to misinterpretations of impliedmeaning of others and usage of formal pedanticlanguage which may be socially inappropriate.For example, an ironic or sarcastic commentmay be taken literally, without any meaninggleaned from the prosody used in its delivery; oran individual may interpret metaphoricalstatements in a literal sense. A statement suchas, “you must be quiet, you’re shouting thehouse down”, becomes very confusing for theindividual.
As a result, individuals with AS mayexperience social exclusion from their peers dueto difficulties in making and sustaining friend-ships. Adults often find it very difficult toobtain and maintain employment; even thoughthey may be capable of performing their workduties and may have specialised knowledge oftheir subject area. Problems with communicat-ing their ideas to, and socially interacting with,others in the work place can produce verydifficult working relationships (Howlin 1997).Specific problems encountered include difficul-ties with interviews, social interactions withinteams, group work and interacting with othersduring unstructured, social time such as breaks
and lunchtimes. For example, Klin andVolkmar argue that mastering specific jobdemands is only part of the challenge facingpeople with ASDs in work situations.
Equal attention should be paid to the socialdemands defined by the nature of the job,including what to do during meal breaks,contact with the public or co-workers, or anyother unstructured activity requiring socialadjustment or improvisation.(Klin and Volkmar 2000 361)
Moreover, they argue thatUnless issues of social presentation are adequatelyaddressed, including what to do in specificsituations such as lunch or free-time periods, thechances for vocational satisfaction are lessened.(Klin and Volkmar 2000 351)
The AS Interactive project aims toaddress some of these issues by constructingsocial scenarios within virtual environments(VEs), which can be used to learn about anddemonstrate social interaction skills in differentsituations. The perceived advantages of usingVEs for this purpose are two-fold. Firstly, VEtechnology allows construction of realistic 3Drepresentations of real world environmentswhich can be interacted with and explored inreal time. This means that we can createinteractive contexts representing a range ofsocial scenarios in which AS users can practisesocial skills. Furthermore, we can control theparameters within the environment (e.g.number of people, cues within the environment,number of solutions to a given problem, etc.).This allows us to control the number of avail-able options and therefore the decisions thatneed to be made by the user. Thus it is possibleto produce different levels for each scenario,involving simple choices to more complexdecisions concerning behaviour initiations andresponses. Secondly, the use of computertechnology may be specifically advantageous tothe AS community; virtual environments offer asecure environment in which they can learnrules of social interaction without having to deal
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
1 3
Digital C
reativity, Vol. 13, No. 1
Applied virtual environments …
with others face-to-face. Users can also practicein the same environment over and over againwithout pressure or concern for how theirbehaviour is received by others. This providesthe potential for repeated learning but also for‘learning by mistakes’. Users can examinealternative outcomes and canmove progressively throughlevels.
Of course, whilst thepossibilities for use of thistechnology may seem attrac-tive, there are many unknownsto this research. For example:what social skills can besupported by this technology?,how do we present these invirtual environments?, howimportant is it that we replicatereal situations in virtualenvironments and how‘realistic’ do these need to be?,how do we know that userswill understand and interpretthese VEs as we expect themto?, and how can we supportlearning of skills that can begeneralised and applied in thereal world?
The AS Interactiveproject is a three-year project funded by theShirley Foundation to develop single user (SVE)and collaborative (CVE) virtual environmentsfor adolescents and adults with AS to learnsocial skills relevant to social interaction inpublic situations, particularly work-basedsituations. The project has two distinct phasesof research. The initial focus has been con-cerned with feasibility investigation of the use ofvirtual environment technology for social skillstraining and establishing the involvement oftraining professionals and users in the designand review process. At the end of this phase wehave constructed an SVE and CVE where social
scenarios may be experienced in a café. Exam-ples of these VEs are given in the context ofdesign decisions and user evaluations of inter-face usability.
The AS Interactive project comprises amulti-disciplinary research team with expertise
in autism, computer science,psychology, and human factorsof computer interface design,together with practitionersworking in AS training(Beardon, Parsons and Neale2001). An integrated researchapproach has been adoptedwhich allows specification forVE design to be informed byuser needs and requirementswhilst taking into accountcharacteristics and limitationsof VE technology (Cobb, Kerrand Glover 2001). Aniterative development processhas been established whichallows further design develop-ments to be informed by userreview and professionalevaluations of the system(Neale 2001a). The advantageof this approach is that thecombined input from these
diverse disciplines allows us to consider ourresearch questions from different perspectivesand to apply a variety of methods to our reviewprocess. Hopefully, this will aid the develop-ment of a useful and usable end result.
Following initial consultation anddiscussions between these groups, the socialinteraction skills we aim to examine in the ASInteractive project are: appropriate use ofpersonal space; choosing and using appropriateresponses for both behaviour and communica-tion; and dealing with unexpected situations orchanges to routine.
individuals withAsperger’sSyndrome mayexperiencesocial exclusionfrom theirpeers due todifficulties inmaking andsustainingfriendships …
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
Dig
ital C
reat
ivity
, Vol
. 13,
No.
1Cobb et al.
1 4
2. Virtual environment development
2.1 User involvement in the design processAn important aspect of the project is theinvolvement of users in the design and review oftechnology development. In previous studiesthe authors have found that the involvement ofteachers and students in technology develop-ment helps to create a sense of ownership andempowerment (Meakin et al. 1998), which has asubstantial influence on usability of the interfacedesign (Brown et al. 1999), and subsequentacceptance and implementation within theexisting teaching programme (Benford et al.2000). However, the process of involving usersin learning technology design is not straightfor-ward; each group of ‘users’(i.e. teachers, stu-dents, training professionals, parents) hasdifferent objectives and can inform the designprocess in different ways. Moreover, theinvolvement of users with special educationalneeds requires the use of a variety of methods toelicit their opinions (many of which will need tobe adapted specifically) and flexibility to ensurethat their involvement is practical, enjoyable andfruitful (Neale, Cobb and Wilson 2001).
On the basis of this previous experience,it was decided that the user groups involved inthe AS Interactive project should be involved atdifferent stages of design: training professionalsshould help to define the social skills we areaiming to support in the virtual environment;teachers could then help with specification ofthe context for presentation of tasks in the VEand in defining specific learning objectivessupporting development of the social skills inquestion; users themselves can help to design thecontent and appearance of the VE and canprovide input to the design of the interface(Neale 2001a). A range of user and professionalgroups are actively involved in the AS Interactiveproject.• Adults with AS drawn from three established
Social Groups based in Sheffield, Nottinghamand Leicester are engaged in the review of VEdesign and usability to help inform design
specification for VE development.• AS training professionals also attend these
sessions to facilitate users in using the systemand to provide feedback to the design team.
• Students aged 16–19 with ASDs, andmatched non-ASD students, drawn from arange of schools in Nottinghamshire partici-pate in formal experimental studies to evaluatethe suitability and usefulness of the VEs forsocial skills training, particularly for this agegroup of students who are in the transitionperiod from school, hopefully seeking to findemployment.
• Teachers at these schools have been invited toparticipate in discussion workshops to reviewcontent and presentation of tasks within theVEs.
At this stage of the project we havecompleted the first full cycle of our design-review process (Cobb, Kerr and Glover 2001).Professionals from autism support organisationshave examined existing VEs to assess thesuitability of this technology for people with ASand advised changes to the appearance of the VEand presentation of information within it beforeit was presented to users with AS. Representa-tive user groups then evaluated the existing VEs(with modifications as suggested by the profes-sional group). The outcomes from these studies,combined with relevant information from thebackground literature reviews, were used toderive guidelines for VE design. Guidelineswere defined, providing recommendations for:content of a learning environment to supportsocial skills development, facilitation of learningstyles, content and interactions suitable for userswith AS, design for usability, making use of VEattributes and VE implementation (Neale2001b).
A workshop was held with a group of 30teachers from a school for children with ASDswho brainstormed ideas for suitable applicationareas, strategies for effective presentation oftasks, and possible barriers to implementation.These confirmed the view that we should place
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
1 5
Digital C
reativity, Vol. 13, No. 1
Applied virtual environments …
our training scenarios within the context ofgeneric situations familiar to everyone.
The first scenario selected for develop-ment aimed to support AS users in learning andmaking decisions about making appropriateversus inappropriate responses and was placed inthe context of a generic café. This context waschosen because it was considered to be familiarto most people such that they would recognisewhat activities they could be expected toperform in this environment. The scenario wasdeveloped in both SVEs and CVEs in order toallow us to assess the interface and usability ofeach type of environment and to begin toconsider how each could be used to supportsocial skills learning. It was considered that theSVE would provide structured training withlimited choices for appropriate/inappropriateresponses available, whereas the CVE wouldrepresent an unstructured situation in which theuser is free to make their own choices as to howthey interact with others.
2.2 Single user virtual environments: the ‘AS Café’The first SVE developed was referred to as theAS Café and is described in this section. One ofthe first objectives of the VE design process wasto develop a suitable user interface. In the case
of the SVE, this required the consideration ofmetaphors for real world actions and behav-iours. For example, how does the user indicatethat they wish to sit down on a virtual chair andstand up again? How do they know whenanother character within the virtual environ-ment is talking to them and how do theyindicate who they want to talk to? How do theyconduct a conversation in a single user virtualenvironment?
A number of design challenges werefaced when building the AS Café, as it was notclear how social situations should be representedin SVEs. Some aspects of social interaction maybe represented using this technology; wholebody position and navigation, for example.Other social cues, such as complex verbalinteraction, are less easy to represent in an SVE.We wanted the user to be able to verballyinteract, by asking questions and makingsocially appropriate decisions as to when to askthe question, what to ask, and who to ask it to.It was therefore necessary to design an interfaceto support this.
A ‘hybrid’ VE interface design wasapplied (Eastgate 2001) in which the 3D VEwas overlaid with a 2D multimedia interface.Figure 1 shows the single user interface design.The screen is split into two main areas, thewindow onto the virtual world and the inter-face.
The interface consists of a speech bubble(on the right) for asking questions, a textfeedback with speaker icon (centre) for commu-nication with the user, and an icon to hear theoverall task (left).
The user begins their experience in theSVE by holding a tray of food at the counterand is then asked to find somewhere to sit.There are no empty tables within the café andso the user needs to find a table that has someempty chairs or spaces. The VE is designed in aflexible manner, so that the user may take anumber of different paths in order to findsomewhere to sit and that each time a person
Figure 1.Single userenvironmentinterface.
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
Dig
ital C
reat
ivity
, Vol
. 13,
No.
1Cobb et al.
1 6
uses the environment they may experience itdifferently.
Users may move towards a table and ifthere is a spare seat at it, their viewpoint isrotated towards the people at the table and theyare asked what they should do next. Users thenhave to respond in one of three ways; they canselect the table or chairs to sit down, they canselect the speech bubble icon to ask a question,or they can just walk away. If they sit downwithout asking the other ‘people’ at the table,then they will be told that the seat is taken andthey should then select the floor to stand upagain. If the user selects the ‘ask question’ icon,a dialogue box then appears with a larger versionof the speech bubble on it. Contained withinthis are questions that the user can ask (seeFigure 2), these being either relevant or irrel-evant to the task, with responses to the latterprompting the user to try again. The questionsare set out on ‘radio’ buttons, which means onlyone can be selected, and when the user exits thedialogue box, code will register which questionwas asked, play the sound file and then activatethe appropriate response. If an appropriatequestion is asked, the user will be told whetherthe seat is available or not and this depends on anumber of factors, including whether it was thefirst time they had asked a question or if theyhad previously found out the availability of theseat. If it is the first time the user has asked aquestion, then they will automatically be toldthat the seat is unavailable. This is so the needto ask a question is highlighted to the user.Once a spare seat has been found the user thenselects the table or chairs to sit down, the tray offood appearing on the table.
2.3 Collaborative virtual environments: the ‘SocialCafé’
A CVE of a café environment, referred to as theSocial Café, was also developed. This wasintended to be a replication of the SVE AS Caféso that the two systems might be compared interms of how they can best be used to support
social skills learning. The main differencebetween the two systems, of course, is that aCVE allows participants to communicatedirectly with other users in a shared virtualspace, whereas in the SVE the user interactswith autonomous avatars. The implications ofthis for the project are that in the SVE theoptions for communication input and responsesfrom others must be pre-defined and pro-grammed into the system. This obviouslyrestricts the number of options available whereasin the CVE the user and other characters caninteract and communicate freely, providing anunlimited choice of options.
The appearance of the Social Café isbased on the AS Café, although direct commu-nication with other avatars eliminated the needto overlay the 2D communication interface (seeFigure 3). The interface is mouse driven andallows the participant to navigate the environ-ment, move or pick up objects and invokecertain behaviours. The user can enter the cafébuilding via opening the doors, navigate aroundthe café and select items from within it. Afterthe user has selected their desired items, theymay place a selection of them onto a tray, carrythem around, and find a suitable table at whichto sit. The user is then free to manipulate theseobjects individually if desired and stand up
Figure 2.Choosingwhat ques-tion to ask.
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
1 7
Digital C
reativity, Vol. 13, No. 1
Applied virtual environments …
again to perform additional tasks. ThroughoutVE use the user may interact verbally with otherinhabitants of the Social Café.
When participating in a CVE, the visualappearance of each user’s embodiment and howtheir actions are portrayed to others has to beconsidered. Each participant will typicallyembody an avatar with a different appearance.This serves both to allow an obvious means ofdistinguishing inhabitants from one another andmay be utilised to purvey a sense of anonymityor customise the way a user wishes to appear toothers. The Social Café allows a sense of gazedirection to be achieved with the avatar’s headmovements being independent of body motion(achieved by moving the mouse in the requireddirection). Different modes of input devicecould also provide further means of non-verbalcommunication, such as motion-trackingequipment allowing users to perform gestures.Users also have the ability to view the environ-ment from a number of different perspectives,such as from eye-level, from behind and above,or looking directly into the embodiment’s face.
It is also advantageous to visually identifythe source of verbal communication in a CVE;in a physical environment this is achieved by asense of the direction of the users’ voices and byvisibly identifying who is talking at any onetime. Although mouth animations are possible
within virtual environments, more exaggeratedrepresentations allow more widespread indica-tions of who is talking at a given moment. Thisis achieved in the Social Café by denoting who isspeaking by the display of ‘rings’ around thehead of the speaker. This may then be coupledwith other attributes, such as gaze direction, tobetter determine who is the subject of thecommunication.
3. Evaluation of virtual environmentdesign and acceptability
3.1 Design, usability and acceptabilityThe SVE AS Café has been through a process offormative evaluations which have been used torefine its design and the further development ofadditional training modules. Adults andadolescents with AS, all members of a ‘socialgroup’ were observed using the SVE with theirgroup facilitator and were interviewed about thisexperience. The main aim of this evaluation wasto examine the usability and acceptability of theinterface design. Users were asked to follow theon-screen instructions which initially promptedthem to find a place to sit down. Their on-screen actions, as well as any communicationbetween them and their group facilitator, wererecorded and analysed to examine which tasksthe user could carry out and which ones theyhad difficulties or required support with.
Most of the tasks were carried outcorrectly, on the first attempt, with little supportfrom the facilitator although it was found thatsome tasks within the VE were more difficult tocarry out than others. These included sitting atthe table (carried out by selecting the table orchair), moving away from the table (navigatingaway from the space), and asking questions(done by interacting with buttons on the 2Doverlay). Navigation was found to be too fastfor some of the users in the VE and as a resultthe speed settings were decreased. Observationof facilitator prompts or incorrect user actionsindicated where usability problems occurred,
Figure 3.The collabo-rative virtualenvironmentcafe.
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
Dig
ital C
reat
ivity
, Vol
. 13,
No.
1Cobb et al.
1 8
and qualitative data from video recordings wereused to explain why these problems occurred.For example, many of the problems with sittingat the table occurred because the user did notrealise that they had to switch from the joystickinput device (that they were using to navigate)to the mouse (which they needed to use to selectobjects), or because they did not know whicharea they had to click on, or because theinteractive area was too small. This data wasused to inform revisions of both the 3D and 2Dinterfaces.
The usability study highlighted aspectsof the SVE which failed to support or structureappropriate behaviour. For example, afterfinding out that a seat was free the users could(and sometimes did) move away from that tablewithout sitting down. Although the VE hasbeen designed to allow the user to choose whatactions they should take and when, they perhapsneeded to be informed that this is not a sociallyappropriate choice of actions.
It was thought that the majority ofproblems observed were because the user did notknow how to interact, rather than their inabilityto do so. As well as carrying out some tasksincorrectly, the participants did not make useout of some of the features of the VE that wouldhave helped them, such as the facility to re-playinstructions. It is proposed that the next versionincludes a training environment to show theuser how to use features of the interface andthen allow them to practise this before using theAS Café.
The user group facilitator had also madecomments relating to use of the SVE to supportlearning. It was suggested that each scenarioshould be presented at three different levels,dependent on the difficulty of the socialinteraction task that the user had to carry out.At the ‘easy’ level the user would just have tofind somewhere to sit, there would be plenty ofempty tables and chairs. The medium levelwould be the current VE, and the difficult level
set some additional challenges, such as wouldrequire the user to borrow a chair from adifferent table. It was also suggested that somekind of recognition for achievement should beincluded in the program, but as yet no decisionshave been made as to what form this shouldtake.
Users generally liked the CVE Social Caféand could navigate the environment reasonablywell, although they did experience initialproblems with use of the virtual space and keptbumping into each other. This may have beenbecause they didn’t have a true sense of theiravatar’s body size or because their viewpointdidn’t show the object they were navigating (i.e.the viewpoint is positioned from the avatar’seyes looking forward). It was suggested that itmay be better if the viewpoint was positionedslightly behind the avatar’s head, although thismay then cause confusion when interacting withother avatars. The solution so far has been toscale up the environment to provide moreexaggerated space. This has helped to improvenavigation.
Some problems were experienced withobject interactions, mostly relating to theunusual behaviour of virtual objects. Forexample, objects placed in mid-air do not falldown and it is possible to pick things up froman unrealistic distance, such as the other side ofthe room. Users did not like this and somecommented that the appearance, behaviour andinterface methods for the Social Café would haveto be much more realistic in order for it tofacilitate learning of social skills. Interestingly,these issues were not raised in relation to theSVE AS Café; possibly the change in perspectivefrom external viewer to inhabitant of the virtualenvironment is influencing users’ attitudestowards, and subsequent expectations of, thevirtual environment as a replication of the realworld. Further investigation of these issues isrequired before we can comment upon this.
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
1 9
Digital C
reativity, Vol. 13, No. 1
Applied virtual environments …
3.2 Experimental studies of use and understand-ing of virtual environments
In addition to the usability studies, experimentalstudies are being used to assess user interpreta-tion of virtual environments with respect to thereal world. These studies will inform ourinterpretations of user performance in virtualenvironments and provide anindication of the potentialtransfer of social skills to realworld execution. Two studieshave so far been conductedwhich have examined userbehaviour in single userenvironments developed oradapted specifically for thepurpose of study.
The first experimentalstudy was conducted toexamine how users behave(verbally and non-verbally)when using an early version ofthe SVE café (Brown et al. 1999) and how theyunderstand and react to ‘people’ within the VE.This experiment compared performance andresponses from participants with ASDs andcontrol participants matched on verbal IQ(VIQ) and performance (non-verbal) IQ (PIQ).Preliminary observations of performance suggestthat users with ASDs found the VE interesting,fun and easy to understand. When askedquestions about the VE and specific elementswithin it, participants generally accepted thatthe VE was representing a real environment andmade appropriate inferences about what theavatars were doing (Parsons and Mitchell 2001).One of the issues highlighted by the study wasthat participants with ASDs may have difficul-ties in respecting personal space of othercharacters within a virtual environment; it wasnoticed that several users navigated between twoavatars positioned at the bar despite their beinga large empty space at the side of them. Infor-mal observations suggest that members of thegroup of pupils with ASDs were more likely to
demonstrate this behaviour (50% of the group),compared to the verbal IQ and non-verbal IQmatched groups (9% and 25% respectively).
As a follow-up to this finding, a furtherstudy investigating the understanding ofpersonal space within SVEs has been carriedout. Pupils with ASDs were again matched with
comparison groups on the basisof gender, age, VIQ and PIQ.All participants were asked tonavigate a route to the counterto get a drink from the waiter.There were eight conditions intotal: four in which it wassocially appropriate to walkstraight ahead to the counter(the direct route to the counterwas bordered by either objectsor people with their backs toyou), and four in which it wassocially inappropriate to do so(the direct route was bordered
by people facing each other, ostensibly engagedin conversation). In each case, there wassufficient space to walk between the people orthe objects en route to the counter, or the usercould choose to take a detour around thepeople/objects, thereby avoiding them alto-gether. Although the results have yet to beformally analysed, all participants (regardless ofgroup membership) did not discriminatebetween the appropriate and inappropriateconditions. Instead, the majority of participantschose either one route or the other (i.e. straightahead to the counter, or around the people/objects), and adhered to that choice acrossconditions. Interestingly, during questioning,many participants (both with and withoutautism) acknowledged that it was sociallyinappropriate to walk through a gap in whichpeople were engaged in conversation. Acommon justification for behaving as they didin the VE was because, “…it’s just a VE, and soit doesn’t really matter what you do”. Despitethis assertion, the majority of participants
a VE … offersthe potentialfor role-play ina simulatedcontrolledenvironment
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
Dig
ital C
reat
ivity
, Vol
. 13,
No.
1Cobb et al.
2 0
avoided walking across grass and flowerbeds enroute to the café, “…because it would not beappropriate to do that – you could break theflowers”. Consequently, one of the mainchallenges facing the project is to investigatewhy some social conventions seem to bereflected in behaviour during VE use (notwalking across flowerbeds), whilst others do not(respecting the personal space of people in acafé).
4. Discussion
The application of VEs for social skills learningin adults with AS is considered advantageousbecause it offers the potential for role-play in asimulated controlled environment. In a CVEusers can interact with others via navigation oftheir avatar and direct verbal communicationusing headset microphones. This offers thefacility to practise real conversations in a role-play scenario without the added intimidation offace-to-face contact. Our research has not yetprogressed far enough for us to comment uponoutcomes from role-play sessions in the CVE,this will be the focus of our next research phase,but early indications suggest that users like thetechnology and consider that it could be usefulto support social skills learning.
SVEs, by their very nature, offer limitedopportunity to explore social interaction.Whilst it is possible that such environments canbe used to replicate life situations and scenarioswhich a user can explore, their real advantage fortraining is in procedural behaviour (for example,learning the sequence of actions required toachieve successful completion of a task). In theAS Interactive project, we decided that SVEscould be utilised for training specific aspects ofsocial interaction in a structured manner. Thelimited number of options available at any givendecision point should make it easier for users tolearn appropriate choices. Moreover, the SVEcan be developed at different levels, representingprogressively more complexity and options
available to the user. This ‘scaffolded’ learningapproach has been successfully utilised ineducation technology and may offer a suitablemethod for building an understanding ofindividual components required for socialinteraction. By focusing on specific aspects ofsocial interaction, we may be able to use thisapproach to examine generalisation of skillsfrom one context to another. For example, wepropose to replicate the task of choosing whereto sit in other social contexts, such as publictransport, to examine whether individualperformance in one context is demonstrated inthe other.
The ultimate aim of the project, ofcourse, is to support learning of social skillswhich can then be applied in the real world.Assessment of user behaviour within a VE isonly useful if we know that they understandhow the training environment represents andapplies to the real world. Results from ourinitial experimental studies have indicated thatusers (both with and without AS) appear toascribe real-world conventions to behaviour insome virtual tasks but not in others (e.g. notwalking across flowerbeds but walking ‘through’personal interaction space). User explanationsfor this behaviour suggest that, at some times,users interpret the VE like a computer game inwhich their actions have no real world conse-quences and so their behaviour ‘does not reallymatter’. What is not clear is why this appliesonly in some situations but not in others. Onepossible explanation is that users are content toapply social rules when decision choices aresimple (e.g. go across the flowerbeds or aroundthem) but when the options becomes morecomplex (involving making assumptions aboutthe social interactions between other characters),it is hard to make realistic decisions and so usersopt for an easy way out, and satisfy themselvesthat it doesn’t really matter anyway because it’sonly a game. It has been suggested that theinterpretation of the VE as a ‘real’ scenario maybe influenced by how tasks are presented to
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
2 1
Digital C
reativity, Vol. 13, No. 1
Applied virtual environments …
users and what cues are used to ‘set the context’.This is obviously an important issue and will beaddressed in follow-up experiments. As a firststep towards this, it will be interesting toexamine how users react to the same task andscenario presented in the CVE; will users stillnavigate through the social space betweenavatars engaged in a real conversation?
We have successfully established collabo-ration with appropriate user groups and theyhave all participated in development of the firstVE prototypes produced. The review of theSVE yielded different information from eachgroup of reviewers. The professional groupcommented upon general design and aesthetics;use of colours, symbols and presentation ofprompts. Users commented upon usability anddesign of the interface. Whilst most usersenjoyed using the VE and needed little assist-ance, usability problems were identified relatingto navigation around the VE, inconsistencies ininteraction metaphors and the need for instruc-tion prompts. Teachers commented uponfunctionality and issues relating to generalisationand presentation of this as learning material.
At the end of the first phase of study wecan conclude that VEs are acceptable to userswith AS and users do sometimes interpret VEsas being representative of real environments.However, there are still many outstandingquestions to address concerning users’ interpre-tation of other people within the virtual envi-ronment, how should we provide the appropri-ate scene setting, and how we can make effectiveuse of progressive learning. In the next phase weintend to build upon our relationship withteachers and training professionals to supportplanning and decision-making with respect tofinal development and implementation. As afirst step toward this we will be installing theSVE and CVE into a school for children withASDs to allow us to conduct in situ evaluations.It is hoped that this will enable us to furtherinvestigate some of the issues raised in ourpreliminary studies with a view to defining
principles and design requirements for theapplication of VEs as part of a social skillstraining programme. A similar approach isbeing taken with adults with AS in NationalAutistic Society run social groups.
Acknowledgements
The authors would like to thank the ShirleyFoundation for funding this research. We arealso greatly indebted to the AS Social Groupmembers in Sheffield, Nottingham and Leicesterand the students and staff at Rosehill School,Foxwood School, The Bluecoat School andBrackenfield School in Nottingham for theirparticipation.
References
Baron-Cohen, S. and Bolton, P. (1993) Autism: thefacts. Oxford University Press, New York.
Baron-Cohen, S., Leslie, A.M. and Frith, U. (1985)Does the autistic child have a ‘theory of mind’?Cognition 21 37–46.
Beardon, L., Parsons, S. and Neale, H. (2001) Aninterdisciplinary approach to investigating the useof virtual reality environments for people withAsperger Syndrome. Educational and ChildPsychology 18(2) 53–62.
Benford, S., Bederson, B. B., Åkesson, K., Bayon, V.,Druin, A., Hansson, P., Hourcade, J. P., Ingram, R.,Neale, H., O’Malley, C., Simsarian, K., Stanton,D., Sundblad, Y. and Taxén, G. (2000). Designingstorytelling technologies to encourage collaborationbetween young children. In Proceedings of HumanFactors in Computing Systems (CHI 2000). ACMPress, pp. 556–563.
Brown, D.J., Neale, H.R., Cobb, S.V. and Reynolds,H. (1999) Development and evaluation of thevirtual city. The International Journal of VirtualReality 4(1) 28–41.
Cobb, S., Kerr, S. and Glover, T. (2001) The ASinteractive project: developing virtual environmentsfor social skills training in users with Asperger’sSyndrome. Proceedings of workshop on Robotic and
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014
Dig
ital C
reat
ivity
, Vol
. 13,
No.
1Cobb et al.
2 2
virtual interactive systems in autism therapy. Univer-sity of Hertfordshire, Hatfield, UK (forthcoming).
Eastgate, R.M. (2001) Structured development ofvirtual environments. PhD Thesis, University ofNottingham.
Frith, U. (1989) Autism: explaining the enigma.Blackwell, Oxford.
Howlin, P. (1997) Autism: preparing for adulthood.Routledge, London.
Klin, A. and Volkmar, F.R. (2000). Treatment andintervention guidelines for individuals withAsperger Syndrome. In Klin, A., Volkmar, F.R. andSparrow, S.S. (eds.) Asperger Syndrome. TheGuilford Press, New York.
Meakin, L., Wilkins, L., Gent, C., Brown, S.,Moreledge, D., Gretton, C., Carlisle, M., McClean,C., Scott, J., Constance, J. and Mallet, A. (1998)User group involvement in the development of avirtual city. In Sharkey, P., Rose, D. and Lindstrom,J-I. (eds.) The 2nd European conference on Disability,virtual reality and associated technologies. Universityof Reading, pp. 1–9.
Mitchell, P., Saltmarsh, R. and Russell, H. (1997)Overly literal interpretations of speech in autism:understanding that messages arise from minds.Journal of Child Psychology and Psychiatry 38 685–691.
Neale, H. (2001a) Using virtual reality to teach socialskills to people with Asperger’s Syndrome: explain-ing virtual reality and user-centred methodology.Proceedings of the 12th Annual Durham InternationalConference on Autism. Autism North (in press).
Neale, H. (2001b) Guidelines for design of virtualenvironments for users with Autism. Proceedings ofworkshop on Robotic and virtual interactive systems inautism therapy. University of Hertfordshire,Hatfield, UK (forthcoming).
Neale, H., Cobb. S. and Wilson, J.R. (2001)Involving users with learning disabilities in virtualenvironment design. Proceedings of the 9th Interna-tional Conference on Human-Computer Interaction(HCI 2001). New Orleans, LA.
Parsons, S. and Mitchell, P. (2001). Virtual reality andpeople with Autistic Spectrum Disorders: informalobservations from the AS interactive project.Proceedings of the 12th Annual Durham InternationalConference on Autism. Autism North (in press).
Pollard, N.L. (1998) Development of social interac-tion skills in preschool children with autism: areview of the literature. Child and Family BehaviorTherapy 20 1–16.
Dow
nloa
ded
by [
Uni
vers
itat P
olitè
cnic
a de
Val
ènci
a] a
t 17:
48 2
7 O
ctob
er 2
014