research autism lorna wing seminar thursday, 24th november ... · acknowledgements > hands...
TRANSCRIPT
Research Autism Lorna Wing Seminar Thursday, 24th November 2011
Does hand held technology improve independence? Helping young people with Asperger syndrome and high functioning autism navigate life’s highways. The ‘HANDS ‘project. An evaluation of multi site EU research collaboration
Joseph Mintz London South Bank Unversity, London
Miklos Gyori ELTE University & Autism Foundation, Budapest
Acknowledgements
> HANDS project has been supported by a grant from the European Commission, within its 7th Framework Programme (Accessible and inclusive ICT; ICT-2007.7.2), contract nr. 224216. Authors are especially grateful for pupils’ and their families’ participation in the project.
Part I. HANDS: the project, the system, and its quantitative efficiency testing.
Miklos Gyori ELTE University, Budapest, Hungary
the HANDS project
the HANDS project
Helping Autism-diagnosed teenagers Navigate and Develop Socially
June 2008 – October 2011
7th Frame Programme of the European Commission
Accessible and inclusive ICT (ICT-2007.7.2)
10 partners from 6 countries
Aalborg, DK London,
UK
Stockholm, SWE
Oslo, NOR
Cluj, RO
Budapest, HU
Aalborg,
DK London,
UK
Stockholm,
SWE
Oslo,
NOR
Cluj,
RO Budapest,
HU =
=
=
=
Autism Foundation School
+
ELTE University
‘Egebakken’ School
+
Aalborg University
‘Helen Allison’ School
+
London South Bank University
‘Svedenskolan’ School
the HANDS project: aims
a mobile-ICT-based support system for HF teenagers with ASD
> specific aims > enhance social integration & participation
> daily life skills, social behaviours & communication
> via supporting well-established intervention approaches
> further potential benefits > on efficiency of intervention
> on efficiency of human resources
> novel ways of data collecting/research
the HANDS system
the HANDS system
two-component software system
mobile component – for the client can be brought into focus situations of intervention
individualised contents
multimedia
easy-to-handle
web-based component – for the professional creating & managing contents ...even from away
supervising mobile activity ...even from away
sharing knowledge
mobile web remote synchronisation
> daily routines
> institutional routines
> social situations
> transportation
> free time
> learning novel skills
> etc.
an example
> high functioning boy (12) with ASD > diabetes > swimming classes are vital but hard to manage for him
quantitative efficiency testing of the HANDS system
key members of team: Ildikó Kanizsai-Nagy (Autism Foundation)
Krisztina Stefanik (Autism Foundation & ELTE Univ.)
Zsombor Várnagy (ELTE University)
developing & testing HANDS
> 2-phase iterative development process >> 2 prototypes
> 3 streams of scientific expertise & research
> challenges and difficulties about development
> quickly evolving technological context
> efficient communication across disciplines
> etc.
> challenges and difficulties about testing & research
> balance between development and research
> balance between quantitative and qualitative methods
> balance between involved research streams
> cross-cultural, cross-institutional and individual differences
overall scheme of testing HANDS Prototype 2
> ‘quasi Randomised Controlled Trial’ > test group: HANDS-assisted educational support
> matched control group: ‘treatment as usual’
> pre-test / post-test & test / control comparisons
> limitations > not totally random assignment
> no blind conditions
> etc.
> active testing > from 15 January 2011 till 15 April 2011
> at all the 4 test-schools
sample
> 4 countries, 6 schools, total N=49 (57) teenagers with ASD
> male/female = 45/4
> age: m= 14.38 years; SD=1.7; range 10,68 – 17,72
> IQ: m= 85.37; SD=15.93; range 53 – 118
> diagnosis: clinical, confirmed by ADI-R and ADOS
> educational context:
autism-specific school or class with an evidence-based approach on a daily basis
HANDS
PROTOTYPE 2
testing + evaluation
Psychology
ELTE Univ. + Autism Found.
testing + evaluation
educational research
LSBU
testing + evaluation
human-computer interaction
Aalborg University
qualitative methods
qualitative methods
quantitative methods
pre-assessment >> ADI, ADOS, IQ
usage >> ‘electronic footprints’ / log data
testing + evaluation
psychology
quantitative methods
adaptive visual design >> eye-tracking
specific effects >> experimental task analysis
skill-level effects >> psychometric testing
adaptive visual design >> eye-tracking
> often atypical attentional processes in autism
> eye-tracking/gaze-tracking: standard technique
> method used here: ‘interactive-dynamic’ (developed by Gyori & Várnagy, 2010, 2011)
> main finding: HANDS mobile is well-designed
specific effects >> experimental task analysis / ExTA
> novel research tool developed by Krisztina Stefanik et al. (2010, 2011)
> adapted from Task Analysis in TEACCH
> combines quantitative measurement with individual focus
> scales a target skill by the number & depth of necessary prompts
> main finding: HANDS-assisted intervention is more effective than traditional in a short run, in supporting specific adaptive behaviours
Making two slices of toast with butter and salt in school settings
0
0,5
1
1,5
2
2,5
3
3,5
4
4,5
0 5 10 15 20 25 30 35 40 45
Steps of the task
Dep
th o
f th
e p
rom
pts
Before
After
skill-level effects >> psychometric testing
> Social Responsiveness Scale, SRS, by Constantino & Gruber, 2005 > 65-item questionnaire, 5 subscales, standardised
> HANDS Follow-up Questionnaire, by Gyori, Stefanik, Kanizsai-Nagy > 55-item questionnaire
> both questionnaires > parent/caregiver & teacher versions > administered pre & post test
> statistical analysis > test/control, before/after comparisons
> key findings > ‘noisy’ results > weak consistency between parents’ & teachers’ ratings > lack of any sample-level effect
skill-level effects >> psychometric testing
usage >> ‘electronic footprints’ / log data
> Data from SRS and HFQ questionnaires as above
+ > Usage data recorded automatically by the HANDS system / ‘electronic footprints’
> statistical analysis
> correlations between usage frequency and effects
> key findings > huge differences in amount/frequency of usage > positive correlations between usage and development in test period > cause effect relationship?
• HANDS has adaptive, ‘autism-friendly’ mobile user surface.
• HANDS shows short-term efficiency of support in specific tasks.
• HANDS has no robust, general and specific long-term effect.
• HANDS seems to have positive and long-term effects in a part of users.
adaptive visual design >> eye-tracking
specific effects >> experimental task analysis
skill-level effects >> psychometric testing
skill-level effects >> psychometric testing
usage >> ‘electronic footprints’ / log data
summary of quantitative results
conclusions from quantitative studies
> on the HANDS support system
> seems effective on both short and longer term…
> in developing highly specific behaviours as well as wider skills
> though not for all teenager-teacher user pairs
> on conditions of success >> Part 2
> on quantitative research methodology
> in spite of challenges & difficulties…
> complex quantitative approach has proven productive and necessary
> has been productively complemented by quantitative methods >> Part 2
> special thanks to
> children and families…
> schools and teachers…
Part II. the ‘educational research’ stream and qualitative studies on the HANDS system
Joseph Mintz London South Bank University
Education: Research Questions Applicability of the HANDS tool to the learning environment
> how HANDS fits in with existing practices of teaching and learning
> what impact it has on such practices
> effects in developing social skills, self management skills and social integration
> Identification of key factors influencing engagement and use
testing + evaluation
education
qualitative methods
Teacher interviews
(mainly HA, but also EGE, SVE and AF)
Class observations
(mainly HA, but also EGE, SVE and AF)
questionnaires (AF, EGE, SVE)
Parent and Child Interviews (HA only)
methods
> interviews
> semi-structured, pre-defined interview guide drawn from research questions > pre-test, in test and post-test period interviewing For Prototype 1: 7 teachers, 10 support staff, 8 parents, 9 children For Prototype 2: 9 teachers, 23 support staff, 6 parents, 10 children
> questionnaires
> 39-item semi-structured questionnaire, matched to interview guide questions For Prototype 1: 14 questionnaires by teachers For Prototype 2: 16 questionnaires by teachers
> observations
> classroom based, matched to teacher interviews. > triangulation with interviews.
The Implementation Experience
Technology can be hard to work with
Software bugs – a mismatch in understanding about what “software testing” means
Significant improvements in technology stability between Prototype 1 and Prototype 2
The Implementation Experience
Technology can be hard to work with
Connectivity issues
Structural issues – can we get the permissions we need?
Battery Issues, Charging Issues, Losing the Phone, Throwing the Phone Issues
Teacher attitude and ability with ICT
The Implementation Experience
But it can be transformational as well
The Implementation Experience
Keeping up with the curve is hard
Windows Dynamic Mobile, iPhone, Android, the Smartphone Revolution
The Implementation Experience
Teenagers and Mobile Devices – what did we see for ASD?
Independence, Adolescence, Group Identity, Mobile Loyalty – but more work is needed on this
Jealousy, Bullying, Fear of Theft/Attack, Inappropriate Behaviour – text bullying – what did we see?
results / 1
> Developing children’s social and life skills
> a mixture of successful and unsuccessful interventions
> to aid a diverse range of skills
> benefits and detriments of both HANDS software & general phone functions cited
It was difficult to arrange use of the device outside the classroom, partly on account of security issues, partly because of the software’s unreliability. (It is difficult to prepare in two ways for a possible problem. Eg., if behavioral aspects are only on the machine and it crashes, then the written form must absolutely be to hand.) These problems cause much less concern outside the classroom, since the structured setting is given there. [AF Teacher Questionnaire].
…..modification of the timetable was equally difficult, because the device must be connected to the computer for updates. [AF, 15156 Teacher Questionnaire]
Interviewer: Do you think he saw it as credible? Is it more than just playing with it, or is he responding to the interventions and doing what they tell him? Parent: Yeah, he definitely would, without a doubt, if he was out on his own, he would use it. if he got into a problem it would give him a great deal of confidence and it would focus him as opposed to him feeling the panic that I think he would feel if he did find himself on his own situation.
[Parent Interview, HA]
Lucy: …we had parents evening, and he actually said that he wanted all his tasks put on the HANDS phone like that, so he didn’t have us nagging at him…. I went right okay, that means every teacher’s got to tell me what every task is in every lesson for me to put on there, but I have said to them that there are things that he does regularly during the week, so he goes to an offsite college on a Friday. He knows he has to have early lunch, come round to the department, get changed, be ready for when the minibus from the college pick him up and that. I said right we can put that on there. Don’t then tell him, nag him as he says… Just say to him check your HANDS phone, you know what’s expected, and just leave him to do it, and maybe that’s a way around it. Because Adam doesn’t like being told what to do [HA Teacher Interview]
results/2
Some key factors affecting engagement and use
User Device Attachment
Fogg (Fogg, 2003; Fogg and Eckles, 2007) proposes that the effectiveness of
specific behavioural messages delivered via mobile devices is enhanced
when there are ongoing repeated positive interactions with a range of
cognitive and social functions on the device.
Observed in 18 cases across PT1 and PT2
Important to note that not seen in all cases
results/2
Some key factors affecting engagement and use
Source Credibility and a focus on the home
Interviews and classroom observation showed that children can be positively
disposed to receiving mobile persuasive interventions from their teacher,
instantiated as a message from their teacher in video, image or text format.
The present study provides evidence to support the exploitation of teacher
credibility in the domain of life and social skills functioning. This very much part
of the school curriculum for young people with ASD, yet in many instances the
context for practising and developing social skills is as much in the home and
in out of school contexts as within the classroom itself.
Observed in 16 cases across PT1 and PT2.
Important to note that not seen in all cases
results/2
Some key factors affecting engagement and use There was some evidence from Prototype 1 that instances of effective use of HANDS
by children were predicated on the individual child recognising that an issue existed
with a particular behaviour and having some level of motivation to engage with
behaviour change.
The Prototype 2 data provides further and stronger evidence to support this
recommendation.
All teachers in the UK school asserted in interviews that in their perception, student
awareness of an issue or difficulty and associated motivation to achieve a specified
behaviour change was an important factor mediating the level of engagement with
HANDS and likelihood of successful response to a particular behavioural intervention
instantiated on HANDS.
In 4 out of 10 teacher-child cases at the UK school, teachers indicated that their
student had difficulties with awareness of specific difficulties for which interventions
were introduced on HANDS, and that this had in their perception been a contributory
factor to lack of response to specific interventions, as shown in the following vignette
involving John.
HANDS could potentially be used to encourage her student, John, a 15 year old male student with a diagnosis of autism, to reflect on his problematic behaviour with another student in the same class.
John and the other student commonly tease and aggravate each other during lessons and break times.
Teacher programmed a series of intervention prompts onto HANDS which asked John “has student X [the other student] had a good day?” which John could select to either answer yes or no, and then a further question “if you said no, was it because you wound him up?” which again John could answer either yes or no to.
Intervention proved problematic, as John did not think that there was anything wrong with the way he socialised with student X or that his behaviour had any influence in the negative outcomes
After this intervention was implemented and John had been using it for a period, John requested that teacher change or stop the intervention because he did not like it, and instead the teacher rewrote the intervention, so that the reminder prompted John with “keep your joking with student X to break times”, a more directed intervention which relied more on leveraging teacher authority, and less on John’s internal awareness of the issue and motivation to change his behaviour.
The teacher’s overall assessment of the intervention was that as John was not at all motivated to change the way he socialised and communicated with student X, the intervention failed to be effective.
results/2
Some key factors affecting engagement and use
Preference for Mobile Device Based Interventions
Our evaluation of Prototype 1 and Prototype 2 indicated that perceived teacher
source credibility is a significant factor mediating engagement.
We also had some weak indications from interview responses for Prototype 1 that
some children may actually prefer to receive persuasive messages from HANDS
than from their teacher.
Accordingly we included specific questions on this area in our interview guide for
Prototype 2 child interviews at the UK school.
Six out of ten of the children commented on whether they preferred support /
prompts from a person or from the HANDS device.
Two students preferred prompts from a person, including parents, teachers and
support staff.
In contrast, the other four students at the UK school felt that they would rather
receive persuasive messages from the HANDS phone than an adult.
results/2
Some key factors affecting engagement and use
Preference for Mobile Device Based Interventions
A potential rationale for this preference is that many children with ASD have
cognitive impairments in processing speeds (Luna et al., 2007).
For example, in one of the cases, the child was using HANDS to support him in
the life skills task of making toast independently. In a classroom observation he is
observed completing the task successfully using HANDS as a support and voices
that he prefers to receive the instructions from HANDS as opposed to from his
teacher. The teacher suggests that this may be because he “can do it in his own
time”, that is that he has longer to process the individual intervention messages
from the mobile device and can control the rate at which the messages are
supplied. In the follow up interview the teacher comments:
I think they felt more relaxed because it was up to them to move to the next step, whereas you present someone with this long list of instructions written down. Someone with autism that you know, it works their minds when they see what they’ve got to do, because it does look a lot of set when it’s written down. Whereas on the HANDS phones they move to the next step when they felt confident enough to do so, and they worked through it didn’t they?[HA Teacher Interview]
conclusions & perspectives
Conclusions & perspectives
a rare/unique attempt to develop & test such an intervention toolset systematically
Many apps out there – very few (if any) with systematic efficacy testing
> though some clear shortcomings revealed
appears as an effective and promising toolkit for some children with ASD
Importance of technical stability and teacher and school technical ability
> many aspects seem transferable to other target populations
> a basis to incorporate other aspect of supportive technology
Next Steps?
> Further integration of qualitative and quantitative data
Further theoretical informed experimental development based on HANDS experiences to develop a screening instrument that would identify which children with ASD are most likely to benefit from the use of mobile devices to develop social and life skills
Larger scale testing of mobile apps for social and life skills development based on such a screening instrument
thank you!
www.hands-project.eu