can adults with autism spectrum disorders infer what happened to someone from their

Can Adults With Autism Spectrum Disorders Infer What Happenedto Someone From Their Emotional Response?Sarah Cassidy, Danielle Ropar, Peter Mitchell, and Peter Chapman

Can adults with autism spectrum disorders (ASD) infer what happened to someone from their emotional response?Millikan has argued that in everyday life, others’ emotions are most commonly used to work out the antecedents ofbehavior, an ability termed retrodictive mindreading. As those with ASD show difficulties interpreting others’ emotions,we predicted that these individuals would have difficulty with retrodictive mindreading. Sixteen adults with high-functioning autism or Asperger’s syndrome and 19 typically developing adults viewed 21 video clips of people reactingto one of three gifts (chocolate, monopoly money, or a homemade novelty) and then inferred what gift the recipientreceived and the emotion expressed by that person. Participants’ eye movements were recorded while they viewed thevideos. Results showed that participants with ASD were only less accurate when inferring who received a chocolate orhomemade gift. This difficulty was not due to lack of understanding what emotions were appropriate in response to eachgift, as both groups gave consistent gift and emotion inferences significantly above chance (genuine positive forchocolate and feigned positive for homemade). Those with ASD did not look significantly less to the eyes of faces in thevideos, and looking to the eyes did not correlate with accuracy on the task. These results suggest that those with ASD areless accurate when retrodicting events involving recognition of genuine and feigned positive emotions, and challengeclaims that lack of attention to the eyes causes emotion recognition difficulties in ASD. Autism Res 2014, 7: 112–123.© 2013 International Society for Autism Research, Wiley Periodicals, Inc.

Keywords: autism; retrodictive mindreading; eye tracking; spontaneous emotion recognition; face processing

Introduction

Successfully interpreting others’ emotional responses is akey for successful social interaction. It is widely reportedthat individuals with autism spectrum disorders (ASD),who experience profound difficulties with social interac-tion [Wing & Gould, 1979], also experience difficulties inemotion recognition. Indeed, Kanner (1943) originallydescribed autism as a disorder of “affective contact,” anddifficulties with processing emotion are part of thecurrent diagnostic criteria for autism (APA, 2000).However, studies of emotion processing in ASD haveshown highly inconsistent results; some finding differ-ences in emotion recognition and others failing to finddifferences between individuals with and without ASD.This is particularly the case for adults with ASD who haveaverage intelligence, who tend to pass simpler emotionrecognition tasks using static, posed expressions [seeGaigg, 2012; Harms, Martin, & Wallace, 2010; Uljarevic &Hamilton, 2013]. To resolve this debate, and assess thesubtle difficulties adults with ASD exhibit, we need todevelop tasks that match the demands of everyday life,while maintaining experimental control. This is thepurpose of the current study.

As stated above, studies using a recognition paradigmwith basic emotions (e.g. happy, sad, angry) often reportceiling effects or fail to find differences between indi-viduals with and without ASD [see Adolphs, Sears, &Piven, 2001; Loveland, Steinberg, Pearson, Mansour, &Reddoch, 2008; Neumann, Spezio, Piven, & Adolphs,2006; Ogai et al., 2003; Rutherford & Towns, 2008;Spezio, Adolphs, Hurley, & Piven, 2007a, 2007b]. In con-trast, emotion recognition research which has used eithermore complex emotions (e.g. guilt), dynamic stimuli, oremotions with lower intensity tend to reveal impair-ments in adults with ASD [e.g. Baron-Cohen et al., 2001a;Golan, Baron-Cohen, Hill, & Golan, 2006; Humphreys,Minshew, Leonard, & Behrmann, 2007; Philip et al.,2010; Roeyers, Buysse, Ponnet, & Pichal, 2001]. Theseresults suggest that stimuli which more closely reflect thedemands of everyday processing are more likely to revealdifferences between adults with and without ASD.

However, there are reasons to question whether thesemore naturalistic tasks mirror the demands of everydaylife. For example, these previous studies [with the excep-tion of Roeyers et al., 2001] used posed, rather than spon-taneous expressions. Spontaneous expressions differ toposed expressions as they are not produced by a direct

From the School of Psychology, University of Nottingham (S.C., D.R., P.C.); School of Psychology, University of Nottingham Malaysia Campus (P.M.)Received June 05, 2013; accepted for publication October 07, 2013Address for correspondence and reprints: Danielle Ropar, School of Psychology, University of Nottingham, University Park, Nottingham, NG7 2RD,

UK. E-mail: [email protected] online 4 December 2013 in Wiley Online Library (wileyonlinelibrary.com)DOI: 10.1002/aur.1351© 2013 International Society for Autism Research, Wiley Periodicals, Inc.

INSAR112 Autism Research 7: 112–123, 2014

RESEARCH ARTICLE

mailto:[email protected]

http://wileyonlinelibrary.com

request by another person [Matsumoto, Olide, Schug,Willingham, & Callan, 2009], but rather occur naturallyduring social interaction. Thus, spontaneous expressionshave lower signal clarity; they are far more subtle thanposed expressions, can portray more than one emotion,and be subject to display rules, such as trying to portray apositive, rather than a negative reaction to a social inter-action partner [Carroll & Russell, 1997; Matsumoto et al.,2009; O’Sullivan, 1982]. This may explain why studieshave found spontaneous expressions to be harder to rec-ognize than posed expressions [Hess & Blairy, 2001; Naab& Russell, 2007; Wagner, 1990; Wagner, Lewis, Ramsay, &Krediet, 1992; Wagner, MacDonald, & Manstead, 1986].Thus, the stimuli predominantly used in previous studiesmay not share the characteristics of emotion expressionsencountered in everyday life, which are more subtle andchallenging to interpret. This may help explain the incon-sistent results of previous research exploring emotion rec-ognition, particularly in the case of adults with ASD.

Another reason to question whether previous emotionrecognition tasks mirror the demands of everyday life isthe predominant focus on recognition. Millikan [2005]has argued that the most common form of emotion rec-ognition is not inferring another’s emotion (as in thetasks described above), rather we more typically observe aperson’s emotional response, and then go about explain-ing this response after the event [e.g. Bartsch & Wellman,1989; Robinson & Mitchell, 1995]. This ability has beentermed retrodictive mindreading [Gallese & Goldman,1998; Goldman & Sripada, 2005; Millikan, 2005].

The only previous study of retrodictive mindreadingwas recently performed by Pillai, Sheppard, and Mitchell[2012], who demonstrated that typically developingadults could systematically infer what happened tosomeone from watching a brief video clip of theirresponse (whether the individual was told a story, a joke,was left waiting, or received a compliment). HoweverPillai et al.’s study did not include any measure ofemotion inference. As those with ASD may have difficultyunderstanding what behaviors are appropriate in certainsocial situations [Baron-Cohen, O.’Riordan, Stone, Jones,& Plaisted, 1999; Loveland, Pearson, Tunali-Kotoski,Ortegon, & Gibbs, 2001], in the current study, we recordparticipants’ estimations of the target’s emotion, in addi-tion to the situational inference. This is to explorewhether those with ASD and typical controls understandwhat emotional responses are appropriate to the givenrange of social situations. This allows us to determinewhether difficulty retrodicting the correct situationalantecedent is due to impaired recognition of emotion asopposed to understanding what kind of reaction wouldbe appropriate in a given social situation (such as beingpolite when receiving an unwanted gift).

In contrast to Pillai et al.’s [2012] study, we investigate adifferent, but commonly experienced social situation—

receiving a gift. This social situation was chosen as itprovides the opportunity to investigate adults with ASD’sunderstanding and recognition of more complex emo-tions. For example, “social emotions” are expressed in thepresence of another person, such as feigning a positiveresponse in order to be polite [Baron-Cohen, Jolliffe,Mortimore, & Robertson, 1997; Kasari, Chamberlain, &Bauminger, 2001] or “cognitive emotions,” which involveunderstanding of belief, such as surprise [Baron-Cohen,1991; Baron-Cohen, Spitz, & Cross, 1993]. We investigatewhether adults with ASD can understand that a personmay pretend to like a handmade novelty, be confused onreceiving an unwelcome gift such as monopoly money,and be genuinely positive on receiving a welcome gift suchas chocolate. Can participants with ASD successfully gaugethese spontaneous emotional responses, in order toretrodict what gift a person received?

The benefit of this retrodictive mindreading task is thatspontaneous emotion recognition and understandingcan be assessed, while having an objectively correctanswer (the situation that caused the response). Typicallydeveloping individuals can systematically retrodictwhat situation caused a reaction from a brief video clip ofthe response [Pillai et al., 2012]. Thus, it appears thatspontaneous emotional responses provide informationwhich typically developing adults can reliably recog-nize, in order to infer the anteceding situations thatproduce them [Matsumoto et al., 2009; Matsumoto &Willingham, 2006]. Given that adults with ASD havedifficulty interpreting complex emotions, we predictedthat they would have difficulty interpreting such sponta-neous emotional responses and thus exhibit difficultieswith retrodictive mindreading.

We also explore the eye movements of participants,in order to investigate whether, as has been proposedby previous research, people with autism have difficultyinferring emotions from the eye region of faces [e.g.Baron-Cohen et al., 1997; Baron-Cohen et al., 2001].Studies of adults with ASD have not always shown overalldifferences in visual attention to social information, suchas the eyes [e.g. Hernandez et al., 2009; Rutherford &Towns, 2008], but rather first fixation, suggesting a delayin looking to socially pertinent information, rather thanan absence [Fletcher-Watson, Leekam, Benson, Frank, &Findlay, 2009; Fletcher-Watson, Leekam, Findlay, &Stanton, 2008; Freeth, Ropar, Chapman, & Mitchell,2010a, 2010b]. This delay in looking to pertinent socialcues could particularly impact on processing of fast-paceddynamic stimuli [Klin, Jones, Schultz, Volkmar, & Cohen,2002; Speer, Cook, McMahon, & Clark, 2007]. Thusour task, which presents dynamic and complex facialexpressions, investigates whether adults with ASD findretrodictive mindreading difficult because of lack ofattention to pertinent social information, in particularthe eye region of faces.

113Cassidy et al./Inferences from reactionsINSAR

MethodParticipants

The ASD group comprised 16 adults (six female, ten male)aged 20–61 years, recruited from adverts in local media,the National Autistic Society, and through variousautism support groups across the UK. All the participantswith ASD had been formally diagnosed by a clinicianaccording to Diagnostic and Statistical Manual of MentalHealth Disorders, 4th Edition (DSM-IV) criteria (AmericanPsychological Association, 1994). Diagnosis was indepen-dently confirmed by the researchers through the AutismDiagnostic Observation Schedule [ADOS, Lord et al.,1989] and Autism Quotient (AQ, Baron-Cohen et al.,2001b). All participants met the criteria for ASD on theADOS and/or the AQ. The typical group comprised 19adults (7 female, 12 male) aged 17–67 years, recruited fromNottingham University campus and the general popula-tion through adverts to local media.

The full Wechsler Abbreviated Subscales of Intelli-gence (Weschler, 1999) was administered to all partici-pants. Groups were matched on gender, age, full scale,verbal, and performance intelligence quotient (IQ) (seeTable 1 below for participant characteristics and groupcomparisons).

Materials

Twenty-one video clips (ranging from 1.3 to 6 sec induration) were selected as stimuli. The videos were pre-sented on the Tobii (1750) eye-tracker (Tobii Systems,Danderyd, Sweden) in high definition (1920 × 1080i). Eyemovements were recorded using Clearview software(Tobii Systems, Danderyd, Sweden) at a rate of 50 record-ings per second.

Stimuli Development

Stimulus videos were collected from 44 University stu-dents who had volunteered to take part in an unrelatedstudy. After approximately 1 hr of testing, each indi-vidual was paid their inconvenience allowance of £12

and asked if they would be willing to continue for a fewmore minutes by being filmed for a study investigatingcommunication.

Participants sat in a comfortable chair and were filmedfrom 2 ft using a Sony HD Camcorder (Sony, Tokyo,Japan), while reading aloud a list of five randomlyselected neutral words. Immediately after, the experi-menter pretended to turn off the video camera, andthanked the person for staying behind. The experimenterthen offered a reward, which was either a box of high-quality chocolates (n = 9), a poor quality handmade gift(n = 17), or a wad of monopoly money (n = 18).Unknown to the recipient, their ensuing reaction to thegift was filmed, and subsequently, their consent wasobtained for using the video in psychology experiments.

Twenty-one videos were selected for use in the currentexperiment according to certain criteria: face remained infull view of the camera, there was a noticeable reaction,and the recipient did not say what the gift was. Twenty-seven videos did not meet these criteria leaving 21 videos(seven different people receiving chocolate, seven receiv-ing Monopoly money, and seven receiving a handmadegift).

These 21 videos were edited using Final Cut Pro 4(Apple, Cupertino, California, USA) to provide a shortclip of each person’s reaction to the gift: in most cases,neutral to the peak of their reaction and back to neutralagain. If the recipient did not return to a neutral expres-sion, the clip consisted of the peak of their expressionheld for a few seconds. Given that the expressions werenot contrived, the duration of the video clips variedsomewhat, from 1.3 sec to 6 sec (mean chocolate = 2.99(standard deviation (SD) = 0.98), mean homemade =3.63 (SD = 1.17), mean monopoly money = 3.73 (SD =1.18)). A one-way analysis of variance (ANOVA) showedno significant difference in the mean length of thevideo clips for each gift (F(2,18) = 0.91, P = 0.42).

Procedure

Participants were given an information sheet aboutthe study and asked to give their consent to take part.

Table 1. Participant Characteristics

AS/HFA group (n = 16) Control group (n = 19)

t-test ResultMean SD Range Mean SD Range

Age (years) 36.44 12 20–61 30.42 13.49 20–67 t(33) = 1.38, P = 0.18Full-Scale IQ 117.06 14.63 91–140 119.95 8.7 105–136 t(33) = .7, P = 0.5Verbal IQ 114 14.63 92–143 114.58 9.64 97–137 t(32) = .13, P = 0.9Performance IQ 115.13 14.45 86–138 119.89 8.99 107–136 t(32) = 1.2, P = 0.23AQ 37.69 9.46 18–47 N/A N/A N/A N/AADOS 9.27 4.13 4–18 N/A N/A N/A N/A

N.B. Verbal and performance IQ is missing for one participant with ASD.

INSAR114 Cassidy et al./Inferences from reactions

Participants sat approximately 40 cm from the eye-tracking screen. A six-point calibration was conductedbefore the start of the experiment, and participants wereasked to remain as still as possible throughout the experi-ment to prevent any deterioration in calibration.

Participants were told that they would see 21 video-taped reactions of people receiving either a box of choco-lates, a wad of monopoly money, or a handmade gift inexchange for doing a big favor for someone. They wereasked to watch each video carefully and: (a) judge whatgift the target had been offered (out of the three options)and (b) estimate the emotion of the target on beingoffered the gift. All responses were verbal and digitallyrecorded.

The researcher was present to monitor the experimentand was not blind to the emotion viewed by the partici-pant. Each trial presentation sequence consisted of a500 ms blank screen preceding a video clip of a personreacting to getting a gift followed by another blankscreen. The researcher would then ask the participant “doyou think the person in the video got a box of chocolates, atacky glitter card made especially for them, or some fakemoney?” The participant was given as much time as theyneeded to respond to the test question. After giving aresponse, the participant was then asked “How do youthink the person felt when they got the [participant’s giftresponse]?” After the participant had responded to both ofthe test questions, the researcher asked if the participantwas ready for the next video and started the next trial bya key press. Participants were debriefed and paid £5 fortheir time. Ethical approval for the study was granted bythe University of Nottingham, School of PsychologyEthics Committee.

Emotion Description Coding

As participants’ estimations of the target’s emotion werefree response, in order to analyze these data, a coding

scheme was developed which adequately captured therange of emotion labels participants used. Participants’estimations of the recipient’s emotion were coded asbelonging to one of four valence categories:

Positive: Any label which had the connotation of beingpositive. For example, happy, glad, pleasantly surprised,pleased.Negative: Any label which had a negative connotation.For example, displeased, unhappy, disappointed, angry,upset.Pretend: Any label which referred to the participant con-cealing negative emotions. For example, hiding disap-pointment, fake smile, politely accepting.Confused: Any label which did not have a positive ornegative connotation. For example, surprised, confused,puzzled, thoughtful.

Inter-Rater Agreement

Inter-rater agreement for the above coding scheme wasestablished in the current experiment for the typical andASD group’s emotion ratings. These emotion ratings werecoded by the experimenter, along with two independentraters blind to methods and hypotheses. Cohen’s Kappawas used to establish inter-rater agreement when using theabove coding scheme to code participants’ estimations ofemotion. Inter-rater agreement was K = 0.76 for typicalparticipants responses and K = 0.79 for the ASD group’semotion responses. In the current study, participants’estimations of emotion were coded by the experimenter.

ResultsBehavioral Results

Are those with ASD less accurate than typicalcontrols when inferring gift? Table 2 shows theconfusion matrices for participants’ gift inferences in the

Table 2 Confusion Matrices Showing Frequency of Correct and Incorrect Gift Inferences in the ASD and Typical Control Group

Correct answer

TotalChocolate Homemade Monopoly money

a) Typical control groupParticipant’s gift response Chocolate 59 45 29 133

Homemade 31 76 29 136Monopoly money 43 12 74 129Don’t Know 0 0 1 1Total 133 133 133

b) ASD groupParticipant’s gift response Chocolate 37 50 22 109

Homemade 40 44 27 111Monopoly Money 26 14 59 99Don’t Know 9 4 4 17Total 112 112 112

Note. Shaded cells indicate correct gift inferences.


typical control (a) and ASD (b) groups. The typical controlgroup gave more correct than incorrect gift responses.However, those with ASD only gave more correct thanincorrect gift responses when inferring who receivedmonopoly money.

To control for response bias (e.g. more don’t knowresponses in the ASD group), the proportion of correctgift responses were calculated as the number of correctresponses, divided by the total number of times a partici-pant offered that gift response (Fig. 1).

A two-way mixed ANOVA was conducted with group asa between-subjects factor with two levels (ASD, typicalcontrol) and percentage of correct gift inferences as awithin-subjects factor with three levels (chocolate, home-made, and monopoly money). Results showed a signifi-cant main effect of gift (F(2,66) = 16.65, P < 0.001);significantly, more correct homemade gifts were inferredthan chocolate (F(1,33) = 8.65, P < 0.01), and signifi-cantly, more correct monopoly money gifts wereinferred than chocolate and homemade (F(1,33) = 21.97,P < 0.001). There was a significant interaction betweengroup and gift (F(2,66) = 3.35, P < 0.05), a difference con-trast showed that this interaction was driven by a signifi-cantly larger group difference in accuracy for chocolateand homemade than monopoly money (F(1,33) = 4.2,P < 0.05). Bonferroni corrected t-tests showed that thosewith ASD were significantly less accurate than controlswhen inferring who received a homemade gift (P < 0.05),but not chocolate (P = 0.09) or monopoly money(P = 0.81). There was also a significant main effect ofgroup; those with ASD made significantly less correct gift

inferences than the typical control group overall (ASDmean = 45.8%, typical mean = 54.45%, F(1,33) = 4.42,P < 0.05).

What was the pattern of participants’ errors?The raw frequencies in Table 2 show that both groupstend to confuse reactions to homemade gifts with choco-late more than monopoly money. These confusionsbetween chocolate and homemade responses are morepronounced in the ASD group. To compare this patternof errors between groups, a three- way mixed ANOVA wasconducted with group as a between-subjects factor withtwo levels (ASD, typical), correct answer (i.e. what giftthe target received) as a within-subjects factor with threelevels (chocolate, homemade, and monopoly money),and participants’ response as a within-subjects factorwith three levels (chocolate, homemade, and monopolymoney).

The three-way mixed ANOVA showed a significantthree-way interaction between group, correct answer,and participants gift response (F(4,132) = 3.5, P < 0.01).Simple main effects analysis showed no significant differ-ence between correct and incorrect gift responses whenparticipants with ASD inferred who received a chocolategift (F(2,32) = 2.1, P = 0.14). There was a significant dif-ference between correct and incorrect gift responseswhen participants with ASD inferred who received ahomemade gift (F(2,32) = 28.64, P < 0.001) and mono-poly money (F(2,32) = 31.72, P < 0.001). Bonferroni cor-rected t-tests showed no significant difference betweencorrect homemade gift and incorrect chocolate inferences(P = 1), but significantly more correct homemade giftthan incorrect monopoly money inferences (P < 0.001)and significantly more correct monopoly money thanincorrect chocolate (P < 0.001) and homemade gift infer-ences (P < 0.001).

Simple main effects analysis showed a significant dif-ference between correct and incorrect gift responseswhen typical controls inferred who received chocolate(F(2,32) = 7.29, P < 0.01), a homemade gift (F(2,32) =35.05, P < 0.001), or monopoly money (F(2,32) = 31.72,P < 0.001). Bonferroni corrected t-tests showed thattypical controls were significantly more likely to give acorrect chocolate response than an incorrect homemade(P < 0.01) or monopoly money response (P < 0.05); togive a correct homemade response than an incorrectchocolate (P < 0.05) or monopoly money response(P < 0.001); and to give a correct monopoly moneyresponse than an incorrect chocolate (P < 0.001) orhomemade response (P < 0.001).

Neither proportion of correct chocolate (r = 0.02,P = 0.9; r = 0.02, P = 0.9), homemade (r = 0.01, P = 0.7;r = 0.03, P = 0.2), or monopoly money responses (r = 0.02,P = 0.9; r = 0.1, P = 0.6) correlated with full-scale IQ or AQscores respectively in the ASD group.

Figure 1. Percentage of correct gift inferences for the typicaland autism spectrum disorder (ASD) group for each gift. The redline denotes chance level (33%).


Are gift and emotion inferences consistent in bothgroups? Tables 3 and 4 show the observed andexpected frequencies of emotion labels participantsoffered alongside their gift inference for correct (Table 3)and incorrect (Table 4) trials (e.g. observed and expectedfrequency of positive emotion ratings for correct (Table 3)and incorrect (Table 4) chocolate responses).1 If thosewith ASD were less accurate when inferring who receiveda chocolate or homemade gift because they did not

understand which emotions were appropriate in responseto these gifts, then emotion and gift inferences would notbe consistent in each group. In Table 3, values with sub-script A denote correct gift inferences with consistentemotion labels, and in Table 4, values with lower casesubscript a denote incorrect gift inferences with consis-tent emotion labels (i.e. positive for chocolate, pretendfor homemade, and confused for monopoly money).

For correct trials, both typical controls (Table 3a) andthose with ASD (Table 3b) rate reactions to chocolate aspredominantly positive and monopoly money as pre-dominantly confused. The observed frequencies are alsofar higher than expected, whereas inconsistent gift andemotion responses (e.g. positive for monopoly money)

1Please see supplementary data Tables S1–S3 for a full breakdown of giftand emotion response combinations for correct and incorrect trials in theASD and typical control group.

Table 3. The Frequency of Emotion Ratings for Correct Gift Inferences in the ASD and Typical Group

Correct gift response (expected frequencies in brackets)

TotalChocolate Homemade Monopoly money

a) Typical control groupEmotion Positive 46A (27.1) 42 (34.9) 8 (34) 96

Pretend 0 (3.4) 12A (4.4) 0 (4.2) 12Confused 9 (19.5) 10 (25.1) 50A (24.4) 69Negative 3 (8.5) 11 (10.9) 16 (10.6) 30Don’t know 1 (0.6) 1 (0.7) 0 (0.7) 2

Total 59 76 74 209b) ASD groupEmotion Positive 30A (13.2) 20 (15.7) 6 (21) 56

Pretend 0 (3.3) 13A (3.9) 1 (5.3) 14Confused 2 (11.1) 7 (13.2) 38A (17.7) 47Negative 2 (4.5) 4 (5.3) 13 (7.1) 19Don’t know 3 (0.9) 0 (1.1) 1 (1.5) 4

Total 37 44 59 140

Note. Frequencies with subscript A denote correct gift and consistent emotion inference.

Table 4. The Frequency of Emotion Ratings for Incorrect Gift Inferences in the ASD and Typical Group

Incorrect gift response (expected frequencies in brackets)

TotalChocolate Homemade Monopoly money Don’t know

a) Typical control groupEmotion Positive 61a (31.5) 13 (25.6) 6 (23.4) 1 (0.4) 81

Pretend 0 (2.7) 5a (2.2) 2 (2) 0 (0) 7Confused 9 (21) 21 (17) 24a (15.6) 0 (0.3) 54Negative 3 (18.3) 21 (14.8) 23 (13.6) 0 (0.2) 47Don’t know 1 (0.4) 0 (0.3) 0 (0.3) 0 (0) 1

Total 74 60 55 1 190b) ASD groupEmotion Positive 63a (33) 16 (30.2) 6 (18.3) 3 (6.4) 88

Pretend 0 (6.7) 15a (6.2) 3 (3.7) 0 (1.3) 18Confused 1 (14.6) 24 (13.4) 14a (8.1) 0 (2.8) 39Negative 1 (9) 7 (8.2) 13 (5) 3 (1.7) 24Don’t know 7 (8.6) 4 (7.9) 4 (4.8) 8 (1.7) 23

Total 72 66 40 14 192

Note. Frequencies with subscript a denote incorrect gift with consistent emotion inference.


are far lower than expected. Although pretend emotionratings were rare, both groups almost exclusively ascribethis rating to homemade gift inferences and above thelevel expected. This pattern is also evident for incorrecttrials in the typical control (Table 4a) and ASD (Table 4b)groups, although both groups rate homemade responsesas less positive (below expected) and more confused(above expected).

To explore whether participants in both groups tendedto attribute similar emotions to each gift, regardless ofwhether the gift inferred was correct or incorrect, likeli-hood ratios were calculated from the observed frequen-cies in Tables 3 and 4 for each group. The likelihood ratiois a chi-square statistic with the same distribution anddegrees of freedom as Pearson’s chi-square. It is basedon maximum likelihood theory, where a model is createdfrom the total observed frequency of consistent (cellsdenoted by subscript A for correct and a for incorrect giftresponses) and total observed frequency of inconsistentgift and emotion responses, where the probability ofobtaining these data is maximized, then compared withthe probability of obtaining these data under the nullhypothesis. The resulting statistic is therefore based oncomparing the observed frequency of consistent andinconsistent gift and emotion responses with those pre-dicted by the model.

Results showed that both groups gave significantlymore consistent and less inconsistent gift and emotioninferences than predicted by the model, when the giftinferred was correct (typical group, observed consis-tent = 108, expected consistent = 59.9, observed inconsis-tent = 85, expected inconsistent = 122.5, Lχ2(1) = 17.8,P < 0.001; ASD group, observed consistent = 81, expectedconsistent = 36.2, observed inconsistent = 76, expectedinconsistent = 82.4, Lχ2(1) = 26, P < 0.001) or incor-rect (typical group observed consistent = 90, expectedconsistent = 49.4, observed inconsistent = 76, expectedinconsistent = 137.2, Lχ2(1) = 6.3, P < 0.05; ASD groupobserved consistent = 92, expected consistent = 46.8,observed inconsistent = 104, expected inconsistent =149.1, Lχ2(1) = 21.3, P < 0.001).

Eye Tracking Results

Analysis. Regions of interest were defined using theEye Tracker Output Utility [Van Heuven, 2010] over aseries of static pictures, representing the movement ofthe target over the course of the video. The Eye trackerOutput Utility [Van Heuven, 2010] calculated thenumber and duration of fixations in each area of interest(Fig. 2). Percentage number and duration of fixationswere calculated as the number/duration of fixations inthe region of interest divided by the total number offixations/total duration of the video. These percentageswere used in analyses to control for differences in stimu-lus duration and number of fixations between partici-pants. On perusal of the raw eye-tracking data, it becameapparent that loss of calibration (indicated by an errormessage that a majority of fixations were being madeoutside the area of the screen) occurred for one partici-pant with ASD and six typical controls. These data wereexcluded from analysis.

Do participants with ASD look less to the eyes? Athree-way ANOVA compared Region of Interest (ROI)(eyes, mouth) for each gift (chocolate, homemade, andmonopoly money) in each group (ASD, typical control).The three-way ANOVA revealed a significant interactionbetween gift and ROI for proportion of fixations (F(2,52) = 12, P < 0.001) and fixation duration (F(2, 52) = 8.3,P < 0.01), indicating that participants tended to look atthe eyes and mouth differently depending on which giftresponse was viewed. Orthogonal contrasts showed thatthe proportion of fixations (F(1,26) = .02, P = 0.9) andfixation duration (F(1,26) = .001, P = 0.9) to the eyes andmouth were similar when viewing reactions to chocolateand homemade gifts. However, when participants viewedreactions to monopoly money, the proportion of fixa-tions (F(1,26) = 28, P = 0.001) and fixation duration(F(1,26) = 17.3, P = 0.001) to the eyes and mouth weresignificantly different compared with when viewing reac-tions to chocolate and homemade gifts; participantstended to look more to the mouth than the eyes whenviewing reactions to chocolate and homemade and more

Figure 2. Example fixation and saccade data over regions of interest (eyes and mouth) for a participants with autism spectrum disorder(ASD) (above left) and typical participants (above right).


to the eyes than the mouth when viewing reactions tomonopoly money (Figs. 3 and 4).

Although participants with ASD looked longer to themouth than the eyes compared with typical controls,this interaction was not significant for proportion offixations (F(1,26) = 3.7, P = 0.07) or fixation duration(F(1,26) = 2.9, P = 0.1) (Figs. 5 and 6). There were no sig-nificant correlations between proportion of fixation orfixation duration to the eyes or mouth and correct giftresponses when viewing the chocolate, homemade, ormonopoly money responses in the ASD or typical group.

Discussion

This study aimed to develop a new naturalistic emotionrecognition task to assess the subtle emotion recognitiondifficulties adults with ASD experience in everyday life.Can adults with ASD successfully infer what happened tosomeone from their emotional response? Adults with ASDwere not significantly less accurate than typical controlswhen inferring who received monopoly money from abrief video clip of a person’s emotional response. However,those with ASD were less accurate when inferring who

Figure 3. Percentage duration of fixations on the eyes andmouth for each gift.

Figure 5. Percentage duration of fixations on the eyes andmouth in the typical and autism spectrum disorder (ASD) group.

Figure 4. Percentage of fixations on the eyes and mouth for gift.

Figure 6. Percentage of fixations on the eyes and mouth in theautism spectrum disorder (ASD) and typical group.


received a chocolate or homemade gift. We did notfind evidence that a subset of individuals with ASDfound the task especially difficult, as neither IQ nor ameasure of self-reported autistic traits correlated withaccuracy.

Problems inferring who received a chocolate or home-made gift could be due to difficulties with recognizingthese emotions or failing to understand which emotionswere appropriate when receiving these gifts. As bothgroups made consistent gift and emotion inferences sig-nificantly above chance, the findings suggest that adultswith ASD understood which emotions were appropriatein response to each gift: positive for chocolate, confusedfor monopoly money, and pretend for homemade. Thus,reduced accuracy when inferring who received a home-made or chocolate gift was most likely due to difficultyrecognizing positive and feigned positive emotions.However, adults with ASD could successfully recognizewho was confused, and thus correctly recognized whoreceived monopoly money.

Understanding which emotions and behaviors areappropriate in different social situations has rarely beenstudied in ASD, and never before in adults with ASD.Children with ASD fail to identify which behaviors areinappropriate in a range of social situations [Baron-Cohen et al., 1999; Loveland et al., 2001], and have dif-ficulty understanding what situation will cause a complexemotion such as surprise (e.g. Jane will be surprised onopening the empty box of coco pops), but not situationswhich cause basic emotions such as happiness andsadness (e.g. having a birthday party as opposed tograzing a knee) [Baron-Cohen et al., 1993]. In the currentstudy, adults with ASD understood what situation wouldcause a complex emotional response (e.g. feigning a posi-tive response to an unwanted gift). These results suggestthat adults with ASD learn what complex emotionalresponses are appropriate in different social situations,but find it difficult to successfully recognize them.

An alternative interpretation of our findings is thatperhaps people do not need to infer the emotion of theperson in order to retrodict what happened to them. Asparticipants with ASD find the task difficult, perhaps theyguess the gift and consequently infer a consistentemotion. The upshot of this critique, termed “Povinelli’schallenge” [see Perner, 2010; Povinelli & Vonk, 2003], isthat it can be applied to any task assessing emotion rec-ognition. Our results also suggest that when participantscorrectly gauge the emotion of the person (e.g. positive),they also tended to correctly infer what gift the personreceived (chocolate). In addition, when participantsincorrectly interpreted the emotion of the person (e.g.positive, as opposed to confused), they also incorrectly,but consistently inferred what gift they received (choco-late, when they really received monopoly money). Thisassociation between gift and emotion inferences is

unlikely to exist if what gift a person received can beinferred by some other means than interpreting theiremotional response.

Why were participants with ASD less accurate wheninterpreting positive and feigned positive emotions, butnot confused? Previous research has shown that complexmental states such as confused are difficult for peoplewith ASD to recognize, particularly from dynamic facialexpressions [e.g. Back, Ropar, & Mitchell, 2007]. It couldbe the case that people with ASD can distinguish verydifferent emotions (positive from confused), but havedifficulty making more subtle distinctions (genuine fromfeigned positive). Analysis of participants’ errors suggeststhat this was the case; participants with ASD were signifi-cantly more likely to mistake feigned positive reactions tothe homemade gift as a genuine positive reaction tochocolate than typical controls. However, both typicalcontrols and participants with ASD did not tend to mis-interpret feigned positive reactions to the homemade giftas confused reactions to monopoly money or vice-versa.Boraston, Corden, Miles, Skuse, and Blakemore [2008]have presented a similar argument for adults with ASDhaving difficulty making subtle distinctions betweengenuine and posed smiles, which involve attention tosubtle cues in the eye region of faces.

Can reduced accuracy in the ASD group be explainedby reduced attention to the eye region of faces?Baron-Cohen [1991] and Baron-Cohen et al. (2001a) haveargued that people with ASD have difficulty inferringemotions from the eye region of faces, which may beparticularly important for recognition of feigned positiveexpressions [Boraston et al., 2008; Ekman & Friesen,1982; Ekman, Friesen, & Davidson, 1990; Ekman, Friesen,& O’Sullivan, 1988]. In the current study, although par-ticipants with ASD spent less time looking to the eyeregion and more to the mouth region than controls, thisdifference failed to reach significance. Looking to the eyesalso did not predict accuracy in either group. Theseresults suggest that other available cues, such as auditoryinformation, may be important when judging an indi-vidual’s emotional response. Evidence of poor emotionrecognition in vocalizations has been found in adultswith ASD [Heaton et al., 2012; Philip et al., 2010]. Fur-thermore, Golan et al. [2006] have suggested that indi-viduals with ASD may be more focused on speechcontent, failing to integrate contextual and facial cues,causing adults with ASD to misinterpret complex emo-tions such as sarcasm as genuine responses. In the currentstudy, those with ASD could have focused more onspeech content (e.g. “thank you,” for chocolate andhomemade, vs. “OK” for monopoly money), whereastypical controls may have integrated speech content withtone of voice and facial cues. This could have resulted inthose with ASD confusing reactions to chocolate andhomemade gifts, but not monopoly money.


Our results demonstrate the impact of emotion recog-nition difficulties in adults with ASD on an importantsocial skill—being able to make sense of another per-son’s behavior. Adults with ASD have difficultyretrodicting what caused an emotional response whenthis involves recognition of subtle emotion responses,requiring integration of cues across different modalities.These results challenge claims that emotion recognitiondifficulties in ASD are primarily due to lack of attentionto the eye region of faces, and stress the importance ofother emotional cues (e.g. auditory, body movement).Difficulties in retrodictive mindreading could impact onthe way adults with ASD interact with others in every-day life. Failure to recognize whether someone is genu-inely happy or trying to put on a brave face could makethe difference between mistakenly congratulating themon a positive event, or correctly consoling them after adisappointing outcome. Such difficulties in social inter-action are a hallmark of adults with ASD’s difficultiesin everyday life. Future research should aim to eluci-date the subtle nature of emotion processing difficultiesin adults with ASD and their wider impact on socialfunctioning.

Acknowledgments

This research was carried out with the support of a PhDstudentship awarded by the University of Nottingham,School of Psychology.

References

Adolphs, R., Sears, L., & Piven, J. (2001). Abnormal processing ofsocial information from faces in autism. Journal of CognitiveNeuroscience, 13, 232–240.

American Psychiatric Association. (1994). Diagnostic and statis-tical manual of mental disorders (4th edition). Washington,DC: American Psychiatric Association.

American Psychiatric Association. (2000). The Diagnostic andStatistical Manual of Mental Disorders, 4th Edition, Text Revi-sion (DSM-IV-TR). Washington: Authors.

Back, E., Ropar, D., & Mitchell, P. (2007). Do the eyes have it?Inferring mental states from animated faces in autism. ChildDevelopment, 78, 397–411.

Baron-Cohen, S. (1991). Do people with autism understand whatcauses emotion? Child Development, 62, 385–395.

Baron-Cohen, S., Jolliffe, T., Mortimore, C., & Robertson, M.(1997). Another advanced test of theory of mind: Evidencefrom very high functioning adults with autism or Aspergersyndrome. Journal of Child Psychology and Psychiatry, andAllied Disciplines, 38, 813–822.

Baron-Cohen, S., O.’Riordan, M., Stone, V., Jones, R., & Plaisted,K. (1999). Recognition of faux pas by normally developingchildren and children with Asperger syndrome or high-functioning autism. Journal of Autism and DevelopmentalDisorders, 29, 407–418.

Baron-Cohen, S., Spitz, A., & Cross, P. (1993). Do children withautism recognize surprise? A research note. Cognition andEmotion, 7, 507–516.

Baron-Cohen, S., Wheelwright, S., Hill, J., Raste, Y., & Plumb, I.(2001a). The “reading the mind in the eyes” test revisedversion: A study with normal adults, and adults with Aspergersyndrome or high-functioning autism. Journal of Child Psy-chology and Psychiatry, 42, 241–251.

Baron-Cohen, S., Wheelwright, S., Skinner, R., Martin, J., &Clubley, E. (2001b). The Autism Spectrum Quotient (AQ):Evidence from Asperger syndrome/ high functioning autism,males and females, scientists and mathematicians. Journal ofAutism and Developmental Disorders, 31, 5–17.

Bartsch, K., & Wellman, H. (1989). Young children’s attributionof action to beliefs and desires. Child Development, 60, 946–964.

Boraston, Z.L., Corden, B., Miles, L.K., Skuse, D.H., & Blakemore,S.-J. (2008). Brief report: Perception of genuine and posedsmiles by individuals with autism. Journal of Autism andDevelopmental Disorders, 38, 574–580.

Carroll, J.M., & Russell, J.A. (1997). Facial expressions in Holly-wood’s portrayal of emotion. Journal of Personality andSocial Psychology, 72, 164–176.

Ekman, P., & Friesen, W.V. (1982). Felt, false and miserablesmiles. Journal of Nonverbal Behaviour, 6, 238–252.

Ekman, P., Friesen, W.V., & Davidson, R.J. (1990). The Duchennesmile: Emotional expression and brain physiology II. Journalof Personality and Social Psychology, 58, 342–353.

Ekman, P., Friesen, W.V., & O’Sullivan, M. (1988). Smiles whenlying. Journal of Personality and Social Psychology, 54, 414–420.

Fletcher-Watson, S., Leekam, S.R., Benson, V., Frank, M.C., &Findlay, J.M. (2009). Eye-movements reveal attention tosocial information in autism spectrum disorder. Neuro-psychologia, 47, 248–257.

Fletcher-Watson, S., Leekam, S.R., Findlay, J.M., & Stanton, E.C.(2008). Brief report: Young adults with autism spectrum dis-orders show normal attention to eye-gaze information—Evidence from a new change blindness paradigm. Journal ofAutism and Developmental Disorders, 38, 1785–1790.

Freeth, M., Ropar, D., Chapman, P., & Mitchell, P. (2010a). Dogaze cues in complex scenes capture and direct the attentionof high functioning adolescents with ASD? Evidence fromeye-tracking. Journal of Autism and Developmental Disor-ders, 40, 534–547.

Freeth, M., Ropar, D., Chapman, P., & Mitchell, P. (2010b). Thegaze direction of an observed person can bias perception,memory and attention in adolescents with and withoutautism spectrum disorder. Journal of Experimental Child Psy-chology, 105, 20–37.

Gaigg, S.B. (2012). The interplay between emotion and cogni-tion in Autism Spectrum Disorder: Implications for develop-mental theory. Frontiers in Integrative Neuroscience, 6,113.

Gallese, V., & Goldman, A. (1998). Mirror neurons and the simu-lation theory of mind-reading. Trends in Cognitive Sciences,2, 493–501.

Golan, O., Baron-Cohen, S., Hill, J., & Golan, Y. (2006).The “reading the mind in films” task: Complex emotion


recognition in adults with autism spectrum conditions. SocialNeuroscience, 1, 111–123.

Goldman, A., & Sripada, C.S. (2005). Simulationist models offace-based emotion recognition. Cognition, 94, 193–213.

Harms, M.B., Martin, A., & Wallace, G.L. (2010). Facial emotionrecognition in Autism Spectrum Disorders: A review of behav-ioral and neuroimaging studies. Neuropsychology Review,20, 290–322.

Heaton, P.F., Reichenbacher, L., Sauter, D., Allen, R., Scott, S.K.,& Hill, E.L. (2012). Measuring the effects of alexithymia, onperception, of emotional vocalizations in Autistic SpectrumDisorder and typical development. Psychological Medicine,42, 2453–2459.

Hernandez, N., Metzger, A., Magne, R., Bonnet-Brilhault, F.,Roux, S., et al. (2009). Exploration of core features of ahuman face by healthy and autistic adults analyzed by visualscanning. Neuropsychologia, 47, 1004–1012.

Hess, U., & Blairy, S. (2001). Facial mimicry and emotionalcontagion to dynamic emotional facial expressions and theirinfluence on decoding accuracy. International Journal of Psy-chophysiology, 40, 129–141.

Humphreys, K., Minshew, N., Leonard, G.L., & Behrmann, M.(2007). A fine grained analysis of facial expression processingin high functioning adults with autism. Neuropsychologia,45, 658–695.

Kanner, L. (1943). Autistic disturbances of affective contact.Nervous Child, 2, 217–250.

Kasari, C., Chamberlain, B., & Bauminger, N. (2001). Social emo-tions and social relationships: Can children with autism com-pensate? In J.A. Burack, T. Charman, N. Yirmiya, & P.R. Zelazo(Eds.), The development of autism: Perspectives from theoryand research (pp. 309–323). Mahwah, NJ: Lawrence ErlbaumAssociates, Inc.

Klin, A., Jones, W., Schultz, R., Volkmar, F., & Cohen, D. (2002).Visual fixation patterns during viewing of naturalistic socialsituations as predictors of social competence in individualswith autism. Archives of General Psychiatry, 59, 809–823.

Lord, C., Rutter, M., Goode, S., Heemsbergen, J., Jordan, H., et al.(1989). Autism diagnostic observation schedule: A standard-ised observation of communicative and social behaviour.Journal of Autism and Developmental Disorders, 19, 185–212.

Loveland, K.A., Pearson, D.A., Tunali-Kotoski, B., Ortegon, J., &Gibbs, M.C. (2001). Judgements of social appropriateness bychildren and adolescents with autism. Journal of Autism andDevelopmental Disorders, 31, 367–376.

Loveland, K.A., Steinberg, J.L., Pearson, D.A., Mansour, R., &Reddoch, S. (2008). Judgments of auditory-visual affectivecongruence in adolescents with and without autism: A pilotstudy of a new task using fMRI. Perceptual and Motor Skills,107, 557–575.

Matsumoto, A., Olide, A., Schug, J., Willingham, B., & Callan, M.(2009). Cross-cultural judgments of spontaneous facialexpressions of emotion. Journal of Nonverbal Behaviour, 33,213–238.

Matsumoto, A., & Willingham, B. (2006). The thrill of victoryand the agony of defeat: Spontaneous expressions of medalwinners of the 2004 Athens Olympic Games. Journal of Per-sonality and Social Psychology, 91, 568–581.

Millikan, R.G. (2005). Some reflections on the TT-ST debate. In S.Hurley & N. Chater (Eds.), Perspectives on imitation: Fromcognitive neuroscience to social science. Cambridge, MA: TheMIT Press.

Naab, P.J., & Russell, J.A. (2007). Judgments of emotion fromspontaneous facial expressions of New Guineans. Emotion(Washington, D.C.), 7, 736–744.

Neumann, D., Spezio, M.L., Piven, J., & Adolphs, R. (2006).Looking you in the mouth: Abnormal gaze in autismresulting from impaired top-down modulation of visualattention. Social Cognitive and Affective Neuroscience, 1,194–202.

Ogai, M., Matsumoto, H., Suzuki, K., Ozawa, F., Fukuda, R., et al.(2003). fMRI study of recognition of facial expressions inhigh functioning autistic patients. Neuroreport, 14, 559–563.

O’Sullivan, M. (1982). Measuring the ability to recognize facialexpressions of emotion. In P. Ekman (Ed.), Emotion in thehuman face (pp. 281–314). New York, NY: Cambridge Uni-versity Press.

Perner, J. (2010). Who took the Cog out of Cognitive Science?—Mentalism in an Era of Anti-cognitivism. In P.A. Frensch & R.Schwarzer (Eds.), Cognition and neuropsychology: Interna-tional perspectives on psychological science (Vol. 1, pp. 451–483). East Essex: Psychology Press.

Philip, R.C.M., Whalley, H.C., Stanfield, A.C., Sprenglemeyer, R.,Santos, I.M., et al. (2010). Deficits in facial, body movementand vocal emotional processing in Autism Spectrum Disor-ders. Psychological Medicine, 40, 1919–1929.

Pillai, D., Sheppard, L., & Mitchell, P. (2012). Can People GuessWhat Happened to Others From Their Reactions? PLoS ONE.7, e49859.

Povinelli, D.J., & Vonk, J. (2003). Chimpanzee minds: Suspi-ciously human? Trends in Cognitive Sciences, 7, 157–160.

Robinson, E.J., & Mitchell, P. (1995). Masking of children’s earlyunderstanding of the representational mind: Backwardsexplanation versus prediction. Child Development, 66, 1022–1039.

Roeyers, H., Buysse, A., Ponnet, K., & Pichal, B. (2001). Advanc-ing advanced mind-reading tests: Empathic accuracy inadults with a pervasive developmental disorder. Journal ofChild Psychology and Psychiatry, 42, 271–278.

Rutherford, M.D., & Towns, A.M. (2008). Scan path differencesand similarities during emotion perception in those with andwithout Autism Spectrum Disorders. Journal of Autism andDevelopmental Disorders, 38, 1371–1381.

Speer, L.L., Cook, A.E., McMahon, W.M., & Clark, E. (2007). Faceprocessing in children with autism. Autism: The Interna-tional Journal of Research and Practice, 11, 265–277.

Spezio, M.L., Adolphs, R., Hurley, R.S.E., & Piven, J. (2007a).Analysis of face gaze in autism using “bubbles. Neuro-psychologia, 45, 144–151.

Spezio, M.L., Adolphs, R., Hurley, R.S.E., & Piven, J. (2007b).Abnormal use of facial information in high-functioningautism. Journal of Autism and Developmental Disorders, 37,929–939.

Uljarevic, M., & Hamilton, A. (2013). Recognition of emotions inautism: A formal meta-analysis. Journal of Autism and Devel-opmental Disorders, 43, 1517–1526.


Van Heuven, W. (2010). Eye Tracker Output Utility (Version 1.2Beta 18) [Computer Software]. Nottingham: University ofNottingham. Retrieved August 22, 2010. Available fromhttp://www.psychology.nottingham.ac.uk/staff/wvh/eou/index.html

Wagner, H.L. (1990). The spontaneous facial expression of dif-ferential positive and negative emotions. Motivation andEmotion, 14, 27–43.

Wagner, H.L., Lewis, H., Ramsay, S., & Krediet, I. (1992).Prediction of facial displays from knowledge of norms ofemotional expressiveness. Motivation and Emotion, 16, 347–362.

Wagner, H.L., MacDonald, C., & Manstead, A.S.R. (1986). Com-munication of individual emotions by spontaneous facialexpressions. Journal of Personality and Social Psychology, 50,737–743.

Wechsler, D. (1999). WASI manual. San Antonio: PsychologicalCorporation.

Wing, L., & Gould, J. (1979). Severe impairments of socialinteraction and associated abnormalities in children:

Epidemiology and classification. Journal of Autism andDevelopmental Disorders, 9, 11–29.

Supporting Information

Additional Supporting Information may be found inthe online version of this article at the publisher’sweb-site:

Table S1. The percentage of emotion and gift responsecombinations for chocolate in the (a) ASD and (b) typicalcontrol group.Table S2. The percentage of emotion and gift responsecombinations for homemade in the (a) ASD and (b)typical control group.Table S3. The percentage of emotion and gift responsecombinations for monopoly money in the (a) ASD and(b) typical control group.


http://www.psychology.nottingham.ac.uk/staff/wvh/eou/index.html

http://www.psychology.nottingham.ac.uk/staff/wvh/eou/index.html

can adults with autism spectrum disorders infer what happened to someone from their

Documents