running head: ados – toddler modulerpetrak/ados-t manuscript 7… · web viewrunning head: ados...

ADOS – Toddler Module 1

Running Head: ADOS – TODDLER MODULE

The Autism Diagnostic Observation Schedule – Toddler Module:

A new module of a standardized diagnostic measure for autism spectrum disorders

Rhiannon Luyster, Katherine Gotham, Whitney Guthrie, Mia Coffing and

Rachel Petrak

University of Michigan Autism and Communication Disorders Center

Pamela DiLavore

University of North Carolina–Chapel Hill

Karen Pierce

University of California – San Diego

Somer Bishop, Amy Esler, Vanessa Hus, Jennifer Richler, Susan Risi and

Catherine Lord

University of Michigan Autism and Communication Disorders Center

Rhiannon Luyster, 07/02/08,

FYI, APA has authors listed by affiliation when work was done – all new affliations are in the author note.


Abstract

The Autism Diagnostic Observation Schedule (ADOS; Lord et al., 2000) is

widely accepted as a “gold standard” diagnostic instrument, but its

downward limits restrict utility in research samples of very young children

at risk for ASD. The purpose of the current project was to modify the ADOS

for use in children between 12 and 30 months of age. A modified ADOS, the

ADOS Toddler Module (or Module T), was used in 362 evaluations.

Participants included 182 children with best estimate diagnoses of ASD,

non-spectrum developmental delay or typical development. A final set of

protocol and algorithm items was selected based on their success in

discriminating the diagnostic groups. The traditional algorithm “cutoffs”

approach yielded high sensitivity and specificity, and a new range of

concern approach was proposed. In sum, the ADOS – Toddler Module may

be a useful component of evaluations for minimally-verbal children between

the ages of 12 and 30 months who are suspected of having an ASD.


Almost ten years ago, the standardization of a revised Autism

Diagnostic Observation Schedule (ADOS), a semi-structured assessment for

the diagnosis of autism spectrum disorders (ASD) (Lord, Rutter, DiLavore &

Risi, 1999) was described. The ADOS has gradually become an integral

part of many research and clinical protocols of children suspected of having

ASD. Due toBecause of the growing understanding of symptoms in the first

two years of life and the desire of researchers and clinicians to have

standardized instruments for use with infants and young toddlers, there is a

need for diagnostic tools that are appropriate for very young children. The

present paper reports on a new Toddler Module of the ADOS. The Toddler

Module retains the original spirit and many of the original tasks of the

ADOS, but is intended for use in children between the ages of 12 and 30

months of age for whom ASD is a concern.

In introducing this new module, it is valuable to review the structure

of the previously published ADOS. The ADOS evaluates social interaction,

communication and play by introducing a series of planned “presses” (Lord

et al., 1989) to the individual in the context of a naturalistic social

interaction with the examiner. Some of the presses are intended to provide

a high level of structure for the participant, while others are intended to

provide less structure. All presses, however, are used to provide contexts

for both initiations and responses, which are then coded in a standardized

manner. An algorithm, which sums the scores of particular items from the


measure, provides a classification indicative of autism, ASD or non-

spectrum conditions. This classification can then be used by a clinician or

researcher as one part of a comprehensive diagnostic process.

The first ADOS was introduced in the late 1980s and was intended for

children who had spoken language age equivalent to at least 36 months. A

revision was published in 2000 that reflected the need for the measure to be

applicable to a wider range of chronological and developmental ages. The

2000 version provided four separate (but overlapping) modules for

individuals of different ages and language abilities. The updated ADOS (i.e.,

Module 1) did indeed extend the usefulness of the original ADOS below a

language age of 3 years, but research has indicated that it remains of

limited value for children with nonverbal mental ages below 16 months

(Gotham, Risi, Pickles & Lord, 2007). For this young population, the ADOS

Module 1 algorithm is over-inclusive, meaning that it classifies about 81

percent (19% specificity) of children with intellectual disabilities and/or

language impairments as having autism or ASD when clinical judgment

deems that they do not. Revised Module 1 algorithms (Gotham et al., 2007)

improve specificity but only to 50%.

However, in recent years, it is precisely this age range (children in the

first two years of life) which has become one of the central concentrations

of autism research efforts. Researchers have used creative methodologies


to explore the early differences in children who are later diagnosed with

ASD, including retrospective videotape analysis, as well as the identification

of infants at high risk for ASD (usually the younger siblings of children

diagnosed with ASD). A number of standardized direct observational

measures have been developed for use with young children at risk for ASD,

such as the Autism Observational Scale for Infants (AOSI; Bryson,

Zwaigenbaum, McDermott, Rombough & Brian, 2008), Screening Tool for

Autism in Two-Year-Olds (STAT; Stone, Coonrod, Turner & Pozdol, 2004)

and the Communication and Symbolic Behavior Scales Developmental

Profile (CSBS-DP; Wetherby, 2001). The ADOS has been of limited use in

these projects, because many of the children fell chronologically or

developmentally below the floor of the measure.

A standardized diagnostic measure applicable for infants and young

toddlers is also needed for early identification efforts. As public awareness

of ASD heightens, parents have been more likely to seek out an evaluation

for their very young children. The average age of parental concern is

between 15 and 18 months (Chawarska et al., 2007; DeGiacomo &

Fombonne, 1998), and some parents (particularly those who already have

one child on the spectrum) have concerns about their child from the earliest

months of life. Early identification has been strongly promoted by federal

and advocacy organizations with the idea that earlier provision of services

will be associated with better outcomes. These findings all point to the


need for professionals to be equipped to handle diagnostic assessments for

very young children. The Toddler Module should be a useful component of

such assessments. One caveat, however, is that diagnostic decisions made

very early in life are less stable than ones made, for instance, at ages closer

to 3 years (Charman et al., 2005; Turner & Stone, 2007). This has been

taken into consideration in recommendations for interpretation of the

Toddler Module scores (discussed below).

The Toddler Module offers new and modified ADOS activities and

scores appropriate for children at chronological ages from 12 to 30 months

who have minimal speech (no or single spoken words), have a nonverbal age

equivalent of at least 12 months and are walking independently.

Communication, reciprocal social interaction, and emerging object use

and/or play skills are all targeted by the module. The ADOS, particularly

Modules 1 and 2 (intended for developmentally younger children), is

designed around the general model that the examiner presents loosely

structured and highly motivating materials and activities (e.g., bubbles,

snack, remote activated toys) in order to see how the child responds, and

whether he/she then makes initiations in order to maintain the interaction.

As in the previously published ADOS modules, each activity of the Toddler

Module provides a hierarchy of presses for the examiner. Eleven activities

are included (see Table 1), and there are 41 accompanying ratings.


[INSERT TABLE 1 ABOUT HERE]

The Toddler Module follows the same basic structure as the Module 1.

It should be conducted while moving around a small, child-friendly room,

and a familiar caregiver should always be present. Simpler cause-and-

effect materials are included as well as toys that require the development of

more representational and/or imaginative play. Similarly, because some of

the Module 1 activities – such as a pretend birthday party – may be new to

younger children, more familiar contexts (i.e., a bath-time routine) have

been substituted.1

Another substantial design change was made because younger

children may make fewer explicit and directed initiations towards an

unfamiliar adult than older children (Sroufe, 1978). Consequently, in the

Toddler Module, we have added more instances of the examiner structuring

an interaction and waiting for a minimal change in the child’s behavior,

such as a shift in gaze, facial expression or vocalization. The new activities

require less complex motor responses than the Module 1 tasks and the

structure of some activities has been exaggerated.

As with other ADOS modules, detailed notes should be taken by the

examiner during administration and coding should be done after the module

is complete. Perhaps even more so than other modules, the success and

1 Inquires about Toddler Module protocols, kits and training should be directed to Western Psychological Services.


validity of the Toddler Module is dependent on the skill of the examiner.

Infants and toddlers, whether typically developing or not, are particularly

sensitive to the introduction of new situations and new people (Bohlin &

Hagekull, 1993). Indeed, this age range is the one associated with the

development of important components of social and environmental

awareness, such as stranger anxiety. As such, the validity of the Toddler

Module assumes the clinical skills needed to navigate the needs of very

young children and carry out the administration and scoring in a reliable

fashion.

Design Decisions

It was necessary to determine at what developmental point children

should receive the Module 1, rather than the Toddler Module. Preliminary

analyses indicated that Module 1 ADOS sensitivity and specificity for

children over the age of 30 months was superior to the Toddler Module.

For this reason, children over 30 months of age were not included in any

further analyses, and the methods below describe only the final

psychometric dataset for this module. Once a child is over the age of 30

months or has spontaneous, non-echoed phrases made up of three

independent units, he/she should receive the Module 1 or Module 2 of the

ADOS, respectively.

Diagnostic Algorithm


A subset of items comprise the diagnostic algorithms (see Table 2),

following the format of the other modules. Different algorithms are

proposed for children who use some words during the administration of the

Toddler Module and those who do not. Because of the similarities in

distribution noted in younger toddlers and older nonverbal toddlers, two

groups were the focus of algorithm selection: (1) all children from 12 to 20

months old and nonverbal children between 21 and 30 months, and (2)

children who use words and who are 21 to 30 months old. The process for

deciding on these groups and determining the algorithms is described

below. These algorithm items are structured according to the domains

used in the revised ADOS algorithms (Gotham et al., 2007): Social Affect

and Restricted, Repetitive Behaviors. All items contribute to one overall

score with a single diagnostic cutoff.


Recent research has indicated that early diagnostic classification

within the autism spectrum (making a distinction between the specific

diagnoses of autism and pervasive developmental disorder – not otherwise

specified, or PDD-NOS) is relatively unstable in young children, even though

diagnoses of ASD more broadly versus other, non-spectrum disorders are

consistent over time. Lord et al. (2006) reported that 14 percent of children

diagnosed with autism at age 2 shifted their diagnosis to PDD-NOS by age


9. Moreover, in children with an age 2 diagnosis of PDD-NOS, 60 percent

shifted into an autism classification by age 9. Turner and colleagues (2006),

using another sample, report similar levels of diagnostic uncertainty within

the autism spectrum but in the opposite direction, as have other

investigations (Kleinman et al., 2008).

Consequently, the Toddler Module includes only two classifications:

ASD or non-spectrum. Because of the newness of these methods, the

relatively small sample sizes and the care required in interpreting these

results, the emphasis for clinical interpretation is on ranges of scores

associated with each algorithm. These ranges are associated with the need

for clinical follow-up (rather than a focus on a cutoff for ASD) and can

reflect little-or-no, mild-to-moderate, or moderate-to-severe concern.

Scores falling into the little-or-no concern range suggest that the child does

not appear to be demonstrating more behaviors associated with ASD than

children in this age range who do not have ASD. Algorithm scores that fall

into the mild-to-moderate range of concern indicate a sufficient number of

ASD behaviors to warrant further ASD-specific evaluation and follow-up, but

scores in this range may not be inconsistent with other, non-ASD conditions.

Algorithm scores falling into the moderate-to-severe range of concern are

more indicative of an ASD.


The focus of the present paper is on ADOS scores that reflect a child’s

current clinical best estimate diagnosis. In the long run, the predictive

validity of these scores is extremely important but is beyond the scope of

this paper. Cutoffs for ASD versus non-spectrum were generated for the

research and statistical purposes of this project. As with the rest of the

ADOS, the algorithm score should never be used as the only source of

information in generating a diagnostic classification. Details about a child’s

developmental history, parent descriptions and current cognitive, social,

language and adaptive functioning across a variety of contexts are all

necessary for appropriate diagnosis and recommendations (National

Research Council, 2001).

Method

Participants

The sample included all children between the ages of 12 and 30

months from three sources: (1) consecutive referrals of children from 12 to

30 months of age from the clinic at the University of Michigan Autism and

Communication Disorders Center, (2) children from University of Michigan

projects studying early development of children with communication delays

and/or at risk for ASD (predominantly younger siblings of children on the

autism spectrum), as well as comparison groups of children recruited for

these projects and (3) children participating in research at the University of


California – San Diego Autism Center of Excellence. “Best estimate” clinical

or research diagnoses were assigned based on clinical impressions of a

clinical psychologist or an advanced graduate student in psychology.

Information from a research version of the Autism Diagnostic Interview-

Revised (ADI-R, a parent interview; Rutter, Le Couteur & Lord, 2003),

modified to be appropriate for toddlers (see Lord, Shulman & DiLavore,

2004) and direct observation (which included the Toddler Module and

standardized language and cognitive testing) was available. Thus, clinical

diagnosis was not independent of the ADOS but algorithms were not

derived until after the samples were collected.

The final sample included data from 162 participants at the University

of Michigan Autism and Communication Disorders Center; and data from an

additional 20 participants from the University of California, San Diego. The

project included children with typical development (TD), non-spectrum

disorders (NS) and ASD. All individuals with NS and TD did not meet

standard ADI-R criteria for ASD (Risi et al., 2006) and received best

estimate diagnoses outside the autism spectrum. Non-spectrum

participants had a range of diagnoses, including 14 children with expressive

language disorders, 5 children with mixed receptive-expressive language

disorders, 9 children with non-specific intellectual disability, 4 children with

Down syndrome, and 1 child with Fetal Alcohol Syndrome. In addition, one

child had been diagnosed with chromosomal abnormalities, one with anxiety


disorder – not otherwise specified, one with communication disorder – not

otherwise specified and one with phonological disorder. These children

were included to demonstrate that the Toddler Module does not consistently

identify ASD in children who did not have a best estimate diagnosis of a

spectrum condition but did have similar developmental levels to the ASD

sample.

As part of ongoing longitudinal studies, many participants from each

site were seen more than once. These children were seen by a new clinician

every six months, who was blind to their previous performance and

tentative diagnosis. Altogether, data were used for 182 individuals, who

were seen 362 times in total. There was an average of 2.01 (SD = 2.48)

assessments per participant. For the majority of the validity and reliability

analyses reported below, data were analyzed separately for two groups

defined by verbal status during the assessment (“verbal” included children

who received scores of ‘0’, ‘1’, or ‘2’ on the item “Overall Level of

Language”; “nonverbal” included children who received scores of ‘3’ or ‘8’

on this item). Score distributions differed by verbal/nonverbal status in

children between 21 and 30 months of age. However, distributions of

scores for participants younger than 21 months did not systematically vary

by verbal/nonverbal status and generally resembled those of nonverbal

participants aged 21-30 months. Therefore, the developmental groups were

assigned as follows : (1) all children between 12 and 20 months of age as


well as nonverbal children between 21 and 30 months of age (hereafter

referred to as “12-20/NV21-30”); and (2) verbal children between 21 and 30

months of age (“V21-30”). Within each group, data were only used for one

time point for each child, so that participants were only represented once in

each developmental group (though the same participant could be included

once in both groups). There were 135 participants in the 12-20/NV21-30

group (112 children between 12 and 20 months and 23 nonverbal children

between 21 and 30 months) and 71 participants in V21-30 group. This set

of groups with one data point per participant was termed “Unique

Participants.” Both “Unique Participants” developmental groups were

created such that average chronological age and/or nonverbal mental age

were approximately equivalent across the three diagnostic groups. See

Table 3 for sample characteristics.


Analyses were also run for data from all assessments for all

participants in order to take advantage of the larger sample size afforded by

including repeated measurements. For these analyses, there were 239 cases

in 12-20/NV21-30 (193 cases from children between 12 and 20 months and

46 cases from nonverbal children between 21 and 30 months), and 122

cases in V21-30 (see Table 4). This set of groups was termed “All Cases.”

For these analyses, groups were generally not equivalent on measures of


mental age and may have been affected by recruitment biases (e.g., non-

spectrum children with more ASD-like symptoms were seen more frequently

than children with non-spectrum diagnoses and fewer ASD-related

behaviors).


Each participant received a minimum of one psychometric evaluation

using the Mullen Scales of Early Learning (Mullen, 1995), which yielded

verbal and nonverbal language age equivalents; for children with repeated

assessments, the Mullen was re-administered every six months. All

participants were ambulatory, and none had sensory (visual or hearing)

disorders or severe motor impairments.

Fourteen Toddler Module administrations from 13 participants (all of

whom were included in this larger sample) were selected to be included in

the inter-rater reliability analyses for the Toddler Module, based on

scorings done by seven independent raters. These administrations were

selected on the basis of the quality of their videotapes and because they

were not known to the reliability coders. Eight of these participants had

best estimate diagnoses of ASD, 3 participants were typically developing, 1

had a diagnosis of mental retardation, and 1 had a diagnosis of Down

syndrome.

Procedures


The Toddler Module was administered as part of an assessment by

clinical research staff and was scored immediately after administration was

complete. Over the course of 43 months, 22 different examiners

participated in this study. All examiners observed and coded numerous

Toddler Modules and had attained three consecutive scorings of at least

80% exact agreement with other reliable coders on item-level scores (at

least two of which had to be their own administrations) prior to becoming

an independent examiner. Inter-rater reliability was computed and

discussed during weekly meetings in order to ensure that drift did not

occur.

Testing was generally administered in a research room, with tables

and chairs appropriate for young children. A familiar caregiver was always

present in the room. Coding of the Toddler Module was based solely on the

behaviors which occurred during the administration of the measure. This

included observations of whether a child was “verbal” (i.e., used phrases or

at least 5 single words or word approximations). Behaviors which occurred

outside the assessment or during administration of another measure were

not considered. Consent, which was approved by the University of

Michigan Medical School Institutional Review Board for Human Subject

Research, was given by parents. Families in longitudinal projects received

oral feedback and a brief report; participants in other studies received a gift

card to a local store.


Inter-rater reliability was assessed using administrations from 13

children (14 sessions), which were independently coded from videotape by

each of seven “blind” raters. Each rater had previously established 80%

item-level agreement with at least one other rater on three consecutive

administrations of the Toddler Module.

Results

Preliminary analyses

Numerous drafts of the Toddler Module were generated and

evaluated, yielding preliminary results and allowing structural decisions

about the measure. To start with, items which were judged to be

appropriate for infants and toddlers were selected from Module 1 of the

ADOS and from the PL-ADOS, an early version of the ADOS intended for

pre-verbal children (DiLavore, Lord & Rutter, 1995). Additional activities

and codes were written based on a review of empirical studies on early

development (Behne, Carpenter, Call & Tomasello, 2005; Phillips, Baron-

Cohen & Rutter, 1992). Some of the items from previous ADOS versions

were re-written to be more appropriate for younger children, and all codes

were structured to include scores of '3'.

Proposed items (some of which are included in the final versions and

some of which have been eliminated) were used during child assessments

and were reviewed and revised during weekly meetings of clinical and

rpetrak, 07/16/08,

This paragraph repeats information from the previous page; it may flow better to move this info there.


research staff. As the project progressed, new codes were added in order to

capture additional aspects of child behavior. New examiners and examiners

who had previously established reliability on ADOS Modules 1 and 2 then

established 80 percent agreement in pairs of raters on each item in order to

ensure that inter-rater reliability could be obtained.

For children who had more than one assessment in the last six months

of the project, all available data, including research diagnosis history over

the most recent months and chart notes, were used by two examiners to

generate consensus best estimate “working diagnoses.” More weight was

given to most recent diagnosis and “blind diagnoses” made by an examiner

not familiar with the child. Distributions of scores on each item were

generated within cells of children grouped by chronological age, verbal

level and diagnosis. Items which appeared to be “too hard” or “too easy” –

that is, where typically developing children were often scoring in the ‘2’ to

‘3’ range or where children with ASD were frequently scoring in the ‘0’ to

‘1’ range – were re-written. Additionally, items where the scores fell only

between ‘0’ and ‘2’ (that is, few children were scoring in the ‘3’ category)

were revised to expand the distribution. Items were eliminated if their

distributions, even with revisions, did not successfully distinguish among

the diagnostic groups (ASD versus typically developing and ASD versus non-

spectrum) using one-way ANOVAs. The few exceptions to this criteria were

items which were low-incidence but clinically significant. When all item


revisions were complete, two researchers (blind to child diagnosis)

reviewed the videotaped administrations and/or notes to re-score the

revised items.

Validity Study

The goal of the validity study was to create a modified set of codes

and algorithm items that could be used with children between 12 and 30

months of age. This was a complex procedure which was completed

through the following steps.

Validity of Individual Items.

Following item revisions and recoding described above, validity was

assessed on a final set of 41 items which either showed markedly different

distributions across diagnostic groups or which had high clinical or

theoretical importance, but rare endorsements (e.g., self-injurious

behavior). The Toddler Module items are generally scored on a 4-point

scale, ranging from ‘0’ (no evidence of abnormality related to autism) to ‘3’

(definite evidence, such that behavior interferes with interaction).

Correlation matrices were generated according to diagnostic group using

data from unique participants; these included the complete item set as well

as verbal and nonverbal mental age, verbal and nonverbal IQ, and

chronological age variables. Items which were highly correlated with each

other were identified, and some items were eliminated from consideration


for the toddler algorithm in order to reduce collinearity. No items were

excluded based on high correlations with any of the participant

characteristics listed above; the strongest association noted was between

scores on the “Overall Level of Language” item and verbal IQ (r=-.71 across

diagnostic groups, n=113).

Exploratory factor analyses were then conducted using Mplus

(Muthen & Muthen, 1998). Due to small sample size, these analyses were

not intended to identify a latent class structure for the item data, but rather

to provide one approach to assessing the potential influence of cognitive

level or chronological age on these data. When analyses were run on ASD

cases only, verbal mental age did not load onto any factor for either

developmental group. Chronological age ceased to load onto any factor

when the sample was divided into the two developmental groups

(12-20/NV21-30 and V21-30).

Validity of Algorithm Items.

Item means and standard deviations were generated across diagnostic

groups, and items were chosen that best differentiated between diagnoses

within each developmental group. This process of choosing candidate

algorithm items was undertaken separately for the “Unique Participants”

and “All Cases” subsets, and it was also repeated within the following

narrow age/language groups: 12 to 14 months (all children were nonverbal),


15 to 20 months (nonverbal only), 15 to 17 months (verbal only), 18 to 20

months (verbal only), and 21 to 30 months (verbal and nonverbal

separately). A pool of 17 items was identified as strong candidates for a

new Toddler Module algorithm based on their differential distributions

across diagnostic group and their relatively low correlations with each other

and with chronological age and IQ. Some of these items were new items in

the Toddler Module and others had been included in previous Module 1

ADOS algorithms. Similarities in diagnostically differential items across the

younger (under 21 months) and nonverbal groups, as well as a distinct

“best” item set for older verbal toddlers, confirmed the validity of the two

developmental groupings used for these analyses.

Next, best items for each developmental group were summed to

generate trial algorithms specifically for the 12-20/NV21-30 and V21-30

groups. Cases missing data from more than 2 algorithm items were

excluded from these analyses. Scores of ‘2’ and ‘3’ were collapsed in

candidate items following the ADOS convention intended to prevent any one

item from exerting undue influence on the total score, and conversely, a

score of ‘1’ on the Unusual Eye Contact item was converted to ‘2’ on the

algorithms in order to reflect the importance of even subtle differences in

eye contact. Receiver Operating Characteristic (ROC) curves (Siegel,

Vukicevic, Elliott & Kraemer, 1989) allow sensitivity and specificity

percentages to be generated for each total score in a scale. For


12-20/NV21-30 cases, sensitivity and specificity was generated for both trial

toddler algorithms as well as the ADOS Module 1, No Words algorithm for

“Unique Participants” and “All Cases” subsets of data. For V21-30 cases,

ROC curve analyses were run for both trial toddler algorithms and the

ADOS Module 1, Some Words algorithm for both “Unique Participants” and

“All Cases” subsets. Specificity was evaluated in comparisons of ASD versus

non-spectrum participants, and again for ASD versus non-spectrum and

typical cases combined, for all possible cutoffs in each of the three possible

algorithms. These algorithms were then re-tested by systematically

omitting items to ensure that each item contributed to the final

differentiations. Within each developmental group, the strongest algorithm

out of the three tested was selected by identifying the cutoff score that

maximized both sensitivity and specificity across “Unique Participants” and

“All Cases” subsets, and that maintained specificity in ASD versus non-

spectrum distinctions as well as ASD versus non-spectrum and typical

combined. The results are shown in Table 5.


For children under 21 months and nonverbal toddlers, the same set of

items that comprise the ADOS Module 1, No Words algorithm also

maximized predictive validity of this measure, though it is important to note

that codes and scores associated with items of the same name in the


Toddler Module and Module 1 are not identical. A cutoff of 12 on this 12-

20/NV21-30 algorithm yielded 91% sensitivity and 91% specificity for ASD

versus non-spectrum comparisons of unique participants. This cutoff also

maintained sensitivity values at 87% or greater and specificity at 86% or

greater when applied to “All Cases” and comparisons of typically developing

children (see Table 5 for details).

For verbal toddlers between 21 and 30 months of age, a new

algorithm was superior to the Module 1, Some Words algorithm. As shown

in Table 5, a cutoff of 10 on this V21-30 algorithm yielded sensitivity of 88%

and specificity of 91% in the ASD versus non-spectrum unique participants.

Sensitivity was maintained at 83% or greater and specificity at 86% or

greater for all other comparisons (including “All Cases”). The V21-30

algorithm is comparable in structure to the ADOS revised algorithms, with

14 items organized into Social Affect (SA) and Restricted, Repetitive

Behaviors (RRB) domains (see Table 2 for a list of items by domain). In the

new V21-30 algorithm, however, only three of these items describe RRBs

versus four RRB items in the 12-20/NV21-30 and other revised algorithms

across ADOS modules. This difference in maximum RRB total score between

the 12-20/NV21-30 and V21-30 algorithms was not theoretically motivated

but rather reflects the selection of items that maximized predictive value of

the new algorithms in these developmental groups.


To improve clinical utility of this measure, ranges of concern were

identified for the new V21-30 algorithm and the 12-20/NV21-30 algorithm

used with young or nonverbal toddlers. Using the “Unique Participants”

data, three ranges of concern were set for each algorithm, such that at least

95% of children with ASD and no more than about 10% of typically

developing children would fall in the two groups suggesting clinical concern

(mild-to-moderate and moderate-to-severe). See Table 6 for results.


For both developmental groups, 82% of children with non-ASD

developmental delays were accurately assigned to the little-or-no concern

range.

In the new V21-30 algorithm, item-total correlations ranged from .49

(“Response to Name”) to .82 (“Quality of Social Overtures”) for the Social

Affect domain, and from .18 (“Hand and Finger Mannerisms”) to .42

(“Unusual Sensory Interests”) for the three items comprising the RRB

domain (the third being “Unusually Repetitive Interests or Stereotyped

Behaviors,” r=.37). Lower correlations for the RRB domain were expected

given the heterogeneous nature of these items. Cronbach’s alpha was .90

for the SA domain and .50 for the RRB domain, indicating strong and

acceptable internal consistency respectively. Correlations between domain

totals and participant characteristics (e.g., chronological age, gender,


mental age, and IQ) were evaluated within the “Unique Participants” subset

only, because of the known effects of recruitment on the composition of the

“All Cases” sample. In the older group of verbal toddlers, domains were

correlated at .64 with each other, and domain total correlations were no

greater than .55 with verbal IQ (SA total, r=-.55; RRB total, r=-.53).

For the younger or nonverbal children receiving 12-20/NV21-30

algorithm, item-total correlations ranged from .35 (“Gestures”) to .81

(“Quality of Social Overtures”) in the SA domain, and from .14 (“Hand and

Finger Mannerisms”) to .44 (“Unusually Repetitive Interests or Stereotyped

Behaviors”) for the four-item RRB domain. Internal consistency was similar

to the older, verbal group findings, with a Cronbach’s alpha of .88 for the

SA domain and .50 for the RRB domain. For “Unique Participants” in this

developmental group, the domains were correlated at .57 with each other;

correlations with participant characteristics did not exceed .60 for the SA

total (r=-.58 with verbal mental age) or .40 for the RRB total (r=-.34 with

verbal mental age).

For both algorithms, SA and RRB domain total scores were

significantly higher for the ASD sample than the non-spectrum or typically

developing groups. The two non-ASD diagnostic groups (non-spectrum and

typically developing) differed from each other significantly in SA scores but


not in RRB scores (one-way ANOVA and Tukey test statistics available from

the authors).

Reliability Study

Reliability of Individual Items.

For reliability analyses, scores indicating that the item was not

applicable (generally these were language-related items) were converted to

zeros, as is done for algorithm use in the other ADOS modules. Three codes

were either rare or considered particularly valuable in interpreting child

behavior (“Stereotyped/Idiosyncratic Use of Words or Phrases,” “Self-

Injurious Behavior,” and “Overactivity”) received a limited range of scores

and were therefore eliminated from the reliability analyses. STATA

software (StataCorp, 2007) was used to generate weighted kappas for non-

unique pairs of raters (i.e., 28 pairs). Kappas could not be computed for the

three items listed above due to limited distributions (although all had

percent agreements equal to 90% or above), but they were retained in the

protocol because of their clinical importance and the existence of data on

their reliability from other modules. Items with weighted kappas below .40

were eliminated. Of the remaining 38 items, 30 weighted kappas were

equal to or exceeded .60 (Mkw = .67). The remainder exceeded .45.

Inter-rater item reliability for all items in the protocol was assessed by

domain by exploring the percent of exact agreement. Because having


reliable ‘3’ scores allows more variation (which is important in treatment

studies), the initial set of analyses retained all scores of ‘0’ to ‘3’. Reliability

between .4 and .74 was considered good, and reliability at or above .75 was

considered excellent (Fleiss, 1986). In previous versions of the ADOS,

condensing the range from ‘0’ to ‘2’, 80 percent exact agreement had been

the goal. For items on the Toddler Module, even using the extended range

of ‘0’ to ‘3’ (which reduces agreement), mean exact (percent) agreement

was .84 across all items and rater pairs. Thirty-nine of 41 items had exact

agreement at or above .75, and every item received at least .71 agreement

across raters. When considered by domain, agreement for codes related to

language and communication was generally excellent: only two items had

reliability that was good (.71 and .74). All items in the remaining domains

had excellent inter-rater reliability. Because the diagnostic algorithm

collapses codes of 2s and 3s (to avoid overly weighting any single item in

the overall diagnosis), a second set of exact agreement analyses were

conducted, collapsing codes of 2 and 3. Mean exact agreement was .87,

and all 41 items had exact agreement at or above .75.

Reliability of Domain Scores and Algorithm Classifications.

Intraclass correlations (ICCs) were computed for protocol total scores,

as well as algorithm domain and total scores. Calculations were made using

both the 12-20/NV21-30 and V21-30 algorithms. ICCs were as follows:

protocol total scores = .96; 12-20/NV21-30 algorithm total = .90; V21-30


algorithm total = .99; 12-20/NV21-30 algorithm SA total = .84; V21-30

algorithm SA total = .99; 12-20/NV21-30 algorithm RRB total = .93; V21-30

algorithm RRB total = .74.

Inter-rater agreement in diagnostic classification using a single cutoff

of 12 (i.e., ASD or non-spectrum) was 97% on the 12-20/NV21-30 algorithm.

Using the V21-30 algorithm with a single cutoff of 10, inter-rater agreement

across diagnostic classifications (i.e., ASD or non-spectrum) was 87%.

Inter-rater agreement in range of concern using the three ranges on the 12-

20/NV21-30 algorithm (little-or-no concern: scores less than 10, mild-to-

moderate concern: scores of 10 to 13, moderate-to-severe concern: scores

of 14 and above) was 70%. On the V21-30 algorithm (little-or-no concern:

scores less than 8, mild-to-moderate concern: scores of 8 to 11, moderate-

to-severe concern: scores of 12 and above), inter-rater agreement for

ranges of concern was 87%.

Test-Retest Reliability.

Test-retest reliability was analyzed using data from 39 children who

had two Toddler Module administrations within 2 months. Reliability was

evaluated using algorithm subtotal scores across the SA and RRB domains,

as well as algorithm total scores. Analyses addressing the 12-20/NV21-30

algorithm, which included 31 participants, yielded high test-retest ICCs for

the SA total (.83), the RRB total (.75), and the algorithm total score (.86).

The mean absolute difference across the two evaluations was 0.90 points


(SD = 3.14) in SA, 0.39 points (SD = 1.54) for RRBs and 1.29 points (SD =

3.55) for the algorithm total score. Out of the 31 children, 24 (77%) were

classified consistently across the two evaluations (using the single cutoff of

10 on the algorithm). Out of the 7 participants who shifted between non-

spectrum and ASD, 3 initially missed the cutoff and then met the cutoff on

the second evaluation, while 4 moved from meeting the cutoff to failing to

meet. Using the three ranges of concern, 23 (74%) children were classified

within the same range across evaluations. Of the 8 participants who

shifted, 1 shifted from the greater level of concern to the lesser one. Seven

shifted from little-or-no concern to a concern range or vice-versa (2 from

little-or-no concern to mild-to-moderate concern, 4 from mild-to-moderate

concern to no concern, and 1 from moderate-to-severe concern to little-or-

no concern).

Data for 8 participants who received the V21-30 algorithm twice

within two months indicated similarly high ICCs for the SA total (.94), the

RRB total (.60), and the algorithm total score (.95). There was a mean

absolute difference across the two evaluations of 0.63 points (SD = 2.13) for

algorithm total scores, 0.38 points (SD = 2.77) for the SA total, and 0.25

points (SD = 1.04) for the RRB total. Using the single cutoff of 10, 2

children shifted classifications across evaluations (1 shifting from meeting

cutoffs to failing to meet, the other vice-versa) and 6 retained the same

classification. Similarly, 5 out of the 8 children remained in the same range


of concern across both administrations. Of the remaining 3 children, 1

increased from mild-to-moderate to moderate-to-severe concern, 1 moved

from mild-to-moderate to little-or-no concern and the other shifted from

little-or-no concern to mild-to-moderate concern.

Discussion

The Toddler Module contributes a new module to the existing ADOS

and permits the use of this standardized instrument with children between

12 and 30 months of age. The Toddler Module maintains the same core

areas of observations, namely, language and communication, reciprocal

social interaction, play and stereotyped/restricted behaviors or interests. It

has acceptable internal consistency and excellent inter-rater and test-retest

reliability. The algorithm, using both the formal cutoff and the ranges of

concern, has excellent diagnostic validity for ASD versus non-spectrum

conditions.

As with other modules of the ADOS, the Toddler Module algorithm

should be interpreted in the appropriate manner. Algorithm classification

based on cutoffs (i.e., non-spectrum or ASD) should be one element of a

comprehensive diagnostic assessment, in which the final diagnostic decision

must be made using the best judgment of the clinician. This is particularly

important when evaluating very young children, for whom the lines of

typical and atypical development can be very unclear and for whom

behavior can change over a few months. Moreover, differential diagnosis


can be particularly challenging in toddlers because symptoms may be

emergeing gradually. An attempt has been made to structure the Toddler

Module algorithm in a manner which – as much as is possible –

accommodates these observations by generating ranges of concern rather

than strict classifications. In addition, because research has indicated that

early specific ASD diagnoses (autism and PDD-NOS versus ASD) have

questionable stability in younger populations, the algorithms provide only

one cutoff for all ASD.

The single cutoffs proposed for the new algorithms should be

interpreted in a fashion consistent with the ADOS: “an individual who meets

or exceeds the cutoffs…has scored within the range of a high proportion of

participants with [ASD] who have similar levels of expressive language in

deficits in social behavior and in the use of speech and gesture as part of

social interaction” (Lord et al., 2000, p. 220). However, in order to warrant

an ASD diagnosis, the individual must otherwise exhibit behaviors

consistent with the criteria as outlined in formal diagnostic criteria

(American Psychiatric Association, 1994). That is, it is possible for a child

to meet a cutoff and not receive a formal diagnosis of ASD according to

clinical judgment. Conversely, it is also possible for a child to score below

the cutoff and for a clinician to judge that the child does meet formal

criteria for an ASD diagnosis. Some aspects of the algorithm scores (i.e.,

negative association with early verbal scores) highlight the importance of


thoughtful clinical interpretation of algorithm results, because certain

features of the child which are non-specific to ASD (like early language

delay) may elevate scores. Because verbal ability in this study was defined

by Mullen (Mullen, 1995) scores, and – as with other measures – the early

Mullen scores are heavily biased to social communication (e.g., “recognizes

own name” and “plays gesture/language game”), the correlations between

Toddler Module scores and early verbal ability scores seemed inevitable.

However, a clearer separation between ADOS scores and eventual language

ability would be ideal.

The ranges of concern which are incorporated into the algorithm are

intended to reflect the diagnostic uncertainty that is often faced when

evaluating very young children, whether because of developmental

variability or confounding conditions (such as global developmental delay or

early language impairment). Nevertheless, by expanding the number of

categories from two diagnostic groupings (ASD and non-spectrum) to three

ranges of concern (little-or-no, mild-to-moderate, moderate-to-severe), more

variation would be expected. Thus, the ranges are intended primarily as

“sign-posts” along a continuous range of scores that show excellent stability

in intra-class correlations, across raters and re-assessments several months

later. Generally, scores which fall into the mild-to-moderate range should

be considered an indicator of behaviors likely to be consistent with an ASD.

Children whose scores fall into this range should receive further ASD-


specific evaluation and follow-up in the next several months. However, 12-

18% of children with non-spectrum conditions and 8% of typically

developing children also scored in this range, so there is considerable

heterogeneity within it. In contrast, algorithm scores falling into the

moderate-to-severe range of concern were more strongly consistent with an

eventual diagnosis of ASD (with only 3-6% false positives). Regardless,

whether using the research-oriented cutoff or the clinically-oriented ranges

of concern, the onus is on the examiner to interpret behaviors and scores

within the broader developmental and assessment context. Moreover, in

cases of diagnostic uncertainty, it is important to be clear with parents

(particularly of very young children) the importance of ongoing monitoring

of child development and thorough follow-up.

The young age of the children receiving the Toddler Module means

that the examiner may face some additional issues in interpreting ADOS

results. In particularSpecifically, some infants and toddlers may be

particularly uncomfortable in the evaluation context, where they are faced

with an unknown adult, unfamiliar toys, and a novel clinic or laboratory

setting. The examiner must, therefore, gauge whether behavior observed in

the ADOS context is representative of behavior in other settings. This is

especially important if something about the ADOS assessment – an unskilled

examiner, the absence of a familiar caregiver, cultural differences in

expected child behavior – might suggest that the child’s behavior is “off”.


Fortunately, because the Toddler Module requires that (barring unique

circumstances, such as children recently placed in foster care) a familiar

caregiver is always present in the room, the examiner should get feedback

from the caregiver about whether the child’s behavior during the ADOS was

representative of day-to-day interactions. If something about the ADOS

administration indicates that the observation did not capture the child’s

true behavioral characteristics, the scores should be interpreted

accordingly and more information should be sought through a home

observation or a repeated assessment.

Results and observations from the Toddler Module may be useful

beyond the diagnostic context. Parents, intervention providers and

teachers often report that the strengths and difficulties noted during the

administration can help in understanding an individual child and developing

programming goals. Therefore, clinicians should make a concerted effort to

thoroughly explain the key observations in behavioral terms (rather than

simply in terms of scores and cutoffs), describing what behaviors were

noted and which were less consistent or absent. When appropriate,

examiners should generate suitable recommendations based on the ADOS

observations which can be applied to educational plans at home and at

school.


The predictive validity of very early diagnosis (under 30 months) is a

question currently being addressed by many investigators (Landa & Garrett-

Mayer, 2006; Wetherby et al., 2004; Zwaigenbaum et al., 2005). The focus

of the Toddler Module development is to provide a standardized method of

quantifying descriptions of behaviors that correspond to experienced

clinicians’ clinical diagnosis of ASD at a given point in time. The Toddler

Module provides information with good to excellent internal consistency

and inter-rater reliability for items, domains and research diagnostic

categories. Stability across raters within clinical ranges was good (87%

agreement) for older, verbal children but less good (70%) for the nonverbal

and younger children. Across time (one to two months), about three-

quarters of children remained in the same clinical range of concern for both

algorithms, and between 63% and 74% (depending on the algorithm)

remained in the same diagnostic category (in part this difference is due to

having three ranges and two diagnostic categories). Thus, variations both

in rater and in time do make a difference in a child’s outcome on the

Toddler Module. Follow-up studies of the long-term predictive value of

these scores will be critical in determining the extent to which they, and

other early measures of diagnostic risk, predict outcome and response to

treatment. In the meantime, consideration of scores as continuous

dimensions and as one marker (along with other measures) of relative risk

of ASD and need for follow-up seems most appropriate. In research, the


diagnostic categories may help in standardizing assessments across studies

and establishing replicable criteria for study inclusion. Again, however,

algorithm classification should be considered in the context of other

information.

In sum, the Toddler Module is a new, standardized module intended

to extend the application of the ADOS to children as young as 12 months of

age. It is appropriate for use with children up to the age of 30 months or

until children acquire phrase speech. It will be important for future

researchers to replicate the psychometric results reported here with larger,

more diverse samples of children with early-appearing, non-spectrum

conditions. We hope that researchers and clinicians alike find it a useful

tool in supporting families and children with autism spectrum disorders and

advancing our understanding of these conditions.


References

American Psychiatric Association. (1994). Diagnostic and statistical manual

of mental disorders (Fourth ed.). Washington, DC: Author.

Behne, T., Carpenter, M., Call, J., & Tomasello, M. (2005). Unwilling versus

unable: Infants' understanding of intentional action. Developmental

Psychology, 41(2), 328-337.

Bohlin, G., & Hagekull, B. (1993). Stranger wariness and sociability in the

early years. Infant Behavior & Development, 16(1), 53-67.

Bryson, S. E., Zwaigenbaum, L., McDermott, C., Rombough, V., & Brian, J.

(2008). The Autism Observational Scale for Infants: Scale

development and reliability data. Journal of Autism and

Developmental Disorders, 38(4), 731-738.

Charman, T., Taylor, E., Drew, A., Cockerill, H., Brown, J., & Baird, G.

(2005). Outcome at 7 years of children diagnosed with autism at age

2: Predictive validity of assessments conducted at 2 and 3 years of age

and pattern of symptom change over time. Journal of Child Psychology

& Psychiatry, 46(5), 500-513.

Chawarska, K., Paul, R., Klin, A., Hannigen, S., Dichtel, L. E., & Volkmar, F.

(2007). Parental recognition of developmental problems in toddlers

with autism spectrum disorders. Journal of Autism and Developmental

Disorders, 37(1), 62-72.


DeGiacomo, A., & Fombonne, E. (1998). Parental recognition of

developmental abnormalities in autism. European Journal of Child &

Adolescent Psychiatry, 7, 131-136.

DiLavore, P., Lord, C., & Rutter, M. (1995). Pre-linguistic Autism Diagnostic

Observation Schedule (PLADOS). Journal of Autism and


Fleiss, J. (1986). Reliability of measurements. In J. Fleiss (Ed.), The design

and analysis of clinical experiments (pp. 2-31). New York: John Wiley

& Sons.

Gotham, K., Risi, S., Pickles, A., & Lord, C. (2007). The Autism Diagnostic

Observation Schedule (ADOS): Revised algorithms for improved

diagnostic validity. Journal of Autism and Developmental Disorders,

37(4), 613-627.

Kleinman, J. M., Ventola, P. E., Pandey, J., Verbalis, A. D., Barton, M.,

Hodgson, S., Green, J., Dumont-Mathieu, T., Robins, D. L., & Fein, D.

(2008). Diagnostic stability in very young children with autism

spectrum disorders. Journal of Autism and Developmental Disorders,

38(4), 606-615.

Landa, R., & Garrett-Mayer, E. (2006). Development in infants with autism

spectrum disorders: A prospective study. Journal of Child Psychology

& Psychiatry, 47(6), 629-638.


Lord, C., Risi, S., DiLavore, P., Shulman, C., Thurm, A., & Pickles, A. (2006).

Autism from two to nine. Archives of General Psychiatry, 63(6), 694-

701.

Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Leventhal, B. L., DiLavore, P.,

Pickles, A., & Rutter, M. (2000). The Autism Diagnostic Observation

Schedule -- Generic: A standard measure of social and communication

deficits associated with the spectrum of autism. Journal of Autism &


Lord, C., Rutter, M., DiLavore, P., & Risi, S. (1999). Autism Diagnostic

Observation Schedule (ADOS) Los Angeles: Western Psychological

Services.

Lord, C., Rutter, M., Goode, S., Heemsbergen, J., Jordan, H., Mawhood, L.,

& Schopler, E. (1989). Autism Diagnostic Observation Schedule: A

standardized observation of communicative and social behavior.

Journal of Autism and Developmental Disorders, 19, 185-212.

Lord, C., Shulman, C., & DiLavore, P. (2004). Regression and word loss in

autism spectrum disorder. Journal of Child Psychology and Psychiatry,

45(5), 936-955.

Mullen, E. (1995). Mullen Scales of Early Learning. Circle Pines, MN:

American Guidance Service, Inc.

Muthen, L. K., & Muthen, B. O. (1998). M-plus user’s guide. Los Angeles:

Muthen and Muthen.


National Research Council. (2001). Educating children with autism.

Washington, DC: National Academy Press.

Phillips, W., Baron-Cohen, S., & Rutter, M. (1992). The role of eye contact in

goal detection: Evidence from normal infants and children with autism

or mental handicap. Development & Psychopathology, 4, 375-383.

Risi, S., Lord, C., Gotham, K., Corsello, C., Chrysler, C., Szatmari, P., Cook,

E. H., Jr., Leventhal, B. L., & Pickles, A. (2006). Combining

information from multiple sources in the diagnosis of autism spectrum

disorders. Journal of the American Academy of Child & Adolescent

Psychiatry, 45(9), 1094.

Rutter, M., Le Couteur, A., & Lord, C. (2003). Autism Diagnosic Interview -

Revised (ADI-R). Los Angeles: Western Psychological Services.

Siegel, B., Vukicevic, J., Elliott, G., & Kraemer, H. (1989). The use of signal

detection theory to assess DSM-III-R criteria for autistic disorder.

Journal of the American Academy of Child & Adolescent Psychiatry,

28, 542-548.

Sroufe, L. A. (1978). Wariness of strangers and the study of infant

development. Annual Progress in Child Psychiatry & Child

Development, 24-50.

StataCorp. (2007). Stata Statistical Software: Release 10. College Station,

TX: StataCorp LP.


Stone, W. L., Coonrod, E. E., Turner, L. M., & Pozdol, S. L. (2004).

Psychometric properties of the STAT for early autism screening.

Journal of Autism and Developmental Disorders, 34(6), 691-701.

Turner, L., & Stone, W. (2007). Variability in outcome for children with an

ASD diagnosis at age 2. Journal of Child Psychology and Psychiatry,

48(8), 793-802.

Turner, L., Stone, W. L., Pozdol, S., & Coonrod, E. E. (2006). Follow-up of

children with autism spectrum disorders from age 2 to age 9. Autism,

10(3), 243-265.

Wetherby, A. (2001). Communication and Symbolic Behavior Scales

Developmental Profile, Preliminary Normed Edition. Baltimore, MD:

Paul H. Brookes Publishing.

Wetherby, A., Woods, J., Allen, L., Cleary, J., Dickinson, H., & Lord, C.

(2004). Early indicators of autism spectrum disorders in the second

year of life. Journal of Autism and Developmental Disorders, 34(5),

473-493.

Zwaigenbaum, L., Bryson, S., Rogers, T., Roberts, W., Brian, J., & Szatmari,

P. (2005). Behavioral manifestations of autism in the first year of life.

International Journal of Developmental Neuroscience, 23, 143-152.


Author Note

Rhiannon Luyster is now at the Autism Consortium, Boston,

Massachusetts. Mia Coffing is now at Vanderbilt University, Nashville,

Tennessee. Rachel Petrak is now at the University of Michigan School of

Public Health. Somer Bishop and Amy Esler are now at the University of

Wisconsin, Madison, Wisconsin. Vanessa Hus is now at the University of

Washington, Seattle, Washington. Jennifer Richler is now at the University

of Minnesota, Minneapolis, Minnesota.

Some of the data from this paper were previously presented at the

2006 International Meeting for Autism Research (IMFAR) in Montreal, the

2nd World Autism Congress & Exhibition in Cape Town, South Africa, the

2007 Society for Research in Child Development conference in Boston,

Massachusetts and at the 2008 IMFAR in London, England. This work was

supported by NRSA F31MH73210-02 from the National Institute of Mental

Health to Rhiannon Luyster, grants MH57167 and MH066469 from the

National Institute of Mental Health and HD 35482-01 from the National

Institute of Child Health and Human Development, and funding from the

Simons Foundation to Catherine Lord. Support was also provided by a

grant from the Department of Education to Amy Wetherby. We thank

Andrea Cohan, Christina Corsello, Pamela Dixon Thomas, LeeAnne Green

Snyder, Alexandra Hessenius, Marisela Huerta, Lindsay Jackson, Jennifer


Kleinke, Fiona Miller, Rosalind Oti and Dorothy Ramos for their assistance

in data collection. We would also like to express our gratitude to the

families and children in the Word Learning, First Words and Toddlers

studies.

Correspondence concerning this article should be addressed to

Rhiannon Luyster, Ph.D., Massachusetts General Hospital, Center for

Human Genetic Research, 185 Cambridge St., 6th Floor, Boston MA, 02114.

Email: [email protected].

Table 1

Toddler Module Activities


1a. Free Play

1b. Free Play -- Ball

2. Blocking Toy Play

3. Response to Name

4a. Bubble Play

4b. Bubble Play -- Teasing Toy Play

5a. Anticipation of a Routine with Objects

5b. Anticipation of a Routine with Objects -- Unable Toy Play

6. Anticipation of a Social Routine

7. Response to Joint Attention

8. Responsive Social Smile

9a. Bathtime

9b. Bathtime -- Ignore

10. Functional & Symbolic Imitation

11. Snack


Table 2

Algorithm Items

Response to Name

Gestures Ignore

Shared Enjoyment in Interaction Requesting

Showing Pointing

Unusual Eye Contact Unusual Eye Contact

Facial Expressions Directed to Others Facial Expressions Directed to Others

Spontaneous Initiation of Joint Attention

Response to Joint Attention

Quality of Social Overtures Quality of Social Overtures

Overall Quality of Rapport

Unusual Sensory Interest in Play Material/ Person

Hand and Finger Movements/PosturingHand and Finger Movements/Posturing

V21-3012-20/NV21-30

Social Affect

Restricted, Repetitive Behaviors

(12-20 months & Non-verbal 21-30 months) (Verbal 21-30 months)

Frequency of Spontaneous Vocalization Directed to Others

Integration of Gaze and Other Behaviors During Social Overtures

Spontaneous Initiation of Joint Attention

Amount of Social Overtures/Maintenance of Attention: CAREGIVER

Integration of Gaze and Other Behaviors During Social Overtures

Note. 12-20/ NV21-30 algorithm is a modified version of the Revised Module 1, No Words algorithm from Gotham, K., Risi, S., Pickles, A., & Lord, C. (2007). The Autism Diagnostic Observation Schedule (ADOS): Revised algorithms for improved diagnostic validity. Journal of Autism and Developmental Disorders, 37(4), 613-627.

Unusual Sensory Interest in Play Material/ Person

Unusually Repetitive Interests or Stereotyped Behaviors

Intonation of Vocalizations or Verbalizations

Unusually Repetitive Interests or Stereotyped Behaviors


Table 3

Description of “Unique Participants” sample in validity analyses

ASD Nonspectrum Typical ASD Nonspectrum ASD Nonspectrum Typical

N (male, female)

20 (19, 1)

24 (22, 2)

68 (47, 21)

15 (12, 3)

8 (3, 5)

24 (21, 3)

11 (10, 1)

36 (24, 12)

Mean Chronological age (SD)

17.70a

(1.90) 15.33b

(2.48)15.82b

(2.69)22.20a

(1.86)25.38b

(2.26)26.00a

(2.62)23.45b

(2.66)22.28b

(1.11)

Chronological age minimim-maximum 13-20 12-19 12-20 21-28 22-28 22-30 21-29 21-25

Mean Nonverbal mental age (SD)

18.05 (2.98)

16.94 (3.10)

18.62 (3.18)

19.87 (3.61)

19.62 (7.11)

25.04 (3.56)

24.82 (5.78)

25.40 (3.32)

Nonverbal mental age minimim-maximum 13-23 12-24 12-26 12-26 12-35 19-33 17-37 20-35

Mean Verbal mental age (SD)

13.05a

(4.15)13.35a

(2.96)16.93b

(4.03)11.03 (5.03)

13.13 (6.58)

21.79 (5.12)

22.32 (7.06)

25.39 (3.94)

Verbal mental age minimim-maximum 6-21 9-19 10-26 1-19 4-26 12-31 12-38 19-36

a, b, c, Superscript letters refer to post hoc Scheffe test: When two groups in the same row and column are marked with different letters, they are significantly different (p < .01) from each other; groups marked with the same letter/ without superscript letters are not significantly different.

12-20 All 21-30 Nonverbal 21-30 Verbal

Note. ASD=autism spectrum disorder; SD=standard deviation. All ages are in months. There were no nonverbal typically developing children between 21 and 30 months.


Table 4

Description of “All Cases” sample in validity analyses

ASD Nonspectrum Typical ASD Nonspectrum ASD Nonspectrum Typical

N (male, female)

55 (53, 2)

47 (45, 2)

90 (61, 29)

32 (27, 5)

14 (9, 5)

59 (56, 3)

24 (23, 1)

39 (26.13)

Mean Chronological age (SD)

17.20 (2.34)

15.87 (2.48)

15.93 (2.69)

23.31 (2.39)

24.71 (2.81)

25.54a

(2.71)24.79a

(2.78)22.46b

(1.37)

Chronological age minimim-maximum 12-20 12-20 12-20 21-29 21-30 21-30 21-29 21-27

Mean Nonverbal mental age (SD)

17.21a, b

(2.92)16.65a

(4.33)18.63b

(3.34)20.59 (3.28)

19.43 (5.26)

24.49 (3.53)

23.75 (4.46)

25.46 (3.84)

Nonverbal mental age minimim-maximum 12-23 12-37 12-28 12-26 12-35 18-33 17-37 17-35

Mean Verbal mental age (SD)

11.87a

(3.79)13.39a

(5.37)17.04b

(4.18)12.61 (4.99)

13.00 (5.17)

21.41a

(5.93)21.35a, b

(5.83)25.18b

(4.61)

Verbal mental age minimim-maximum 6-21 6-38 10-26 1-25 4-26 11-31 12-38 10-36

a, b, c, Superscript letters refer to post hoc Scheffe test: When two groups in the same row and column are marked with different letters, they are significantly different (p < .01) from each other; groups marked with the same letter/ without superscript letters are not significantly different.

12-20 All 21-30 Nonverbal 21-30 Verbal

Note. ASD=autism spectrum disorder; SD=standard deviation. All ages are in months. There were no nonverbal typically developing children between 21 and 30 months.


Table 5

Sensitivity and specificity of the algorithm cutoffs used with the ADOS-Toddler Module

(n=35) (n=34) (n=35) (n=101) (n=24) (n=24) (n=47)Sens Spec Sens Spec Sens Sens Spec

91% 91% 91% 94% 88% 88% 94%

(n=87) (n=64) (n=87) (n=153) (n=58) (n=22) (n=58) (n=59)Sens Spec Sens Spec Sens Spec Sens Spec

87% 86% 87% 91% 83% 86% 83% 91%

a Using a cutoff of 12. b Using a cutoff of 10.

Note. ASD=autism spectrum disorder; NS=non-spectrum; TD=typically developing; Sens=Sensitivity; Spec=Specificity.

V21-30 algorithmb

ASD vs NS ASD vs NS, TD ASD vs NS ASD vs NS, TD(n=11)Spec

91%

(Verbal 21-30 months)

"Unique Participants" sample

(12-20 months & Non-verbal 21-30 months) (Verbal 21-30 months)

12-20/ NV21-30 algorithma V21-30 algorithmb

12-20/ NV21-30 algorithma

(12-20 months & Non-verbal 21-30 months)

"All Cases" sample

ASD vs NS ASD vs NS, TD ASD vs NS ASD vs NS, TD


Table 6

Percent of participants falling into ranges of concern by diagnostic group

ASD NS TD ASD NS TD(n=35) (n=34) (n=67) (n=24) (n=11) (n=36)

Little-or-no concern Little-or-no concern

Scores: 0-9 Scores: 0-7

Mild-to-moderate concern Mild-to-moderate concern

Scores: 10-13 Scores: 8-11

Moderate-to-severe concern Moderate-to-severe concern

Scores: 14+ Scores: 12+

Note. ASD=autism spectrum disorder; NS=non-spectrum; TD=typically developing.

84 -- --

92

12 18 8

V21-30(Verbal 21-30 months)

2 12 8

12-20/ NV21-30(12-20 months & Non-verbal 21-30 months)

Range

4 82

77 6 3

Range

3 82 89

running head: ados – toddler modulerpetrak/ados-t manuscript 7… · web viewrunning head: ados...

Documents