Developmental Developmental Screening and Screening and Assessment:Assessment:

What Are We Thinking?What Are We Thinking?

Glen P. Aylward, Ph.D., ABPPGlen P. Aylward, Ph.D., ABPPSouthern Illinois UniversitySouthern Illinois University

School of MedicineSchool of MedicineSpringfield, ILSpringfield, IL

Q 1: Is there a “Gold Q 1: Is there a “Gold Standard” in Standard” in

Developmental Evaluation?Developmental Evaluation?• reference standardreference standard

• Flynn effect (.3-.5 pt/year)Flynn effect (.3-.5 pt/year)

• Bayley Scales (1969; 1993; 2006)Bayley Scales (1969; 1993; 2006)

• BSID—>BSID II (MDI 12 pts lower, PDI BSID—>BSID II (MDI 12 pts lower, PDI 7 points)7 points)

• BSID-IIBSID-IIBSID-III (mental 6 pts BSID-III (mental 6 pts higherhigher; ; motor 8 pts motor 8 pts higherhigher))

• Mean 7 pt increase; comparability is Mean 7 pt increase; comparability is limitedlimited

• Length/pragmaticsLength/pragmatics

Q2: Is There Agreement as Q2: Is There Agreement as to What Qualifies as a to What Qualifies as a Developmental Delay?Developmental Delay?

• ““precision issue”precision issue”

• 20% delay ?20% delay ?

• 2 standard deviations below the mean 2 standard deviations below the mean for a reference group?for a reference group?

• Score compared to “local norms”?Score compared to “local norms”?

• A ratio/criterion measure?A ratio/criterion measure?

• Acceptance of psychometrically poor Acceptance of psychometrically poor teststests

• Recommend SD cutoffsRecommend SD cutoffs

Q3: Does Development Q3: Does Development (DQ)= Intelligence (IQ)?(DQ)= Intelligence (IQ)?

• NeurologicNeurologicmotormotorsensorimotorsensorimotor cognitivecognitive

• SkillSkillfunctionfunctionintegrated functional integrated functional unitunit intelligence intelligence

• Complexity increases in concert with ageComplexity increases in concert with age• Skill, function = potentialSkill, function = potential• Different streams, different ratesDifferent streams, different rates• Younger than age 2: simple cognitive Younger than age 2: simple cognitive

functions—only after discrete functions are functions—only after discrete functions are combined do we predict later “intelligence”combined do we predict later “intelligence”

Canalized BehaviorCanalized Behavior• Species-specific, prewired, self-Species-specific, prewired, self-

rightingrighting• Fixed behavior patternFixed behavior pattern• Not highly complexNot highly complex• More canalized, less affected by More canalized, less affected by

adverse circumstancesadverse circumstances• Less canalized, weaker self-righting, Less canalized, weaker self-righting,

greater likelihood of disruptiongreater likelihood of disruption• Sensorimotor behaviors are strongly Sensorimotor behaviors are strongly

canalizedcanalized• Impact on test results/predictionImpact on test results/prediction

Integrated FunctionsIntegrated Functions• Individual developmental skill/ability Individual developmental skill/ability

is not most importantis not most important• Integration of abilities into functional Integration of abilities into functional

units that control these abilitiesunits that control these abilities• Ability to integrate Ability to integrate

functionsfunctionsinformation processing, information processing, memory, discrimination, attentionmemory, discrimination, attention

• Musicians [skills]Musicians [skills]section of section of orchestra [function]orchestra [function]integration of integration of sections (conductor)sections (conductor) concert concert

IQ/DQ Ambiguity• BSID-III Cognitive Composite

• Mullen Scales Early Learning Composite

• SB-V NVIQ, VIQ, FSIQ

• K-ABC/2 Mental Processing Composite; Mental Processing Index

• WPPSI-III FSIQ

• MSCA General Cognitive Index (GCI)

• DAS General Cognitive Ability (GCA)

• Cattell IQ

Q4: Is a Ratio DQ useful?Q4: Is a Ratio DQ useful?• Ratio DQ– MA/CA x 100Ratio DQ– MA/CA x 100• Rate of developmentRate of development• Not comparable at different age levels Not comparable at different age levels

b/c the standard deviation (variance) of b/c the standard deviation (variance) of the ratios does not remain constantthe ratios does not remain constant

• CI’s vary tremendouslyCI’s vary tremendously• Interpretation is difficultInterpretation is difficult• ““MA” is totally dependent on test usedMA” is totally dependent on test used• Similar issues with “developmental age”Similar issues with “developmental age”• Better to use 1.5, 2 SD < ‘average’Better to use 1.5, 2 SD < ‘average’

Q5: Is Caretaker Report Q5: Is Caretaker Report Sufficient for Sufficient for

Developmental Screening?Developmental Screening?• AAP (2006) policy statement regarding AAP (2006) policy statement regarding

surveillance and screeningsurveillance and screening• 1/3 of developmental screening 1/3 of developmental screening

instruments (excluding those targeting instruments (excluding those targeting ASD) were parent completedASD) were parent completed

• Earlier, parent report considered a Earlier, parent report considered a Stage I or “prescreening” techniqueStage I or “prescreening” technique

• Evolved to being considered Evolved to being considered comparable to hands-on screeningcomparable to hands-on screening

• ? Evidence-based use? Evidence-based use

Caretaker ReportCaretaker Report• Little is known as to how parent completed Little is known as to how parent completed

questionnaires are affected by: 1) child-questionnaires are affected by: 1) child-related, or 2) environmental variablesrelated, or 2) environmental variables

• Accuracy depends on developmental area Accuracy depends on developmental area assessed, populationassessed, population

• ? Different tests for different populations? Different tests for different populations

• How questions are answered (y/n, Likert, How questions are answered (y/n, Likert, etc.)etc.)

• Considerations:Considerations:

-- Length, detail-- Length, detail

-- Age range encompassed-- Age range encompassed

-- Presence/absence of examples of behavior-- Presence/absence of examples of behavior

-- Test behaviors or milestones-- Test behaviors or milestones

Caretaker ReportCaretaker Report• Diamond & Squires (1993): current Diamond & Squires (1993): current

behaviors, recognition (vs recall), behaviors behaviors, recognition (vs recall), behaviors should occur frequently, parents need skills should occur frequently, parents need skills to be able to complete questionnaireto be able to complete questionnaire

• Screening risk status of infant Screening risk status of infant most most predictive of agreement < 2-years; at 2, predictive of agreement < 2-years; at 2, race (marker of SES) predictiverace (marker of SES) predictive

• Camp (2007) spectrum bias: better/worse Camp (2007) spectrum bias: better/worse identification depends on base rates of identification depends on base rates of problemsproblems

• Items most predictive often are those with Items most predictive often are those with poorer agreement (puzzle board, stacks 6 poorer agreement (puzzle board, stacks 6 cubes)cubes)

Q6: How Problematic Are Q6: How Problematic Are Test Refusals?Test Refusals?

• Behaviors have an impact: frequently Behaviors have an impact: frequently negativenegative

• More pronounced with younger childrenMore pronounced with younger children

• Possibilities: a) Declines to respond to any Possibilities: a) Declines to respond to any item; b) specific types of items, or c) stops item; b) specific types of items, or c) stops when items become too difficultwhen items become too difficult

• Occasional refusals—41% of young childrenOccasional refusals—41% of young children

• State of arousal, affect, motivation, State of arousal, affect, motivation, temperament, physiological issuestemperament, physiological issues

• Score refusals as failures, prorate scores, or Score refusals as failures, prorate scores, or consider testing to be invalid?consider testing to be invalid?

Test RefusalsTest Refusals• Potential causes:Potential causes:

--Reaction to poor underlying --Reaction to poor underlying skills/attempt to avoid failureskills/attempt to avoid failure

--Oppositional behavior--Oppositional behavior

--Shyness, anxiety--Shyness, anxiety

--Temperament--Temperament

--Poor attentional skills/high activity level--Poor attentional skills/high activity level

--Fatigue/malaise--Fatigue/malaise

--Temper displays/crying--Temper displays/crying

--Parental behaviors--Parental behaviors

Test RefusalsTest Refusals• Verbal production tasks, gross motor Verbal production tasks, gross motor

activities, end of testingactivities, end of testing

• More in children born at biologic risk, More in children born at biologic risk, low SES low SES

• Those who refuse any aspect of testing Those who refuse any aspect of testing differ from those who refuse some differ from those who refuse some items or who refuse more difficult itemsitems or who refuse more difficult items

• High rates of refusal at one age High rates of refusal at one age associated with similar behaviors at associated with similar behaviors at later ageslater ages

Test Refusals--Test Refusals--ImplicationsImplications

• Those who refuse to comply often Those who refuse to comply often have decreased scores in several have decreased scores in several areas of function--untestableareas of function--untestable

• Risk for lower test scores and higher Risk for lower test scores and higher rates of problems at ages 7-8 years rates of problems at ages 7-8 years in many areasin many areas

• Source of clinical informationSource of clinical information

• Not in place of quantitative; rather, in Not in place of quantitative; rather, in conjunction withconjunction with

• Causes for + finding: cognitive impairment, Causes for + finding: cognitive impairment, emerging LD, language dysfunction, emerging LD, language dysfunction, environmental risk, testing issues, environmental risk, testing issues, combinationcombination

• Clinicians vs. techniciansClinicians vs. technicians

• Play-based ‘assessment’Play-based ‘assessment’

• Examples: form board; naming pictures, Examples: form board; naming pictures, stacking cubesstacking cubes

• Training to taskTraining to task

Q7: Is There a Role for Q7: Is There a Role for Qualitative Information?Qualitative Information?

Quality ControlClinicians vs. Technicians

•Quality of assessment may be compromised because of the questionable proficiency of examiners

•Not clear who is qualified– Conceptual and factual knowledge of

normal development– Awareness of significance of

pathognomonic indicators– Well versed in administration & scoring– (speed, best response, stop, eliciting

report)

Q8:What About Q8:What About Prediction?Prediction?

• Prediction tells us if early alarm or reassurance has any basis

• Prediction is difficult because:– Rapid developmental change– Intervening variables (environmental, biologic)– Interventions (EI, medical, social)– Testing itself has impact on developmental

trajectory (observational effect)– Emergent, latent, delayed, deficient, disordered– Moving target– Aspects of tests used at T1 T2 Tn

– Domain/area of development

Prediction

•Stable performance: high risk>low risk> moderate risk

•How does one define prediction (co-positivity/co-negativity; ORs, correlations)

•Time span/interval

•What predicts what?– Single composite measure may not be

appropriate; sub-domains of function

Q9: Is There a Summary?Q9: Is There a Summary?• Consider tests as reference standards; Consider tests as reference standards;

be aware of psychometric issuesbe aware of psychometric issues

• Evaluation is a balance between Evaluation is a balance between concepts and pragmaticsconcepts and pragmatics

• Percent delay is not accurate; criterion Percent delay is not accurate; criterion based, based, >> 1.5, 2, 3 SDs below average 1.5, 2, 3 SDs below average

• Consider what can be assessed at Consider what can be assessed at different ages (skill=capacity)different ages (skill=capacity)

• Ratio DQ’s not accurateRatio DQ’s not accurate

• Serial screening/assessmentSerial screening/assessment

SummarySummary

• We need to better understand strengths, We need to better understand strengths, weaknesses, and variables that affect weaknesses, and variables that affect caretaker reportcaretaker report

• Consensus on test refusals: should we Consensus on test refusals: should we include, prorate, or invalidate scores?include, prorate, or invalidate scores?

• Clinicians Clinicians need to testneed to test

• Environment affects different skills and at Environment affects different skills and at different timesdifferent times

• Wear sunscreen and eat fiberWear sunscreen and eat fiber