Interpreting and Integrating Assessment Data for Instruction

Presented by:Ellen FleishmanJeff KirshBeth KriegerAminah LucioJoel Seltzer

2009-2010

Supervisors of SchoolPsychologists

Purpose:

IDEIA seeks to assure the link between assessment and instruction.

“The purpose of psychoeducational assessments in the schools is to explore and systematically study aspects of the student’s academic skill development, intellectual functioning, strengths and weaknesses in cognitive/learning processes and social/adaptive functioning.”

Evaluations Assess:

Academic Achievement:

Reading Math Written expression CommunicationOther: Social-emotional,

behavioral and adaptive functioning

Overall Cognitive Functioning:

Reasoning Perceptual-motor skills Language Executive function Visuo-spatial skills Memory

Special Education Eligibility Criteria: Disability

Classifications

Autism Deaf-blindness Deafness Hearing

impairment Emotional

disturbance Learning disability Mental retardation

Multiple disabilitiesOrthopedic impairmentOther health impairmentSpeech or language impairmentTraumatic brain injuryVisual impairment

Learning Disability: IDEIA definition

According to Section 300.7b(10) of the Individuals with Disabilities Education Act (IDEIA), specific learning disability (SLD) is: General, i.e., a disorder in one or more of

the basic psychological processes involved in understanding or in using language, spoken or written, that may manifest itself in an imperfect ability to do the following:

Listen Think Speak Read

WriteSpellMathematical calculations

Learning Disability: IDEIA definition

Inclusive Definition: …includes conditions such as perceptual handicaps, brain injury, minimal brain dysfunction, dyslexia, and developmental aphasia.

Exclusive Definition: does not include learning problems that are primarily the result of visual, hearing or motor disabilities; mental retardation; emotional disturbance; or of environmental, cultural or economic disadvantage.

Our Intelligence Forebears

Alfred Binet (1908; in Matarazzo, 1972):“…In intelligence there is a fundamental

faculty, the alteration or the lack of which, is of the utmost importance for practical life. This faculty is judgment, otherwise called good sense, practical sense…the faculty of adapting one’s self to circumstances.”

Intelligence Forebears

David Wechsler (1943; in Matarazzo, 1972):

“Intelligence, as a hypothetical construct, is the aggregate or global capacity of the individual to act purposefully, to think rationally, and to deal effectively with his environment.”

On Linking Psychology to Instruction

Lightner Witmer (1897; as quoted in Reynolds & Gutkin, 1998) developed the first child psychology clinic and advocated for the training of:

“The psychological expert who is [knowledgeable] in practical psychology…and capable of treating the many difficult cases that resist the ordinary methods of the school room.”

Theoretical Approaches to Intelligence and Intelligence

Testing

Progress in test development and design

Progress in theories of intelligence

General Ability

Dichotomous Abilities

Multiple Intelligences(Incomplete)

MultipleIntelligences(“Complete”)

Interacting Cog/Non-cog Factors

Spearman OriginalGf-Gc;Simultaneous-Successive

ThurstonePMAs

Carroll Three-Stratum; Horn-Cattell Gf-Gc

(e.g.)

Stanford-Binet Wechsler (Rs) K-ABC KAIT

CAS DAS SB-4 WJWISC-IIIWAIS-III

WJ-R

Cross-Battery Approach

WoodcockGf-Gc Info.Proc. Model

CONTINUUM OF PROGRESS IN THEORIES

Most of psychology is here

CONTINUUM OF PROGRESS IN MEASUREMENT

Source: McGrew, K. (2003) iapsych.com

CHC Theory as a Problem Solving Model

CHC Theory is a combination of the theories of three researchers

Cattell Horn (his work was an extension of Cattell’s original Gf-

Gc formulation) Carroll

McGrew (2004) states: “CHC Theory of Intelligence is the tent that houses the two most prominent psychometric theoretical models of human cognitive abilities.”

This model serves as the theoretical foundation for some of the latest cognitive assessment (WJ-III, Stanford Binet V, KABC-II, WISC IV/WAIS IV/WPPSI, and the upcoming DAS II) instruments and is gaining acceptance by assessment specialists (Fiorello & Primerano, 2005).

CHC/Gf-Gc Theory and The IEP: Instructional

ImplicationsWHAT IS CHC THEORY? The major description of

intelligence available today Based on the analyses of

hundreds of data sets that were not restricted to a particular test battery

The theory is empirically based The CHC abilities represent broad

domains of ability

CARROLL’S (1993) THREE-STRATUM THEORY OF COGNITIVE ABILITIES

g GeneralIntelligence

FluidIntelligence

CrystallizedIntelligence

GeneralMemory &Learning

BroadVisualPerception

BroadAuditoryPerception

BroadRetrievalAbility

BroadCognitiveSpeediness

ProcessingSpeed (RTDecisionSpeed)

G

ener

al(S

trat

um

III

)

B

road

(Str

atu

m I

I)

Nar

row

(Str

atu

m I

)

69 narrow abilities found in data sets analyzed by Carroll

Gf Gc Gy Gv Gu Gr Gs Gt

Adapted from K. S. McGrew & D. P. Flanagan (1998). The Intelligence Test Desk Reference (ITDR):Gf-Gc Cross-Battery Assessment. Boston: Allyn & Bacon.

HORN-CATTELL Gf-Gc THEORY

Gf Gq Gsm Gv Ga Gs CDS GrwGc Glr

Flu

id

Inte

llig

enc

e Cry

stal

lize

d

Inte

llig

ence

Qu

anti

tati

ve

Kn

ow

led

ge

Sh

ort

-Ter

mM

emo

ry

Vis

ual

P

roce

ssin

g

Au

dit

ory

Pro

cess

ing

Lo

ng

-Ter

mR

etri

eval

Pro

cess

ing

Sp

eed

Co

rrec

tD

ecis

ion

Sp

d

Rea

din

g/

Wri

tin

g

Bro

ad(S

trat

um

II)

N

arro

w(S

trat

um

I)

69 narrow abilities found in data sets analyzed by Carroll (1993) as suggested by McGrew (1997) and McGrew & Flanagan (1998)

Adapted from K. S. McGrew & D. P. Flanagan (1998). The Intelligence Test Desk Reference (ITDR):Gf-Gc Cross-Battery Assessment. Boston: Allyn & Bacon.

BROAD (II) Gf-Gc ABILITIES SUBSUME NARROW(I) ABILITIES: Gf EXAMPLE

General SequentialReasoning

(RG)

Induction(I)

QuantitativeReasoning

(RQ)

Speed ofReasoning

(RE)

PiagetianReasoning

(RP)

GfFluid

Intelligence

FLUID INTELLIGENCE (Gf) NARROW ABILITIES

General Sequential Reasoning (RG): Ability to start with stated rules, premises, or conditions, and to engage in one or more steps to reach a solution to a problem.

Induction (I): Ability to discover the underlying characteristic (e.g., rule, concept, process, trend, class membership) that governs a problem or a set of materials.

Quantitative Reasoning (RQ): Ability to inductively and deductively reason with concepts involving mathematical relations and properties.

Piagetian Reasoning (RP): Seriation, conservation, classification and other cognitive abilities as defined by Piaget.

Speed of Reasoning (RE): (Not clearly defined by existing research.)

Short-Term Memory (Gsm) Narrow Abilities

Memory Span (MS): Ability to attend to and immediatelyrecall temporally ordered elements in the correct orderafter a single presentation.

Working Memory (MW): Ability to temporarily store andperform a set of cognitive operations on information thatrequires divided attention and the management of thelimited capacity of short-term memory.

Learning Abilities (L1): A number of factors that arespecific to particular kinds of learning situations andmemory [Also listed under Glr] (Not clearly defined byexisting research.)

Gsm - SHORT-TERM MEMORYThe ability to hold information in immediate awareness and then use it within a few seconds

Summary of Relations between CHC Abilities and Specific Areas of Academic Achievement

CHC Ability

Reading Achievement

Math Achievement

Writing Achievement

Gf Inductive (I) and general sequential reasoning (RG) abilities play a moderate role in reading comprehension.

Inductive (I) and general sequential (RG) reasoning abilities are consistently very important at all ages.

Inductive (I) and general sequential reasoning abilities is related to basic writing skills primarily during the elementary school years (e.g., 6 to 13) and consistently related to written expression at all ages.

Gc Language development (LD), lexical knowledge

(VL), and listening ability (LS) are important at all ages. These abilities become increasingly more important with age.

Language development (LD), lexical knowledge (VL), and listening abilities (LS) are important at all ages. These abilities become increasingly more important with age.

Language development (LD), lexical knowledge (VL), and general information (K0) are important primarily after age 7. These abilities become increasingly more important with age.

Gsm Memory span (MS) is important especially when

evaluated within the context of working memory.

Memory span (MS) is important especially when evaluated within the context of working memory.

Memory span (MS) is important to writing, especially spelling skills whereas working memory has shown relations with advanced writing skills (e.g., written expression).

Gv May be important primarily for higher level or

advanced mathematics (e.g., geometry, calculus).

Ga Phonetic coding (PC) or “phonological

awareness/processing” is very important during the elementary school years.

Phonetic coding (PC) or “phonological awareness/processing” is very important during the elementary school years for both basic writing skills and written expression (primarily before age 11).

Glr Naming facility (NA) or “rapid automatic

naming” is very important during the elementary school years. Associative memory (MA) may be somewhat important at select ages (e.g., age 6).

Naming facility (NA) or “rapid automatic naming” has demonstrated relations with written expression, primarily the fluency aspect of writing.

Gs Perceptual speed (P) abilities are important

during all school years, particularly the elementary school years.

Perceptual speed (P) abilities are important during all school years, particularly the elementary school years.

Perceptual speed (P) abilities are important during all school years for basic writing and related to all ages for written expression.

Note. The absence of comments for a particular CHC ability and achievement area (e.g., Ga and mathematics) indicates that the research reviewed either did not report any significant relations between the respective CHC ability and the achievement area, or if significant findings were reported, they were weak and were for only a limited number of studies. Comments in bold represent the CHC abilities that showed the strongest and most consistent relations with the respective achievement domain. Information in this table was

reproduced from McGrew and Flanagan (1998) and Flanagan, McGrew, and Ortiz (2000) with permission from Allyn & Bacon. All rights reserved.

Relationship between Assessment and Intervention

Clinical Aspect: Differential diagnosis Treatment recommendations Educational Planning

Salient Features of Psycho-Educational Assessments

Psychoeducational Test Interpretation: Psychometric/Normative Cognitive Processing and

Neuropsychological analysis Clinical Behavioral

Testing Caveats I

Tests are samples of behavior Tests do not directly reveal traits or

capacities Tests must have adequate

reliability and validity Test scores can be affected by a

myriad of factors that makes interpretation difficult

Testing Caveats II

Test results must be interpreted in context.

Test results depend on the examinee’s cooperation and motivation

Tests supposedly measuring the same ability may produce different scores

Individual subtests, groups of subtests, factor indexes, etc. do not necessarily measure unique cognitive processes, abilities, or constructs.

Relationship between Assessment and Intervention

Behavioral Analysis: Data Collection Integration of observed behavior

across time and settings Hypothesis generation linking test

behavior to class performance

Neuropsychological Models of Assessment

Cognitive/Neuropsychological Aspects Executive function Neurodevelopmental factors, brain

and hemispheric function Profile and Subtest Pattern Analysis Report writing reflects this process

fMRI: Brain Imaging During Reading

Note: Shaywitz, S. (1998). "Current concepts: Dyslexia." The New England Journal of Medicine, 338(5), 307-312.

Dyslexic reader Non-impaired reader

Relationship between Assessment and

Intervention/Instruction

Engagement of the parent, teacher, and other relevant persons in the Assessment Process:

Contextualization Dynamic Assessment Ecological Assessment

Contextualization

Cultural factors Race/Ethnicity Familial background and history Language SES School climate and culture

Contextualization: Assessment of ELLs

Determining need for bilingual assessment

Determining language dominance Language difference/acquisition Language deficiencies

Recommendations for ELLs ESL Bilingual class/service

Standards for Educational & Psychological Testing for ELLs

Any test that employs language is, in part, a measure of their language skills… test results [for ELLs] may not reflect accurately the qualities and competencies intended to be measured (AERA, et al., 1999, p. 91).

Dynamic Assessment

Intelligence testing as cognitive processing Qualitative factors Testing the limits

RTI-Response to Intervention CBA- Curriculum Based

Assessments

Qualitative Factors

Attention Concentration Distractibility Anxiety Frustration Impulsivity Perseverance

Ecological Assessment

Observations and interview data provided by: Parent Teacher Providers Peers School/Classroom milieu School records (including achievement

test results, predictors etc.)

Exosystem Mesosystem

Chronosystem

Macrosystem

Bronfenbrenner’s Ecological Theory of Development

FamilySchool & classroom

Workplace &

Church

Peer group

Microsystems

School system

Political philosophy

Nat

iona

l cus

tom

sE

conomic patterns

Soci

al c

ondi

tion

sC

ultural values

Com

mun

ity

Mass m

ediaM

edic

al in

stitu

tions

Connection: Assessment results need to be approached through various lenses

We need results from the psychologist and the teacher for a complete picture.

Psychoeducational Report Writing: Linking Assessment to

Instruction

Based on the student’s cognitive processing, strengths, weaknesses and interests, how will the curriculum content be instructionally DIFFERENTIATED?

Consider Ecological Theory of Functioning

Consider students’ social emotional functioning

Psychoeducational Report Writing

Individual comprehensive reports need to identify strengths and weaknesses in relevant skill areas and discuss non-cognitive factors affecting performance, and hypothesize how cognitive factors may impact instruction.

Bridging Assessment to the IEP

IEP Present Levels of Performance

(PLOP) Should Include:

Input from student, parents and teachers Unique strategies, needs and

modifications What has worked and what hasn’t? ELL’s: Language acquisition and

acculturation Student preferences and interests Transition

Case of Morgan

Morgan is a 10 year old fifth-grader She is bright, vivacious and outgoing Her favorite school extracurricular activities

are music and dancing She frequently forgets what she is doing;

she has trouble sustaining attention and remembering things in school even for a very short time

She has great difficulty organizing work for large school projects

Note: “Morgan,” from McCloskey, Perkins & Divner (2009), Assessment and Intervention for Executive Function Difficulties. NY: Routledge

Determining Educational Benefit

Present Level of Performance Assessment Results

Needs & Concerns Goals and objectives Accommodations & Modifications Services/Placement Progress Made on Annual Goals

NOW…Determine: Is this IEP Reasonably Calculated to provide Educational Benefit? Is there a clear alignment between the assessment results, identified needs, goals, and services? If there is, you should be able to draw a line that will connect each item in each column.

Present Levels of Performance

Needs &Concerns

Goals & Objectives Accommodations

&Modifications

Services&

Placement

Progress

Toward

Goals

Morgan is a bright, vivacious, outgoing 5th grader.

She enjoys music and dancing.

Morgan has weak working memory skills related to short attention span; in the classroom she frequently forgets what she is doing and becomes easily distracted and distracts peers.

She has great difficulty with organizing work for large school projects.

Morgan has weak working memory skills related to short attention span; in the classroom she frequently forgets what she is doing.

She has great difficulty with organizing work for large school projects.

In one year, given positive behavioral strategies, e.g., role-playing, self-monitoring and modeling, Morgan will be able to work in a group setting, without distractions, independent of teacher prompts 9 out of 10 times over a two-month period.

In one year, given organizational strategies, e.g., checklists, chunking, process charts, Morgan will schedule, plan and pace the components of a 5-step project. She will complete all 5 steps of two school projects at the expected due date with 90% accuracy.

Modeling *Role-playingChecklistsVerbally repeat steps to understand directionsProvide clear rules and expectationsProvide seating with minimal distractionsChunking assignments into manageable piecesVerbal promptsInstructional cues and remindersProcess charts (graphic organizer)Time and ½ *Separate location* Directions read*

CTTCounseling

YES

YES

Was the student’s program reasonably calculatedreasonably calculated to result in educational benefiteducational benefit? YES or NO?YES or NO?

Note: “Morgan,” from McCloskey, Perkins & Divner (2009), Assessment and Intervention for Executive Function Difficulties. NY: Routledge

Linking Assessment to Instruction

Resources for IEP Goal Writing-- ATDR (Achievement Test Desk Reference)

Flanagan, et al., (2002) PRIM- Pre-Referral Intervention Manual LIM- Learning Intervention Manual

www.hes-inc.com CHC Theory www.iapsych.com/chcpp