1

Great Expectations: Great Expectations: Practice and Policy for Practice and Policy for Developing a Quality Developing a Quality Outcomes Assessment Outcomes Assessment SystemSystem

A Tale of Two StatesA Tale of Two States

Meredith CathcartMeredith CathcartAnne KuschnerAnne KuschnerLarry Edelman Larry Edelman Patty Salcedo Patty Salcedo

Measuring Child and Family Outcomes 2008Measuring Child and Family Outcomes 2008

2

California’s Child Outcomes System: Quality Assessment to Quality Programs

Overview of California’s DRDP Assessment System

Policies and practices that support quality assessment systems- Demonstration of an online learning module

designed for new assessors

Use of data to improve services and outcomes

3

The Context of Diversity in California

Size Geographic

Ethnic Linguistic

Socio-economic

California Department of Education/Special Education Division

• Overall Direction

• Policy Development

• Funding

• Monitoring and Enforcement

• State and Federal Reporting

• Inter- and Intra-agency Coordination

Training and Technical

Assistance for ECSE

Statewide ECSE

Conferences & Leadership

Training

ECSE Support for Regular

Child Development

LEA Partners for Research &

Training in Assessment

SEEDS SEECAP CPIN DR Grantees

•Overall Leadership and Coordination of Assessment Efforts

•Instrument Research and Development

•Training and Technical Assistance in Assessment

•Data Collection and Analysis

•Support for Use of Data

Desired Results Access Project

5

Child Development Division (CDD)

Special Education Division(SED)

Desired Results Developmental Profile - Revised (DRDP-R) for:

• Infant/toddler

• Preschool

• School-age

• Preschool Desired Results Developmental Profile – Revised for Children with IEPs (PS DRDP-R)

• Desired Results Developmental Profile access (DRDP access)

California Department of EducationDesired Results System Assessment Instruments

Purposes of the DRDP Assessment System

Assure that preschool-age children with IEPs and their families benefit from California’s ECSE services and programs

Provide information about children’s development in relation to research and state standards so that programs, services, and supports are designed to maximize children’s potential

Establish a “common ruler” for describing children’s progress across the state

6

Goals of the DRDP Assessment System

Improve results for children and families Provide families with information about child

progress and ways to support their children’s learning

Provide teachers with authentic assessment information to inform program planning

Provide program administrators, staff, and stakeholders with data to improve overall program design

Enable California’s preschool special education system to comply with federal reporting requirements

7

8

Principles Guiding the Development of the DRDP Instruments

Desired Results are for all children Content of the instruments is based on child

development research Instruments utilize an authentic assessment model

and are based on observation in typical environments

Multiple sources of data should be gathered over time

Assessments are universally designed Instruments include adaptations for children with

disabilities

Basics of California’s System

In addition to the SPP, California must meet IDEA requirements for statewide assessment:- Every 3,4,and 5 year old is assessed 2x per year- Assessment windows cover 6 month intervals

The SELPA is the unit of accountability for reporting results

IEP teams determine which DRDP instrument to use and which adaptations should be in place

Data is reported using a statewide web-based system, SEDRS

T/TA system supports fidelity at all levels

9

SystemsChange

Collaboration

FamilyParticipation Making

Meaningof Data

ReportingData

TechnicalAssistance

Instruments

CaliforniaGoals

& Federal

Reporting

Training

SystemsChange

Collaboration

FamilyParticipation Making

Meaningof Data

ReportingData

TechnicalAssistance

Instruments

CaliforniaGoals

&Federal

Reporting

Training

……and so on…and so on…

The DRDP Instruments for The DRDP Instruments for Preschool Special EducationPreschool Special Education

12

Navigating the DRDP Instruments

Our first online module in learning to use the DRDP instruments

13

2

Navigating the DRDP Instruments

The Desired Results access ProjectNapa County Office of Educationwww.draccess.org

Funded by the California Department of Education, Special Education Division

Larry Edelman,Desired Results access Project

14

DRDP Adaptations

1. Augmentative or Alternative Communication System

2. Alternative Mode for Written Language

3. Visual Support

4. Assistive Equipment or Device

5. Functional Positioning

6. Sensory Support

7. Alternative Response Mode

Training, Technical Assistance, and Training, Technical Assistance, and Professional DevelopmentProfessional Development

16

17

Training Facilitates Quality Assessment Practices

Teachers understand the requirements and responsibilities of the DRDP Assessment System

Teachers feel confident and competent using the DRDP assessment tools

The quality of data collected increases with fidelity to the use of the instrument

18

What Worked for the First 2 Years of Implementation

Creating a phase-in of the system over several years

Creating a training cadre drawn from highly qualified, experienced statewide trainers- Built on collaboration between ECSE T/TA contractors,

SELPAs, and CDE

Systematically mentoring less experienced trainers Conducting a mandatory Train-the-Trainer institute Piloting the training sessions Training trainers on the use of technology Insisting on minimum standards for training facilities

Current Training System Components

Hold two statewide trainings annually for new assessors

Support and build capacity for local training events Provide online training modules for assessors and

data entry personnel Provide online guidance documents, teacher

resources, and discussion forum and listservs Disseminate updates at statewide and local

conferences and events Communicate with SELPA and link with CDE

19

A Brief Tour of www.draccess.org

20

Making Meaning of DRDP DataMaking Meaning of DRDP Data

21

What Data is Collected?

Every preschool child with an IEP is assessed on the ten DRDP Indicators twice per year

Current demographic information is gathered on each child at each assessment period

Raw data is submitted by assessors or SELPAs to SEDRS

Scale scores are generated by SEDRS and reported back to SELPAs, district administrators, and service providers

22

Primary Uses of Assessment Information

Serves multiple purposes:- accountability- monitoring of child progress- program planning

Provides immediate, useful feedback about an individual child’s progress and status

Assessment scores are easily understood by staff and families, e.g.:- plot a child’s developmental trajectory - display status relative to typical development

23

24

The DRDP Contributes to:

Formative assessment:- information is gathered on an ongoing basis- observation of child behavior is conducted in typical routines and

activities- information is gathered from families, child care providers, and/or

related service providers

Summative assessment- data relative to the three OSEP child outcomes is aggregated by

CDE and sent to OSEP- data for each SELPA is reported to the public according to OSEP

requirements- data for programs is be provided through the use of DRDP scale

scores

25

The blue colored band represents the typical developmental range for the child’s age

The black line marks the place represented by the child’s scale score in that particular indicator

26

This report depicts the change between two assessment periods.

27

Group Measures into Indicators

28

Facilitated Process: Reflective Data Analysis (pilot 2008-09)

When data analysis is done within the context of the team it yields useful information to programs and adds to increased data quality.

Build time to review and analyze data into team meetings.

Build a classroom and program summary Ask questions about the completeness and

accuracy of the data Ask questions about the implications of the results

for program design, services, and staff development

Some Questions about Completeness and Accuracy of Data

If children have a developmental age less than 3 years they should NOT be assessed on the PS DRDP-R: - Are IEP teams using an appropriate decision-making

process?- What training is needed for IEP teams?

Assessors should be able to rate 90% of Measures: - Do teachers understand the developmental constructs

being measured by the DRDP?- What are the barriers to observing collecting data?

Most children with visual impairments should be assessed with visual supports:- What is the use of this adaptation in your program?- Is more support needed for teachers?

29

DRDP Results

By Indicator

By OSEP Outcome

By Developmental

Area

By Learning Foundations/St

andards

Specialty Curriculum

General Curriculum (commercial or locally

developed)

Learning Foundations /Standards

Child Development Research

Curriculum Linkage

Functional

Academic

Age level

Standards-based

IEP goals

Current ActivitiesCurrent Activities

31

Continue activities to support quality professional development

Modify the DRDP access instrument to align with Learning Foundations/preschool standards and ongoing revisions to the PS DRDP-R

Increase support to local communities on professional development, data reporting, and data analysis

Expand activities to interpret results in useful ways at local and state levels

Create additional reports in the data collection and analysis software (SEDRS) to assist teachers, parents, and staff

32

Current Activities

Thank you!Thank you!