Monitoring Child Outcomes: The

Good, the Bad, and the Ugly

Anne LucasMeasuring Child and Family Outcomes

August 27 – 28, 2008

2

Public Policy Context

• Age of accountability – IDEA 2004• Accountability increasingly means

looking at results, not just process• Office of Special Education

Programs (OSEP) under pressure to produce outcome data on children participating in early intervention and early childhood special education programs and for holding states accountable to requirements

3

The IDEA Accountability Trail

Compliance to Procedural Safeguards

Compliance Monitoring

CIFMS

Identification Pro

cedura

l Saf

eguar

ds

ACCESS

Results

1980’s

1969

2004

1997

SPP/APR & Determinatio

ns

CIMP

Data-

based

decisi

ons

Compliance and Results

Compliance, Results, Data-based Decisions and Technical

Assistance

IDEA

4

IDEA 2004

• Requires that states have a general supervision system that focuses on:– Improving educational results and

functional outcomes for all children with disabilities

– Ensuring that States meet the program requirements of IDEA, with a particular emphasis on those requirements that are most closely related to improving results and outcomes for children with disabilities

5

IDEA 2004

• States must have in place a General Supervision System that monitors implementation of IDEA by early intervention programs (whether or not they receive Part C funds), and local education agencies statewide.

[IDEA 2004, Section 616(a)(2)]

6

State Performance

Plan

Policies, Procedures, and Effective

Implementation

Data on Processes

and Results

Targeted Technical

Assistance & Professional Development

Effective Dispute

Resolution

Integrated Monitoring Activities

Improvement, Correction,

Incentives & Sanctions

Components: State General Supervision System

Fiscal Management

7

Interrelationship of Components

• Although the components are described separately (puzzle pieces), they connect, interact and articulate to form a comprehensive system

(see Developing and Implementing an Effective System of General Supervision: Part C at www.accountabilitydata.org)

• Each component informs and gains information from the others

8

Implementation of Components

• States have flexibility regarding how they:– Implement each of these components– Interrelate the components with other

initiatives and needs within their state (including their organizational structure and other factors relevant to how they are accountable)

– Design their Own Model of General Supervision based on what’s required and desired

9

Interrelationship of Components

The National Early ChildhoodTechnical Assistance Center

Based on the Big 8 of General Supervision . Teal boxes represent the Big 8 components. 3/16/08

Integrated On-site/Off-site Monitoring Activities

Record Reviews

Surveys (Family/Provider) Interviews (Family/Provider)

Data on Process and Results

Analysis and verification of data from state data system

and other sources that inform performance and compliance with Part C

requirements and SPP/APR and state monitoring

indicators

Complaints/Disputes Self Assessment

Desk Audit

Contract Management

Follow-up/ Verify

Correction

Targeted T & TA

Corrective Action / Improvement Plans Addresses infrastructure, supervision, personnel, resources, T & TA, etc.

Sanctions, if necessary

Incentives/ Rewards

for Good Performance

Compliance

Noncompliance

(Findings)

IDEA Part C Requirements (Federal Law and Regulations)

SPP/APR and Other State Monitoring Indicators Includes SPP/APR Indicators that are relevant to local EI programs and other state priority

indicators that are closely aligned with results

State Policies, Procedures and Fiscal Management

Contracts and Agreements

Training and TA

Reporting to Public Status Determination

PROGRAM

PERFORMANCE

Other On-site Activities

10

State General Supervision System

• State General Supervision System must include multiple methods to:– Ensure implementation of IDEA 2004– Identify and correct noncompliance (e.g., off

site and onsite monitoring, complaints)– Facilitate improvement – Support practices that improve results and

functional outcomes for children and families

11

Monitoring

States are responsible for monitoring all Part C requirements using the all components of the

General Supervision System (e.g., policies/procedures, IAs, contracts, complaints)

. . . but . . .

this is different from collecting and analyzing “monitoring” data to determine local program

performance on an annual basis

12

How are states integrating child and family outcomes into their local

monitoring processes?

13

Monitoring Indicators

• Some states include child and family outcome compliance and quality indicators in their monitoring of local EI programs or LEAs – Adapting SPP/APR indicators (e.g., COSF

completed at entry and exit for each child, family survey provided to families at time of annual IFSP meeting); and/or

– Developing other state priority indicators (e.g., COSF ratings match assessment results)

14

Collecting Monitoring Data

• States use variety of methods to collect monitoring data on child and family outcomes:– Data system– Self-assessment– On-site monitoring or data verification visits

15

State Presentations

Wyoming and Utah

16

Resources

SPP/APR Calendar (C-9 and B-15)http://spp-apr-calendar.rrfcnetwork.org/

NECTAC General Supervision Webpage http://www.nectac.org/topics/quality/gensup.asp

Anne Lucasalucas3@comcast.net