1 trends in child outcomes (c-3 / b-7) and family outcomes (c-4) analysis and summary report of all...

1

Trends inChild Outcomes (C-3 / B-7)and Family Outcomes (C-4)

Analysis and Summary Report of All States’ 2008-2009Annual Performance Reports

Christina Kasprzak,Robin Rooney, Siobhan ColganLynne Kahn, Kathy Hebbeler (NECTAC / ECO)

November 30, 20104:00 PM EST

2

1. National data on child and family outcomes (C3/B7 and C4)

2. Challenges related to collecting and reporting on this indicator

3. Improvement activities

Webinar Focus

Quick Poll 1

Who is joining uson the call today?

4

Child Outcomes Data

Summary of 2010 APR Data

5

Part C State Approaches (n=56) to Measuring Child Outcomes

• Child Outcomes Summary Form (COSF)

– 41 (73%) states

• Single assessment statewide

– 7 (13%) states

• Publishers’ online assessment systems

– 3 (5%) states

• Other approaches

– 5 (9%) states

6

HI

GU

AS

MP

Legend: COSF Publishers’ on-line systems One tool statewide Other

State Approaches to Child Outcomes Measurement Part C Program

Early Childhood Outcomes Center –August 2010

7

7

619 State Approaches (n=59)to Measuring Child Outcomes

• Child Outcomes Summary Form (COSF)– 36 (61%) states

• Single assessment statewide– 9 (15%) states

• Publishers’ online assessment systems– 6 (10%) states

• Other approaches– 7 (12%) states

8

MH

HI

GU

PWFM

AS

MP

Legend: COSF Publishers’ on-line systems One tool statewide Other

State Approaches to Child Outcomes Measurement Section 619 Programs

Early Childhood Outcomes Center –August 2010

9

The number of Part C childrenwith outcome data is increasing!

Part C Total Number of Children States Included in Progress Data

# of children reported

# of StatesFFY07

# of StatesFFY08

99 or less 13 6

100-499 25 16

500-999 6 13

1000-1999 9 11

2000+ 3 10

Range = 5-6452 Range = 11-7998

10

The number of 619 childrenwith outcome data is increasing!

619 Total Number of Children States Included in Progress Data

# of children reported

# of StatesFFY07

# of StatesFFY08

99 or less 12 8

100-499 14 6

500-999 10 12

1000-1999 8 7

2000-2999 5 10

3000+ 9 15

Range= 3-10,157 Range= 3-9,967

Quick Poll 2

Number of childrenin the data

12

Missing Data

• ECO additional analysis

• State efforts to identify missing data

• State efforts to reduce missing data

13

National Conference Callon Data Quality – Coming Soon

• What do you know the quality of your state’s outcomes data?

• Do you know how much missing data you have? How much is reasonable?

Missing data is still a major problem for many states. Join us to learn about how much progress has been made and how your state compares to the national numbers.

14

15

16

Part C Progress data trends FFY07FFY08

17

619 Progress data trends

FFY07FFY08

Quick Poll 3

ProgressData Trends

19

Part C - Category ‘e’ by % served

20

FFY2008 Summary Statements

Baseline FFY08Part C 619

SS1 SS2 SS1 SS2Outcome 1

63% 63% 76% 61%

Outcome 2

68% 54% 76% 53%

Outcome 3

69% 62% 75% 65%

21

Part C - Summary Statement 2 by % Served

22

What States are Doingfor Improvement

• Continuing training and TA on data collection system

• Enhancing data systems• Developing data analysis • Identifying and addressing

data quality issues• Identifying areas for program

improvement

Open Discussion

Questions?Comments?

24

Family Outcomes Data

Summary of 2010 APR Data

25

State Approaches to Family Outcomes Measurement Part C Program

Legend: ECO Family Outcomes Survey State-developed survey NCSEAM survey

HI

GU

AS

MP

Early Childhood Outcomes Center – August 2010

26

Survey timing and family population

Timing

Family Population

TotalAll familiesFamilies with ≥6

months*

Point in time (or time period) 25 12 37

Based on child participation 8 9

17

-at exit (3) (6)

-at IFSP meeting(s) (3) (2)

-IFSP and/or exit (2) (1)

Total 33 21 54

*One State used ≥9 months, and one State used ≥12 months

Quick Poll 4

Surveydistribution

28

Response rate variables

29

Response rate by state size

30

Representativeness: Comparison data

• Thirty-nine states (70%) reported the source of data used: – Part C population/ 618 data: 31

states– Program population data: 3 states – Target population: 3 states

– State data (not specified): 2 states

• Remaining 17 states did not specify

Quick Poll 5

Addressingrepresentativeness

32

Criteria used for evaluating representativeness

• Forty-six states (89%) reported the criteria they used for determining representativeness – Race/ ethnicity: 73% (41 states)– Geography (district, county, region): 50% (28 states)– Sex: 21% (12 states)– Child’s age: 20% (11 states)– Disability/ eligibility category: 9% (5 states) – Length of time in services: 9% (5 states)– Program size : 9% (5 states)

• Previous years: 2009: 44/56 (78.6%) and 2008: 37/56 (66.1%) reported criteria used

• Mean number of criteria used this year: 2.7 Previous years: 2009: 2.6 criteria and 2008: 2 criteria.

33

Performance and trends

Early intervention has helped…

A. Families know their rights: 84% +3% from last year

B. Families effectively communicate children’s needs: 85%

+2% from last year

C. Families help their children develop and learn: 90% +2% from last year

34

Performance by Survey Type

35

Performance by scoring criteria

36

Performance by state size

37

Performance by percent served

38

Were data representative?

• Forty-four states reported whether their data were representative (79%) – Yes, some data provided: 36% (20

states)– Yes, no data provided: 14% (8 states)– No: 11% (6 states)– Varied results: 18% (10 states)

• No conclusions re: representativeness reported among the remaining 12 states (21%)

39

State Highlights

• Analyzing and reporting outcomes among subgroups

• Improvement activities based on detailed analysis (e.g. by subgroups of families)

• Data collection improvement activities

• Partnering w/parent organizations

40

Suggested Formats for February 2011 APR Reporting

http://www.fpg.unc.edu/~eco/pages/ fed_req.cfm#ECOSuggestedFormats

(this link is also available from

the webinar series page)

Open Discussion

Questions?Comments?

42

Full APR analysis Reportsare Available Online

• Part C:• http://www.nectac.org/~pdfs/calls/2010/

partcapr/2010_spp_partc_report.pdf

• Part B:• http://www.nectac.org/~pdfs/calls/2010/

partcapr/2010_spp_partc_report.pdf

(these links are also available from the webinar series page)

43

Thank you for participating.

Presentations from this series and their related resources are made available on the NECTAC website at:

http://www.nectac.org/~calls/2010/partcapr/partcapr.asp

Trends inChild Outcomes (C-3 / B-7)and Family Outcomes (C-4)