critical markers of high quality child outcomes data
DESCRIPTION
Critical Markers of High Quality Child Outcomes Data. ECO Advisory Board March, 2012. DRAFT. Topics. Why identify critical markers of high quality data? Anticipated use 3 areas of focus Draft critical markers - PowerPoint PPT PresentationTRANSCRIPT
Critical Markers of High Quality Child Outcomes Data
ECO Advisory BoardMarch, 2012
Topics
• Why identify critical markers of high quality data?
• Anticipated use• 3 areas of focus• Draft critical markers• Where these might be
discussed in the APR• Reactions?
2Early Childhood Outcomes Center
Looking for…
Proposed markers that suggest a state’s EC outcomes systems may produce high quality data
Use markers to:
- systematically track over time - track internally (within states)
- to produce a national picture
3Early Childhood Outcomes Center
Anticipated Use
• Tracking state progress (based on APR info)• Compare data from each state to a series of
critical markers for summaries– X % of states met standard on each marker nationally– X% of states met standard on 7 out of 10 markers
• NOT a state by state report card• Share detailed info with each state, upon
request
4Early Childhood Outcomes Center
3 Areas
• Completeness of data
5Early Childhood Outcomes Center
Completeness of data
• Missing data concepts– Are whole forms missing? (Have data
from all kids expect to have data from)
– Are the forms that you have complete?
6Early Childhood Outcomes Center
3 Areas
• Completeness of data
• Accuracy of data
7Early Childhood Outcomes Center
Accuracy of Data
• Do the data reflect the “true” performance of children in the program– Unknowable, so….
• Are the patterns in data what you would expect to see if the data were accurate?
• Do they make sense? Or, are there red flags that raise more questions?
8Early Childhood Outcomes Center
3 Areas
• Completeness of data
• Accuracy of data
• State efforts related to tracking quality of data
9Early Childhood Outcomes Center
Criteria for Selection of Markers
• Important – necessary for or indicative of
high quality data• Accessible to ECO
– Information to determine presence or absence of marker is available
10Early Childhood Outcomes Center
Draft Markers: Completeness
• State calculates and publicly reports number of children missing outcomes data
• Percent of missing outcomes data is less than 5%. – Is this reasonable?
• Percent of missing data by proxy calculation is less than. – 40% of exiters for Part C– 20% of child count for 619
11Early Childhood Outcomes Center
Part C percent of exiters 2009-10
12Early Childhood Outcomes Center
Draft Markers: Accuracy/Patterning
• State % in a is not overly high (GT 5%)• State % in b is not overly low (LT 5%) or high (GT 50%)• State % in c is not overly low (LT 5%) or high (GT 50%)• State % in d is not overly low (LT 5%) or high (GT 50%)• State % in e is not overly low (LT 5%) or high (GT 65%)
13Early Childhood OutcGTomes Center
Knowledge and SkillsPart B 619 proportion of children that make no
progress (progress category a)
14Early Childhood Outcomes Center
Knowledge and SkillsPart B 619 proportion of children who improved functioning but not sufficiently to move nearer to functioning comparable to same age
peers(progress category b)
15Early Childhood Outcomes Center
Knowledge and SkillsPart B 619 proportion of children who improved functioning to a
level nearer to same age peers but did not reach it(progress category c)
16Early Childhood Outcomes Center
Knowledge and SkillsPart B 619 proportion of children who improved functioning to
reach a level comparable to same aged peers(progress category d)
17Early Childhood Outcomes Center
Knowledge and SkillsPart B 619 proportion of children who maintained
functioning at a level comparable to same aged peers(progress category e)
18Early Childhood Outcomes Center
Percent of states meeting the Critical Markers for 2009 - 10
19Early Childhood Outcomes Center
Accuracy of data Completeness of data
Both
Part C 66 59 47
Part B 619 63 66 41
Draft Markers: State Quality Review of Data
• State conducts data quality checks• State’s own analyses provide evidence of
high quality data
20Early Childhood Outcomes Center
21Early Childhood Outcomes Center
Completeness/ Missing Data
Accuracy/ Patterns
Location in Suggested APR Templatehttp://www.fpg.unc.edu/~eco/pages/fed_req.cfm#ECOSuggestedFormats
22Early Childhood Outcomes Center
State Quality Review of Data
Location in Suggested APR Templatehttp://www.fpg.unc.edu/~eco/pages/fed_req.cfm#ECOSuggestedFormats
Questions for you
• Are these markers important?– Related to high quality data– Things you value and might track or are already tracking
them?– Would tracking them help you improve your system?
• Is there anything else you can think of as a good marker of high quality data
23Early Childhood Outcomes Center
Summary – Draft Critical Markers
• State calculates and publicly reports number of children missing outcomes data
• Percent of missing outcomes data is less than 5%• Percent of missing data by proxy calculation is less than X • State % in a is not overly high (GT 5%)• State % in b is not overly low (LT 5%) or high (GT 50%)• State % in c is not overly low (LT 5%) or high (GT 50%)• State % in d is not overly low (LT 5%) or high (GT 50%)• State % in e is not overly low (LT 5%) or high (GT 65%) • State conducts data quality checks• State’s own analyses provide evidence of high quality data
24
Comments, Reactions, Questions?
25