data workshop: analyzing and interpreting data kathy hebbeler, eco at sri lynne kahn, nectac/eco at...
TRANSCRIPT
![Page 1: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/1.jpg)
Data Workshop: Analyzing and Interpreting Data
Kathy Hebbeler, ECO at SRILynne Kahn, NECTAC/ECO at FPG
Cornelia Taylor, ECO at SRIAnn Bailey, NCRRC
Presented at the Measuring and Improving Child and Family Outcomes Conference
New Orleans, LA September 2011
![Page 2: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/2.jpg)
1. Using Data
2Early Childhood Outcomes Center
![Page 3: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/3.jpg)
Using data for program improvement = EIA
Evidence
Inference
Action
3
![Page 4: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/4.jpg)
Evidence
• Evidence refers to the numbers, such as
“45% of children in category b”
• The numbers are not debatable
4
![Page 5: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/5.jpg)
Inference
• How do you interpret the #s?• What can you conclude from the #s?• Does evidence mean good news? Bad
news? News we can’t interpret?• To reach an inference, sometimes we
analyze data in other ways (ask for more evidence)
5
![Page 6: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/6.jpg)
• Inference is debatable -- even reasonable people can reach different conclusions
• Stakeholders can help with putting meaning on the numbers
• Early on, the inference may be more a question of the quality of the data
6
Inference
![Page 7: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/7.jpg)
Action
• Given the inference from the numbers, what should be done?
• Recommendations or action steps• Action can be debatable – and often is• Another role for stakeholders• Again, early on the action might have to do with
improving the quality of the data
7
![Page 8: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/8.jpg)
Good Data??
Programs
Bad/Weak Good/Strong
Data
Bad/Weak ?? ??
Good/Strong Program Improvement
Questionable Good
Program Improvement
Good Better
8Early Childhood Outcomes Center
![Page 9: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/9.jpg)
Good Data??
Programs
Bad/Weak Good/Strong
Data
Bad/Weak ?? ??
Good/Strong Program Improvement
Questionable Good
Program Improvement
Good Better
9Early Childhood Outcomes Center
![Page 10: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/10.jpg)
Good Data??
Programs
Bad/Weak Good/Strong
Data
Bad/Weak ?? ??
Good/Strong Program Improvement
Questionable Good
Program Improvement
Good Better
10Early Childhood Outcomes Center
![Page 11: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/11.jpg)
2. Primer on how to present data
11Early Childhood Outcomes Center
![Page 12: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/12.jpg)
• What do you want your audience to notice? Where do you want their eyes to go?
• Alternate presentation of the same data. What works?
• Chartjunk and minimal ink• When pies don’t work?
12Early Childhood Outcomes Center
![Page 13: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/13.jpg)
Early Childhood Outcomes Center 13
Social relationships Knowledge and skills Action to meet needs
SS1 0.70973347275746 0.742180533377054 0.755546347980436
SS2 0.61772636892334 0.537547577335669 0.596725744958282
10%
30%
50%
70%
Estimated Summary Statement Data for Part C, 2009-10
![Page 14: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/14.jpg)
Early Childhood Outcomes Center 14
SS1: Greater than Expected Growth
SS2: Exit like Same-Aged Peers
Social relation-ships
0.70973347275746 0.61772636892334
Knowledge and skills
0.742180533377054 0.537547577335669
Action to meet needs
0.755546347980436 0.596725744958282
10%
30%
50%
70%
Estimated Summary Statement Data for Part C, 2009-10
![Page 15: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/15.jpg)
Early Childhood Outcomes Center 15
Social relationships Knowledge and skills Action to meet needs
SS1 0.70973347275746 0.742180533377054 0.755546347980436
SS2 0.61772636892334 0.537547577335669 0.596725744958282
10%
30%
50%
70%
Estimated Summary Statement Data for Part C, 2009-10
![Page 16: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/16.jpg)
Early Childhood Outcomes Center 16
Social relationships Knowledge and skills Action to meet needs
SS1 0.70973347275746 0.742180533377054 0.755546347980436
SS2 0.61772636892334 0.537547577335669 0.596725744958282
5.0%
15.0%
25.0%
35.0%
45.0%
55.0%
65.0%
75.0%
Estimated Summary Statement Data for Part C, 2009-10
Wasted ink
![Page 17: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/17.jpg)
Early Childhood Outcomes Center 17
a b c d e0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
Social relationships
Knowledge and skills
Action to meet needs
No 3-D
![Page 18: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/18.jpg)
Early Childhood Outcomes Center 18
2%
18%
19%
29%
32%
Outcome 1
a b c d e
2%
22%
24%35%
18%
Outcome 2
a b c d e
![Page 19: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/19.jpg)
3. Making Comparisons
19Early Childhood Outcomes Center
![Page 20: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/20.jpg)
• What constitutes a meaningful difference?– Statistical significance refers to sampling from
a population– Your child outcomes data is the population.
• What constitutes stability in your data?
20Early Childhood Outcomes Center
![Page 21: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/21.jpg)
Comparing to state numbers
• Influence of big entities on totals• Demonstration with an example of how a
huge change in a small entity makes almost no difference on a – e, but how that same proportionate change in a big entity makes a big difference (i.e., why state numbers will often look a lot like the biggest subunit.
21Early Childhood Outcomes Center
![Page 22: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/22.jpg)
Early Childhood Outcomes Center 22
a b c d e Total N228 0.07 112 0.04 1003 0.32 1361 0.44 410 0.13 311427 0.02 290 0.24 249 0.21 329 0.27 317 0.26 121281 0.01 1470 0.20 1110 0.15 1842 0.25 2751 0.38 725412 0.01 420 0.27 92 0.06 200 0.13 833 0.54 155711 0.00 688 0.28 379 0.15 564 0.23 821 0.33 246335 0.01 408 0.14 84 0.03 583 0.21 1706 0.61 281612 0.00 543 0.16 674 0.20 1121 0.33 1006 0.30 335637 0.02 701 0.28 687 0.28 614 0.25 429 0.17 246836 0.03 271 0.21 384 0.29 304 0.23 319 0.24 13141 0.00 100 0.30 51 0.15 96 0.29 88 0.26 33615 0.03 65 0.13 48 0.10 142 0.29 215 0.44 4856 0.01 298 0.29 320 0.31 336 0.32 84 0.08 10448 0.01 112 0.11 148 0.14 330 0.32 447 0.43 104561 0.03 382 0.18 378 0.18 714 0.33 619 0.29 215452 0.01 1161 0.19 1179 0.19 1994 0.33 1677 0.28 6063
1.7% 19.1% 18.5% 28.7% 32.0%
![Page 23: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/23.jpg)
Early Childhood Outcomes Center 23
a b c d e228 0.07 112 0.04 1003 0.32 1361 0.44 410 0.13 Total N27 0.02 290 0.24 249 0.21 329 0.27 317 0.26 311481 0.01 735 0.20 1845 0.15 1842 0.25 2751 0.38 121212 0.01 420 0.27 92 0.06 200 0.13 833 0.54 725411 0.00 688 0.28 379 0.15 564 0.23 821 0.33 155735 0.01 408 0.14 84 0.03 583 0.21 1706 0.61 246312 0.00 543 0.16 674 0.20 1121 0.33 1006 0.30 281637 0.02 701 0.28 687 0.28 614 0.25 429 0.17 335636 0.03 271 0.21 384 0.29 304 0.23 319 0.24 24681 0.00 100 0.30 51 0.15 96 0.29 88 0.26 131415 0.03 65 0.13 48 0.10 142 0.29 215 0.44 3366 0.01 298 0.29 320 0.31 336 0.32 84 0.08 4858 0.01 112 0.11 148 0.14 330 0.32 447 0.43 104461 0.03 382 0.18 378 0.18 714 0.33 619 0.29 104552 0.01 1161 0.19 1179 0.19 1994 0.33 1677 0.28 2154
1.7% 17.1% 20.5% 28.7% 32.0% 60631.7% 19.1% 18.5% 28.7% 32.0%
![Page 24: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/24.jpg)
Early Childhood Outcomes Center 24
a b c d eTotal N
228 0.07 112 0.04 1003 0.32 1361 0.44 410 0.13 311427 0.02 290 0.24 249 0.21 329 0.27 317 0.26 121281 0.01 1470 0.20 1110 0.15 1842 0.25 2751 0.38 725412 0.01 420 0.27 92 0.06 200 0.13 833 0.54 155711 0.00 688 0.28 379 0.15 564 0.23 821 0.33 246335 0.01 408 0.14 84 0.03 583 0.21 1706 0.61 281612 0.00 543 0.16 674 0.20 1121 0.33 1006 0.30 335637 0.02 701 0.28 687 0.28 614 0.25 429 0.17 246836 0.03 271 0.21 384 0.29 304 0.23 319 0.24 13141 0.00 100 0.30 51 0.15 96 0.29 88 0.26 33615 0.03 33 0.13 80 0.10 142 0.29 215 0.44 4856 0.01 298 0.29 320 0.31 336 0.32 84 0.08 10448 0.01 112 0.11 148 0.14 330 0.32 447 0.43 104561 0.03 382 0.18 378 0.18 714 0.33 619 0.29 215452 0.01 1161 0.19 1179 0.19 1994 0.33 1677 0.28 6063
1.7% 19.1% 18.6% 28.7% 32.0%1.7% 19.1% 18.5% 28.7% 32.0%
![Page 25: Data Workshop: Analyzing and Interpreting Data Kathy Hebbeler, ECO at SRI Lynne Kahn, NECTAC/ECO at FPG Cornelia Taylor, ECO at SRI Ann Bailey, NCRRC Presented](https://reader036.vdocuments.net/reader036/viewer/2022062322/5697bfd71a28abf838caeaec/html5/thumbnails/25.jpg)
25
Find more resources at: www. the-eco-center-org
Thank you!!