Using Data to Improve Adult Ed Programs

Administrators’ Workshop

Workshop Objectives

Participants will be able to understand the data-driven decision making process.

Participants will be able to identify what types of data can be collected.

Participants will be able to identify what data reports are most useful to teachers and administrators.

Participants will be able to identify what major barriers hinder the data collection process.

Participants will be able to analyze data report examples.

What is Data-Driven Decision Making?

Data-driven decision making is the process of making choices based on appropriate analysis of relevant information.

Why use data to make decisions?

More access to better information enables educational professionals to test their assumptions, identify needs, and measure outcomes.

Administrators and teachers are using data-driven decision making to: provide more individualized instruction to students track professional development resources identify successful instructional strategies better allocate scarce resources communicate better with parents and the community.

What data should we collect and use to make decisions? There is an abundance of information stored.

Examples are: student records student assessment human resources student progress special education curriculum management.

What data reports are most useful to instructors?

TABE or CASAS test results for entire class Student attendance information Teacher developed test results for entire class Progress reporting information for class Student retention information for class Student performance gains (LCP/OCP) for

class

What data reports are most useful to administrators?

TABE or CASAS test results for program area Student attendance information for all programs Progress reporting information for all programs Student retention information for programs/school site Student performance gains (LCP/OCP) for programs

and for school Cost effectiveness of program (Income versus cost of

salaries, fringe, supplies, equipment)

Administrators use all types of data: attendance enrollment student performance (LCP’s, OCP’s) student/teacher/parent satisfaction surveys.

Test results are used to assess progress, allocate resources, and create school improvement plans.

Information is organized numerically rather than alphabetically.

The information includes objective descriptions of data, visual displays of information, and query tools.

What common data report formats are most useful to administrators?

What are the major barriers to using data

based decision-making?

Lack of training Interoperability—systems that are unable to share

or exchange data Lack of understanding of what to do with the data Absence of clear priorities on what data should be

collected Failure to collect data in a uniform manner Outdated technology systems Low quality data – inaccurate or incomplete Timing of data collection

What are the major misconceptions about effective use

of data in decision making?

Build it and they will use it. Teachers need to know how to analyze

data and use query systems. Test scores determine the quality of a

school and the student’s education.

What is necessary for the systematic use of data for decision making?

Collection, integration and dissemination of data

Analysis and reporting of data Process and procedures for acting on the

data Review Analysis Planning

Schools need both organizational and individual capacity for improvement: Leadership Professional development.

Administrators need training with the opportunity to apply skills learned using their own institutional data.

Dialogue with peers keeps the process going. School-based training for faculty and staff is necessary.

Instructors need training in different instructional strategies to apply when the data shows that traditional methods are not working.

What types of skills are needed to implement systemic data processes?

Who are the key decision makers at the school site who should be involved in the data-driven decision making process?

Administrators are the change agents at the school site.

Administrators model data use and encourage it by sharing the benefits and successes.

Site-based specialists or support teams assist administrators and teachers with data mining and analysis.

Who are the key decision-makers at the classroom level who should be involved?

In addition to using data for determining

instruction, teachers can engage students in

the decision making process by helping

them: view appropriate reports; set learning goals; make decisions about how to meet their goals.

Where do we begin?

The process: Develop a leadership team Collect various types of data Analyze data patterns Generate hypotheses Develop goal-setting guidelines Design specific strategies Define evaluation criteria Make the commitment

What information does our institution need to make decisions that will improve student achievement?

What learning strengths and weaknesses are evident in the data?

Which groups or subgroups of students are having difficulty learning?

What instructional changes might improve student learning?

What professional development is need to improve student learning?

What materials and equipment are needed to support changes in instruction?

Example 1 – Making decisions based on data A school examines its student performance

results and finds: As a whole, the school is doing better in

reading than in math Students are doing better in basic computation

than in problem solving As a subgroup, Hispanic male students

perform the lowest on grades and tests at most grade levels

Example 1- continued

Using this information, the school improvement team decides to find the following: An instructional intervention that specifically

addresses basic math computation Interventions that have been especially

effective in improving the performance of Hispanic male students

Example 1 - continued

Professional development is planned to provide instruction on the new intervention: Staff development days are planned for

teachers to learn the new intervention before it is implemented.

Regular short meetings are planned to give teachers time to discuss their efforts and troubleshoot problems while implementation occurs in the classroom.

Example 2: FCAT Classes

Name Course Days LCP Earned

Credit Earned

Pre-Class Score

Post-Class Score

Grade ESE LEP

Maria Intensive Reading

T/R 1 .5 248 300 B No No

Maribel Intensive Reading

T/R 1 .5 223 296 B No Yes

Kasie Intensive Reading

T/R 1 .5 245 286 A Yes No

Elysia Intensive Reading

T/R 1 .5 260 301 B No No

Eric Intensive Reading

T/R 1 .5 246 185 B No Yes

Thordis Intensive Reading

T/R 1 .5 297 306 A No Yes

Totals/

Average6 3 253 279 3.33

Example 3 – Budget Data Across Sites

Site Fee

Support

Enrolled

FS Percent

Enrolled

9-12 Enrolled

9-12 Percent Enrolled

Grades 30/31

Enrolled

Grades

30/31

Percent

Enrolled

Total

Enrollment

Budget Cost

per

Student

ACE

1139 12% 291 25% 715 62% 1,145 $619,537 $541

ACE

288 9% 322 33% 567 58% 977 $682,826 $698

ACE

3134 16% 360 44$ 327 40% 821 $481,238 $586

ACE 4 12 2% 166 27% 438 71% 616 $460,099 $746

ACE

5192 25% 181 24% 387 51% 760 $619,698 $815

Totals/AVG

565 13% 1,320 31% 2,434 56% 4,319 $2,863,398 $677

Example 4 – Co-Enrolled Completion/Retention Semester II

Course Title Enrollment LCP’s Earned Students with no LCP’s

Percent Completing

Percent Not Retained to Completion

English I 12 8 4 67% 33%

English II 8 5 3 63% 37%

English III 9 2 7 22% 78%

English IV 17 2 15 12% 88%

Totals/

Averages46 17 29 33% 67%

Data Review

Other types of information teachers may want to review: Student records with demographic data (LEP,

ESE, grade level completed, courses taken, GPA)

FCAT scores TABE scores Other test records

More Data Review

Administrators may collect and analyze other reports: Class attendance School surveys – satisfaction or quality

surveys Individual program/class surveys of

satisfaction or delivery of services Student exit surveys W-26 reports (high school students

withdrawing to attend postsecondary programs)

Let’s Review the DATA!

Using the handouts provided, break into small groups and discuss the data presented on each handout following directions provided.

What assumptions can be made about the data presented?

Did the data serve to confirm ideas you had already pondered?

What other types of data could you collect about your program(s)?

References/Links

Data-Based Decision Making – Resources for Educators. http://www.ael.org/dbdm/Tutorial.cfm?&ider=Deve4060

D3M: Helping schools distill data. http://eric.uoregon.edu/search_find/data_analysis.html

The Toolbelt: A collection of data-driven decision-making tools for educators. http://www.ncrel.org/toolbelt/