2012 USER CONFERENCE 1 2012 USER CONFERENCE
Using EXPLORE and PLAN Results to Predict College/Career Readiness
2012 DataDirector User Conference October 11-12, 2012
Stan Masters and Mike Husband Lenawee ISD
Adrian, Michigan
2012 USER CONFERENCE 2 2012 USER CONFERENCE
POP • Purpose
– Share reports created for local schools around college and career readiness
• Objective – Identify local uses for EXPLORE and PLAN
• Procedure – PowerPoint prompts
– Sample reports
– Help documents
2012 USER CONFERENCE 3 2012 USER CONFERENCE
WE MUST UTILIZE AN INQUIRY APPROACH
TO DATA ANALYSIS
WE MUST USE MULTIPLE SOURCES
OF DATA
We need a data warehouse
for our 21st century schools
WE MUST FOCUS ON DATA TO INCREASE STUDENT ACHIEVEMENT
Talking Points for the Purpose of Implementing
a Data Warehouse in Lenawee Schools
2012 USER CONFERENCE 4 2012 USER CONFERENCE
Our Process
LISD share reports with Local District Staff through DataDirector
Reports are generated by Instructional Data Services Coordinator
Data is uploaded by Data Entry Specialist
Schools give CDs and passwords to LISD
PLAN and EXPLORE CDs and passwords received by school
2012 USER CONFERENCE 5 5
College Readiness Benchmark Scores
Early Indicators of College Readiness
EXPLORE PLAN ACT
English English Composition 13 15 18
Reading Social Sciences 15 17 21
Math Algebra 17 19 22
Science Biology 20 21 24
ACT Subject Area Test College Course(s)
2012 USER CONFERENCE 6
2012 USER CONFERENCE 7
“On Target”
met or exceeded CRB
“Nearly On Target”
<2 points from CRB
“Off Target”
>2 points from CRB
2012 USER CONFERENCE 8 2012 USER CONFERENCE
Predictions Using EXPLORE to PLAN to ACT
Source: http://www.act.org/research/policymakers/pdf/ReasonableGrowth.pdf
Average Growth Points Between Tests
“On Target”
(met or exceeded CRB)
“Nearly On Target”
(<2 points from CRB)
“Off Target”
(>2 points from CRB)
Test EXPLORE to PLAN
PLAN to ACT
EXPLORE to PLAN
PLAN to ACT
EXPLORE to PLAN
PLAN to ACT
English 2-3 2-3 3-4 1 3-4 1
Math 2-3 2-3 1-2 1-2 2-3 1-2
Reading 1-2 4-5 3-4 2-3 3-4 1-2
Science 1-2 2-3 1-2 2-3 2-3 1
10-11 PLAN Expected ACT 10-11 PLAN Expected ACT 10-11 PLAN Expected ACT 10-11 PLAN Expected ACT
Last name First name English English Reading Reading Mathematics Mathematics Science Science
16 16-20 17 17-21 14 14-18 16 16-20
17 17-21 17 17-21 20 21-25 21 22-26
22 23-27 20 21-25 21 22-26 21 22-26
16 16-20 14 14-18 16 16-20 20 21-25
10-11 Expected 10-11 Expected 10-11 Expected 10-11 Expected
EXPLORE PLAN EXPLORE PLAN EXPLORE PLAN EXPLORE PLAN
Last name First name English English Reading Reading Mathematics Mathematics Science Science
9 10-13 11 12-15 11 12-15 11 12-15
14 15-18 12 13-16 14 15-18 15 16-19
14 15-18 14 15-18 17 18-21 16 17-20
13 14-17 12 13-16 15 16-19 16 17-20
13 14-17 10 11-14 11 12-15 13 14-17
2012 USER CONFERENCE 10 2012 USER CONFERENCE
Analysis of EXPLORE to PLAN
• English and Reading – Stronger relationship
• if low in English, then low in Reading
– language usage vs. reading comprehension?
• Math achievement is low – 8th grade math vs. Algebra I?
• Low achievement in Science – 8th grade Science course?
2012 USER CONFERENCE 11 2012 USER CONFERENCE
Analysis of PLAN to ACT
• Stronger relationship in Reading in English – curriculum vs. test?
– lower CRB scores?
• Strongest relationship in Math – instruction in Algebra II?
• Weaker relationship in Science – process vs. content?
• Very few perform well on PLAN and miss CRB on ACT
2012 USER CONFERENCE 12
Identify students who need assistance with the testing formats
• Needs identified by students on PLAN test
– Writing – Reading – Math – Study Skills
• Identify students who need assistance with the testing formats
– Writings using ACT rubric – Analyzing data in graphs, charts,
and tables – Use of released items from MDE – Use of release practice items
from ACT – Strategies for completing timed
portions of ACT – Close and critical reading
strategies from MS/HS Literacy Team
2012 USER CONFERENCE 13 2012 USER CONFERENCE
PLAN Student Profile Report
• Test Scale Scores – color coded
• Self-Reported GPA on 4.0 scale
• Self-Reported Post-Secondary plans
• Self-Reported High School Coursework
• Self-Reported Educational Attainment:
– Mother and Father
2012 USER CONFERENCE 14 2012 USER CONFERENCE
EXPLORE Student Profile Report
• Test Scale Scores – color coded
• Self-Reported grades on coursework aligned with test scores
• Self-Reported Post-Secondary plans
Last First PLAN
English PLAN
Reading PLAN Math
PLAN Science GPA Plans HSCourses Ed Level - Mom Ed Level - Dad
14 22 17 17 1.6 OTJT with military
College prep Some college >4-yr degree
21 24 23 23 3.4 4-yr coll./uni. College
prep Some college CTE
9 11 16 15 1.5 Undecided CTE/Other I don't know I don't know
10 13 15 16 2.8 Nothing after
HS CTE/Other 2-yr degree HS/equivalent
19 17 18 21 1.9 >4-yr degree College
prep Some college HS/equivalent
21 21 18 21 3.9 >4-yr degree College
prep 2-yr degree HS/equivalent
20 18 18 21 2.3 CTE school CTE/Other 4-yr degree 2-yr degree
2012 USER CONFERENCE 16 2012 USER CONFERENCE
Questions to Consider
• What is the relationship between CCR test scores and grades?
• What is the relationship between CCR test scores and post-secondary plans?
• What is the relationship between post-secondary plans and parents’ educational attainment?
2012 USER CONFERENCE 17 2012 USER CONFERENCE
Answer Sheet Assessment
• Thanks to Cathy Krakowiak at Howell Schools!
– Use score as number grid item
– Create a separate assessment for each score
– Development of performance bands with benchmark score as highest score
– Creates prebuilt pie graphs
2012 USER CONFERENCE 18 2012 USER CONFERENCE
EXPLORE, PLAN, ACT
English
2012 USER CONFERENCE 19 2012 USER CONFERENCE
PLAN and EXPLORE Item Analysis
• Use test booklets from 11-12 testing
– Order extra materials for your staff (no cost)
– Review items from the booklet and the student responses
2012 USER CONFERENCE 20 2012 USER CONFERENCE
Item Analysis of EXPLORE and PLAN • Robert Hanvey at SCCRESA
– developed program that pulls raw data from ACT – File format allows DataDirector to “read”
the data like an Answer Sheet Assessment
• Some formatting needed – Create report from DD with UIC, birthdate, FN, LN – Match test form
• EXPLORE 04 A/B • PLAN 31 A/B
• Answer sheet organized by sections aligned to test’s content areas and subscores
2012 USER CONFERENCE 21 2012 USER CONFERENCE
Fixing Bad Bubbling
2012 USER CONFERENCE 22
2012 USER CONFERENCE 23 2012 USER CONFERENCE
Making It Formative
• Get the scores into the hands of the students
• Develop multiple worksheet Excel file
• Data feeds into form with student names
• Teacher have students identify “simple mistakes” and “need help” with wrong answers
2012 USER CONFERENCE 24 2012 USER CONFERENCE
Sample Report
2012 USER CONFERENCE 25
Practice ACT Test
• Reports include:
– Scale Scores
– Sub Area Scores
– Composite Score
– Customized Report
• Form 64E from ACT website
• Test questions are in a document housed within the Shared Assessment
• Send us an electronic file of student responses
2012 USER CONFERENCE 26 2012 USER CONFERENCE
College Readiness Benchmark Standards • Compare Standards in each subject area
– below, at, and beyond benchmark
• Review written and taught curriculum
– Which unit in the course/grade level?
– How was it assessed in the classroom?
– What were the students’ scores on the classroom assessment?
• Explore lesson plans and activities
2012 USER CONFERENCE 27 2012 USER CONFERENCE
College Readiness Standards and the
Michigan Mathematics Content Expectations
Bill Aten Mathematics Consultant
Charlevoix-Emmet ISD and Great Lakes M/S Center
Charlevoix, MI
2012 USER CONFERENCE 28 2012 USER CONFERENCE
College
and Career
Readiness
Source: College and
Career Readiness
Workshops, Fall 2011
CRB=22
2012 USER CONFERENCE 29 2012 USER CONFERENCE
Future Work
2012 USER CONFERENCE 30 2012 USER CONFERENCE
ACT Moodle Course
Using College and Career Readiness Standards as a Tool for School Improvement
2012 USER CONFERENCE 31 2012 USER CONFERENCE
Data Mining
2012 USER CONFERENCE 32
Existing Summative and Formative Classroom Tests
Not Aligned with Expectations
Classroom Summative and Formative Tests Aligned to Expectations
Common Classroom Summative and Formative Assessments Aligned to Expectations
Common Formative and Summative Assessments Aligned to Expectations and Delivered Online Through DataDirector
Implementing Assessments with DataDirector
Adapted from St. Clair RESA
2012 USER CONFERENCE 33
What are some other data sets to consider for “College and Career Readiness”?
• Standards Met
• Credits Earned
• Courses Taken
• Grades Earned
• EDP Completion
• Resume Completion
• Interview Completion
• Application Completion
2012 USER CONFERENCE 34 2012 USER CONFERENCE
Assessment Calendars
2012 USER CONFERENCE 35 2012 USER CONFERENCE
Data Driven Dialogue
Predictions
Observations
Inferences Adapted from Deb Clancy, Washtenaw ISD, 2008, based upon the work of Nancy Love, “Using Data/Getting Results” (2002)