© 2010 american institutes for research ® january 6 & 13, 2011 presenters susan bowles...
Post on 22-Dec-2015
215 views
TRANSCRIPT
©2010 American Institutes for Research®
January 6 & 13, 2011
Presenters
Susan Bowles Therriault Lindsay Fryer
National High School Center
Introduction to theVirginia Early Warning System (VEWS)
www.betterhighschools.org
©2010 American Institutes for Research®
Who We Are
USED-funded Content Centers including the
National High School Center
State Education Agencies (SEAs)
Local Education Agencies (LEAs)
IES
Regional
Labs
16 Regional Comprehensive Centers (RCCs)
USED-funded Special Ed Technical Assistance and
Dissemination Centers
6 Regional Resource Centers (RRCs)
www.betterhighschools.org2
©2010 American Institutes for Research® 3
Objectives
• Provide an overview of the background and research base for early warning systems
• Discuss the evolution of the Virginia early warning system (VEWS) tool
• Introduce the 7-step Virginia early warning system implementation process
• Demonstrate the VEWS tool
• Introduce the Eight Elements of High School Improvement
• Discuss how the VEWS data and implementation process can support Eight Elements of High School Improvement
www.betterhighschools.org
©2010 American Institutes for Research®
Sobering Statistics…
• Nearly one-third of all high school students leave the public school system before graduating
• 1.2 million students drop out of high school each year – that’s 12 million over the next decade
• 7,000 students drop out of high school every day
• 15% of the high schools in the U.S. produce 50% of our dropouts – schools Balfanz and Legters call “dropout factories”
Balfanz, R. and Legters, N. (2006, July 12). The graduation rate crisis we know and what can be done about it. Retrieved online from http://web.jhu.edu/CSOS/graduationgap/edweek/Crisis_Commentary.pdf
www.betterhighschools.org4
©2010 American Institutes for Research®
Economic Consequences
• A new high school dropout in 2000 had less than a 50% chance of getting a job
• That job earned less than half of what the same job earned 20 years ago
• Lack of education is strongly correlated with welfare dependency and incarceration
• Cutting the number of dropouts in half would reap $45 billion in revenues and decreased costs (Levin et al., 2007)
www.betterhighschools.org5
©2010 American Institutes for Research®
Virginia Data
Diploma Graduates83.2%
GED Earners
3.7%
Cer-tifi-
cates of
Comple-tion
0.3%
Still En-
rolled2.7%
Dropouts8.1%
Long Term Absences0.4%
Unconfirmed Status1.6%
Diploma Graduates
GED Earners
Certificates of Completion
Still Enrolled
Dropouts
Long Term Absences
Unconfirmed Status
Data taken from the Virginia Department of Education’s State-Level Cohort Report, Four Year Rate for the Class of 2009 See:http://www.doe.virginia.gov/statistics_reports/graduation_completion/cohort_reports/index.shtml
www.betterhighschools.org6
©2010 American Institutes for Research®
Background Information
In February, 2009, the Board of Education strengthened Virginia’s accountability program by:
• Requiring high schools—beginning with the 2011-2012 school year—to meet an annual benchmark for graduation
• Revising Virginia’s accreditation standards to create a graduation and completion index for high schools
7 www.betterhighschools.org
©2010 American Institutes for Research® 8
Early Warning Systems
Early warning systems (EWS) rely on readily available data housed at the school to:
– Predict which students are at-risk for dropping out of high school
– Target resources to support off-track students while they are still in school, before they drop out
– Examine patterns and identify school climate issues
www.betterhighschools.org
©2010 American Institutes for Research® 9
9th Grade is a Critical Year
• Ninth grade is a “make or break year”
– More students fail 9th grade than any other high school grade
– A disproportionate number of students who are held back in 9th grade subsequently drop out
• Monitoring students’ progress throughout 9th grade—and even during the first semester—provides powerful indicators that can predict whether students will complete high school:
– Engagement
– Course performance
– Chicago’s “On-Track” Indicator (CCSR End-of-Year)
www.betterhighschools.org
©2010 American Institutes for Research® 10
Key Indicators
Engagement
• Attendance/absenteeism
Course Performance
• Course grades
• Number of credits earned
CCSR End-of-Year Indicator
• Core course performance & accumulated credits
www.betterhighschools.org
Research from several U.S. school districts provides a strong foundation for defining 9th grade warning signs that students might drop out, but local adaptation is key.
©2010 American Institutes for Research® 11
“High-Yield” Academic Indicators: Attendance
www.betterhighschools.org
0-4 5-9 10-14 15-19 20-24 25-29 30-34 35-39 40+0%
20%
40%
60%
80%
100%87%
63%
41%
21%
9%5% 2% 1% 0%
Four-Year Graduation Rates for CPS Students Entering High School in 2001, by 9th Grade Absences (Allensworth & Easton, 2007)
Days Absent Per Semester (Course Cutting Counted as Partial Days)
Perc
ent G
radu
ated
in F
our Y
ears
©2010 American Institutes for Research® 12
“High-Yield” Academic Indicators: Course Failures
www.betterhighschools.org
0 1 2 3 4 5 6 7 8 More than 8
0%
20%
40%
60%
80%
100%
85%
70%
55%
42%33%
25%
13% 11%7%
2%
Four-Year Graduation Rates for CPS Students Entering High School in 2001, by Freshman Course Failures (Allensworth & Easton, 2007)
Semester Course Failures
Perc
ent G
radu
ated
in F
our Y
ears
©2010 American Institutes for Research® 13
“High-Yield” Academic Indicators: GPA
www.betterhighschools.org
0.0 0.5 1.0 1.5 2.0 2.5 3.0 3.5+0%
20%
40%
60%
80%
100%
1%6%
28%
53%
72%
86%93%
97%
Four-Year Graduation Rates for CPS Students Entering High School in 2001, by Freshman GPA (Allensworth & Easton, 2007)
Freshman GPA
Perc
ent G
radu
ated
in F
our Y
ears
©2010 American Institutes for Research®
Chicago’s “On-track” Indicator
Students are “on-track” if they:
1. have not failed more than one semester long core course, AND
2. have accumulated enough credits for promotion to the 10th grade.
www.betterhighschools.org
Number of Semesters with Fs in Core
Courses
# of Credits Accumulated Freshman Year
Less than 5 5 or more
2 or more courses Off-track Off-track
0 or 1 courses Off-track On-track
©2010 American Institutes for Research®
CCSR End of Year Indicator
www.betterhighschools.org
Source: Allensworth & Easton (2005)
©2010 American Institutes for Research® 16
“High Yield” 9th Grade Indicators
Indicators Benchmark (flagged)
Absenteeism Missing 10% or more of instructional time
Course failures One or more failed courses
Grade point average
2.0 or lower (on a 4-point scale)
CCSREnd-of-Year Indicator
Fail two or more semester core courses, or accumulate fewer credits than the number required for promotion to the 10th grade
www.betterhighschools.org
©2010 American Institutes for Research® www.betterhighschools.org
Evolution of the VEWS Tool
17
©2010 American Institutes for Research®
VEWS Pilot Team
A team of school divisions, education organizations, and VDOE consultants assisted the VDOE with the pilot of the VEWS Tool
– School divisions: Franklin City, Lee County, Richmond City, and Pulaski County
– Organizations: Appalachia Regional Comprehensive Center and the National High School Center
– Validation of the Virginia Pre-High School Index
– Development of an implementation guide based on the pilot divisions’ experience
– Refinement of the VEWS Tool based on pilot divisions’ feedback and VDOE suggestions
18 www.betterhighschools.org
©2010 American Institutes for Research® 19
VEWS Tool Features
Based on this work over the past two years, the National
High School Center in collaboration with ARCC and VDOE
conceptualized the current VEWS Tool. This version allows
users to:
– Import data
– Accommodate local contextual factors for high schools
(e.g., semesters vs. trimesters vs. quarters)
– Align with the Virginia pre-high school index
– Modify the benchmarks/thresholds based on analysis of
longitudinal data (i.e., indicator validation)
– Integrate behavioral indicators (referrals and suspensions)
www.betterhighschools.org
©2010 American Institutes for Research® 20
VEWS Tool Features
Features, continued:
– House an inventory of dropout prevention interventions
– Assign students to intervention programs and monitor student response and progress in the interventions over the course of the school year
– Create pre-set school-level summary reports, detailed student-level reports, and individual student reports
– Create customized student-level reports
www.betterhighschools.org
©2010 American Institutes for Research®
Risk Indicators in the VEWS Tool
21 www.betterhighschools.org
Indicator Time Frame Benchmark (red flag)
Pre-High School Before school begins Virginia DOE validated pre-high school index
Attendance First 20 or 30 days, per grading period, end of year (annual)
Missed 10% or more of instructional time
Course failures Per grading period, end of year (annual)
Failed one or more semester courses (any subject)
Grade point average Per grading period, end of year (annual)
Earned 2.0 or lower (on a 4-point scale)
CCSR “on track” indicator End of year (annual) Failed two or more semester core courses, or accumulated fewer credits than the number required for promotion to the next grade
Referrals and suspensions Per grading period, end of year (annual)
Locally validated
©2010 American Institutes for Research® 22 www.betterhighschools.org
7-Step VEWS Implementation Process
STEP 1Establish roles
and responsibilities
STEP 2Use the EWS Tool
STEP 3Review the EWS
data
STEP 4Interpret the EWS
data
STEP 5Assign and
provide interventions
STEP 6Monitor students
STEP 7Evaluate and
refine the EWS process
©2010 American Institutes for Research®
Step One: Establish Roles and Responsibilities
• VEWS teams need to include individuals who have:
– Authority to make decisions
– Knowledge of diverse students
– Expertise to manage and analyze data
• VEWS team are required to:
– Meet regularly
– Communicate EWS/dropout prevention issues to
groups/individuals outside of the team
– Solicit feedback from stakeholders (leaders, staff,
students, parents)
– Monitor students’ progress23 www.betterhighschools.org
©2010 American Institutes for Research®
Step Two:Use the VEWS Tool
• In order to be used as an effective tool to support VEWS team work:
– Data must be regularly entered/imported throughout the school year
– At least one individual should be responsible for ensuring the VEWS tool is loaded with the latest data
– VEWS Team members must be trained to understand the use of the tool
– Reports must be used to make decisions about students
– Students must be assigned to interventions and progress monitored
www.betterhighschools.org
©2010 American Institutes for Research®
Step Three: Review VEWS Data
• VEWS data are reviewed and monitored to identify students at risk for dropping out and to understand patterns in student engagement and academic performance
• Questions to ask about VEWS data:
– Student-level patterns: What do your data tell you about individual students who are at-risk?
– School-level patterns: What do your data tell you about how the school is doing?
• Are students who were flagged from the beginning remaining “off-track” through the year?
• Are students who were flagged at one reporting period back “on-track” at the next?
www.betterhighschools.org
©2010 American Institutes for Research®
Step Four: Interpret VEWS Data
• The VEWS team must look BEYOND the indicators and dig deeper into reasons for student disengagement with school and academic failure
– Indicators are just observable symptoms, not root causes
– It is important to examine additional data from a variety of sources not included in the tool (e.g., talking to classroom teachers, parents, individual students, other adults in the school)
• Looking at data beyond those in the VEWS Tool can help identify individual and common needs among groups of students and raise new questions for understanding students’ reasons for being off-track for graduation
www.betterhighschools.org
©2010 American Institutes for Research®
Step Four: Interpret VEWS Data (cont.)
• Understanding characteristics of students at-risk of dropout is important because:
– Decisions to persist or drop out are affected by multiple contextual factors - family, school, neighborhood, peers
– Personal and school factors contribute to success or failure during the freshman year
– Attendance and course performance problems are distinct indicators in the VEWS but are highly interrelated, and both can signal disengagement
– Student background characteristics are less important in explaining failures than behaviors in high school
www.betterhighschools.org
©2010 American Institutes for Research®
Step Five:Assign and Provide Interventions
• The VEWS team matches individual students to specific interventions after having gathered information about:
– Potential root causes for individual flagged students
– The available dropout prevention and academic and behavioral support programs in the school, district, and community
• A tiered approach can be used to match students to interventions based on their individual needs
28 www.betterhighschools.org
©2010 American Institutes for Research®
Tiered Approach toDropout Prevention
Tier Three Individualized
Tier Two
Targeted
Tier One
Universal
www.betterhighschools.org29
©2010 American Institutes for Research®
New Hampshire's Model
www.betterhighschools.org
Examples: Positive Behavioral Supports (PBS) model and Universal Leadership Team
including a diverse representation of “opinion-leaders”
Examples: Intervention Team of specialists and administrators will
focus on students who exhibit challenging behaviors and who are
at risk
Examples: RENEW (Rehabilitation, Empowerment, Natural supports, Education and Work) facilitators
Individualized
Targeted
School-wide
30
©2010 American Institutes for Research®
Minnesota’s Model
www.betterhighschools.org
Examples: Extracurricular activities, school-to-work programs,
or positive discipline programs
Examples: Interventions build specific skills, school-within-a-
school model, or mentoring programs (15% of students)
Examples: Individualized behavior plans, wrap-around services, alternative programs (5% of
students)Targeted
Selected
Universal
31
©2010 American Institutes for Research®
Example Programs that incorporate this strategy:
America’s Choice
First Things First
School Development Program
Talent Development High School
Example Programs that incorporate this strategy:
America’s Choice
Talent Development High School
Step Five (cont.):Assign and Provide Interventions
• Focus on achievement in core courses • Content recovery courses• Tutoring as an academic support • Tiered approaches• Attendance and behavior monitors• Advisories and team teaching• Counseling and mentoring• Small learning communities and school within a
school for greater personalization• Partnerships between high schools and feeder
middle schools• Ninth grade transition programs• Support for students with disabilities outside
of school• Career and college awareness• Family engagement• Community engagement
Example Programs that incorporate this strategy:
America’s Choice
Check and Connect
Coca-Cola VYP
Interpersonal Relations Personal Growth Class
NGP
Quantum Opportunities Program
School Development Program
Talent Development High School
Twelve Together
Example Programs that incorporate this strategy:
Academic Literacy Program
Positive Behavioral Interventions and Supports (PBIS)
Rehabilitation, Empowerment, National supports, Education, and Work (RENEW)
RTI
Strategic Instruction Model
Example Programs that incorporate this strategy:
ALAS
Check and Connect
Coca-Cola VYP
Interpersonal Relations Personal Growth Class
PBIS
Project COFFEE
Talent Development High School
Teen Outreach Program
Example Programs that incorporate this strategy:
Middle College High School
NGP
STEP
Example Programs that incorporate this strategy:
ALAS
Check and Connect
First Things First
Interpersonal Relations Personal Growth Class
Project COFFEE
Twelve Together
Example Programs that incorporate this strategy:
Career Academies
First Things First
Middle College High School
NGP
Project COFFEE
Talent Development High School
Example Programs that incorporate this strategy:
Project GRAD
Example Programs that incorporate this strategy:
Career Academies
Ninth Grade Success Academies
STEP
Example Programs that incorporate this strategy:
Check and Connect
PBIS
Example Programs that incorporate this strategy:
Career Academies
Learning to Work Program (NYC)
Lifelong Options Program (LOP)
Middle College High School
Project COFFEE
RENEW
Twelve Together
Example Programs that incorporate this strategy:
ALAS
First Things First
Support Center for Adolescent Mothers (Family Growth Center)
NGP
Talent Development High School
Example programs using these types of strategies:
• ALAS• America’s Choice• Career Academies• Check and Connect• Support Center for Adolescent
Mothers (Family Growth Center)
• Middle College High School• NGP• Quantum Opportunities
Program• Teen Outreach Program
32
©2010 American Institutes for Research®
Step Six: Monitor Students and Interventions
• The VEWS team monitors students who are participating in interventions to:
– Make necessary midcourse corrections by identifying students’ whose needs are not being met
– Identify new interventions that will to meet students’ needs
– Use data to monitor the effectiveness of interventions offered
• Increase knowledge about the general effectiveness of interventions
• Improve the matching of students to interventions
– Communicate with families and appropriate stakeholders and solicit their involvement in the monitoring process
33 www.betterhighschools.org
©2010 American Institutes for Research®
Step Seven: Evaluate and Refine the VEWS Process
• Refine the VEWS Implementation Process
– During the school year
– At the end of a school year
• Identify short- and long-term needs and solutions
– Student needs
– Organizational needs (school and/or division)
www.betterhighschools.org34
©2010 American Institutes for Research®
Step Seven: Evaluate and Refine the VEWS Process
• Validate the indicators to maximize predictive power of the system
www.betterhighschools.org
Displayed Early Warning Sign in 9th Grade?
Graduated in 4 (or 5) Years?
YES NO
YESFalse Positive(or EffectiveIntervention)
AccuratePrediction
NOAccurate
PredictionFalse Negative
35
©2010 American Institutes for Research®
VEWS Tool Demonstration
www.betterhighschools.org
©2010 American Institutes for Research® www.betterhighschools.org
Eight Elements ofHigh School Improvement
• The Eight Elements of High School Improvement comprise a framework for mapping comprehensive and systemic school, district, and state high school improvement efforts
• The Eight Elements are:
– Rigorous Curriculum and Instruction
– Assessment and Accountability
– Teacher Effectiveness and Professional Growth
– Student and Family Involvement
– Stakeholder Engagement
– Effective Leadership
– Organization and Structure
– Sustainability
©2010 American Institutes for Research® 38 www.betterhighschools.org
The Eight Elements
• Each of the Eight Elements is broken into subcategories to improve users’ ability to address high school improvement issues
• For example, each element is organized in the following manner:
Element• Areas of Focus
• Indicators of Effectiveness
©2010 American Institutes for Research® 39 www.betterhighschools.org
Areas of Focus andIndicators of Effectiveness
©2010 American Institutes for Research® 40 www.betterhighschools.org
The Eight Elements and VEWS
• How can VEWS Tool support high school improvement efforts in terms of the Eight Elements?
– The VEWS Tool helps provides data/evidence of areas of need
• VEWS student level reports
• VEWS school level reports
– The Eight Elements provide a framework for addressing these needs
©2010 American Institutes for Research® 41 www.betterhighschools.org
EXAMPLEThe Eight Elements and VEWS
• VEWS Implementation Guide Example
– Step 7: Evaluate and refine the VEWS process
• Guiding Questions - Longer-Term Question:
• Do the VEWS data reveal systemic problems in the district or school (e.g., risk factors that are prevalent from year to year, feeder patterns, schools with persistent problems, groups of students who are consistently identified)?
• How can the VEWS Tool reports (evidence) be used to identify areas of need?
©2010 American Institutes for Research® 42 www.betterhighschools.org
Eight Elements Self-Assessment
• A Coherent Approach to High School Improvement: A District and School Self-Assessment
– Users rate themselves on implementation their schools’ implementation of each of the Eight Element indicators
– Results allow users to see where they need to focus high school improvement efforts
• Self-Assessment and VEWS
– The VEWS Tool provides supporting evidence for user ratings
– VEWS Tool reports allow users to track performance over time (after self-assessment baseline)
©2010 American Institutes for Research® 43 www.betterhighschools.org
A Coherent Approach to High School Improvement: A District and School
Self -Assessment
©2010 American Institutes for Research® 44 www.betterhighschools.org
A Coherent Approach to High School Improvement: A District and School
Self-Assessment
• Once users rate themselves on implementation of each indicator and area of focus, they can use the Next Steps section of the tool to strategically plan future high school improvement efforts based on the ratings
©2010 American Institutes for Research® 45 www.betterhighschools.org
• The Eight Elements break school improvement issues into manageable parts/steps that can be addressed
EXAMPLE Aligning Improvement Strategies to the
Eight Elements
Element 4. Student and Family Involvement
Area of focus Student and Family Supports
Indicators of effectiveness
4.5 Students are supported as they transition in and out of high school
Element: 2. Assessment and Accountability
Area of focus: Multiple Assessment Strategies
Indicators of effectiveness
2.2 Students with special needs and those at risk of failure are identified early and are matched with appropriate interventions
©2010 American Institutes for Research® 46 www.betterhighschools.org
EXAMPLE The Eight Elements and VEWS
©2010 American Institutes for Research® 47 www.betterhighschools.org
EXAMPLEThe Eight Elements and VEWS
©2010 American Institutes for Research® 48 www.betterhighschools.org
EXAMPLEPossible Improvement Strategies
• Self-assessment implementation of indicator ratings:
– 2.2 Students with special needs and those at risk of failure are identified early and are matched with appropriate interventions: little to no evidence
– 4.5 Students are supported as they transition in and out of high school: little to no evidence
• Areas of need
– Many students need transition supports
– Students with disabilities need interventions to address their needs
©2010 American Institutes for Research® 49 www.betterhighschools.org
EXAMPLEPossible Solutions and Next Steps
• Possible solutions and next steps:
– Implement a transition intervention in your school for both SWDs and all students
– Keep track of interventions and students assigned to interventions using the VEWS tool
– Monitor over time if students are getting back on track for graduation through this intervention using the VEWS tool
©2010 American Institutes for Research® 50 www.betterhighschools.org
EXAMPLEMonitoring Transition Services
©2010 American Institutes for Research® 51 www.betterhighschools.org
EXAMPLE Monitoring Interventions for SWDs
June 30, 2010
©2010 American Institutes for Research® 52 www.betterhighschools.org
Eight Elements: New Online Self-Assessment
• The new online version of the self-assessment includes several additional features:
– A cumulative rating system to assess the degree of implementation of each indicator
– Printable reports that highlight the level of implementation for each element and focus area
– Printable summary reports that provide an overall look at the degree of implementation of the Eight Elements
– The ability to save self assessment data so users can update reports as school improvement efforts are implemented
©2010 American Institutes for Research® 53 www.betterhighschools.org
For More Information about…..
• VEWS & the Eight Elements
Virginia Department of Education Websitehttp://www.doe.virginia.gov/support/school_improvement/early_warning_system/index.shtml
• Questions about VEWS and Eight Elements of School Improvement
Dr. Kathleen Smith, Director, Office of School Improvements, Virginia Department of EducationEmail: [email protected]
• National High School Center Products http://www.betterhighschools.org
A Coherent Approach to High School Improvement: A District and School Self-Assessment http://betterhighschools.org/pubs/documents/EightElementsSelfAssessmentTool.pdf
COMING SOON: Online Self-Assessment Tool