preparing to implement common core state standards june 6, 2011
TRANSCRIPT
3 ©2011 U.S. Education Delivery Institute and Achieve
The public sector in general – and education in particular – face increasing pressure for results
Productivity imperative for the education
sector
Pressure for enhanced learningoutcomes
Pressure to prepare students to meet workforce needs
Recession and budget cuts: pressure to utilize
public funds wisely
4 ©2011 U.S. Education Delivery Institute and Achieve
The Prime Minster’s Delivery Unit (PMDU) was founded in 2001 to help the British government take on similar challenges
Key activities of the PMDU
Monitor and report on the delivery of the Prime Minister’s top priorities
Identify key barriers that prevent improvements and actions needed to strengthen implementation
Strengthen departmental capacity to deliver through better planning and sharing knowledge about best practice
Selected targets that the PMDU oversaw
Education:▪11-year-old English proficiency▪11-year-old Math proficiency▪14-year-old English proficiency▪14-year-old Math proficiency
Health:▪Heart disease mortality▪Cancer mortality▪Max waiting time for non-emergency
surgery▪Emergency room waiting time▪Physician appointments
Crime:▪Street crime▪Burglary▪Car crime▪Offenses brought to justice
Transportation▪Road congestion▪Train punctuality
5 ©2011 U.S. Education Delivery Institute and Achieve
Within four years, the government was on track to hit over 80% of its high-priority targets
Targets on track, percent
December 2004
17
83
December 2003
47
53
July 2004
62
38
6 ©2011 U.S. Education Delivery Institute and Achieve
The US Education Delivery Institute has distilled the PMDU’s delivery approach to 15 essential elements
Develop a foundation for delivery
Understand the delivery challenge
Plan for delivery
A. Evaluate past and present performance
B. Understand drivers of performance and relevant activities
A. Determine your reform strategy
B. Set targets and establish trajectories
C. Produce delivery plans
A. Establish routines to drive and monitor performance
B. Solve problems early and rigorously
C. Sustain and continually build momentum
Drive delivery
A. Define your aspiration
B. Review the current state of delivery
C. Build the delivery unit
D. Establish a “guiding coalition”
2 3 41
Create an irreversible delivery culture
5
A. Build system capacity all the timeB. Communicate the delivery messageC. Unleash the “alchemy of relationships”
7 ©2011 U.S. Education Delivery Institute and Achieve
EDI was founded to bring this approach to state systems of K-12 and higher education
EDI focuses on supporting states in their implementation efforts to: Achieve college- and career- readiness for all students Increase postsecondary access and degree completion Close equity gaps
Higher Education Systems The California State University
System The Connecticut State University
System The Kentucky Council on
Postsecondary Education The Louisiana Board of Regents The University System of Maryland
The University of Missouri System The State University of New York The Pennsylvania State System of
Higher Education The Tennessee Board of Regents The University of Wisconsin System
K12 Systems Delaware Kentucky Louisiana Massachusetts Tennessee
System Partners
8 ©2011 U.S. Education Delivery Institute and Achieve
EDI has worked with Achieve to adapt the approach for Common Core implementation
Organize To Implement
AspirationInternal leadership team Timeline Budget Gap analysis
Stakeholder communications
Critical Questions
Where are we now?
What would success look like in 2014-15?
What are our strategies to achieve success?
How will the strategies be implemented through the field to the classroom?
How will we connect strategies to expected outcomes?
How will be monitor progress and stay on track?
Take
Action:
Implementation
Actions
Align instructional materials
Train educators
Transition technology and assessment system
Transition accountability and data reporting system
Align teacher preparation, evaluation, and licensing
Inform student transitions to higher education
Covered in workbook AnticipatedDesired student outcomes
9 ©2011 U.S. Education Delivery Institute and Achieve
The Common Core workbook is organized according to the essential elements of this adapted approach
Diagnostic questions to help your team gauge the extent to which you have already addressed the action(s) in question
Brief narratives that provide principles and potential options for putting the relevant action(s) in place
Case stories that illustrate the principles in the narrative
Exercises that will help flesh out your implementation strategy and put the relevant action(s) in place.
Each chapter covers one or more essential elements of the approach, and includes…
Workbook Table of Contents Completed
11 ©2011 U.S. Education Delivery Institute and Achieve
Why Do We Need an Implementation Timeline?
▪In many states, there are more moving pieces than ever before happening concurrently: Coordination is essential
▪Look across the areas of work and funding to consider how they interact
▪Faster is not better: What matters most is being thoughtful, realistic, and comprehensive
▪Implementation will depend on state/district needs and capacity
▪Detail is important: Year-by-year is not enough
▪Communicate your timeline publicly: This effort is too large and complex for “whisper down the lane”
12 ©2011 U.S. Education Delivery Institute and Achieve
What Might a Timeline Include?
▪Foundational Elements (e.g. establish leadership team, build guiding coalition, develop communications plan)
▪Align Instructional Materials
▪Train educators and administrators
▪Transition assessment system
▪Transition technology to support assessment and accountability systems
▪Align teacher preparation, evaluation and licensing programs
▪Align the transition space between K-12 and postsecondary
▪Monitor and sustain progress
13 ©2011 U.S. Education Delivery Institute and Achieve
Discussion: Our State Timelines
Objective: additions and revisions to your timeline for CCSS implementation
Instructions: ▪Using the template and sample
timelines, discuss the following:– Which type of timeline are you
following (accelerated or other)?
– In light of the sample, how might your timeline need to change?
– Which tasks will be state-led, district-led, or a hybrid?
– Will you stagger by grade or content area?
▪If you have not yet created a timeline, use this time to discuss what one would look like for your state
References: Workbook pages 3.10 – 3.16 Time: 15 minutes
Template
Sample timelines in workbook (most critical in orange)
15 ©2011 U.S. Education Delivery Institute and Achieve
The workbook contains a rubric that will help you assess your system’s capacity to implement its timeline
16 ©2011 U.S. Education Delivery Institute and Achieve
Exercise: Review your system’s capacity to implement its timeline
Objectives: ▪A self-assessment of your
system’s capacity to implement its timeline in five key areas
▪Based on your self-assessment, selection of a topic to focus on for the rest of the meeting (instructional materials or professional development)
Instructions: ▪Using the rubric in the workbook,
make your own judgment about your system’s capacity to implement its timeline
▪“Vote” on your judgments▪Discuss and agree on a judgment
for each of the five areas References: Handouts, workbook pages 2.3 – 2.8 Time: 30 minutes
Rubric
Self-assessment worksheet
Template
19 ©2011 U.S. Education Delivery Institute and Achieve
A delivery chain helps you to understand how you can reach the field
Definition
A delivery chain is the set people or organizations, and the relationships between them, through which a strategy (in our case, instructional materials or professional development) will be implemented.
A delivery chain has one question at its core: Starting from the intent of the leaders in your system and ending with the desired change in behavior on the front line (better teaching practice that improves student outcomes), how – and through whom – will your strategy be implemented?
20 ©2011 U.S. Education Delivery Institute and Achieve
There are a few core principles for drawing a delivery chain
▪For the strategy you are considering, where does the chain begin and end, and what are the levels between?
▪At each level, who are all the people or organizations that could conceivably be involved in implementing the strategy? How many of each are there? What role does each play?
▪What are the most important lines of direct influence from the beginning to the end of the chain? How will they work?
▪Are there secondary or more indirect relationships that involve others? How important are they?
Questions to ask
▪A mapping of each person or organization and its location relative to others (e.g., state level, LEA level), with an indication of how many of them there are (e.g., 150 superintendents)
▪Short summary of the role played by each person or organization
▪Lines between people/organizations that represent relationships of influence
▪Brief description of each relationship of influence
Visual elements to include
21 ©2011 U.S. Education Delivery Institute and Achieve
As applied to the Common Core, delivery chains will help you to map your strategy for influencing instructional practice
StateRegion/ County District School Classroom
Chief
Curriculum/ instruction team
Website
Regional centers
Curriculum directors
Curriculum committees
Manage
Principals Teachers
Teachers
TeachersMarket by
9/2012
Post by 9/2011
Train
and d
istr
ibute
by
9/2
01
1
Coaches Teachers
Train
and d
istr
ibute
by
9/2
01
3
Consult
Train and Distribute by 9/2013
1
1
1 65,000
65,000
65,000
65,000
1500
400150
150
15
Train
and d
istr
ibute
by
5/2
01
2
Train
and d
istr
ibute
by
9/2
01
2
Sample delivery chain: instructional materialsWhat percent of teachers will change their behavior as a result?
10% (small districts)
15% (large districts)
5% (mix)
15% (large districts)
22 ©2011 U.S. Education Delivery Institute and Achieve
Once you have drawn a delivery chain, it is important to identify weaknesses and address them
Typical challenges Potential solutions
Individual relationships
▪Weak personal relationships▪Low leverage
▪Identify and replicate stronger relationships of this type
▪Identify alternate routes to the end of the chain
Complexity ▪Too many actors necessary to get something done
▪“Rationalize” chain▪Identify alternate routes to
the end of the chain
Funding flows ▪Mismatch between resource flows and delivery chain
▪Redesign chain to take advantage of leverage from resource flows
Feedback loops
▪Few or no feedback loops ▪Create feedback loops▪Use feedback loops to exert
influence
Choke-points ▪Overreliance on a few key actors
▪Build capacity/cooperation of key actors
▪Identify alternate routes to the end of the chain
23 ©2011 U.S. Education Delivery Institute and Achieve
It is helpful to anchor these weaknesses in specific points on the delivery chain
Sample delivery chain with weaknesses identified: instructional materials
StateRegion/ County District School Classroom
Chief
Curriculum/ instruction team
Website
Regional centers
Curriculum directors
Curriculum committees
Principals Teachers
Teachers
Teachers
Coaches Teachers
1
1
1 65,000
65,000
65,000
65,000
1500
400150
150
15
What are the potential weaknesses?
Historically difficult relationship and loose authority structure – curriculum directors not likely to listen to regional centers
1
In smaller districts, principals may not have capacity to train their teachers
2
A large proportion of teachers are not accustomed to using any state website to receive curricular resources
3
1
2
3
24 ©2011 U.S. Education Delivery Institute and Achieve
Breakout session: Draw and analyze your delivery chain
Objectives: ▪A brief description of your strategy
for ensuring that the right instructional materials or PD are available
▪A map of your delivery chain for ensuring that instructional materials or PD make it to the field and change classroom behavior at scale
▪An analysis of weaknesses in the chain, and what you plan to do about them
Instructions: ▪Using the handouts as a
discussion starter, decide what your state’s ideal strategy is
▪Map the delivery chain on poster paper and record weaknesses on flipcharts
Instructional materials handout
PD handout
References: Handouts, workbook page 5.11 Time: 90 minutes Analysis
worksheet
Delivery chain template
Potential weaknesses handout
27 ©2011 U.S. Education Delivery Institute and Achieve
There are four main types of metrics that you can use to measure progress
Alignment
User satisfaction
Classroom practice
Impact on student outcomes
Description
Extent to which teachers and principals have received instructional materials aligned to the CCSS
Extent to which teachers and principals find aligned instructional materials helpful
Extent to which teachers receiving aligned instructional materials use them to change what they do in the classroom
Extent to which teachers receiving aligned instructional materials achieve better results for their students
Sample metrics (instructional materials)
▪Number of teachers and/or principals who have received aligned instructional materials
▪Number of teachers and/or principals expressing satisfaction with aligned instructional materials
▪Self-reporting of changed practice by teachers who have received aligned instructional materials (versus those who have not)
▪Formative or summative assessment data, comparing teachers who have received aligned instructional materials with those who have not
▪Observations of practice for a sample of teachers that have and have not received aligned instructional materials
28 ©2011 U.S. Education Delivery Institute and Achieve
The delivery chain will help you to decide which metrics to focus on
Sample delivery chain with metrics identified: instructional materials
StateRegion/ County District School Classroom
Chief
Curriculum/ instruction team
Website
Regional centers
Curriculum directors
Curriculum committees
Principals Teachers
Teachers
Teachers
Coaches Teachers
1
1
1 65,000
65,000
65,000
65,000
1500
400150
150
15
What are the metrics we will use?
Number of districts undergoing training
1
Number of teachers undergoing training
2
Number of teachers who have accessed web portal
3
Number of teachers who 1) are using the new materials and 2) are satisfied with them
4
Number of teachers whose classroom behavior is changing, as reported by principals and educator leaders
5
Difference in gain on formative assessments for teachers using materials vs. not
6
1
2
2
2
3
4
4
4
4
5
5
29 ©2011 U.S. Education Delivery Institute and Achieve
Once you have prioritized your metrics, you need to ensure that collection mechanisms are in place for each
Metrics from delivery chain (instructional materials) Potential ways to measure them
Number of districts undergoing training
1 ▪Fold into district monthly reporting
Number of teachers undergoing training
2 ▪Fold into district monthly reporting
▪Attendance reports from professional associations and universitiesNumber of teachers who have
accessed web portal3 ▪Website analytics engine
Number of teachers who 1) are using the new materials and 2) are satisfied with them
4 ▪Add relevant questions to existing school climate survey
Number of teachers whose classroom behavior is changing, as reported by principals and educator leaders
5 ▪Extrapolate from sample focus groups of principals, as well as existing principal advisory group
Difference in gain on formative assessments for teachers using materials vs. not
6 ▪Formative assessment data combined with survey self-reporting of adoption in classrooms
30 ©2011 U.S. Education Delivery Institute and Achieve
Tennessee’s school climate survey is a rich source of data that can be tapped for rapid feedback
Tennessee Teaching, Empowering, Leading, & Learning (TELL) Survey
▪Established in 2011 as part of the state’s Race to the Top plan
▪First statewide survey of principals and educators at this scale
▪Contains a variety of questions on topics including:– Collaborative instructional
planning– School and teacher leadership– Facilities and resources– Professional development
▪In first administration, over 77% of the state’s educators participated, giving 1,605 out of the 1,745 schools access to anonymous data on their educators’ responses
31 ©2011 U.S. Education Delivery Institute and Achieve
Breakout session: Develop your metrics for feedback
Objectives: ▪Identification of priority metrics
that will help you collect feedback on whether your strategy is working
▪Identification of ways to gather data on these metrics that are feasible in your state
Instructions: ▪Using the handouts and your
existing delivery chain as a starting point, identify a balanced set of metrics (across the four types) that cover the “pain points” in the chain
▪Brainstorm a list of ways to gather data on these metrics that will be feasible References: Handouts, workbook page 6.12
Time: 65 minutes
Template
Potential metrics handouts
Worksheet
33 ©2011 U.S. Education Delivery Institute and Achieve
Routines are crucial to ensuring that you keep your effort on track
▪ Regularly scheduled checkpoints to assess whether your implementation effort is on track
▪ Engine that drives implementation forward: Without routines, implementation will stall or be replaced by more urgent concerns on the agenda
▪ A source of structure and discipline to create order in complex education systems
What are routines?
▪ Monitor performance: Understand if system is on track to deliver on its aspirations
▪ Diagnose problems: Surface issues that are inhibiting progress and analyze data from delivery chain to pinpoint causes
▪ Address problems: Provide a venue to discuss and decide how to overcome challenges
What purpose do routines serve?
34 ©2011 U.S. Education Delivery Institute and Achieve
In the Prime Minister’s Delivery Unit, three routines kept things on track
Definition Purpose
Prime Minister Notes (monthly)
▪ Progress briefing for the prime minister
▪ Consists of a brief summary, followed by a short report
▪ Update the prime minister on progress against targets, key actions required, and warning signs of risks
▪ Identify areas where prime minister needs to make decisions or recommendations
Stocktakes (quarterly)
▪ Regular meeting of prime minister, leaders from relevant departments, and key officials
▪ Evaluate delivery of specific set of activities ▪ Update the prime minister on progress▪ Enable prime minister to hold individuals
accountable ▪ Provide focus, clarity and a sense of urgency ▪ Make decisions on key actions or new policy
needed ▪ Remove barriers to cross-departmental work
Delivery reports (semi-annual)
▪ Comprehensive assessment of the status of all of the system’s key priority areas
▪ From delivery leader to prime minister
▪ Update prime minister on comparative progress against all priorities
▪ Outline what success looks like for priorities over the next 6 months
▪ Identify key actions that need to be taken▪ Act as a reference document against which to
chart progress
35 ©2011 U.S. Education Delivery Institute and Achieve
All routines focused on the recent data and the likelihood that a target would be delivered
PMDU Assessment Framework
Red
Amber/Red
Amber/Green
Green
Highly problematic – requires urgent and decisive actionProblematic – requires substantial attention, some aspects need urgent attention
Mixed – aspect(s) require substantial attention, some goodGood – requires refinement and systematic implementation
Recent performanc
e against
trajectory and
milestones
Likelihood of
delivery
Degree of challenge
Quality of planning, implementation and performance management
Capacity to drive progress
Stage of delivery
L/M/H/VH
1/2/ 3/4
Key
Judgement Rating Rationale Summary
36 ©2011 U.S. Education Delivery Institute and Achieve
Progress on each target was compared in a league table
Red
Amber/Red
Amber/Green
Green
Key
Highly problematic; requires urgent action
Problematic; requires action
Mixed; some good, some action required
Good; requires only refinement
Overall judgment
Quality of planning, implementation and performance management
Capacity to drive progress
Degree of challenge
Stage of delivery
Likelihood of delivery
July 2004
A PSA 1 L G G 3 G
GB PSA 2 L G AG 2
C PSA 3 H AG AG 3 G
D PSA 4 H G AG 3 AG
A PSA 5 VH
G AG 2 AG
B PSA 6 H AG AG 3 AG
C PSA 7 H AG AG 2 AG
D PSA 8 H AG AG 3 AG
A PSA 9 H AG AG 2 AG
B PSA 10 VH
AG AG 2 AG
C PSA 11 VH
AG AG 2 AG
D PSA 12 H AR AG 3 AG
A PSA 13 VH
AR AG 2 AR
B PSA 14 VH
AG AR 2 AR
C PSA 15 VH
AG AR 2 AR
D PSA 16 VH
AR AR 2 AR
A PSA 17 VH
AR AR 2 AR
B PSA 18 H AG AR 3 R
C PSA 19 H AG AR 2 R
D PSA 20 VH
AG AR 3 R
A PSA 21
VH
R R 2 R
= 1
= 1
3
4
5
6
= 7
= 7
= 7
= 10
= 10
12
13
= 14
= 14
= 16
= 16
= 18
= 18
20
21
DeptRank (out of 21)
Assessment criteria
Degree of challenge:
L = low
H = high
M = medium
VH = very high
37 ©2011 U.S. Education Delivery Institute and Achieve
One PARCC state uses a combination of quantitative and qualitative information to make interim assessments of progress
Example: Interim data for one strategy in 3rd Grade Reading Goal
Leading indicators for a strategy
Qualitative judgments of the likelihood that each of the strategy’s projects will deliver the promised impact on the goal
+
A quarterly data set that can serve as the evidence base for a performance conversation about the strategy
# STRATEGY LEADING INDICATOR Q12010-11
2
Ensure Prek-3 teachers statewide receive consistent professional development related to curriculum standards.
Teacher retention of teachers <5 years service, grades PK-3 87.8%
38 ©2011 U.S. Education Delivery Institute and Achieve
Massachusetts uses a modified version of this approach in its bimonthly notes on delivery priorities
Example: note for college and career readiness goal
Immediate actions for the commissioner
Likelihood of delivery for each core strategy in current and prior periods, based on most recent data and qualitative assessmentAdditional detail on the evidence underlying the likelihood of delivery for each core strategy
40 ©2011 U.S. Education Delivery Institute and Achieve
Exercise: Analyze and develop your own routines for monitoring progress
Objectives: ▪Identification of existing routines
in your agency that are or could potentially be used to monitor progress for your instructional materials or professional development strategy
▪Decisions on specific ways in which you can improve existing routines or add new ones if necessary
Instructions: ▪Identify and analyze your existing
routines using the template▪Evaluate how well they fit
together to give an overall picture of performance, and decide whether to add more References: Handouts, workbook page 11.7
Time: 25 minutes
Template
Analysis worksheet
42 ©2011 U.S. Education Delivery Institute and Achieve
We have covered a section of the Common Core workbook in detail during this gathering
Organize To Implement
AspirationInternal leadership team Timeline Budget Gap analysis
Stakeholder communications
Critical Questions
Where are we now?
What would success look like in 2014-15?
What are our strategies to achieve success?
How will the strategies be implemented through the field to the classroom?
How will we connect strategies to expected outcomes?
How will be monitor progress and stay on track?
Take
Action:
Implementation
Actions
Align instructional materials
Train educators
Transition technology and assessment system
Transition accountability and data reporting system
Align teacher preparation, evaluation, and licensing
Inform student transitions to higher education
Covered at this instituteDesired student outcomes
43 ©2011 U.S. Education Delivery Institute and Achieve
Now we will revisit our capacity review and think through next steps
Organize to implement
Align instructional materials
Train educators
ElementRating (1-4)
What we have accomplished Next steps
▪ Internal leadership team
X ▪ ▪
▪ Strategies to achieve success
X ▪ ▪
▪ Understanding how the strategies will be implemented through the field to the classroom
X ▪ ▪
▪ Strategies to achieve success
X ▪ ▪
▪ Understanding how the strategies will be implemented through the field to the classroom
X ▪ ▪
44 ©2011 U.S. Education Delivery Institute and Achieve
Exercise: Revisiting the capacity review and capturing next steps
Objectives: ▪Review of progress made and immediate next steps for
the areas we have covered▪Identified broader next steps for CCSS implementation
Instructions: ▪Using the capacity review ratings for the areas we
have covered, reflect on progress in the last two days and immediate next steps in these areas
▪Discuss the following questions:– Does our timeline for CCSS implementation still feel
right? – How well is the state working with and leveraging
the work of its leading districts?– What lessons can the state draw from leading
systems?– What are the first things we need to focus on when
we return home?– What additional or outside support do we need? Material: N/A
Time: 45 minutes
Template