a ccountability & e valuation. overview of this session defining accountability & evaluation...
TRANSCRIPT
ACCOUNTABILITY & EVALUATION
Overview of this Session
• Defining Accountability & Evaluation
• MTSSS & Program Evaluation Issues
• Example of Evaluation in MTSSS
• Discussion– Current status– Next steps
What does…
• Accountability mean to you?
• Evaluation mean to you?
Accountability in Florida
• Increasing accountability focus the last decade
• Examples include:– School grading– AYP– Special education rules– DA– FEAPS & Teacher evaluation systems
Impact of Accountability
Criticisms• Lack of educator
involvement• Controversy• Consequence driven• Compliance driven• Conflicting
requirements• Duck and cover
approach
Positives• Establishes and
maintains standards for performance
• Reinforces use of data to monitor student outcomes
• Reinforces need to examine resource use
• Student outcome rather than process focus
• Success stories
(Hall & Hord)
Accountability & Evaluation Issues
• Compliance driven versus informative evaluation– Evaluation often done to meet accountability
requirements– Evaluation can serve to help integrate and improve
school and district services
• Evaluation is fundamental to MTSSS• MTSSS has the potential to:
– Be viewed as one more thing we have to do OR– Help address accountability & evaluation demands
through the multi-tier framework
Mission and Vision
Multi-Tiered System of Student Supports - Inter-Project Collaboration
The collaborative vision of the Florida Problem-Solving/Response to Intervention (FL PS/RtI) and the Florida Positive Behavior Support/Response to Intervention for Behavior (FLPBS/RtI:B) Projects is to:• Enhance the capacity of all Florida school districts to successfully
implement and sustain a multi-tiered system of student supports with fidelity in every school;
• Accelerate and maximize student academic and social-emotional outcomes through the application of data-based problem solving utilized by effective leadership at all levels of the educational system;
• Inform the development, implementation, and ongoing evaluation of an integrated, aligned, and sustainable system of service delivery that prepares all students for post-secondary education and/or successful employment within our global society.
Important MTSSS Evaluation Issues
• Stakeholders should be involved in all aspects of planning and carrying out the evaluation process as well as in decision-making
• Goals through planning should drive the process
• Information obtained to:– Determine where you currently are (needs)– Take ongoing looks at how things are working– Make decisions about what to keep doing and
what to change or eliminate
MTSSS Eval Issues cont.
• The data you collect should be driven by the questions you want to answer– Are students meeting expectations?
Academically? Behaviorally? Social-emotionally?– Are we implementing MTSSS with fidelity?– Do we have the capacity to implement
successfully?– Do staff buy into implementing MTSSS?
*Example questions
EXAMPLE QUESTIONS AND DATA SOURCES
How Are Students Performing?
Examples of data sources• Academics
– FCAT– FAIR– Core K-12– End of Course Exams
• Behavior– Attendance– Tardies– Suspensions– Discipline referrals
• Global Outcomes– Graduation Rates
www.flrtib.org
Are Schools Implementing MTSSS with Fidelity?
Examples of data sources• Curriculum and Instruction/Intervention
– Principal walkthroughs– Lesson plans– Intervention Documentation Worksheets
• Components of MTSSS and Data-Based Problem-Solving*– BOQ, PIC, BAT– SAPSI, Tier I & II CCCs, Tier III CCCs
* See http://flpbs.fmhi.usf.edu/ and http://floridarti.usf.edu for more information
Ro
les
Rep
rese
nte
d
Pro
ble
m Id
enti
fica
tio
n
Pro
ble
m A
nal
ysis
Inte
rven
tio
n D
evel
o...
Pro
gra
m E
valu
atio
n/..
.0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
0.680.76
0.54
0.28
0.64
Tiers I & II Observation Checklist
Percent Present
Roles Present and Problem-Solving Steps
Per
cen
tag
e o
f R
ole
s/C
om
po
nen
ts P
rese
nt
Do We Have the Capacity to Implement MTSSS with Fidelity?
Examples of data sources• Leadership Team structure and functioning
– Organizational charts– Minutes/meeting summaries– SAPSI, BOQ, PIC
• Staff knowledge and skills– FEAPs & teacher evaluation system– Staff development evaluations– Work samples
• Resources allocated to match needs– SIP, DIP– Master calendar/schedule– School rosters– Resource maps
Marzano Evaluation Domains
• Classroom Strategies and Behaviors• Preparing and Planning• Reflecting on Teaching• Collegiality and Professionalism
Do Staff Buy Into Implementing MTSSS?
Examples of data sources• Leadership vision and commitment– SAPSI, BOQ, PIC– Required and non-required plans
• Staff buy in– SAPSI, BOQ, PIC– District/school staff and climate surveys– Dialogue– Brief interviews with key personnel
District commitment SBLT support Faculty involvement SBLT present Data to assess commitment
1 2 3 4 5
0
1
2
3
Sunshine Elementary: Self-Assessment of Problem Solving Implementation (SAPSI) Data Consensus Building
BOY
EOY
Item Description
Indi
cato
r Sta
tus
3= Maintaining2= Achieved1= In Progress0= Not Started
Discussion
• What are you currently doing to examine these areas in your district or school?– What are the critical questions you ask?– What data sources do you have to answer them?– What questions do you already have that you cannot
answer with available data?– How do you use the data you collect to inform decisions?
• What areas need to be addressed as you return to your districts to plan? What are the priorities?– What critical questions do you need to start asking?– What data sources do you need?– How can you better use the data to inform decisions?
Our Plan for Moving Forward
• Learn more about your needs for evaluation support through needs assessment and DAPP protocol
• Develop resources for evaluating critical domains necessary to implement MTSSS such as:– Leadership– Coaching– Fidelity of implementation
• What additional supports can we provide?