iowa support system for schools and districts in need of assistance phase ii: diagnosis aea month...

Iowa Support System for Schools and Districts in Need of Assistance

Phase II: DiagnosisAEA

Month Day, Year

Iowa Support System for Schools and Districts in Need of

Assistance ©2009

Today, you will . . .

Increase your understanding of the Diagnosis Phase.

Access several tools for diagnosis.

Experience working with a gap analysis.

Work through a format for writing “if…then” statements and a theory of change that set the stage for action plan goals.

2


Assistance ©2009

The Phases of the Support for Schools and Districts Identified in Need of Assistance (SINA & DINA)

Phase I – The Audit

Phase II – The Diagnosis Phase III – The Design Phase IV – The Implementation (and

Monitoring) Phase V – Monitoring and Assessment

3


Assistance ©2009

Guiding Principles for Iowa Support Teams

Build on school/building strengths Encourage shared school leadership Analyze data to make decisions Focus on teaching and learning through a systems approach Integrate quality professional development practices Promote alignment of all components of the system Integrate existing state initiatives Reflect collaborative efforts throughout the structure Build capacity and accountability at all levels Implement evidence-based strategies

4


Assistance ©2009

Our Norms for Today

Share experiences to enrich others. Ask questions. Learn by doing. Set aside any preconceived notions about gap

analysis and diagnosis. Apply to your own work.

ThinkThink S-A-L-S-A! S-A-L-S-A!

Adapted from Adapted from Training Manual for Assessing Impact: Evaluating Staff DevelopmentTraining Manual for Assessing Impact: Evaluating Staff Development by Joellen Killion, by Joellen Killion, ©2003©2003

5


Assistance ©2009

Guiding Questions for Facilitators of Diagnosis

What is the severity of the problem? Does the audit profile accurately reflect all of your

building and/or district data? What level of involvement have staff members had in

the analysis of building and/or district data? Who is involved in/leads the school/district

improvement process? What role will central office personnel play in this

process? How is professional development organized in this

building/district? Is the focus on factors that we can control?

6


Assistance ©2009

Guiding Questions for Facilitators of Diagnosis

How well are curriculum, instruction, and assessment aligned?

What opportunities exist for the staff to implement the Iowa Professional Development Model (IPDM)?

What are current initiatives at the building and/or district level?

What is the history of the staff’s level of support in the school/district improvement process?

What are the potential barriers for initiating and implementing needed changes in this building/district?

7


Assistance ©2009

Selection of Building/District Leadership Team

Be sure to include: Building principal(s) as involved participants in buildings and

central office personnel in districts Content specialist(s) Special need areas (e.g., ELL, Special Education) A cross representation by grade level, experience, and expertise

in buildings and by building, experience, and expertise in the district

Those with credibility with colleagues Those with a balance of skills including but not limited to the

following: leadership, organization, facilitation, data analysis, presentation

8


Assistance ©2009

Building/District Leadership Team

Consider the following: Establish a common understanding of

purpose/function of the team Clarify role, function, and responsibility

of team members Define ground rules/norms for working

together Develop team operations (e.g., agenda,

minutes, time commitment, logistics) Identify supports available (e.g., central

office, AEA staff, other resources)

9


Assistance ©2009

Definition of Diagnosis

As part of the support framework for Schools and Districts in Need of Assistance (SINA & DINA), the Diagnosis Phase reviews prioritized areas from the audit summary. Through a comparison of the current reality with the desired state, a gap analysis is completed. The root causes that are contributing to the area(s) for further study are identified. If/then statements based on possible solutions and a theory of change are created as a final step to set the stage for the goals or action plan steps in the design phase. All of this is summarized on the Diagnosis Worksheet.

10


Assistance ©2009

Six Steps of Diagnosis

1. Review and refine current reality2. Determine the desired state3. Complete a gap analysis4. Determine the root cause(s)5. Identify and prioritize solutions (Instructional/Professional Development solutions must be research-based.)6. Create “if…then”* statement(s) and a theory of change *Some districts/schools call this “determining opportunities for improvement” or

“hypothesis/prediction statements.”

11


Assistance ©2009

Step 1: Review and Refine Current Reality

Review the audit. Review student achievement data. Request additional data as needed. Organize and analyze multiple sources/points of

data for deeper understanding. Request additional data as needed. Document prioritized areas from the audit and the

refined current reality on the Diagnosis Worksheet.

12


Assistance ©2009

Data To Consider For Step 1: Review and Refine Current Reality

Student Data – achievement, perception, demographic, participation, and enrollment.

Professional Practices Data – strategies, certification, professional development, teacher perception, leadership.

Programs and Structures Data – academic programs, support, curriculum, school / districtenvironment.

Family and Community Partnership Data – parent involvement, parent perception, community involvement, community perception.

13


Assistance ©2009

Tools to Consider for Step 1: Review and Refine Current Reality

Diagnosis Worksheet Questions for Areas of Concern/Red Flags Analyze and Report Data Response Sheet - Iowa Professional

Development Model (adapted) Item Analysis Summary - Iowa Professional Development

Model (adapted) Performance Profile Analysis – Iowa Testing Services (adapted) Questioning Your Data Questioning Your Data – Genny Eric High School Pareto Diagram Questioning Your Data – Hilltop Central

14


Assistance ©2009

Activity for Step 1:Review and Refine Current Reality

Review of the Audit Profile– Use the audit profile and the guidance for

reviewing the building/district profile to determine strengths and challenges revealed in the profile.

– Focus your discussion on those areas identified as areas for further study.

– Create/Revise “the current reality” based on information provided.

15


Assistance ©2009

Activity for Step 1: Review and Refine Current Reality

Data Analysis – Review the data using the tool marked

“Analyze & Report Data Response Sheet” (Iowa Professional Development Model)

– Discuss in small groups the findings– Share out a summary of the discussion and the

findings to the whole group– Summarize the “current reality” on the

Diagnosis Worksheet for Westlake Middle School

16


Assistance ©2009


Data Analysis – Another Activity – Review the data using the tool marked

“Questioning Your Data – Genny Eric High School”

What do YOU SEE in these data? (Individual) What do WE SEE in these data? (Small Group) What are the “big six” that your group sees in these

data? (Small Group) Share with large group

17


Assistance ©2009


Data Analysis – Optional Activity – Review the data using the tool marked

“Questioning Your Data – Hilltop Central”

What do I SEE in these data? What questions do I have about what I see?

18


Assistance ©2009

Using Data as a Lever for Change

Data can uncover problems that might otherwise remain invisible.

Data can convince people of the need for change Data can confirm or discredit assumptions about

students and school practices Data can get to the root cause of problems, pinpoint

areas where change is most needed, and guide resource allocations.

Data can help schools evaluate program effectiveness and keep the focus on student learning results.

19


Assistance ©2009

Using Data as a Lever for Change

Data can provide the feedback teachers and administrators need to keep going and stay on course.

Data can prevent over-reliance on standardized test scores.

Data can prevent one-size-fits-all and quick-fix solutions.

Data can give schools the ability to respond to accountability questions.

Data can help build a culture of inquiry and continuous improvement.

20


Assistance ©2009

Questions about Step 1 of the Diagnosis Phase…

21


Assistance ©2009


1. Review and refine current reality

2. Determine the desired state3. Complete a gap analysis4. Determine the root cause(s)5. Identify and prioritize solutions (Instructional/Professional Development solutions must be research-based.)6. Create “if…then”* statement(s) and/or theory of change *Some districts call this “determining opportunities for improvement” or “hypothesis/prediction

statements”

22


Assistance ©2009

Definition of Desired State

Desired state is the picture of your school/district that you wish to reach after the completion of your school/district improvement process (your action plan).

The state we have all agreed to reach and be held accountable to is . . . Determined collaboratively Based on research, best practices, and

promising practices “Doable” in the near future

23


Assistance ©2009

Step 2: Determine the Desired State

Consult expert content specialist(s) Review research Develop the picture of your desired state Document the desired state on the

Diagnosis Worksheet

24


Assistance ©2009

Step 2:Determine the Desired State

Examples of Research to Review Iowa Content Networks Expert Content Specialists United States Department of

Education Clearing House - http://www.whatworks.ed.gov/

25


Assistance ©2009

Tools to Consider for Step 2: Determine the Desired State

Structured Response Sheet – Every Child Reads (adapted)

Content Network Tool for Reviewing Research

Diagnosis Worksheet

26


Assistance ©2009

Activity for Step 2: Determine the Desired State

Provide the research/evidence to identify the expectations of the identified areas of concern.

Develop common expectations based on research for all areas of concern.

Document the desired state.

27


Assistance ©2009


28


Assistance ©2009


1. Review and refine current reality2. Determine the desired state

3. Complete a gap analysis4. Determine the root cause(s)5. Identify and prioritize solutions (Instructional/Professional Development solutions must be research-based.)6. Create “if…then”* statement(s) and/or theory of change. *Some districts call this “determining opportunities for improvement” or “hypothesis/prediction statements”

29


Assistance ©2009

Step 3: Complete a Gap Analysis

Summarize Current Reality Summarize Desired State Identify deficits or the gap between desired state

and current reality Identify themes and patterns across deficit

areas/gaps, using guiding questions Document the gap analysis on the Diagnosis

Worksheet

30


Assistance ©2009

Guiding Questions for Step 3: Complete a Gap Analysis - Identify themes and patterns across deficit areas/gaps, using guiding questions

What are the relative strengths? What are the relative weaknesses? What are the trends, themes, and patterns? Consider the triangulation of data to verify the

gap analysis. ITBS/ITED

Diagnostic Measure

Teacher Observation

31


Assistance ©2009

Tools to Consider for Step 3: Complete a Gap Analysis

Gap Identification Worksheet Forced Field Analysis Planning Template Fishbone Graphic Organizer Diagnosis Worksheet

32


Assistance ©2009

Activity for Step 3:Complete a Gap Analysis

For this activity we will use the Force Field Analysis to help us process the determination of the gap.

33


Assistance ©2009


34


Assistance ©2009


1. Review and refine current reality2. Determine the desired state3. Complete a gap analysis

4. Determine the root cause(s)5. Identify and prioritize solutions (Instructional/Professional Development solutions must be research-based.)6. Create “if…then”* statement(s) and/or theory of change. *Some districts call this “determining opportunities for improvement” or “hypothesis/prediction statements”

35


Assistance ©2009

Definition of Diagnosis

As part of the support framework for Schools and Districts In Need of Assistance (SINA & DINA), the diagnosis phase reviews prioritized areas from the audit summary. Through a comparison of the current reality with the desired state, a gap analysis is completed. The root causes that are contributing to the area(s) for further study are identified. If/then statements and or a theory of change based on possible solutions are created as a final step to set the stage for the goals or action plan steps in the design phase.

36


Assistance ©2009

Definition of Root Cause

The deepest underlying cause, or causes, of positive or negative symptoms within the area of concern that, if dissolved, would result in elimination, or substantial reduction, of the symptom.

Source: Root Cause Analysis – School Leader’s Guide to Using Data to Dissolve Problems – Paul G. Preuss

37


Assistance ©2009

Root Cause Patterns

Chained or Linear Combined Contributing

38


Assistance ©2009

Root Cause Pattern: Chained

A B CResult

X =

Chained Causes: Interrupt the chain and eliminate the result

39


Assistance ©2009

Root Cause Patterns: Combined

Combined Causes: The common presence of multiple factors brings about the result. Eliminate any one factor and the result disappears.

A B

C

ResultY

40


Assistance ©2009

Root Cause Patterns: Contributing

Contributing Causes: Multiple factors each contribute to a certain percentage of the result. To totally eliminate the result, all contributing causes must be removed.

A

C

BResult

Z

41


Assistance ©2009

When Is a Cause a Root Cause?

1. Would the problem have occurred if the cause had not been present? If no, then it is a root cause! If yes, then it is a contributing cause.

2. Will the problem reoccur as the result of the same cause if the cause is corrected or dissolved? If no, then it is a root cause! If yes, then it is a contributing cause.

42


Assistance ©2009


3. Will correction or dissolution of the cause lead to similar events? If no, then it is a root cause! If yes, then it is a contributing cause.

43


Assistance ©2009


Other indicators that you have found the root cause: You run into a dead end asking what caused the proposed

root cause. Everyone agrees this is a root cause. The cause is logical, makes sense, and provides clarity to

the problem. The cause is something that you can influence and

control. If the cause is dissolved, there is realistic hope that the

problem can be reduced or prevented in the future.

44


Assistance ©2009

Insights on Root Causes

Look for triangulation of data

Aim to dissolve a problem, not just the symptom Think proactively, as well as reactively Realize some causes are contributing, not root

ITBS/ITED

Diagnostic DataTeacher

Observation

45


Assistance ©2009

Step 4: Determine the Root Cause(s)

Given what you identified in your gap analysis, determine the critical cause/effect relationships from your prioritized areas

Document the root cause(s) on the Diagnosis Worksheet

46


Assistance ©2009

Tools for Step 4: Determine the Root Cause(s)

Diagnostic Tree Pareto Diagram Fishbone - Cause and Effect Diagram Relations Diagram Five Whys – Iowa Evaluator Approval

Training Program (adapted) Diagnosis Worksheet

47


Assistance ©2009

Activity for Step 4 : Determine the Root Cause(s)

Process the cause/ effect relationships within the prioritized areas to identify the root causes.

Using the Diagnostic Tree, identify possible causes in each of the domains.

Review the evidence for supporting/not supporting these causes.

Determine those that are evidence-based and/or “doable.”

48


Assistance ©2009


49


Assistance ©2009


1. Review and refine current reality2. Determine the desired state3. Complete a gap analysis4. Determine the root cause(s)

5. Identify and prioritize solutions (Instructional/Professional Development solutions must be research-based.)6. Create “if…then”* statement(s) and/or theory of change. *Some districts call this “determining opportunities for improvement” or “hypothesis/prediction statements”

50


Assistance ©2009

Step 5: Identify and Prioritize Solutions

Sources to Consider:•Consult expert content specialist(s)•Review research•Request information from the Iowa Support Team facilitator associated with successful SINA schools•Utilize Iowa Content Networks•Access USDE Clearinghouse

51


Assistance ©2009


Identify possible solutions correlated to root causes - if the solution is related to instruction or professional development, it should be research based

Prioritize based on research (e.g., greatest impact on student achievement, response to intervention)

Document the solution(s) on the Diagnosis Worksheet

52


Assistance ©2009

Step 5:Identify and Prioritize Solutions

Possible Solution Categories Academic

– Alignment of Curriculum, Instruction & Assessment with Standards– Content & Strategy Selection

Quality Educator– Professional Development– Leadership – Teaming and Collaboration Opportunities– Culture & Climate

Building/District System– Instructional and Operational Policies & Practices– Resource Allocation (e.g., time for the content area, dollars for materials)– Family & Community– Communication

53


Assistance ©2009

Step 5:Identify and Prioritize Solutions

Guidelines for Prioritizing Solutions: Will the implementation of this solution address our

school’s/district’s area(s) of concern and underlying reasons for them?

Will the proposed solution have a great impact on students’ learning?

Are resources (i.e., people, time, money) available to implement this solution?

54


Assistance ©2009


Guidelines for Prioritizing Solutions: Do the advantages outweigh the disadvantages if we

implement this solution? Could this solution be modified without impacting the

effectiveness to best meet the needs/interests of our students, their families, our teachers, our classrooms, our school, and our district?

Is everyone who is part of the challenge also a part of the solution?

55


Assistance ©2009

Activity for Step 5:Identify and Prioritize Solutions

For this activity the Nominal Group Technique will be used to prioritize the solutions.

Questions to ask yourself about each solution:– Do we have evidence to support this solution? Is there

research that shows this solution will have an impact?– Is this an area over which we can have control and can

influence change (e.g., proximal vs distal)?– Do we have the resources to implement this solution?– Is this solution sustainable?

57


Assistance ©2009

Activity for Step 5:Identify and Prioritize Solutions

Other questions you might consider for each solution:

Will the implementation of this solution address our school’s/district’s areas of concern and underlying reasons for them?

Will the proposed solution have a great impact on students’ learning? Are resources (i.e., people, time, money) available to implement this

solution? Do the advantages outweigh the disadvantages if we implement this

solution? Could this solution be modified without impacting the effectiveness to

best meet the needs/interests of our students, their families, our teachers, our classrooms, our school, and our district?

Is everyone who is part of the challenge also a part of the solution?

58


Assistance ©2009


1. Review and refine current reality2. Determine the desired state3. Complete a gap analysis4. Determine the root cause(s)5. Identify and prioritize solutions (Instructional/Professional Development solutions must be research-based.)

6. Create “if…then”* statement(s) and/or theory of change

*Some districts call this “determining opportunities for improvement” or “hypothesis/prediction statements.”

60


Assistance ©2009

Step 6: Create “if…then” statement(s)

Based on prioritized solution(s) write “if…then” statement(s) “If…then” statements may either result in student

achievement goal statements or action plan steps toward a student achievement goal.

Document “if…then” statements on the Diagnosis Worksheet

61


Assistance ©2009

Step 6:Create “if…then” statement(s)

Example: Possible Root Cause: Students have inadequate

instructional time to master identified math skills. Possible solution: Classroom schedules are adjusted to

accommodate increased instructional time in math. This may be accompanied by a district/building expectation for a specific amount of instructional time in math.

If…then statement: If classroom schedules are adjusted to increase the amount of instructional time in math, then student achievement in math should improve.

62


Assistance ©2009


Example: Root cause: Student achievement in reading comprehension is

low because professional development lacks specific reading strategies targeted to student needs.

Solution: Provide professional development on specific reading strategies targeted to student needs.

If…then statement: If all teachers are provided professional development based on research-based reading strategies targeted to student needs, then student achievement in reading will improve.

63


Assistance ©2009


Example: Root Cause: The articulated building/district assessment plan

lacks formative assessments to guide math instruction. Solution: Identify/develop required formative assessments that

align to the math curriculum and regularly guide instruction. If…then statement: If the required formative

assessments in math align with curriculum and resulting information is used to change instructional practices, then student achievement in math will improve.

64


Assistance ©2009

Tools for Step 6:Create “if…then” statement(s)

If . . . Then Worksheet ORID Questioning PowerPoint – Iowa

Evaluator Approval Training Program (adapted)

Diagnosis Worksheet

65


Assistance ©2009

Activity for Step 6:Create “if…then” statement(s)

For this activity, create “If…Then” statements to help the group process the relationships between the root causes and the prioritized solutions.

66


Assistance ©2009

Step 6: Complete a Theory of Change

A Theory of Change . . . Identifies the components of a program (what

the program does) Specifies the relationship among the

components to explain how the change occurs (sequence of actions)

Delineates the underlying assumptions upon which the program is based

68


Assistance ©2009

Tools for Step 6:Create a Theory of Change

Creating a Theory of Change – Blue and Green Card Activity

Diagnosis Worksheet

69


Assistance ©2009

Activity for Step 6:Theory of Change ActivityActivity for Step 6:Theory of Change Activity

To further your understanding of the theory of change, sequence the blue and green cards to show a possible causal link among the components of the Youth Development Program.

(Hint: Identify the starting and ending points first, then sequence all the other cards between those two.)

70


Assistance ©2009

Theory of Change ActivityTheory of Change Activity

How do the blue and green cards differ?– The blue cards represent _____________.– The green cards represent ____________.

71


Assistance ©2009

What process do you use?

In collaboration with stakeholders:– Affinity process– Flow chart– Brainstorm

72


Assistance ©2009

Activity: Creating Your Own Theory of ChangeActivity: Creating Your Own Theory of Change

Create a theory of change for Westlake Middle School. When you are satisfied, map the actions on the large chart paper.

Post your chart.

73


Assistance ©2009

AssessingAssessing EvaluabilityEvaluability

Initial Outcomes

Activities / Processes

Inputs/

Resources

Assumptions

Intermediate Outcomes

Intended Results

•Who?

•What?

•When?

•Where?

•Knowledge

•Skills

Teacher Admin.

•Attitudes

•Aspirations

•Behaviors

Teacher Admin.

•Standards

•Indicators

•Data Points

SMART Goals:Increased

Student Achievement

Backmapping Model

Theory Of Change

Initial Experiences•Coaching•Follow Up•Monitoring

74


Assistance ©2009

Activity: Assumptions Card Stack & ShuffleActivity: Assumptions Card Stack & Shuffle

Write down assumptions you have about your theory of change.

– One assumption per card– Assumptions can be written for other people’s theory of

change Cards are stacked by separate theory of change and

elements. As a table group, cards are read aloud and placed

“on the table.” Discuss your findings.

75


Assistance ©2009


1. Review and refine current reality2. Determine the desired state3. Complete a gap analysis4. Determine the root cause(s)5. Identify and prioritize solutions (Instructional/Professional Development solutions must be research-based.)6. Create “if…then”* statement(s) *Some districts call this “determining opportunities for improvement” or “hypothesis/prediction

statements”

78

End of Diagnosis Training Slides

Proceed to Design Phase

iowa support system for schools and districts in need of assistance phase ii: diagnosis aea month...

Documents

iowa support teamsbuild

yeariowa support system

need of assistancephase

staffs level of support

staff development

guiding questions

building andor district

improvement process