Utilizing Program Evaluation to Identify and Promote

Transformational Change Initiatives

AASA-NCPEA-CWC 2014 Nashville, TN

February 13, 2014

Dr. Nick Osborne, Dr. Marleis Trover, Dr. John Dively, Dr. David Grace EASTERN ILLINOIS UNIVERSITY

At various times, school districts need to distinguish worthwhile programs, initiatives and courses from ineffective ones and launch new programs and or courses or revise existing ones so as to achieve certain desirable results.

What Is Program Evaluation?

Evaluation means asking good, critical questions about programs to improve programs and help them be accountable for the wise use of resources…i.e.. Time, Money etc.

What does Evaluation mean to

you?

JUST…..NOT ENOUGH

TIME

Drug Education

Neighborhood Schools

Non Graded Primary

Block Scheduling

Non Traditional Curriculum

After School Programs

Worthy or Wasteful??

Looping

Middle School

Summer School

Advisory Councils

Expanded Internships

Legal Seminars

QUALITY REVIEW

Policy Centers

Satellite Cohorts

Reorganization Studies

Mentoring

Expanded Interns

FILTER

FILTER

TIME IS PRECIOUS

Administrative Roundtables

Getting Started: Essential Questions

• Why do you want to evaluate the program?

• Who will use the results?

• How will they use the results?

• Clearly identify the Program/Course/Activity? What does it include? Exclude?

• How much time and money do you have for the evaluation?

Uses of Program Evaluation •Instrumental or allocated uses

direct use of evaluation to make decisions or changes

•Persuasive uses

persuade funders of program’s success

•Conceptual uses

to shape the thinking of the policy-shaping community

•Symbolic uses

used for other than that for which it may have been

intended. (showing the public that evaluation IS being

done)

“I think you should

be more explicit

here in Step Two.”

A common problem is that activities and strategies

often do not lead to the desired outcomes. Check

your ‘if-then’ statements and ensure that they

make sense and lead to the outcomes you want to

achieve.

The Malcolm Baldrige Model makes the connections more EXPLICIT.

Collecting the Data

CHECKING THE LEVEL OF PERFORMANCE

Program Evaluation Questions !!!

1. Why do we have the program,

course, activity?

2. What needs do we have that the

program/course is meeting?

3. What “burning” question do we

want answered by this program?

4. What will we have to give up during

the day to implement the program? 11

Program Evaluation Questions !!!

5. What TIME for the instructional

staff, etc., is involved?

6. What COSTS are involved?

7. Will it cause political or inter

district/community controversy?

8. Can we foresee any unintended

outcomes?

12

Program Evaluation Questions !!!

9. What might be some downsides to the

program?

10. What data do we need to collect along

the way? Who will do what? and when?

11. How will we know we achieved our

goal?

12. Will we keep, delete, or revise the

program, course or activity? 13

LLT Training

• "Change is hard because people

overestimate the value of what they have—and…

underestimate the value of what they may gain by giving that up."

Change….

James Belasco and Ralph Stayer

Flight of the Buffalo (1994)

LLT Training

Data Collection

What data? Collection Method? Who Collects?

Analysis

Where are we? What have we learned?

Disposition

How do we feel about it?

Targets and Vision

Where do we desire to be? Change What?

Action Plan

What are we going to do about it?

Evaluation

Is it working? Do we need to realign and redirect?

Components in Change Management

LLT Training

Managing Change

The road to performanceexcellence

Where Are We?

Where Do We want

(Need) To Be?

How Are We Going

To Get There?

Data Collection

and Analysis

VISION & Targets

Strategies

How Do We Feel About It?

Disposition

DISPOSITIONAL DIALOGUE GROUP

PURSUE THE CHANGE!

DO NOTHING!!

SPIN ZONE!!

MANAGING THE DISCUSSION REGARDING CHANGE

RESULT

The skills used by the leader (s) to facilitate a positive Dispositional Dialogue is the catalyst that guides visioning for improved conditions.

Wise leaders utilize and partner with the persuasive members of the group!

The Opinion Leaders!!

RESISTANCE FORMULA

*Formula adapted from Beckhard and Pritchard (1992), Changing the Essence.

A X B X D

SUCCESSFUL CHANGE = R

A = Degree of dissatisfaction with the present situation

B= Desire for a new situation

D= Practicality of the change effort

R= Resistance (Cost and Pain of Change)

A x B x D SUCCESSFUL CHANGE = R

In Order For Change To Be Successful..

There Must Be:

A Enough Shared Dissatisfaction with the Status Quo.

B A Vision of How to Proceed.

D Thorough Knowledge of the First Steps for Moving

Ahead and the ability to proceed!

RESISTANCE FORMULA

Formula adapted from Beckhard and Pritchard (1992), Changing

the Essence

To help the stakeholders recognize there exists a need. (N )

To Foster a vision for the organization. (V)

To Facilitate the Dispositional Dialogue. (D)

To Assess the abilities and skills of the organization to pursue the change initiatives. (A)

To clearly understand the cost and the related pain of the proposed change initiative. (C/P)

Role of Leadership

The Path to Change*

N= Recognizing a Need

D= Disposition toward the Need

V= Having a Vision

A= Ability or Skills to Change

MUST BE GREATER THAN THE:

C/P = Cost and Pain of Change *Formula adapted from Beckhard and Pritchard (1992), Changing the Essence.

N+D+V+A> C/P

FORMULA

Application of Formula

• Need = Our 8th grade math scores are presently at the 48th percentile.

• Disposition= So What! Individual and Group

• Vision = To perform at the 70th percentile

• Ability = What is the capacity of our organization to reach the vision?

• Change = action which results in reaching the vision. What must we do differently?

N+D+V+A> C/P