Evaluating Trainig Programs

Why to Evaluate?

The Four Levels

During program evaluation 1. Reaction 2. Learning

Post program evaluation Behavior Results Kirkpatrick

According to Kirkpatrick:

“What is quality training?”

“How do you measure it?”

“How do you improve it?”

Evaluating

“The reason for evaluating is to determine the effectiveness of a training program.” (Kirkpatrick, 1994, pg. 3)

The Ten Factors of Developing a Training Program

1. Determine needs

2. Set objectives

3. Determine subject content

4. Select qualified applicants

5. Determine the best schedule

The Ten Factors of Developing a Training Program

6. Select appropriate facilities

7. Select qualified instructors

8. Select and prepare audiovisual aids

9. Co-ordinate the program

10. Evaluate the program

Why do we need to evaluating?

1.“To justify the existence of the training department by showing how

it contributes to the organizations’ objectives and goals.”

Reasons for Evaluating

2. “To decide whether to continue or discontinue training programs.”

3. “To gain information on how to improve future training programs.”

(Kirkpatrick, 1994, pg. 18)

The Four Levels

Reaction Learning Behavior Results

Reaction:

- what you want to find out

- Within this process we measure the reaction of the participants in the training program.

- Perception if they learned anything

Examples for REACTION

How much did you know about this subject before taking this workshop?Nothing Some A lot1 2 3 4 5

How much do you know about this subject after participating in this workshop?Nothing Some A lot1 2 3 4 5

How likely are you to use some or all of the skills taught in this workshop in your work/community/ family?

Not Very

Likely Likely Likely

1 2 3 4 5

Learning:

is the change in the participants’ attitudes, or an increase in knowledge, or greater skills received, as a result of the participation of the program.

*What did the participants learn in the program?

Learning

The measuring of learning in any training program is the determination of at least one of these measuring parameters:

Did the attitudes change positively? Is the knowledge acquired related and

helpful to the task? Is the skill acquired related and helpful to

the task?

EXAMPLE FOR LEARNING;Which of the following should be considered when evaluating at the Reaction Level? (more than one answer possible)

___Evaluate only the lesson content ___Obtain both subjective and objective

responses ___Get 100% response from participants ___Honest responses are important ___Only the course instructor should

review results.

Behavior

* Level 3 attempts to evaluate how much transfer of knowledge, skills, and attitude occurs after the training.

* Is the learning conveyed, transferred from the classroom to the real world?

* How the training affects performance

Example

1. Ask the program participant to what extent performance was improved as a result of the program. __ Large extent __ Some extent __ Not at allIf “Large extent” or “Some extent”, ask to please explain.

2. If “Not at all”, indicate why not: ___ Program content wasn’t practical ___ No opportunity to use what I learned ___ My supervisor prevented or discouraged me to change ___ Other higher priorities ___ Other reason (please explain)3. Ask, “In the future, to what extent do you plan to change your

behavior?” ___ Large extent ___ Some extent ___ Not at all

(Ask) please explain:

Kirkpatrick classified four changes to occur:

Desire to change

Knowledge of what to do and how to do it

Work in the right climate

Reward for (positive) change

What happens when all conditions are met?

Realize an opportunity to use the behavioral changes.

Make the decision to use the behavioral changes.

Decide whether or not to continue using the behavioral changes.

When evaluating change in behavior, decide:

When to evaluate

How often to evaluate

How to evaluate

Guidelines for evaluating behavior: Use a control group Allow time for change to occur Evaluate before and after Survey/interview observers Get 100% response or sampling Repeat evaluation, as appropriate Consider cost versus benefits

Results

Level 4 is the most important and difficult of all - determining final results after training.

Evaluation Questions:

Improved quality? Increased production? Decreased costs? Improved safety numbers? Increased sales? Reduced turnover? Higher profits?

Guidelines for evaluating results: Use a control group. Allow time for results to be achieved. Measure before and after the program. Repeat the measurements, as needed. Consider cost versus benefits. Be satisfied with evidence if proof is not

possible.