11

IMPLEMENTING KIRKPATRICKS FOUR LEVELS

TU101

By Dr. Donald L. Kirkpatrickdleekirk1@aol.com

www.donaldkirkpatrick.comwww.smrhub.com/usa

Session Objectives:Understanding of:

1. Requirements for an effective training program2. Overview of the Four Levels

3. Guidelines for implementing each level

22

Requirements for an Effective Program

1. Based on __ __ __ __ __.2. Aimed at __ __ __ __ __ __ __ __ __ __3. Scheduled at the right __ __ __ __.4. Held at the right __ __ __ __ __.5. For the right __ __ __ __ __ __.6. Conducted by an effective __ __ __ __ __ __.7. Using effective __ __ __ __ __ __ __ __ __ __.8. __ __ __ __ __ __ __ __ __ __ are reached.9. Participants are __ __ __ __ __ __ __ __ __.10. Program is __ __ __ __ __ __ __ __ __.

33

Why Evaluate?

Should the program be continued?

How can the program be improved?

How can trainers justify their existence?

44

EVALUATION PRINCIPLES

1. Evaluation methods and forms can be borrowed from other organizations and used or adapted to your situation.

2. Evaluation results cannot be borrowed even if the program is the same.

55

The Four Levels

REACTION how training participants react to it. A customer satisfaction measure.

LEARNING the extent to which participants change attitudes, increase knowledge, and/or increase skill

BEHAVIOR the extent to which change in behavior occurred.

RESULTS the final results that occurred as a result of the training.

66

Guidelines for Evaluating Reaction

Determine what you want to find out. Design a form that will quantify reactions. Encourage written comments. Get 100% immediate response.. If desirable, get delayed reactions. Develop acceptable standards. Measure future reactions against the

standard.

77

Tabulation of Reactions from 20 SupervisorsReaction Sheet

1. How do you rate the subject (interest, benefit, etc)?5 Excellent 10 Comments & Suggestions:4 Very Good 53 Good 3 Rating = 4.12 Fair 11 Poor 1

2. How do you rate the conference leader (subject knowledge, ability to communicate, etc)?5 Excellent 8 Comments and Suggestions:4 Very Good 43 Good 5 Rating = 3.82 Fair 21 Poor 1

88

Guidelines for Evaluating Learning

Measure before and after attitudes, knowledge, and/or skills.

Use a paper and pencil test for knowledge and attitudes.

Use a performance test for skills. Get 100% response.

99

DECISION MAKINGManager

QUALITY Subordinates ACCEPTANCE

Past FutureAve Range Ave Range25% 10-50% 16% 5-40% 1. Manager decides and sells decision37% 10-50% 30% 10-50% 2. Manager asks for input, considers it,

decides, and sells decision22% 10-50% 25% 5-50% 3. Manager leads subordinates in making

a consensus decision16% 0-40% 29% 0-60% 4. Manager empowers subordinates to ____ ____ make the decision100% 100%

ServiceMaster Executives August, 1996

10

10

Change in Behavior Requires:

1. Desire to Change2. Necessary Knowledge and Skills3. The Right Job Climate4. Encouragement and Help5. Rewards for Change

11

11

The Job Climate

Department HeadSupervisor

CLASSROOM CLIMATEKnowledge PreventingSkills DiscouragingAttitudes Neutral

EncouragingRequiring

12

12

Guidelines for Evaluating Behavior

Measure on a before / after basis if practical

Allow time for behavior change to take place (perhaps 3-6 months)

Get 100% response or a sampling Repeat at appropriate times Consider cost vs. benefits

13

13

Behavior Change(3-6 months after program)

Survey or Patterned Interview1. Explain purpose of survey/interview.2. Review program content.3. Ask the program participant:

- To what extent have you changed behavior as a result of the program you attended?__ Large extent __ Somewhat __ Not at all

If Large extent or somewhat, please explain:

4. If not at all, please indicate why not:___ Program content wasnt practical___ No opportunity to use what I learned___ My boss prevented or discouraged me to change___ Other higher priorities___ Other reason (please explain)

5. In the future, to what extent do you plan to change your behavior?___ Large extent ___ Somewhat ___ Not at all

Please explain:

14

14

Guidelines for Evaluating Results

Measure on a before and after basis Allow time for possible results to take

place (suggest 6-12 months) Repeat at appropriate times Use a control group if practical Consider cost vs. benefits (ROI) Consider Evidence if Proof not possible

15

15

Suggestions for Evaluating Training Programs

1. Measure reaction to all programsa. Develop your own formb. Gather datac. Establish standards of acceptable performanced. Improve programs

16

16

Suggestions for Evaluating Training Programs (Cont.)

2. Expand evaluation to measure: Learning Behavior Results (including ROI)

Consider: Available evaluation resources Frequency of programs Cost of programs Cost of the evaluation Executive interest / demand

17

17

References

Kirkpatrick, D.L. Evaluating Training Programs: The Four Levels, 3nd

Ed., Berrett-Koehler Publishers, Inc. San Francisco, CA, 1998

Kirkpatrick, D.L. and Kirkpatrick, J.D. Transferring Learning to Behavior: Using the Four Levels to Improve Performance, Berrett-Koehler Publishers, Inc. San Francisco, CA, 2005

Kirkpatrick, D. L. Consulting, . www.donaldkirkpatrick.com,

dleekirk1@aol.com.