evaluating training programs level 5: return on investment kelly arthur richard gage-little dale...
TRANSCRIPT
Evaluating Training Programs
Level 5: Return on Investment
Kelly Arthur
Richard Gage-Little
Dale Munson
E-Poster Objectives
Define the fifth level of evaluation Explain how to find Return on Investment
(ROI) for training programs Identify benefits and disadvantages
2 of 21
Building on Kirkpatrick’s Four Levels
1. Reaction
2. Learning
3. Behavior
4. Results
5. Return on investment (ROI)
3 of 21
Why Was It Created?
The fifth level was created because Kirkpatrick’s four levels only focus on the effectiveness of training events and not monetary benefits.
Instructional Developers needed a way to demonstrate cost-value and accountability for their training.
4 of 21
What Is the Fifth Level?
Known as ROI, the fifth level was published by Jack Phillips to quantify the monetary value of training investments.
ROI answers the question, “For every dollar invested in training, how many dollars does the employer get back?”
In other words, show me the money.
5 of 21
What Does it Do?
ROI converts Kirkpatrick’s fourth level data (business results) into monetary values and compares it with the cost of training. ROI is only a financial measure of training’s value, it does not provide information to improve training.
ROI% = Benefits – Costs of Training x 100 Costs of Training
6 of 21
Why Should ROI Be Used?
“Many programs have failed to deliver what wasexpected; consequently, program sponsors haverequested ROI calculations/justifications.”
“The concern for accountability in all functions inorganizations is increasing; thus, the training anddevelopment function becomes one of many support efforts under scrutiny.”
Jack Phillips
7 of 21
Isolate training effects
2
Convert data into
$$$
3
Tabulate program
costs
4
CalculateROI
5
Collect data
1
Identifyintangiblebenefits
6
How to Determine ROI (animation)
8 of 21
Step 1: Collect Data
Hard data – objective and easy to measure– Output (units produced, items sold, etc.)
– Quality (scrap, waste, rework, product defects)
– Time (equipment downtime, employee overtime)
Soft data – subjective and difficult to measure– Work habits (employee absenteeism, tardiness)
– Work climate (employee grievances, turnover)
– Attitudes (employee loyalty, self-confidence)
– New skills (decisions made, problems solved)
9 of 21
Step 2: Isolate Training Effects
Because many factors influence performance, it is necessary to determine the amount of improvement that is directly attributable to training.
Example: After attending a training program, sales people sold 20 percent more widgets over the next six months. Was it the training? Or, maybe it was the new reorganization or new incentive program.
10 of 21
Step 3: Convert Data into $$ Values
In this step, level four data are converted into monetary values and compared against program costs.
Level four measurements: – Documented improvements in the standard of work
and/or care delivered to clients.– Service targets being met more effectively.– Work being accepted by others (repeat business,
testimonials, complementary letters).– Greater measurable productivity, improved safety
records, etc.
11 of 21
Step 4: Tabulate Program Costs
Costs can be categorized into three types:
1. One-time costs: – Analysis and design– Program development – Equipment and software
2. Cost per offering: – Facility rental– Instructor's salary
3. Cost per participant:– Travel – Per diem expenses– Tuition
12 of 21
Step 5: Calculate ROI
Formulas
1. ROI = Total Cost of Training Number of Students
2. ROI (%) = Training Costs - Benefits x 100 Training Costs
13 of 21
Step 6: Identify Intangible Benefits
Intangible benefits are activities or conditions that are extremely difficult to quantify.
Examples: Improved teamwork Improved customer service Increased job satisfaction Reduction in customer complaints Improved communications Increased flexibility Employee morale
14 of 21
The ROI Formula in Action
Krispy Kreme’s evaluation research determined that there was a 10% increase in the number of donuts sold after a new selling skills training program was conducted. Other data revealed that each 1% increase in sales is equal to increased annual revenue of $25,000 for a total of $250,000 in benefits. The cost of training was $75,000. What was the ROI?
Benefits - Costs of training x 100 = %ROI Costs of training
15 of 21
Let’s Work it Out (animation)
($250,000 - $75,000)(100)
$75,000
= ($175,000)(100)
$75,000
($17,500,000)
$75,000= 233%
So, for every $1 invested in training, the organization realized a benefit of $2.33.
Benefits - Costs of training x 100 = %ROI Cost of training
16 of 21
Some Benefits of ROI Evaluations
ROI also provides: A comprehensive measure of training programs Comparability between training programs Benefits and cost in percentage terms
17 of 21
Some Limitations of ROI Evaluations
It is estimated that only 14% of organizations actually evaluate ROI. Why? Many companies don’t perform level four evaluations Difficult to isolate training benefits ROI formula is only an estimation
18 of 21
Summary
ROI is the fifth level of evaluation It provides a monetary value for training
investments ROI is only concerned with how much a
training is “worth” The ROI formula can be used to determine if
the benefits of training outweigh the costs of training
ROI is difficult to determine effectively
19 of 21
Helpful URLs
http://www.learnativity.com/roi-learning.html
http://www.performancexpress.org/0205/mainframe0205.html
http://www1.astd.org/news_letter/October/Links/phillips.html http://www.performancexpress.org/0208/mainframe0208.html
http://www.arches.uga.edu/~rali/edit8350/ROI/default.htm
20 of 21
References
Long, Larry. (1999). ROI: Capturing the Big Picture. Technical Training. Retrieved November 12, 2002 from the World Wide Web http://www.astd.org/virtual_community/comm_evaluation/long.pdf
Martin, John. (2000). ROI: Do It & Prove It. ASTD. Retrieved November 12, 2002 from the World Wide Web http:www.astd.org/CMS/templates/index.html?template_id=1&articleid=26100
Phillips, Jack. (1996) Measuring ROI: The Fifth Level of Evaluation. Technical & Skills Training. Retrieved November 12, 2002 from the World Wide Web www.astd.org/virtual_community/ comm_evaluation/phillips.pdf
21 of 21