evaluate training effectiveness

TRAINING NEED ANALYSIS

What is training?

• TRAINING IS TO IMPROVE WORK PERFORMANCE– ON THE JOB TRAINING OR– OFF THE JOB TRAINING

RETURN ON TRAINING

IMPROVEMENT VALUE

REDUCE COSTS

INCREASE REVENUE

REDUCE CAPITAL

TRAINING INVESTMENT

ACTUAL COSTS

OPPORTUNITY COSTS

INVESTMENTS

RETURN ON TRAINING

VALUE OF IMPROVEMENT

TRAINING INVESTMENT

THIS IS A RATIO JUSTIFYING THE INVESTMENT ON

TRAINING

What was the return on the investment?

Did it have an impact on the business?

Did they apply it on the job?

Did they learn what was required?

Did they like it?Level 1

Reaction

Level 4Impact

Level 5ROI

Level 3Application

Level 2Learning

Source: Jack J. Phillips. Return on Investment. Houston: Gulf Publishing Co., 1997.

Combined Evaluative Approach

5

Benefit-Cost Ratio = Program Benefits

Net Program Benefits*

Program CostsROI =

*Net Program Benefits = Program Benefits – Program Costs

X 100%

Source: ROI Institute.Measuring the ROI of Training and Performance Improvement Programs. Arlington, VA: ASTD, 2004. Page 2.9.

Program Costs

ROI Calculations

6

Customer Perspective

Financial Perspective

Learning &

Growth Perspective

Process Perspective

VISION&

STRATEGY

Balanced Scorecard Source:. Kaplan and Norton. The BalancedScorecard. Boston: HBS Press, 1996.

7

Balanced Scorecard

Balanced Scorecards tell you …..the knowledge, skills and systems that your employees will need (their learning and growth) …. to innovate and build the right strategic capabilities and efficiencies (the internal processes) ……that deliver specific value to the market (customers), ……which will eventually lead to higher shareholder value (the financials).

FinancialProcessLearning & GrowthCustomer

Vision &

Strategy

Source: David P. NortonBalanced Scorecard Report. “Use Strategy Maps to Communicate Your Strategy.” Page 169

8

• Human Capital Capability Scorecard™— predicts an organization’s ability to achieve future business results by analyzing data on an array of human capital indicators. 

• Generates prioritized, fact-based recommendations for improving business results through more effective development and management of employees.  

 • Statistically analyzes the relationship between HCCS

scores (by P&L center, branch, etc.) to identify those factors that are the most powerful predictors of an organization’s performance

• Serves as a blueprint for creating employee investment strategies, and a baseline to evaluate the effectiveness of these strategies. 

McBassi Methodology

9

We modified the Human Capital Capability Scorecard™ and our data collection to those elements with both of the following characteristics:

- Most relevant to employee development, and - Could be provided by a single individual within the organization

ABA CLO Summit Application

10

• The evidence clearly points to a positive relationship between human capital (learning) practices and investments, and financial returns.

• This relationship is most consistent for the measure that captures training and development expenditures per employee.

• The financial outcome most consistently associated with an array of human capital measures is net income per employee.

ABA CLO Summit Results

11

Median Net Income Per FTE (10/02 to 9/03), by Top Half/Bottom Half on Human Capital Variables

$0

$20,000

$40,000

$60,000

$80,000

Resources: T&DExpenditures Per FTE

Resources: Employeeto Trainer Ratio

Average SystemsMaturity Score

Average WorkforceOptimization Score

Average LearningCapacity Score

Human Capital Variable

Med

ian

Net

Inco

me/

FTE

Institutions in Top Half on HC Variable Institutions in Bottom Half on HC Variable

ABA CLO Summit Sample Results

12

Conclusion

There is every reason to believe that an institution’s increased focus on, and commitment to, human capital factors, especially learning and development, should lead to improved financial performance.

Which side of the fence are you on?

13

Conclusion

There is every reason to believe that an institution’s increased focus on, and commitment to, human capital factors, especially learning and development, should lead to improved financial performance.

Which side of the fence are you on?

14

Step 1 – identifying critical performance gaps

• Gaps is obtained through comparing actual against industry standards

• Critical is how material is the impact on the corporate objectives.

• Use ROE format as a coporate summary• Critical is derived from pareto and sensitivity

analysis

Define the Performance Gaps

DESIRED PERFORMANCE (Optimals)

- ACTUAL PERFORMANCE (Actuals)

= POSSIBLE TRAINING NEED

Describe Discrepancy

P. 31

Is it critical?

• Why is it important?• What if you did nothing?• How big is it? (Quantify if possible)• Who cares?• “Is the cost of the discrepancy high

enough that it seems worth pursuing a solution?”

P. 34

Step 2 - calculate the KPI of each critical performance areas

• Calculate KPIs for actual, standards and industry

• Compare KPIs to evaluate true performance.• The KPI may be used in ROI calcualtion.

Step 3 - Analysing further the critical performance gaps

• Is it true performance gaps?• Is the gap caused by factors other than

competency.• Benchmarkings to recognise best practices• Use input output analysis• Task and subtask analysis• Interviews, observations, focus groups,

documentations

Determine Cause(s)

Is it a problem of skill or

a problem of will?

I don’t wanna!I don’t know how.

I don’t wanna!

Yes, it is a skill deficiency

Arrange Formal Training

Arrange Practice

Arrange Feedback

Used to do it?

Used often?

no

yes

no

yes

Other questions

Change the Job

Arrange on-the-job training

Transfer or terminate

Simpler way?

Potential?

If a skill deficiency..

• Provide training• Provide practice• Provide feedback• Simplify the task• Develop a job aid• OJT• Transfer• Terminate

And one last question...

Obstacles?Remove

Obstacles

Step 4: To train or not to train?

• First determine cause(s)• Only then look at

possible solutions• Seek integrated

solution systems that get to the root of the problem

P. 39

Calculate cost

Select best solution(s)

Implement

Cause Solution

• If skill or knowledge……….training• If lack feedback……………..feedback, standards• If not motivated…………….rewards,

consequences• If unclear expectations…..std, measure, discuss• If job environment…………change environment• If potential……………………change personnel

P. 39

EVALUATE PERFORMANCE

LP HP

WP LP

COMPETENCY

MOTIVATION

Step 5 - calculate the ROI training

• The purpose of this step is to justify the training BEFORE the training start.

• Calculate the benefits generated by the training divided by the capital employed in the training.

• We obtain our ROI figures through the input output model.

Benefit-Cost Ratio = Program Benefits

Net Program Benefits*

Program CostsROI =

*Net Program Benefits = Program Benefits – Program Costs

X 100%

Source: ROI Institute.Measuring the ROI of Training and Performance Improvement Programs. Arlington, VA: ASTD, 2004. Page 2.9.

Program Costs

ROI Calculations

29

SUMMARY

1. Identify critical performance gaps using some benchmarks.

2. Identify key job holders and conduct competency profiling.

3. Identify best practices4. Identify effective training alternatives5. Use projected ROI to justify training