Evaluating Training EffortOrganisations are under pressure to justify

various expenses.

Business heads and training managers are under pressure to prove the effectiveness of training.

Two basic issues to address when evaluating:-

How to design the evaluation study? What to measure?

www.a2zmba.com

Measuring Training Effectiveness Basically three things that we can measure:

Participants’ reactions to the program;

What the trainees learned from the program;

To what extent on-the-job behavior changed.

www.a2zmba.com

Designing the Evaluation StudyTime Series:- Series of measures

before and after the training program.

Controlled Experimentation:-

Formal methods for testing the effectiveness of a training program, preferably with before-and-after tests and a control group.

www.a2zmba.com

What to Measure? Kirkpatrick’s TaxonomyReactions: Through “ Happy

sheets questionnaire” or “360-degree questionnaire”.

Learning

Behavior

Results

www.a2zmba.com

360 Degree Questionnaire Benefits of using a 360-degree format include:

Workplace behaviour improvements

The pre-program assessment can be used by the participant as a focus for their learning.

The participant is given a much broader appreciation of the application of their behaviours.

The reporting after the post assessment allows the individual to see how they have grown or developed as a result of the training program.

It provides the facilitator with hard data for proving the cost-effectiveness of the training intervention to Management.

www.a2zmba.com

Issues against

The Kirkpatrick framework represents a trainer’s notion of what constitutes effective evaluation, not a business manager’s.

The second issue deals with hierarchy, the assumption that

each level has more value than the previous one.

The third issue deals with causality, the assumption that each level is caused by the previous one.

Kirkpatrick’s taxonomic model is conceptually flawed as it does not fully identify all the constructs underlying the phenomena of interest. Furthermore, the results provided by the model can lead to incorrect decisions since major intervening variables are not included.

www.a2zmba.com

Other Approaches….QAI India QAI is a leading global consulting

organization (Asia’s largest and world’s third largest organization) addressing ‘Operational Excellence' in IT, BPO and Knowledge intensive organizations.

Effectiveness of a training program is measured at 5 levels:

I. Need for trainingII. Satisfaction of participantsIII. Learning at the classroomIV. Change in BehaviorV. Improvement in results

www.a2zmba.com

ROI approach by Jack PhillipsROI = Net Program Benefits X 100 % Program Costs

ROI couches results and expectations in the language of business and is often a hurdle

a project must clear in order to secure funding. Given that learning is

increasingly perceived as an investment by the organization, it seems only natural

that it should be evaluated in the same way as other large investments, in terms

of costs and benefits

www.a2zmba.com

Software Companies:

They use Heuristic Evaluation Quality Score (HEQS)

Steps of HEQS are:

www.a2zmba.com

1. Identify the application, parameters and the severity ratings

2. Standardize the scope of evaluation

3. Provide a knowledge transfer (optional)

4. Standardize the time for evaluation

5. Standardize the evaluation format

6. Proceed to evaluate

7. Use the individual evaluations to arrive at a benchmark

8. Use this benchmark to measure the skills of each evaluator

9. Derive Insights

www.a2zmba.com

www.a2zmba.com

• The IBM Certified Trainer (ICT) program is a comprehensive program for Trainers' Preparation and Certification.

• Validates skills using three proficiency levels: IBM Authorized to Teach. (IAT) IBM Professional Trainer (IPT) IBM Training Expert (ITE)

• E.g. Blended learning programs encompasses multiple learning strategies and delivery formats, ranging from those that are solely technology-based

to those that support interactions between people. Training effectiveness is measured by a control group of employees who receive no training. Provides real-world-based assessments with

feedback• Increase in its annual investment in learning programs that are

blended (with increases between 30 and 40 percent each year for the past four years).

   

www.a2zmba.com

Management Development Programmes(MDPs) have been institutionalised.

Wipro has different programmes catering to different life-cycle stages of the manager.

Different people may get different amounts of training at different times.

360 degree feedback.Training Quality measured against six sigma.

Manufacturing CompanyFeatherlite IndiaTraining given on various

topicsTraining conducted every 3

months or 6 months depending on requirements

In-house and external trainers

Happy sheet Questionnaire known as “ Survey Forms”

Monthly Performance Appraisal

Interview Rating Sheets

www.a2zmba.com

ConclusionMeasuring Training Effectiveness is different for

software and manufacturing companies.The purpose of Evaluation is to improve the

probability of investment providing a good return to the business.

Measurement of training effectiveness is a hallmark of good management.

www.a2zmba.com