evaluate the effectiveness of your online learning & training programs
TRANSCRIPT
PRESENTER NAMEPresenter Job Title Here
Evaluating the Effectiveness of Your Online Learning & Training Programs
Presented with Paula Yunker of Limestone Learning
Housekeeping
2
1. Sound & Audio
2. Poll Questions
3. Q&A
4. Presentation Recording & Slides
5. Post-Webinar Satisfaction Survey
Your Hosts
3
Paula YunkerManaging Partner &
Instructional Designer
Nimritta ParmarMarketing
Coordinator
Limestone Learning
4
Lambda Solutions
5
Poll: How important do you believe learning evaluation is?
6
• Not at all
• Somewhat important
• Important
• Very Important
• I’m not sure
Poll: To what degree does your organization use evaluation?
7
• Not at all
• Occasionally
• Most of the time
• All of the time
• I’m not sure
Evaluating the Effectiveness of Your Learning & Training Programs
Why is Evaluation Important?
Training should have a purpose:
• To inform
• To teach
• To improve change
Evaluation will identify if learning
has occurred, and if improvements
have been made
9
Aligning Learning to Organization Goals
• For training to be valued by leaders and stakeholders, all training efforts need to focus on the organizational goals
• Consider what your stakeholders are expecting, and the impact on business training needs to have to meet expectation
10
Training
Business Goals
Strategies for Aligning Learning
11
• Step 1: Identify the business need
• Step 2: Create the learning plan
• Step 3: Measure the business results
12
Ask your key stakeholders and business
partners:
• What are the desired results?
• What are the goals & outcomes?
• What needs to change to meet these
goals & business outcomes?
Step 1: Identify the Business Need
13
• What KPI’s should change?
• What needs to be learned to make those
changes happen?
• What will success look like to you?
Step 1: Identify the Business Need (cont.)
14
Develop a learning and evaluation plan to meet the learning requirements based on your answers to Step 1:
• What needs to be learned to achieve business outcomes?
• What required KPI’s are needed?• What learning experiences and practice
are needed to become proficient?
Step 2: Create the Learning Plan
15
• When and how will these learning experiences and practices be best accomplished?
• How and when will the learning be measured?
• How will learning be applied to the job, and how will that application be reinforced on the job?
Step 2: Create the Learning Plan (cont.)
16
To identify how learning aligns to business outcomes, and meets stakeholder/business partners needs:
• What changes in business results & key performance indicators have been realized from the training program?
Step 3: Measure the Business Results
17
• For every dollar invested in training, how many dollars does the organization get in return? Is the ROI significant enough to justify training?
• What key intangible benefits (soft data) have been gained?
Step 3: Measure the Business Results (cont.)
18
Where do you need to be with data & analytics, and how do you get there?
Take a BIG PICTURE view: Stakeholders want to know if programs have helped make money, save money, or avoid costs.
This brings us to … DATA
Start by collecting data easy to obtain
19
• System logins by department – are there some departments or organization groups that login significantly less than others?
• Course completions by department –are there any learners not completing courses?
• Pre and post test and quiz results
Now consider the tougher questions..
20
How can additional data be helpful?
Calculating Success Measures (using sales training as an example):
1. Measure for each sales
rep, if possible
2. Isolate variables that
effect revenue
3. Corroborate pre-test, test,
and retention quizzes
4. If knowledge is retained,
proceed.
5. Wait 3 months to
compare sales
6. Isolate data for considerations
of seasonality, product, etc.
Let’s step back – Typical Challenges
21
• Lack of buy-in from the top• No ownership of the project• Inability, or unwillingness to
measure and track evaluation on the job
• Lack of reinforcement on the job
Overcome the Challenges: Implementation Strategies
22
What’s needed to gather & measure data, and evaluate the results?
• Expertise• Resources• Organizational Support
Implementation Strategies
23
Evaluation tools that support and measure learning on the job
• Create resources that reinforce learning (job aids, reference handouts, etc.)
• Create follow-up questions• Ask similar questions to managers and supervisors, internal
clients, and customers to get a fuller picture
Implementation Strategies
24
Learning Action Plan
• A contract between the learner and manager, that provides structure to assist managers and employees to put learning into practice
• Gives learners the opportunity to reflect on their learning experience
• Signed by both parties, and provides accountability
Poll: Does your organization use some form of Learning Action Plan?
25
• No• Yes• I’m not sure
Overcome your Challenges
26
Before Learning:• Create pre-evaluation that is
directly aligned to learning content, and the learning objectives.
• Examples: self-assessment surveys, written pre-tests, blog/wiki questions, reading assignments
Overcome the Challenges: Implementation Strategies
27
During Learning: • Design formative evaluation tools, and place in strategic spots
throughout content. • Examples: team exercises, brainstorming, flipchart discussions,
learning games, reflective exercises
After Learning:
• Design an end of course summative evaluation to evaluate how successful each learner is at meeting learning objectives
Evaluation Best Practices & Tips
28
• Make sure you’re measuring learner satisfaction AND engagement/relevance to the job
• Incorporate statements/questions that are learner centric• Consider using a 7, 9, or 11 point rating scale to provide a
richer level of feedback• Don’t jump to conclusions!
Recommended Resources
29
• Evaluation websites• Books/reference material• Kirkpatrick Evaluation LinkedIn
group• Training and certification• Industry and best practice
research sites• Limestone Learning blog
Questions?
30
Follow-up
31
• You will receive thewebinar recording & slides
• Post-webinar satisfaction survey
• Further questions – don’t hesitate to contact us.
