clo webinar: how to measure informal learning and engage managers to optimize on-the-job impact
DESCRIPTION
Informal learning measurement (social networks, performance support, coaching) is on the rise. It is important to think about measuring this investment and comparing it against formal programs to determine its mix and place. In addition, the root cause of low learning impact is lack of manager support. This webinar will address why this happens and how to overcome it. In this webinar, you will:1. Define informal learning and a learning taxonomy to incorporate it.2. Discuss when and how to measure informal learning.3. Understand the reasons for scrap learning and low manager support.4. Discuss tools and methods to increase manager support and optimize learning impact.TRANSCRIPT
You can listen to today’s webinar using your computer’s speakers or you may dial into the teleconference.
If you would like to join the teleconference, please dial 1.866.469.3239 and enter access code 669 720 923.
You will be on music hold until the seminar begins.
How to Measure Informal Learning and Engage Managers to Optimize On-the-Job Impact
#CLOwebinar
Speaker: Jeffrey BerkChief Operating OfficerKnowledgeAdvisors, Inc.
Moderator: Daniel MargolisManaging EditorChief Learning Officer magazine
How to Measure Informal Learning and Engage Managers to Optimize On-the-Job Impact
#CLOwebinar
• Q&A
– Click on the Q&A panel (?) in the bottom right corner
– Type in your question in the space provided
– Click on “Send.”
Tools You Can Use
#CLOwebinar
• Polling
– The poll will appear on the right side of your screen
– Select the best option for each question
– Click on “Submit”
#CLOwebinar
Tools You Can Use
Frequently Asked Questions1. Will I receive a copy of the slides after the webinar?
YES
2. Will I receive a copy of the webinar recording?
YES
Please allow up to 2 business days to receive these materials
#CLOwebinar
#CLOwebinar
Daniel MargolisManaging EditorChief Learning Officer magazine
How to Measure Informal Learning and Engage Managers to Optimize On-the-Job Impact
#CLOwebinar
Jeffrey BerkChief Operating OfficerKnowledgeAdvisors, Inc.
How to Measure Informal Learning and Engage Managers to Optimize On-the-Job Impact
Informal Learning and Manager Engagement
Jeffrey BerkChief Operating Officer
KnowledgeAdvisors
Informal Learning Measurement Approaches
8April 18, 2011 © 2010, KnowledgeAdvisors
What is “Informal Learning”Research on Informal Learning tacticsMeasurement in Informal Learning
Informal Learning
9April 18, 2011 © 2010, KnowledgeAdvisors
Informal Learning: Knowledge transfer that occurs without the assistance of structured curriculum.
Source: KnowledgeAdvisors
What Informal Learning is Supported?
10April 18, 2011 © 2010, KnowledgeAdvisors
73% 56% 54% 46% 13%% of respondents supporting each type of informal learning
# of
resp
onde
nts 38
29 2824
7
05
1015202530354045
Mentoring & Coaching (M&C)
Communities of Practice (CoPs)
Virtual Knowledge Sharing (VKS)
Performance Support Systems
(PSS)
Other
Source: KnowledgeAdvisors Informal Learning Research, 2009
How Much is Spent on Informal Learning?
11April 18, 2011 © 2010, KnowledgeAdvisors
% supporting each method
$70K $85K $170K$110KAvgspend
73%
56% 54%46%
13%
0%
10%
20%
30%
40%
50%
60%
70%
80%
Mentoring & Coaching
Communities of Practice
Virtual Knowledge
Sharing
Performance Support Systems
Other
Source: KnowledgeAdvisors Informal Learning Research, 2009
Informal Learning Measurement
12April 18, 2011 © 2010, KnowledgeAdvisors
Are your Execs asking for quantitative measures?
Are you measuring business benefits?
While only 21% of respondents indicated that their Executives are asking for quantitative measures, twice that
number expect to be asked soon.Source: KnowledgeAdvisors Informal Learning Research, 2009
Measurement Correlates to Budget
13April 18, 2011 © 2010, KnowledgeAdvisors
Measurementdiscipline
L&D Spendperson
/ COPs M&C VKS EPSS
Those who measure $958 $168,000 $84,000 $250,000 $250,000 Those who don’t measure $1004 $68,929 $82,679 $97,917 $129,286
Overall $996 $71,042 $83,026 $109,615 $167,750
% delta ( over ) -5% 143% 6% 155% 93%
• The group measuring Informal Learning spends less per person on L&D annually.
• However, their Informal Learning spend is notably higher than the non-measurement group in most areas.
• We might conclude that measuring impact of Informal Learning enables organizations to demonstrate outcomes and justify greater allocation of L&D $ to these critical areas.
Source: KnowledgeAdvisors Informal Learning Research, 2009
Informal Learning Measurement : What?
14April 18, 2011 © 2010, KnowledgeAdvisors
Support Systems (Manager, Resources, Rewards)
Usability
Availability/ Accessibility
Fulfills objectives
Effectiveness
Value / Belonging
Quality
Engagement
Right solution for needs
Business Results / ROI
Performance impact
Application to the job
Professional growth
Solution Experience Benefits
Source: KnowledgeAdvisors Informal Learning Measurement White Paper, 2010
Informal Learning Measurement : When?
15April 18, 2011 © 2010, KnowledgeAdvisors
1. Become aware /
consider
3. Engage
4. Use, learn,
practice, contribute
5. Complete and/or
recommit
At each interaction
Periodically
At the conclusion
2. Search, access, sign in or join
Source: KnowledgeAdvisors Informal Learning Measurement White Paper, 2010
Informal Learning Measurement : How?
16April 18, 2011 © 2010, KnowledgeAdvisors
Systems Based• Web analytics• Supporting system
statistics• Performance data
Tools Based• Needs analysis• Usability studies• Feedback links• Content ratings• Surveys• Assessment
People Based• Observation• Interviews• Focus groups• Case studies
High Tech High TouchSource: KnowledgeAdvisors Informal Learning Measurement White Paper, 2010
Informal Learning Measurement
17April 18, 2011 © 2010, KnowledgeAdvisors
Easy wizard for administration of informal learning measurement
Pop Up Experience
Micro Poll Experience
Robust reports
Manager Engagement Measurement Approaches
18April 18, 2011 © 2010, KnowledgeAdvisors
What is “Scrap Learning”Research from the War on “Scrap”Measurement to Combat the “Scrap”
Scrap Learning
19April 18, 2011 © 2010, KnowledgeAdvisors
60% of learning is not applied to the job…
That’s Scrap. Scrap learning is learning not applied on the job. It is the amount of training that is wasted.
What causes scrap learning?
Source: KnowledgeAdvisors , Metrics that Matter
Causal Factors of Scrap
20April 18, 2011 © 2010, KnowledgeAdvisors
Source: KnowledgeAdvisors , Metrics that Matter
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
content not practical
prevented or discouraged from
using
no opportunity other high priorities
other
Data from thousands of on‐the‐job evaluations showsthe major barriers to impact occur on‐the‐job
This is where the power of training leaks out of the organization
Manager Support is Needed
21April 18, 2011 © 2010, KnowledgeAdvisors
Manager support comes both before and after training. Key steps include:
Evaluate learner readiness
Follow up on expectations
Provide performance resources
Get involved after trainingTraining
Expectation setting
Measurement is a difference maker to achieve manager support and reduce scrap learning!
Research: Learner Readiness
April 18, 2011 © 2010, KnowledgeAdvisors
Only 21% of organizations pre-evaluate learners “most of the time” or “all of the time” prior to sending them to training.
Do you pre-assess learners to determine their readiness for training?
Research: Expectation Setting
April 18, 2011 © 2010, KnowledgeAdvisors
75% of companies indicate that managers set post-training performance expectations with learners less than 25% of the time.
How often do you set expectations prior to training?
Research: Manager Involvement
April 18, 2011 © 2010, KnowledgeAdvisors
A large minority (44%) of managers have little involvement in how their employees use training on the job. Scrap learning can be reduced by getting managers involved.
Research: Manager Follow Through
April 18, 2011 © 2010, KnowledgeAdvisors
Fewer than 35% of managers follow-through to validate that training actually accomplished anything.
Research: Resource Provisions
April 18, 2011 © 2010, KnowledgeAdvisors
Fewer than 25% of managers provide resources to support learners application of training on the job.
Research: Manager Support Summary
April 18, 2011 © 2010, KnowledgeAdvisors
Here are the general estimates of manager support for selected activities:
Evaluate learner readiness
Follow up on expectations
Provide performance resources
Get involved after trainingTraining
Expectation setting
21%
25%
52%
35%
25%
Measurement is a difference maker to achieve manager support and reduce scrap learning!
Measure Learner Readiness Before
28April 18, 2011 © 2010, KnowledgeAdvisors
Source: KnowledgeAdvisors , Metrics that Matter
Implement a pre-evaluation to managers. Keep it simple and concise.
Case: Telecom Co.
When: Strategic, costly programs
How: LMS sends pre-evaluation to manager via integration with Metrics that Matter
Purpose: Used for L&D to obtain ‘right ‘ learners in class and remind managers of their role in the learning process.
Measure Learner Readiness During
29April 18, 2011 © 2010, KnowledgeAdvisors
Source: KnowledgeAdvisors , Metrics that Matter
Implement end of class, follow-up and manager evaluations to critique support tools.
Case: Payroll Processor
When On customer service program
How: Post event evaluations revealed high satisfaction but low manager report
Purpose: Used to convince management the problem wasn’t the training. They got funds to do manager performance coaching as a result.
Sharing between Learner and Manager
30April 18, 2011 © 2010, KnowledgeAdvisors
Answer Sharing
Answer Sharing is to automatically route student responses to theirmanager or coach.
The learner can identify who they would like to route their responses to via an email address.
When the learner completes the evaluation for a training program, then the manager/coach then receives an email with the responses to those taggedquestions.
Goal Setting and Periodic Check Ins
31April 18, 2011 © 2010, KnowledgeAdvisors
Check-in to Change
Program managers can enable and oversee a process over timethat facilitates the collaboration between program participant and manager.Employees are asked to enter goals. These goals are then stored and tracked for both the employee and the manager to review. Automated ‘Check-ins’ occur to facilitate feedback on goal status, the attainability of their goals, and also share possible obstacles and successes amongst learner and manager. This is done via an online portal
Contact InformationFor further information on the information contact:
Jeffrey BerkChief Operating Officer+1 312 [email protected]
Join Our Next CLO Webinar
Preparing Your Workforce for Tomorrow’s Challenges
Thursday, April 21, 2011
CLO Webinars start at 2 p.m. Eastern / 11 a.m. Pacific
Register at www.clomedia.com/events
Join the CLO Network: http://network.clomedia.com/
#CLOwebinar