evaluation proposal: leadership and management training ... · pdf fileevaluation proposal:...

Evaluation Proposal:

Leadership and Management

Training Evaluation

Rapid Learning Institute Platform Training

999 Anywhere Someplace, USA

p. 999-999-9999 f. 111-111-111

Darlene [email protected]

[Web address]

EVALUATION PROPOSAL: LEADERSHIP AND MANAGEMENT TRAINING EVALUATION - [SELECT DATE] 1

Executive Summary

Training Need

ARC Industries is a customer service based organization that is geographically disbursed across North

America. The business goal of the organization is to leverage employee talent and growth through improved

leadership and management. Prior Annual Manager Satisfaction Surveys given to employees relative to the

Leadership and Management support of management staff has yielded a need for improvement in these areas.

The organization is seeking a cost-effective training platform solution to encourage the development of

Leadership and Management Skills among their management levels at various locations.

Highlights – Training Need

The results of prior Annual Manager Satisfaction Survey Results correlated to business growth reveal a need

for Leadership and Management Training to support the increases to staffing. The summary of results support

that as contract sales have increased, employee’s satisfaction with management has decreased. Additional

recent analysis has also noted a trend in turnaround of employees that correlates to business growth and low

manager satisfaction survey results.

Objectives

Improve employee satisfaction with managers by inspiring Leadership and Management Skills in

Managers at all levels.

Improve employee retention levels through managerial support

To provide a cost effective Leadership and Management training solution for managers that can be

distributed among diverse geographical locations.

Evaluation Goals

This proposal conveys an evaluation strategy to assess the benefit of a training platform for topics of Interest,

these being “Building Trust & Credibility” and “Communication’. Results of this proposed analysis should

yield the following goals for future use of this training platform:

Level 1 Satisfaction

Learner Satisfaction of Training Content in the 90th percentile or higher

Level 2 Learning

Comprehension of training content with mastery noted by a score of 80% or higher for each training

module.

Level 3 Behavioral Transfer of Training to evaluate a training platform

A minimum of 5% increase of noted behavior of training participants over the control group

An improvement of 5% of frequency of behaviors for training participants over the control group.

Identification of additional training needs in response to surveys and comments of support staff and

peers.

Level 4 Benefits to the Organization

A conveyance of reported improvements to behavior as a result of training.

An increase in employee satisfaction with management as noted by comparison of the current and the

prior year Arcs Annual Manager Satisfaction Survey


A higher retention rate of personnel as indicated by comparison of the prior years data and the control

group compared to training participants

Training Audience and Scope

ARC has Training Managers strategically placed in offices at various parts of the U.S. and Canada. The

company employs a total of 1862 employees and 400 management staff at diverse levels and locations.

Management Staff minimally possess a 4-year college education in Management and some are seasoned to

higher degrees and project management certifications. Given the geographic disbursement of management

relative to training staff it was noted that training would need to be provided via an online forum and that

training would need to provide a consistent message and timing to determine the potential effects on the rate

of investment (ROI) to support this type of training to stakeholders. In addition, a cost effective solution

would consist of a per license investment of a training platform that supplied expert content relative to

Leadership and Management. The training considered in this evaluation is realized from an online training

platform provided by the Rapid Learning Institute. The training product the company provides seems to be an

ideal solution that can be managed under the guidance of existing Training Managers. A pilot study is

proposed using only a sample of the Leadership Training provided by the Rapid Learning Institute. The focus

of this analysis will be employed for the first two learning goals of the program with a focus on “Building Trust

and Credibility” and “Communication”.

A full training evaluation is necessary to determine if the initial Leadership training provided will meet the

organizational needs of Leadership and Management and to determine if a blended learning strategy may be

employed via internal trainers to complement any gaps to the online training support system. A group of

managers will be used for the pilot studies for a 6 – 10 month period. A sample of managers from each training

location will be used as a control group for long-term comparison of realized business goals influenced by

training.

Participants

Eighty managers have been selected for participation in the study, half will participate in the training and the

other half will serve as a control group. The participants selected for this pilot analysis have been selected

based on the following criteria:

Have been in a managerial role in the company for a minimum of three years

Have yielded management satisfaction rates in line with the majority of organizational results from the Annual Survey.

Have worked with the same unit of support staff for over a 1-year period.

Rapid Learning Institute Product Overview

The Rapid Learning Institute provides fee paid instruction for Leadership and Management Training. Each

module is conveyed in small learning chunks of 10 minutes or less. The modules are self- paced and a failed

quiz can be repeated. Each license is purchased for a low cost individual fee and training managers can keep


track of online use and results. Trainees can revisit any of the training modules, as their needs require. This

availability provides a robust environment for reinforcement through the self-regulation of trainee needs and

goals once formal required training sessions are completed.

Training can be self-completed or in combination with internal corporate training staff to facilitate the

complementary training support material the Rapid Learning Center provides for Internal Training Staff. This

content includes the following for each Core subject module: A Discussion Guide, An Interval Reinforcement

Plan, a replica of the online quiz with an answer key, and an online resource for the manual entry of scores for

retake testing or other deemed training needs. In addition, the online training system maintains data tracking

of training for the organization to include:

E-Learning Module Usage Summary by User

E-Learning Module Usage Summary by Program

Access History

Assigned Training Tracking

Test Scores per Module

Each of these reports is available for online access and export to a printed report.

Rapid Learning Institute Information

About: http://rapidlearninginstitute.com/

Training website: http://my.rapidlearninginstitute.com/leadership/customer/account/login

Access Codes/ User Name: [email protected] Passcode: fkurjack

Process Overview The following evaluation strategy uses the widely accepted evaluation methods guided by Kirkpatrick’s Four Level

Evaluation Model. For the purpose of this pilot evaluation Levels 1 and Level 2 evaluation method are initially employed

within the formal training period, and respectively represent Trainee Reactions to the Training and Learning Evaluations

(Kirkpatrick & Kirkpatrick, 2006). Level 3 surveys of behavior will be employed longitudinally over the span of 8

months. A control group of managers will also be utilized for the Level 3 surveys to eliminate any bias that results from

the organizational culture or unanticipated external influences. The goal of having a control group is to isolate the effect

of training. Level 4 evaluations will begin at 10 months post-training and will consist of external data such as employee

retention rates, an annual satisfaction survey and correlation to Level 3 data. A detailed Evaluation Process plan can be

noted in Appendix F.

Measurement Instruments and Administrative

Procedures

Level 1 and Level 2 Evaluation Procedures

http://my.rapidlearninginstitute.com/leadership/customer/account/login


The Initial Pilot of the Leadership and Management Training Product that The Rapid Learning Institute provides will

consist of learning modules for “Building Trust and Credibility” and “Communication”. Training will take place via two

forums of delivery over the course of 2 weeks for each training topic, resulting in 4 weeks of blended training. Initially,

training modules will be assigned in a manner that distributes the learning content evenly over the course of a week. Due

to the nature of a Pilot Training Program, initial online training will take place in a designated training area within the

organization and directed by the guidance of the training team. The Week 2 training will be conducted in house

conducted by trainers on staff. All trainers will have been updated to the Materials and Training Content offered by the

Rapid Learning Institute for the purpose of this trial.

Level 3 and Level 4 Evaluation Procedures

Level 3 Evaluations will consist of a long-term evaluation plan beginning 4 months post training with replica

assessments 8 months post training. Assessments for all trainee participants and the control group will be given to

support staff and supporting managers. Standard practice of evaluating behavior recommends the use of control groups,

the time allowance or behavioral change to occur, repeat evaluations and surveys of subordinates and supervisors

(Kirkpatrick & Kirkpatrick, 200, p. 53). The analysis will assess the frequency and noted improvement of Level 3

Behaviors to ascertain transfer of training to workplace behaviors relative to the training topics of “Building Trust and

Credibility” and “Communication”.

The accumulation of Level 3 data will be interpreted and extend to Level 4 evaluations to determine the organizational

benefit of training as noted by behaviors highlighted in these surveys representative of training content (Kirkpatrick &

Kirkpatrick, 2006, p.65) .In addition, the standard annual survey will be deployed at the 10 month mark of post training

and compared to prior years.

Data Collection Level 1 and Level 2

Upon conclusion of each module summarized in Appendix A Table 1, a Level 2 learning assessment will be provided via

the online training and scores will be maintained to include the name and group associated with the trainee. Training

assessments can be found in Appendix A Table 2. Upon conclusion of each training module, a facilitator will guide

trainees to a quick Level 1 Satisfaction assessment. The assessment will be obtained via online manual input and stored

in an internal database. The data for Level 2 assessments for each module will be stored by the Rapid Learning Institute

and be available for export into a database.

Week 2 will consist of in-house training sessions led by Training managers. Training managers will employ the tactics

dictated by the Trainer’s Guide (Your Interval Reinforcement Plan (IRP) in Appendix C for each learning module

associated with the topics of “Building Trust and Credibility” and “Communication”. In house face-to-face module

training will be evenly balanced over the course of a week. The face to face assessments will mimic Level 2 assessments

taken via the online learning (See Appendix A) and an overview of additional supporting material will assist in the

learning to include the creation of a Personal Action Plan and Implementation Log noted in Appendix C for each

training topic module included under the topics of “Building Trust and Credibility” and “Communication”.

All level 1 Reaction data will be collected in an anonymous fashion to allow for honest feedback and collected

immediately upon conclusion of a module to ensure 100 percent of responses are obtained and utilize a Likert scale for

quantifiable measurement (Kirkpatrick & Kirkpatrick, 2006, pp. 28, 58).


Data Collection Level 3 and Level 4

Level 3 Behavioral data will be collected from both support staff and supporting managers of both training participants

and the control group via a mandatory anonymous online survey obtained in the local training center. Surveys will utilize

a Likert scale for quantifiable measurement and results will not be shared with managers, as the goal is to measure the

effectiveness of training, and encourage honest responses from those surveyed (Kirkpatrick & Kirkpatrick, 2006, p. 28).

Each survey will convey the manager’s name and date of survey. Surveys will be administered at 4 months, and repeated

at 8 months post-training to allow for the opportunity of behavioral change. Level 3 surveys can be found in Appendix

B2. External data will be collected 9 months post training via the online training platform provided by the Rapid

Learning Institute. Data collected will reflect the following: E-Learning Module Usage Summary by User and Program

and access history of managers. This data will serve as an objective measure to demonstrate the independent behaviors

of trainees to seek out training resources provided by this learning platform. Other data collected at the 10 month mark

will consist of ARCs Standard Manager Annual Satisfaction Review which will be administered to the support staff of

training participants and the control group.

All data collection methods can be visualized via the timeline represented by the Gantt Chart noted in Appendix F. Evaluation Instruments Level 1 and Level 2

Level 1 data will be obtained using a 5-point Likert rating system. The 5-point Likert rating system will provide an

appropriate range of responses relative to the learning content, materials and relevance of the training to professional

needs. Each question also contains a section for optional comments or suggestions. See sample Level 1 assessments in

Appendix A for trainees and Appendix B to include the evaluation of face-to-face training which considers the trainee

feedback relative to trainers.

Level 2 assessments gained face to face will be compared with the results of online assessment (see Appendix D). The

results of this comparison will determine if there was any learning improvement relative to the additional face-to-face

training. If the additional training involvement by training staff proves beneficial to reinforcement and learning, this type

of training may be televised via communication systems to remote locations. If any gaps are realized relative to the

online training platform, supplemental training may be recommended. .

Evaluation Instruments Level 3 and Level 4

Level 3 Behavioral Data collected from surveys from both supporting staff and support managers from the 4-month and

8-month timeframes will be interpreted via a 4-point Likert rating and results will be separated according to the topics of

“Building Trust and Credibility” and one for “Communication”. The rating will utilize the following point system based

on the results: Always = 4 point, Often = 3 points, Occasionally = 2 points, Seldom = 1 points and Never = 0.

Comments received relative to specific behaviors for both training participants and the control group will be collected

for each item number and placed into the classifications of “Building Trust and Credibility” and “Communication. In

addition, items that have been checked as “noted improvement” will be recorded for each item number and classified

according to training participants and control groups and sub classified under the headings of “Building Trust and

Credibility” and “Communication.

The accumulation of Level 3 data will be interpreted and extend to Level 4 evaluations to determine the organizational

benefit of training as noted by behaviors highlighted in these surveys representative of training content (Kirkpatrick &

Kirkpatrick, 2006, p.65 . Level 3 assessments will also obtain comments relative to items for each topic of “Building

Trust and Credibility” and “Communication” and any noted improvements.


Level 4 assessments will also utilize external data, such as employee retention rates over the 4 month and 8 month

assessment periods that correspond to the timeline for delivery of Level 3 evaluations. These will be compared to

retention rates associated with the staff of training participants and the control group prior to training. The goal of

correlation to external retention rate data and noted results of Level 3 evaluations will provide some tie in to the

monetary benefits of training to the organization relative manager satisfaction rates and retention rates of supporting

staff. In addition, the ARCs Standard Manager Annual Satisfaction Review .will be deployed to support staff at the 10-month

mark of post training and compared to prior years. This data will isolate training participants separately and compare

results to those selected for the control groups. This final Level 4 analysis will not reflect the results of the entire

organization but will have sole focus on accessing previous data of those managers selected to participate in this study.

External data will also evaluate the reference to training modules by managers over the 9 month post-training to

determine the following:

E-Learning Module Usage Summary by User

E-Learning Module Usage Summary by Program

Access History

This latter data will serve as an objective measure and tie into the value of the training itself from the perspective of the

managers and their motivation to improve workplace behaviors.

Reporting Process

Level 1 and Level 2

The Level 1 Reaction data of the pilot group will be averaged into a score that is an accumulation of module scores for

each main learning topic. Therefore, there will be an accumulated rating for the topic of “Building Trust and Credibility”

and one for “Communication” reported using an average of the accumulated ratings of the pilot group for each learning

module. The accepted standard will consist of a score of 4.0 or above in order to validate further use of the training

platform. Level 2 Learning Evaluations will be averaged in the same manner as Level 1 data and the accumulated

averages will consist of reporting a total cumulative grade for the Pilot Group relative to the training topics of “Building

Trust and Credibility” and “Communication”. The standard acceptable grade for the group for each of these two

categories will be a score of 80-100 in order to validate further use of the training platform. Any inconsistencies or

anomalies resulting from the data will require the direction of deeper analysis before a final decision is made.

Level 3 and Level 4

Level 3 Behavioral Data collected from surveys from both supporting staff and support managers from the 4-month and

8-month timeframes will be interpreted via a 4-point Likert rating and results will be separated according to the topics of

“Building Trust and Credibility” and one for “Communication”. Data will be averaged for each topic category for both

training participants and the control group. A comparative analysis of behavioral indicators of frequency for training

participants relative to the control group will be compared for each item number associated to a topic in the categories

of “Building Trust and Credibility” and “Communication” for identification of behavioral improvements that may

resulted from training. Any percentages of improvement relative to training participants compared to the control group

will be highlighted for validity, where an improvement of 5% or higher will yield significance to the benefit of training

and convey to Level 4 Results. Results consistent to both groups will be negated from Level 4 results but may serve to

deeper analysis of Level 3 results. Any inconsistencies or anomalies resulting from the data will require the direction of

deeper analysis.


Comments received relative to specific behaviors will be documented and evaluated for consistencies within training

participants and between the control group. Positive and negative comments relative to each associated item number will

be ranged for frequency, and classified as positive or negative into the classifications of “Building Trust and Credibility”

and “Communication. In addition, items that have been checked as “noted improvement” will be classified according to

frequency of responses and classified according to Training Participants and the Control Group. An item-by-item

analysis will compare the results between training participants and the control group. Areas of consistency between the

two groups will be negated as a benefit of training. Deeper investigation may pursue to yield the results of consistent

improvement between training participants and the control group. Areas of noted improvement in the training group

that yield a minimum of a 5% increase of responses of “noted improvement” over the control group will be attributed

to training and convey to Level 4 results.

Level 4 analysis also seeks to utilize external data of retention rates as correlates to pre-training and post training at the 4

and 8-month interval for training participants and the control group. Data collected for each group will be averaged for

the categories of: Control Group pre-training retention average, Training participants Pre-training retention average,

Control Group 4 month Post Training Retention Average, Training Participants 4 month Post Training Retention

Average, Control Group 8 month post training Retention Average, Training Participants 8 month post training retention

average. Retention rate data for both the control group and training participants will be correlated to pre-training 4

month post training and 8-month post training data. See Appendix E for sample proposed reports of Level 3 and Level

4 Analysis.

Additional Level 4 reporting will result from data collected via the redistribution of a replica of ARCs Standard Manager

Annual Satisfaction Review. The reported results will demonstrate the opinions of support staff relative to the control

group participant’s and training participants to determine any differences between these two groups relative to the

categories of Leadership and Management that may have resulted from the training when compared to the report of the

prior year for each manager participating in the study. Data will be accumulated and averaged for both the control group

and training participants. The results of the two groups will demonstrate comparison of improvements of current and

the prior year within and between groups. Reporting samples can be visualized in Appendix E. A final report will be

drafted to reflect findings of the full evaluation of all Levels 1-4, for stakeholder review.


REFERENCES:

Kirkpatrick, Donald; Kirkpatrick, James (2006). Evaluating Training Programs: The Four Levels. Retrieved from

http://www.eblib.com

Rapid Learning Institute. (n.d.). Retrieved September 1, 2015, from Rapid Learning Institute Corporate Website:

http://rapidlearninginstitute.com

http://www.eblib.com/


Appendix A: Measurement Instruments Online Learning


LEVEL 1 – SATISFACTION Please complete this form and provide your input to assist us to evaluate our efforts. Any comments and suggestions you provide will better assist us to meet your needs in future training programs.

State Your Title/Position in the company: [ ]

CONTENT OBJECTIVES:

How satisfied are you with the content of this training module to

meet your training needs.

Very

Satisfied

Satisfied Neutral Dissatisfied Very Dissatisfied

4 3 2

COMMENTS:

How satisfied are you that the training content provided in this

training module will assist you in performing your job.

Very

Satisfied


4 3 2

COMMENTS:

How satisfied are you with the content as it relates to the stated

learning objectives?

Very

Satisfied


4 3 2

COMMENTS:

RELEVANCE


How satisfied are you with the relevance of the course content to

convey to the tasking needs of your job?

Very

Satisfied


4 3 2

COMMENTS:

How satisfied are you that the objectives stated in this course are

relevant to the tasking needs of your job?

Very

Satisfied


4 3 2

COMMENTS:

How satisfied are you that the skills you learned in this training can

be applied to your job?

Very

Satisfied


4 3 2

COMMENTS:

MATERIALS

How satisfied are you with the materials (handouts and summary

sheets) you were provided for use in this training?

Very

Satisfied


4 3 2

COMMENTS:


LEVEL 2 – LEARNING

Table 1: Overview of The Rapid Learning Website learning modules

Projects Stages Individual Module Summary Learning Content Trainees Materials for Meeting Leaders

Building Trust and Credibility

Leadership Credibility 1: The Confidence Base

What does it take to be a credible leader? Is it charisma? Raw intelligence? Some intangible trait that some people are born with and others are not? The fact is, credibility often boils down to one key factor. In this program you’ll discover what that is and learn: Why “leadership skill” is NOT what gets people promoted into management, the number one source of credibility for leaders, the key to increasing, and sustaining, your credibility as a

Online Instruction

Online Quiz

Support Material & Handouts

Handouts

Summary Sheet

Personal Action Plan & Implementation Log

Discussion Guide

Interval Reinforcement Plan (IRP)

Quiz Answer Guide (Print Version)

Track Training / Enter Quiz Scores

How satisfied are you that the training materials reflected the needs

of the training objectives?

Very

Satisfied


4 3 2

COMMENTS:

COMMENTS:

Please provide your overall feedback or suggestions (if any)



leader and just how fragile credibility can be.

Leadership Credibility 2: The Fallibility Paradox

Leadership credibility takes a long time to build but can be destroyed in seconds. The question is, how and why does it happen? What should managers do – and never do – to maintain a strong reputation as a trustworthy, reliable leader? In this Quick Take, you will learn: One of the most common, and most preventable, mistakes leaders make that destroys their hard-earned credibility, what we mean by the Fallibility Paradox – and why it’s the key to maintaining credibility over the long-term, and what caused one leadership success story to end in abject failure.

Online Instruction

Online Quiz


Handouts


Quiz (Print Version)

Summary Sheet

Discussion Guide Interval Reinforcement Plan

(IRP) Quiz Answer Guide (Print

Version)


Six Managerial Styles You Need to Lead Effectively

The best managers realize that to be a truly effective leader you need to deploy a variety of management styles depending on who you're dealing with and what you're trying to accomplish. They know that a "one size fits all" approach to leadership simply doesn't work. In this Quick Take you’ll learn: Why “one-trick-pony” managers have limited value to the organization, that great managers master a repertoire of six management styles and, how to correctly deploy the right style depending on the situation.

Online Instruction

Online Quiz




Handouts

Summary Sheet

Discussion Guide






The Power of Predictability

The #1 job of the human brain is to predict what will happen, so that we can be prepared. But managers can unintentionally create a work environment where unpredictability reigns. When that happens, employees feel stressed, anxious and demotivated. In this Quick Take you will learn how the brain creates “predictive models” that are designed to tell us what’s going to happen, why we feel good when our predictions are right – and bad when they’re wrong and how you can use the principle of predictability to create a more productive and positive work environment for your people.

Online Instruction

Online Quiz




Handouts

Summary Sheet



Discussion Guide




Communication Communication: How to Get Buy-In for Change

What’s the best way to introduce a potentially disruptive change? Should you announce the change and immediately implement it? Announce the change and delay the implementation? Or Announce the change and gradually phase it in? In this Quick Take, you will learn a proven way to introduce and implement disruptive change, why this method will maximize employee buy-in, and the psychology behind people’s acceptance of change. .

Online Instruction

Online Quiz


Handouts

Summary Sheet


Discussion Guide




Persuasion: Getting Employees, Bosses and Colleagues to See the Light

You can’t be a successful leader without mastering the art of persuasion. You have to get subordinates, peers, and even bosses to see things your way – that is, the way you believe best serves organizational goals. But it’s hard. People see things differently than

Online Instruction

Online Quiz


Handouts

Summary Sheet

Discussion Guide






you do. In this Quick Take you will learn the most persuasive way to communicate hard truths to employees, colleagues and senior management, how to present evidence in a way that’s less likely to lead to a debate, and why we’re more persuasive when we communicate in the brain’s “native language.”


Communication: How to Be an Idea Catalyst, Not an Idea Killer

Leaders get where they are by being competent and confident. In certain situations projecting confidence and authority is necessary -- for example, in a crisis, when a time-sensitive project is behind schedule, or when rallying the troops behind a new initiative. But sometimes too much confidence can backfire, by shutting down contributions by the people on your team. In this Quick Take, you will learn when it’s important for a leader to appear capable and confident and when it’s important to dial it back

Online Instruction

Online Quiz


Handouts

Summary Sheet


Discussion Guide




The FACE Method: How to Ensure Crystal Clear Communication

How is it that two people can walk away from the same conversation with a completely different understanding of what was said? The reason is that we too often hear what we want to hear and assign our personal expectations and meaning to conversation. Fortunately, there’s a way to avoid this kind of frustrating – and sometimes costly - miscommunication. In this Quick Take you will learn the #1 reason miscommunication occurs between managers and employees, and the

Online Instruction

Online Quiz


Handouts

Summary Sheet


Discussion Guide






FACE Method – a simple approach you can use to gain clarity with your employees and colleagues.

Handling Excruciatingly Difficult Conversations

A lot of people love playing a leadership role until … the day comes when there’s a problem in the organization and somebody needs to sit down with an employee and have a really difficult conversation – say, for example, he or she has terrible body odor that coworkers complain about. You don’t want to do it. You’re looking up and down the ranks to find someone else who’ll break the news. But guess what? They’re all looking at YOU. This Quick Take will give you a template you can apply whenever you have to confront an employee about a performance problem, the use of foul language, inappropriate physical contact, excessive use of perfume, unconscious rude behavior, or a host of other problems.

Online Instruction

Online Quiz


Handouts

Summary Sheet


Discussion Guide




Fact-Based Decision-Making: The Five Whys Technique

Managers get paid to make difficult decisions – about hiring, firing, promotions, salaries, resource deployment, operations, new products, new markets and more. A decision-maker’s worst enemy is lack of information or inaccurate information. In this Quick Take you will learn the Five Whys technique that can help you quickly get the information you need to make fact-based decisions, what it means to have a “threaded conversation”, and

Online Instruction

Online Quiz


Handouts

Summary Sheet


Discussion Guide






how the Five Whys can make you a better manager.

Table 2: Overview of The Rapid Learning Website learning objectives and assessment per module

Training Focus Learning Module

Assessment

Answer Key

Building Trust

and Credibility

Leadership

Credibility 1: The

Confidence Base

1. B

2. False 3. D 4. C 5. B 6. False 7. C



Assessment

Answer Key

Building Trust

and Credibility

Leadership

Credibility 2: The

Fallibility

Paradox

1. A 2. FALSE 3. D 4. C 5. A 6. TRUE

Building Trust

and Credibility

Six Managerial

Styles You Need

to Lead Effectively

1. A 2. B 3. FALSE 4. B 5. D 6. C 7. TRUE



Assessment

Answer Key

Building Trust

and Credibility

The Power of

Predictability

1. A 2. A 3. C 4. B 5. D

Communication

Communication:

How to Get Buy-

In for Change

1. B 2. A 3. D 4. C 5. B



Assessment

Answer Key

Communication

Persuasion:

Getting

Employees, Bosses

and Colleagues to

See the Light

1. C 2. B 3. B 4. D 5. A

Communication

Communication:

How to Be an Idea

Catalyst, Not an

Idea Killer

1. C 2. FALSE 3. B 4. FALSE 5. F



Assessment

Answer Key

Communication

The FACE

Method: How to

Ensure Crystal

Clear

Communication

1. C 2. A 3. D 4. C 5. B 6. A 7. B

Communication

Handling

Excruciatingly

Difficult

Conversations

1. C 2. A 3. D 4. C 5. B 6. A 7. B



Assessment

Answer Key

Communication

Fact-Based

Decision-Making:

The Five Whys

Technique

1. TRUE 2. B 3. FALSE 4. C 5. TRUE 6. B 7. B


Appendix B: Measurement Instruments Face to Face Learning


LEVEL 1 ONSITE FACE TO FACE TRAINING Please complete this form and provide your input to assist us to evaluate our efforts. Any comments and suggestions you provide will better assist us to meet your needs in future training programs.

State Your Title/Position in the company: [ ]

CONTENT OBJECTIVES:

How satisfied are you with the content of this training module to

meet your training needs.

Very

Satisfied


4 3 2

COMMENTS:

How satisfied are you that the training content provided in this

training module will assist you in performing your job.

Very

Satisfied


4 3 2

COMMENTS:

How satisfied are you with the content as it relates to the stated

learning objectives?

Very

Satisfied


4 3 2

COMMENTS:

RELEVANCE


How satisfied are you with the relevance of the course content to

convey to the tasking needs of your job?

Very

Satisfied


4 3 2

COMMENTS:

How satisfied are you that the objectives stated in this course are

relevant to the tasking needs of your job?

Very

Satisfied


4 3 2

COMMENTS:

How satisfied are you that the skills you learned in this training

can be applied to your job?

Very

Satisfied


4 3 2

COMMENTS:

MATERIALS

How satisfied are you with the materials (handouts and summary

sheets) you were provided for use in this training?

Very

Satisfied


4 3 2


COMMENTS:

How satisfied are you that the training materials reflected the

needs of the training objectives?

Very

Satisfied

Satisfied Neutral Dissatisfie

d

Very Dissatisfied

4 3 2

COMMENTS:

FACILITATOR

How satisfied are you that the facilitator clearly conveyed the

training objectives.

Very

Satisfied


d

Very Dissatisfied

4 3 2

COMMENTS:

How satisfied are you that the facilitator of this training assisted

you in the creation of a Personal Action Plan & Implementation

of that plan?

Very

Satisfied


d

Very Dissatisfied

4 3 2

COMMENTS:


How satisfied are you that the facilitator of this training clearly

communicated training content?

Very

Satisfied


4 3 2

COMMENTS:

Please provide your overall feedback or suggestions (if any)


Appendix B2: Measurement Instruments Level 3 Behavioral Transfer

Manager Name: Date: / /

Leadership/Management Questionnaire (Support Staff) The purpose of this survey is to evaluate the effects of training. The survey is anonymous and results will not be shared with the manager participating in this study, so please be frank in your answers. Read each item carefully. Reflect on the behavior described. Decide what level of frequency your manager demonstrates for each item since your last annual assessment. Place an (x) under your corresponding choice. If you have noticed any improvement in these areas place an (x) under the column that indicates noted improvement. If you have any comments or suggestions relative to an item listed, please indicate these below in the comments section.

OBSERVED FREQUENCY

Has demonstrated improvement

in this area (note with an

“ x “).

Alw

ays

Oft

en

Occ

asio

na

lly

Seld

om

Ne

ver

Communication

1 Demonstrates clear communication of expectations during transition and or change.

2 Communicates in a convincing manner.

3 Clearly communicates ideas, solutions or crisis.

4 Ensures clarity of meaning in discussions to avert miscommunication.

5 Handles difficult conversations relative to behavior or performance with tact and clear resolution.

6 Communicates decisions clearly with supporting justification.

7 Clearly communicates directions and/or expectations.

Credibility

8 Demonstrates leadership during transition or change of policies, procedures or tasks

9 Demonstrates expertise in area(s) relative to his/her managerial role.

10 Delegates duties or questions to those with matched expertise.

11 Demonstrates effective management of his/her staff.

12 Creates a predictive and positive work environment.

13 Provides prompt resolutions to staff member concerns.

COMMENTS

Relative to any of the numbered items above, please provide any comments or suggestions you have relative to your experience (s) with this manager.

1

2

3

4

5

6

7

8

9

10

11

12

13

ADDITIONAL COMMENTS


Manager Name: Date: / /

Leadership/Management Questionnaire (Supporting Manager Assessment) The purpose of this survey is to evaluate the effects of training. The survey is anonymous and results will not be shared with the manager participating in this study, so please be frank in your answers. Read each item carefully. Reflect on the behavior described. Decide what level of frequency your manager demonstrates for each item since your last annual assessment. Place an (x) under your corresponding choice. If you have noticed any improvement in these areas place an (x) under the column that indicates noted improvement. If you have any comments or suggestions relative to an item listed, please

indicate these below in the comments section.

OBSERVED FREQUENCY

Has demonstrated improvement

in this area (note with an

“ x “).

Alw

ays

Oft

en

Occ

asio

na

lly

Seld

om

Ne

ver

Communication

1 Demonstrates clear communication of expectations during transition and or change.

2 Communicates in a convincing manner.

3 Clearly communicates ideas, solutions or crisis.

4 Ensures clarity of meaning in discussions to avert miscommunication.

5 Handles difficult conversations relative to behavior or performance with tact and clear resolution.

6 Communicates decisions clearly with supporting justification.

7 Clearly communicates directions and/or expectations.

Credibility

8 Demonstrates leadership during transition or change of policies, procedures or tasks

9 Demonstrates expertise in area(s) relative to his/her managerial role.

10 Delegates duties or questions to those with matched expertise.

11 Demonstrates effective management of his/her staff.

12 Creates a predictive and positive work environment.

13 Provides prompt resolutions to staff member concerns.

COMMENTS

Relative to any of the numbered items above, please provide any comments or suggestions you have relative to your experience (s) with this manager.

1

2

3

4

5

6

7

8

9

10

11

12

13

ADDITIONAL COMMENTS


Appendix C: Sample Face to Face Training Material


Appendix D: Reporting Samples Levels 1 and 2

0

1

2

3

4

5

6

Content Materials Relevance

Level 1 Reaction Data Online Training

Building Trust & Crediility Communication

0

1

2

3

4

5

6

Online Face to Face

Level 1 Reaction Results



’

80%

82%

84%

86%

88%

90%

92%

94%

96%

98%

Online Level 1 Face to Face

Level 2 Learning Results



Appendix E: Reporting Samples Levels 3 and 4

4.2

54.8

4

0

1

2

3

4

5

6

Online Face to Face

Level 3 Behavioral Data Training Participants


0

1

2

3

4

5

6

Online Face to Face

Level 3 Behavioral Data Control Group



0

1

2

3

4

5

6

7

8

Supporting ManagerCredibility

Supporting ManagerCommunication

Support Staff Credibility Support StafCommunication

Level 3 Behavioral Data - Frequency and Improvement

Survey Data Frequency Average Noted Improvement Column1

0

1

2

3

4

5

6

Control Group Training Participants

Retention Rates

Pretraining 4 months post training 8 Mos Post-training


0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Support Manager Credibility Support ManagerCommunication

Support Staff Credibility Support StaffCommunication

Level 3 Behavioral Survey - Comments

Positve Negative


Leadership Nonparticipants

Leadership PilotTraining

Management Nonparticipants

Management PilotTraining

ARCS Annual Manager Assessment

2014 2015


Appendix F: Planning Charts

Evaluation Plan for Leadership and Management Training

Project Scope: 1) Building Trust and Credibility 2.) Communication

Goal Purpose Analysis Level Objectives to Measure Trainee satisfaction Level 1 Overall satisfaction of training content, delivery and

facilitation (instructor and online).

Learning Comprehension Level 2 Comprehension of training content.

Transfer of Behavior Level 3 Influence of training on identified behaviors associated to training.

Results of Training Level 4 The benefit of training vs no training on behavior, employee satisfaction and retention rates.

Process & Data Collection

Facilitator Steps Time Range (from Start)

Ferri-Kurjack Finalize Evaluation Plan with Stakeholders Week 1

Training Team Conduct online training session – “Building Trust and Credibiity”

Week 2

Training Team Collect Level 1 and Level 2 data - “Building Trust and Credibiity” – online training

Week 2

Training Manager Conduct face to face training - “Building Trust and Credibiity”

Week 3

Training Manager Collect Level 1 and Level 2 data - “Building Trust and Credibiity” – face to face training

Week 3

Training Team Conduct online training session – “Communication” Week 4

Training Team Collect Level 1 and Level 2 data - “Communication” – online training

Week 4

Training Manager Conduct face to face training - “Communication” Week 5

Training Manager Collect Level 1 and Level 2 data - “Communication” – face to face training

Week 5

Training Team Collect Level 3 Data 4 month post-training (subordinate staff and supporting manager).

Week 21

Training Team Collect Level 3 Data 8 month post-training (subordinate staff and supporting manager).

Week 37

Ferri-Kurjack Analyze data Levels 1-3 Week 39-45

Ferri-Kurjack External data collection –retention rates prior and post training

Week 41

Ferri-Kurjack Analyze retention rate data Week 46-48

Training Team Deploy ARCs Standard Manager Annual Satisfaction Review (support staff of study participants)

Week 45

Training Team Collect data resulting from the ARCs Standard Manager Annual Satisfaction Review

Week 47


Ferri-Kurjack Analyze data received from ARCs Standard Manager Annual Satisfaction Review

Week 49-51

Ferri-Kurjack Report Findings Week 53

Analysis Target Findings Calculation Method

Eliminating External Influences

Level 1 Learner satisfaction rates of 90% or above

Likert Rating Taken in anonymity

Level 2 Comprehension of training content with mastery scores 80% or higher

Module Assessments Training assessments are scored via online computer and matched to face to face assessments

Level 3 A minimum of 5% increase of noted improvement of behaviors o training participants over the results of the control group

Surveys to subordinates and supporting manager

Level 3 data is obtained in anonymity Results are not shared with managers evaluated Evaluation questions are aligned to training content

An improvement of behaviorial frequency of 5% of training participants over the control group.

Identification of additional training needs in response to surveys and comments of support staff and peers.

Noted use of training content outside of the formal training content

Usage Data collected via Rapid Learning Institute

Reports identifiable usage of training participants

Level 4 A conveyance of reported improvements to behavior as a result of training.

Accumulation and assessment of Level 3 data

Obtained in anonymity not shared with manager study participants

An increase in employee satisfaction with management as noted by comparison of the current and the prior year Arcs Annual Manager Satisfaction Survey

Arcs Annual Manager Satisfaction surveys (current and prior year) data specific to study participants

Limited focus to study participants.

A higher retention rate of personnel as indicated by comparison of the prior years data and the control group compared to training participants

Retention rate data for all study participants for current (4 & 8 month) post-training to prior training data

Limited focus to study participants.


Report Data

High Level Findings

Stakeholder Interest

Communication of Results

Improved behaviors to support employee satisfaction and retention rates

Employee satisfaction and increases to retention rates during business growth

Finding report for stakeholders Stakeholder meeting to convey preliminary findings

Satisfaction and use of training content

Validity of training through manager buy in and value


ENLARGE TO FULL SCALE

evaluation proposal: leadership and management training ... · pdf fileevaluation proposal:...

Documents