instructional designprobased.org/wp-content/uploads/2018/12/instruction-design.pdfinstructional...

37
1 INSTRUCTIONAL DESIGN

Upload: others

Post on 17-Oct-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

1

INSTRUCTIONAL DESIGN

Page 2: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

2

Project Based Learning in VET

Intellectual Output No: 4

Instruction Design

Output Coordinator:

Mazhar Zorlu Mesleki ve Teknik Anadolu Lisesi, İzmir- TURKEY 2019

“Funded by the Erasmus+ Program of the European Union. However, European Commission and

Turkish National Agency cannot be held responsible for any use which may be made from the

information contained therein”

This PBL Module Book is just a case study and is based on the "Perception and Modeling Tools"

vocational Lesson.

However, there are other approaches that use the PBL method that the reader can take into

account.

The development of this module book has been possible due to Erasmus+ KA2 program funds, with

2017-1-TR01-KA202-045991 project number.

© All Rights Reserved. - 2019

Page 3: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

3

Contents

ABSTRACT .......................................................................................................................5

1. INTRODUCTION .........................................................................................................6

2. THEORETICAL BACKGROUND ....................................................................................7

INSTRUCTIONAL DESIGN IN THE REAL WORLD ......................................................9

A QUICK GUIDE TO FOUR INSTRUCTIONAL DESIGN MODELS ............................10

ADDIE MODEL ............................................................................................................13

ARCS MODEL ..............................................................................................................14

GAGNE’S 9 EVENTS OF INSTRUCTION ....................................................................18

STEPS OF INSTRUCTİONAL DESİGN ........................................................................22

3. TRAINING EVALUATION ...........................................................................................24

Understanding Kirkpatrick's Four Levels ........................................................................25

Level 1: Reaction ...........................................................................................................25

Level 2: Learning ...........................................................................................................26

Level 3: Behavior...........................................................................................................26

Level 4: Results .............................................................................................................27

4. PRACTICAL ID APPLICATION ..................................................................................28

Page 4: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

4

The information provided in this textbook is intended to be pure guidelines for an Instructional Design

implementation. The practical application documented are mostly based on the project titled “The

Industrial Automation Technology (IAT) Extension Project for Central Asian and Middle East Countries”,

which was jointly implemented among The Turkish Ministry of National Education (MoNE), Turkish

Cooperation and Coordination Agency (TIKA) and Japan International Cooperation Agency (JICA) in

2012-2015.

The objective of this textbook is to offer an easy-to-follow guide that encourages training designers to

plan and conduct evaluation based on their needs.

The development of this textbook has been possible due to Erasmus+ KA2 program funds, with

2017-1-TR01-KA202-04599 project number.

Page 5: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

5

ABSTRACT

Although there is not a magic training matching every learning outcome, by considering

targets of training courses it is possible to define the most proper learning strategy for the

specific learning outcome to be addressed.

Every organization has its own process for developing training courses.

Some organizations prefer to work on the entire training at once, while others prefer to

identify small segments within the training and develop them concurrently through

collaboration. No matter which approach an organization follows, there are a few steps that

remain mandatory in every Instructional Design (ID) process. Understanding the purpose of

each mandatory step will help us successfully achieve the desired output at every step,

irrespectively of our approach!

ID carefully considers how students learn and what materials and methods will most effectively help

individuals achieve their academic goals. The principles of instructional design consider how educational

tools should be designed, created and delivered to any learning group, from grade school students to

adult employees across all industry sectors.

Evaluation of the impact of training often seems second priority – of far less importance than training

design and implementation. But evaluation of the impact never diminishes in importance.

This textbook provides an approach to ID models and to evaluating training using Kirkpatrick’s model of

four levels of evaluation.

Page 6: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

6

1. INTRODUCTION

It must begin with an end in mind

This is a maxim popularized by Stephen Covey, as you might have noticed.

It is the second habit that Stephen Covey covers in “The 7 Habits of Highly Effective People” book.

Did you know that many authors write the endings of books first, before they get going with the

beginning? That is what is known as beginning with the end in mind.

Professional athletes visualize everything they do, before they do it.

Boston Red Sox legend Ted Williams, one of the greatest

hitters of all time, used to practice his swing in the mirror over

and over each day, constantly working on it.

Figure 1: Ted Williams (1918-2002)

Figure 2: Derek Jeter (1974 -..)

When New York Yankees star Derek Jeter was a child, he had a

goal to be shortstop for the New York Yankees, a goal he

achieved when he was 22 years old. They began with the end in

mind.

Page 7: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

7

We borrowed it since it tells us something simple but powerful. The maxim suggests that every design

must achieve something specific—a defined goal.

It often starts with a question: what do I intend to achieve? Or questions that derive from such

question. What should learners know, understand and apply in real-life environment? What will inspire

students to learn and strive for excellence? What student learning outcomes will the prepared material

serve?

Beginning with an end in mind allows us to design an instructional material efficiently—without waste

of time and energy. We no longer need to jump from area to another only to find some pieces missing.

2. THEORETICAL BACKGROUND

DEFINITION

Instruction is a systematic process that involves teacher,

learners, materials, and learning environment in order to

achieve successful and identified learning goals.

Figure 3: Learning Definitions

The “system” refers to an orderly, logical method of identifying, developing, and evaluating a set of

strategies aimed at attaining a particular instructional goal.

Training refer to those instructional experiences that are focused upon individuals acquiring very

specific skills that they will normally apply almost immediately.

Page 8: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

8

The terms Instructional Design, Instructional Technology, Learning Experience Design, Educational

Technology, Curriculum Design are often used interchangeably.

Instructional design (or instructional systems design) models provide frameworks to facilitate

gaining new knowledge, skills or attitudes. Instructional designers use these models to guide

the creation of engaging learning activities based on the science of how people learn. Simply

put, instructional design is the creation of instructional materials.

The foundation for instructional design was laid during World War II when hundreds of thousands

needed to be taught very specific tasks in a short amount of time. Individual aspects of these complex

tasks were broken down, so that soldiers could better understand and comprehend each step of the

process. This approach was later taken and built upon leading to the development of instructional

design, a field of study that marries education, psychology and communications to create the most

effective teaching plans for specific groups of students. This is vital because it ensures that students

receive instructions in a form that is meaningful to them, helping them better understand the topics

and concepts being taught.

Instructional designers carefully consider how learners from all walks of life learn and what materials

and methods will most effectively help individuals achieve their academic goals in a variety of ways.

Generally speaking, an instructional designer’s job is to help ensure that students walk away from a

course with a specific set of knowledge and skills.

They work with traditional paper materials, such as handouts and manuals, as well as

eLearning technologies and multimedia. Their work can be seen in elementary and secondary

schools to universities and adult training facilities. They're also found outside the academic

sector in a range of industries including health care, retail and the military.

Page 9: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

9

INSTRUCTIONAL DESIGN IN THE REAL WORLD

For the corporate sector, instructional design plays an integral role that many don’t often see.

When new training programs are introduced within companies, instructional designers are the

ones that systematically collect, process and analysis data, determining if employees were

properly educated on the new topics introduced. If an area of the training doesn’t meet the

previously set standards, then it’s an instructional designer’s duty to revamp the

lecture/course (face to face or at distant, e-Learning) to help make sure that learners are able

to understand the topics down the road. This process helps ensure that companies are

working efficiently and using their resources wisely.

Instructional designers often work as part of a team, but their importance can't be

overestimated. Consider an eLearning course for example. An instructional designer will play

a part in developing this course, along with a multimedia designer, eLearning developer and a

quality assurance employee.

It ensures students learn efficiently by creating high quality learning materials that consider

the strengths and weaknesses of students.

It allows us to communicate the purpose and reason behind a strategy. A framework gives us

the birds-eye view of all the major components that must be included in the course/lecture.

Above all, instructional design yields result. Those in this field create lesson/course plans

intended to engage students, so they're more likely to achieve their goals. Evaluation is a key

final phase of instructional design implementation, so instructors can ensure that the learning

sessions have been effective in meeting pre-set objectives.

Page 10: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

10

A QUICK GUIDE TO FOUR INSTRUCTIONAL DESIGN MODELS

There are many widely used models that designers acknowledge and use to structure and plan

their lecture/course training. We selected the four of them to give an overview.

1. Bloom’s Taxonomy

2. ADDIE Model

3. ARCS model

4. Gagne’s Nine Events of Instructions

Each framework has its own advantages and disadvantages, and the choice of which to use

will depend on which model works best for the company, and learners.

Bloom’s Taxonomy

Bloom’s Taxonomy is a hierarchical ordering of cognitive skills and

was created by Benjamin Bloom in 1956 who was an educational

psychologist at the University of Chicago, published six levels of

learning. These 6 levels can be used to structure the learning

objectives, lessons, and assessments of the courses.

Figure 4: Benjamin Bloom (1913 -1999)

Page 11: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

11

• Knowing: Retrieving, recognizing, and recalling relevant knowledge from long‐

term memory.

• Understanding: Constructing meaning from oral, written, and graphic messages

through interpreting, exemplifying, classifying, summarizing, inferring, comparing,

and explaining.

• Applying: Carrying out or using a procedure for executing or implementing.

• Analysing: Breaking material into constituent parts, determining how the parts relate

to one another and to an overall structure or purpose through differentiating,

organizing, and attributing.

• Synthesizing: Putting elements together to form a coherent or functional

whole; reorganizing elements into a new pattern or structure through generating,

planning, or producing.

• Evaluating: Making judgments based on criteria and standards through checking

and critiquing.

To create an accurate conclusion, you must have completed a thorough evaluation.

However, we don’t always start with lower order skills and step all the way through the entire

taxonomy for each concept we present in our courses.

In case the course level is “Introduction to…” course, many learning objectives may target the

lower order Bloom’s skills, because students are building foundational knowledge.

If students have a solid foundation in much of the terminology and processes for the course,

we can skip the many remembering and understanding level objectives. Students should be

able to master higher-order learning objectives. Too many lower level objectives might cause

boredom or apathy. In order to understand how Bloom’s works with learning objectives,

there are “verb tables” to help identify which action verbs align with each level in Bloom’s

Taxonomy.

Page 12: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

12

Bloom’s

Level

Key Verbs (keywords) Example Learning Objective

Create design, formulate, build,

invent, create, compose,

generate, derive, modify,

develop

By the end of this lesson, the student will be

able to design an original homework problem

dealing with the principle of conservation of

energy.

Evaluate

choose, support, relate,

determine, defend, judge,

grade, compare, contrast,

argue, justify, support,

convince, select, evaluate.

By the end of this lesson, the student will be

able to determine whether using conservation

of energy or conservation of momentum

would be more appropriate for solving a

dynamics problem.

Analyse

classify, break down,

categorize, analyse,

diagram, illustrate, criticize,

simplify, associate.

By the end of this lesson, the student will be

able to differentiate between potential and

kinetic energy.

Apply

calculate, predict, apply,

solve, illustrate, use,

demonstrate, determine,

model, perform, present.

By the end of this lesson, the student will be

able to calculate the kinetic energy of a

projectile.

Understand

describe, explain,

paraphrase, restate, give

original examples of,

summarize, contrast,

interpret, discuss.

By the end of this lesson, the student will be

able to describe Newton’s three laws of

motion to in her/his own words

Page 13: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

13

Remember

list, recite, outline, define,

name, match, quote, recall,

identify, label, recognize.

By the end of this lesson, the student will be

able to recite Newton’s three laws of motion.

ADDIE MODEL

ADDIE stands for Analysis, Design, Develop, Implement, and Evaluate. It was developed

in the 1970’s. Remarkably it appears that the ADDIE model wasn’t specifically

developed by any single author.

The ADDIE model relies on each stage being done in the given order but with a focus

on reflection and iteration for the changes before moving to the next one.

Figure 5: Schematic view of ADDIE Model

Here is a brief description of each step of the ADDIE process:

• Step #1 Analysis — “Why is the training needed?” and "How can we improve the

situation and achieve business goals through training?". The instructional

designers (IDs) answer this question after collecting information and understanding

the needs and expectations of the learners. Analysis drives design and the

development process.

Page 14: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

14

• Step #2 Design — With the training plan done, IDs then get to the design phase.

This includes a strategy, delivery methods, appropriate media, duration,

assessment.

• Step #3 Development — IDs come out with course content in this phase to develop

the course materials. The development process should be iterative. Once a course

have been created It should be tested to ensure there are no basic errors -

grammar, spelling, syntax etc.

• Step #4 Implementation — it’s time to share them with the learner. The course is

released/rolled-out, delivered, to the learners, and its impact is monitored.

• Step # 5 Evaluation — Is the course providing the expected results? IDs collaborate

with the client and evaluate the impact of the course based on learner feedback,

surveys, and even analytics.

Once the evaluation is complete, the results are converted into actionable improvements. The

whole ADDIE process is repeated.

ARCS MODEL

Figure 6: John Keller (1938 -..)

John Keller is the founder of the ARCS Model of

Motivation, which is based upon the idea that there

are four key elements in the learning process which

can encourage and sustain learners’ motivation. These

four elements form the acronym ARCS of the model

and stand for Attention, Relevance, Confidence and

Satisfaction

Page 15: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

15

John Keller is the founder of the ARCS Model of Motivation, which is based upon the idea that

there are four key elements in the learning process which can encourage and sustain learners’

motivation. These four elements form the acronym ARCS of the model and stand for Attention,

Relevance, Confidence and Satisfaction

Figure 7: Application of ARCS Model (In Mazhar Zorlu VET)

1. Attention

Figure 8: One interesting picture to draw attention

Page 16: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

16

• Keller attention can be gained in two ways:

a. Perceptual arousal – uses surprise or uncertainly to gain interest. Uses novel, surprising,

incongruous, and uncertain events;

b. Inquiry arousal – stimulates curiosity by posing challenging questions or problems to be

solved.

Methods for grabbing the learners’ attention include the use of:

• Active participation -Adopt strategies such as games, roleplay or other hands-

on methods to get learners involved with the material or subject matter

• Inquiry – Pose questions or problems for the learners to solve, e.g. brainstorming activities.

• Specific examples – Use a visual stimuli, story, or biography.

• Humour -Maintain interest by use a small amount of humour (but not too much to be

distracting)

2. Relevance

• Establish relevance in order to increase a learner’s motivation. To do this, use concrete

language and examples with which the learners are familiar. Six major strategies described by

Keller include:

o Experience – Tell the learners how the

Experience – Tell the learners how the new learning will use their existing skills. We best learn

by building upon our pre-set knowledge or skills.

• Present Worth – What will the subject matter do for me today?

Page 17: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

17

• Future Usefulness – What will the subject matter do for me tomorrow?

• Needs Matching – Take advantage of the dynamics of achievement, risk taking, power, and

affiliation.

• Modeling – First of all, “be what you want them to do!” Other strategies include guest

speakers, videos, and having the learners who finish their work first to serve as tutors.

• Choice – Allow the learners to use different methods to pursue their work or allowing s choice

in how they organize it.

3. Confidence

Figure 9: Confidence

• Help students understand their likelihood for success. If they feel they cannot meet the

objectives or that the cost (time or effort) is too high, their motivation will decrease.

• Provide objectives and prerequisites – Help students estimate the probability of success by

presenting performance requirements and evaluation criteria.

• Feedback – Provide feedback and support internal attributions for success.

• Learner Control – Learners should feel some degree of control over their learning and

assessment. They should believe that their success is a direct result of the amount of effort

they have put forth.

Page 18: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

18

4. Satisfaction

• Learning must be rewarding or satisfying in some way, whether it is from a sense of

achievement, praise from a higher-up, or mere entertainment.

• Make the learner feel as though the skill is useful or beneficial by providing opportunities to

use newly acquired knowledge in a real setting

GAGNE’S 9 EVENTS OF INSTRUCTION

Gagne's Nine Levels of Learning provide a step-by-step approach that can help managers,

trainers, and facilitators structure their training so that their students or teams get the most

from their learning opportunities.

Robert Gagne was an educational psychologist who pioneered the science of instruction in the

1940s. His book "The Conditions of Learning," first published in 1965, identified the mental

conditions that are necessary for effective learning

Figure 10: Robert Gagne (1916 -2002)

Page 19: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

19

1. Gain attention of the students Ensure the learners are ready to learn and participate in

activities by presenting a stimulus to gain their attention.

Methods for gaining learners’ attention include:

— Stimulate students with novelty, uncertainty and surprise

— Pose thought-provoking questions to the students

— Have students pose questions to be answered by other students

2. Inform students of the objectives Inform students of the objectives or outcomes to help

them understand what they are to learn during the course. Provide objectives before

instruction begins. Methods for stating the outcomes include:

— Describe required performance

— Describe criteria for standard performance

— Learner establishes criteria for standard performance

3. Stimulate recall of prior learning Help students make sense of new information by relating

it to something they already know or something they have already experienced.

Methods for stimulating recall include:

— Ask questions about previous experiences

— Ask students about their understanding of previous concepts

4. Present the content Use strategies to present and cue lesson content to provide more

effective, efficient instruction. Organize and chunk content in a meaningful way. Provide

explanations after demonstrations.

Ways to present and cue lesson content include:

— Present vocabulary

Page 20: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

20

— Provide examples

— Present multiple versions of the same content, e.g., video, demonstration, lecture,

podcast, group work

— Use a variety of media to address different learning preferences

5. Provide learning guidance Advise students of strategies to aid them in learning content

and of resources available.

Methods to provide learning guidance include:

— Provide instructional support as needed – as scaffolds (cues, hints, prompts) which can

be removed after the student learns the task or content

— Model varied learning strategies – mnemonics, concept mapping, role playing,

visualizing

— Use examples and non-examples – in addition to providing examples, use non-examples

to help students see what not to do or the opposite of examples

— Provide case studies, analogies, visual images and metaphors – case studies for real

world application, analogies for knowledge construction, visual images to make visual

associations, metaphors to support learning

6. Elicit performance (practice) Activate student processing to help them internalize new

skills and knowledge and to confirm correct understanding of these concepts.

Ways to activate learner processing include:

— Elicit student activities – ask deep-learning questions, refer to what students already

know or have students collaborate with their peers

— Elicit recall strategies – ask students to recite, revisit, or reiterate information they have

learned — Facilitate student elaborations – ask students to elaborate or explain details

and provide more complexity to their responses

Page 21: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

21

— Help students integrate new knowledge – provide content in a context-rich way (use

real-world examples)

7. Provide feedback Provide immediate feedback of students’ performance to assess and

facilitate learning.

Types of feedback include:

— Confirmatory feedback – Informs the student they did what he or she were supposed

to do

— Corrective and remedial feedback – informs the student the accuracy of their

performance or response

— Remedial feedback – Directs students in the right direction to find the correct answer

but does not provide the correct answer

— Informative feedback – Provides information (new, different, additions, suggestions) to

a student and confirms that you have been actively listening – this information allows

sharing between two people

— Analytical feedback – Provides the student with suggestions, recommendations, and

information for them to correct their performance

8. Assess performance to evaluate the effectiveness of the instructional events, you must

test to see if the expected learning outcomes have been achieved. Performance should be

based on previously stated objectives.

Methods for testing learning include:

— Pretest for mastery of prerequisites

— Use a pretest for endpoint knowledge or skills

— Conduct a post-test to check for mastery of content or skills

— Embed questions throughout instruction through oral questioning and/or quizzes

Page 22: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

22

— Include objective or criterion-referenced performances which measure how well a

student has learned a topic

— Identify normative-referenced performances which compares one student to another

student

9. Enhance retention and transfer to the job to help learners develop expertise, they must

internalize new knowledge.

Methods for helping learners internalize new knowledge include:

— Generating examples

— Create concept maps or outlines

— Create job-aids, references, templates, or wizards

STEPS OF INSTRUCTİONAL DESİGN

An Instructional Designer designs learning (instructional) materials usually in the form

of units, modules, courses and programmes. But this is not the most important part of

their job, which is to help people make sense of these materials and other online

resources.

Their role is to turn information into clear, meaningful and contextual content, and to

point learners in the right direction

An Instructional Designer will commonly undertake all or some of the following steps:

1. Conduct a needs analysis to identify the needs of the targeted group, which may

be very wide and diverse.

2. Determine which needs can be met by learning and how they can be satisfied.

Page 23: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

23

3. Define learning objectives/learning outcomes that will be the dipstick (yardstick,

gauge stick) of successful learning.

4. Acquire information about the targeted group to assess entry skills and

knowledge, attitudinal and motivational factors, and behaviour patterns.

5. Based on all of the above devise an instructional strategy (a blue-print for the

learning) and select the techniques (case studies, story-telling, scenarios) and

delivery media (online, face-to-face, mobile, blended) for the learning provision.

6. Follow-up on the learning outcomes to see if they have been achieved and,

where relevant, performance has improved as a result.

What instructional design doesn’t do

1. It doesn’t put information in front of people. Instead it makes sense of it.

2. It doesn’t use technology to increase learning significantly. Instead it creates

learning environments that mimic as closely as possible real life learning and the

workplace.

3. It doesn’t produce materials that address as many learning styles as possible.

Instead it identifies the learning needs of the targeted group and helps everyone

to meet their needs, by allowing people to learn at their own speed and in ways

of their own choosing.

Page 24: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

24

3. TRAINING EVALUATION

Any time we deliver training to trainees, it is needed to know how effective it's been. Are

trainees putting their learning into practice? Is it positively impacting their role and the wider

organization? It acts as a check to ensure that the training can fill the competency gaps within

the organisation.

Kirkpatrick's Four-Level Training Evaluation Model can help you to answer questions like

these. We can use it to objectively analyse the impact of training, to work out how well your

team members learned, and to improve their learning in the future.

Donald Kirkpatrick, professor emeritus, university of Wisconsin began working on evaluating

the effectiveness of training very early in his life. His early work on the same was published in

the year 1959 in a journal of American Society of Training Directors. He laid out four levels for

evaluation of any training. This model is arguably the most widespread for evaluation in use.

It is simple, very flexible and complete.

Figure 11: Donald Kirkpatrick (1924-2004)

Page 25: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

25

Some of the benefits of the training evaluation are as under:

• Evaluation ensures accountability - Training evaluation ensures that training programs

comply with the competency gaps and that the deliverables are not compromised upon.

• Check the Cost - Evaluation ensures that the training programs are effective in improving

the work quality, employee behaviour, attitude and development of new skills within the

employee within a certain budget.

• Feedback to the Trainer/Training - Evaluation also acts as a feedback to the trainer or the

facilitator and the entire training process. Since evaluation accesses individuals at the level

of their work, it gets easier to understand the loopholes of the training and the changes

required in the training methodology.

Understanding Kirkpatrick's Four Levels

The four levels as described by Kirkpatrick are as follows:

Level 1: Reaction

We want trainees to feel that training is valuable. Measuring how engaged they were, how actively they contributed, and how they reacted to the training helps you to understand how well they received it.

It also enables us to make improvements to future programs, by identifying important topics that might have been missing.

Questions to ask trainees include:

• What did you like most about this training?

• Did you feel that the training was worth your time?

• What were the biggest strengths and weaknesses of the training?

• Did you like the venue and presentation style?

Page 26: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

26

• Were the training activities engaging?

• What are the three most important things that you learned from this training?

It is identified how to measure participants(trainees)' reactions. Many trainers use “Training Evaluation Form” to do this, but we can also watch trainees' body language during the session, or ask for verbal feedback. After the feedback is analysed, the changes are considered to make in response.

Level 2: Learning

Level 2 focuses on measuring what trainees have and haven't learned. This demonstrates how training has developed their skills, attitudes and knowledge, as well as their confidence and commitment.

To measure how much trainees have learned, it is started by identifying what we want to evaluate. Generally, Pre/Post tests are used to measure the objectives before and after training.

Before the training begins, test your trainees to determine their knowledge, skill levels and attitudes. Then, when the training is finished, test your trainees a second time to measure what they have learned, or measure their learning with interviews or verbal assessments.

Level 3: Behavior

This level helps us to understand how well trainees apply their training. It can also reveal where they might need help. But behaviour can only change when conditions are favourable.

Imagine that you're assessing your team members after a training session. You can see little change, and you conclude that they learned nothing, and that the training was ineffective.

Effectively measuring behaviour is a longer-term process that should take place over weeks or months following the initial training.

Questions to ask include:

• Did the trainees put any of their learning to use?

Page 27: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

27

• Are trainees able to teach their new knowledge, skills or attitudes to other people?

• Are trainees aware that they've changed their behaviour?

One of the best ways to measure behaviour is to conduct peer observations, job checklists and interviews. Another is to integrate the use of new skills into the tasks that we set our team, so that trainees have the chance to demonstrate what they know.

Level 4: Results

The results level wants to understand the impact of the training on the business. It looks to abstract metrics like customer satisfaction and also aims to measure financial outcome.

Level 4 will likely be the most costly and time-consuming. The biggest challenge will be to identify which outcomes, benefits, or final results are most closely linked to the training, and to come up with an effective way to measure these outcomes in the long term.

Here are some outcomes to consider, depending on the objectives of your training:

• Increased employee retention.

• Increased production.

• Reduced waste.

• Higher quality ratings.

• Increased customer satisfaction.

• Fewer staff complaints.

One of the best ways to measure “results” is to conduct job checklists and metric repots. CONCLUSION

Training can be expensive, and this investment has to be made in the right place. Evaluation not only contributes to the quality of the training project but also demonstrates the accountability of training management, the success of the programme and improved performance for the organization.

Page 28: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

28

4. PRACTICAL ID APPLICATION

4.1. Problem-Situation

XX City has one public bus corporation, which owns 90 percent of the buses in the city, and

three small private bus companies.

Buses are the principal means of transportation for the citizens of XX City. Over the past few

years, the number of accidents caused by buses operated by the public bus corporation has

increased dramatically, becoming the primary cause of traffic congestion. Not only bus

passengers but also people using roads have complaints about these conditions.

The causes of many of the accidents involving the public bus corporation were technical:

• Buses are old and in disrepair,

• Regular maintenance checks are not performed,

• Repairs are hampered by shortages of spare parts and equipment,

• Mechanics are poorly trained.

Accidents are also caused by the bus drivers’ failure to observe traffic laws, written below:

• Excessive speed on poorly maintained roads

• Disregard of traffic signals.

The public bus corporation has requested the cooperation for improving the public bus’s

services.

Page 29: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

29

4.2. Problem Analyses

Chart 1: Problem Tree of the Public Bus Corporation of XX City

Passengers have little trust in the

public bus

Passengers are injured and killed in

accidents.

Transportation for XX city citizens is

impeded.

Traffic jams are caused.

Accidents ivolving the public bus corporation

occur frequently.

Drivers do not comply with traffic

rules.

Buses are prone to break down.

Roads are not well maintained.

Buses are old.

Buses are in poor repair.

Periodic inspections

are not carried out.

Mechanics are

poorly trained.

Page 30: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

30

4.3. Core Problem

Chart 2: Problem Tree of the Core Problem

Passengers have little trust in the

public bus

Passengers are injured and killed in

accidents.

Transportation for XX city citizens are

impeded.

Traffic jams are caused.

Accidents involving the public bus corporation

occur frequently.

Buses are prone to break down.

Buses are old. Buses are in poor

repair.

Periodic inspections

are not carried out.

Mechanics are

poorly trained.

Drivers do not comply with traffic

rules.

Roads are not well maintained.

Page 31: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

31

4.4. Objective Analyses

Chart 3: Problem Tree of the Objectives Tree

Passengers have incrased trust in the

public bus

Fewer passengers are injured and killed in accidents.

Smoothness of transportation for XX city

citizens is improved.

Traffic jams are reduced.

Accidents involving the public bus corporation

are reduced.

Drivers comply with traffic rules.

Buses seldom break down.

Roads are properly maintained.

Fewer old buses. Mechanic’s skills are improved.

Mechanic tehnicians

acquire necessary

Maintenance facilities are

improved

Drivers are trained.

Old buses are replaced with new

ones.

Periodic inspections are

carried out .

(Direct Ends)

(Core Objective)

(Direct Means)

Page 32: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

32

4.5. Approaches

Chart 3: Identifying Approaches

•Vehicle improvement approach

• Safe driving approach

Accidents involving the public bus corporation are reduced.

Drivers comply with traffic rules.

Buses seldom break down.

Roads are properly maintained.

Fewer old

buses.

Mechanic’s skills are improved.

Passengers have incrased trust in the

public bus corporation.

Fewer passengers are injured and killed in accidents.

Smoothness of transportation for XX city

citizens is improved.

Mechanic tehnicians

acquire necessary skills.

Traffic jams are reduced.

Maintenance facilities are

improved

Drivers are trained.

Old buses are

replaced with new

ones.

Periodic inspections are

carried out .

•Driver

training

approach

•Road improvement approach

Page 33: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

33

Narrative Summary Objectively Verifiable Indicators

Means of Verification Important Assumptions

Overall Goal

Traffic jams in XX city are reduced.

1 The number of passengers

injured or killed in accidents

involving the public bus

corporation falls by 60 percent per

annual by the year 2030

2 The time required to cross XX

city decreases by 10

percent by the year 2030

1 Traffic accident records of the

Traffic Department of XX city

Police Department.

2 Investigation reports of the

Traffics Safety Centre of XX

University.

The public transportation policy of

the Ministry of

Transport of that country remains

unchanged

regarding the public bys

corporation.

Project Purpose

Accidents involving the public bus

corporation are reduced.

The number of the bus

corporation's accidents in L City

declines by 50 percent by march

2020.

Traffic accident records of the

Traffic Department of XX city.

The total number of vehicles in XX

city does not increase drastically.

Page 34: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

34

Outputs

1 Drivers observe traffic rules.

2 Maintenance facilities are better

equipped.

3 Mechanics skills are improved.

4 Buses are properly maintained.

5 Old buses are replaced.

1-1 By the year 2021, more than

70 percent of the bus public bus

corporation's drivers observe 75

percent of the important driving

rules set by the officials

2-1 The operation rate of the bus

maintenance equipment reaches

80 percent by the year 2020.

2-2 The number of buses that

cannot be repaired due

to parts shortages declines by

50 percent by the

year 2021.

1-1 Project survey reports on

drivers' compliance with traffic

rules.

2-1 Project survey reports on the

operation rate of bus

maintenance equipment.

2-2 Project survey reports on bus

repair.

3-1 Project records of the Final

Examination of the mechanic

training program.

The road maintenance conditions

in XX city do not worsen.

Page 35: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

35

3-1 60 percent of the mechanics of

the public bus corporation

undergo maintenance training by

the year 2021.

4-1 The operation rate of buses

meeting the standards increases

by 30 percent by the year 2021

5 -1 The number of buses used

beyond a prescribed number of

years or mileage declines to

less than 10% by the year 2022.

4-1 Operation records of the

public bus corporation.

5-1 Vehicle maintenance records

of the project.

Page 36: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

36

Activities

1-1 Specify the rules and practices

that bus drivers should observe.

1-2 Formulate a driver-training

program.

1-3 Produce training materials.

1-4 Train driving instructors.

1-5 Train drivers.

1-6 Check periodically for

compliance with important rules.

2-1 Draw up an improvement plan

for maintenance facilities.

2-2 Purchase necessary

equipment.

......................

Inputs:

[A Side]

Experts

Provision of Equipment and

materials

Training of driving instructors in

……….

[B Side]

Counterpart personnel

Provision of Equipment and

materials

Running expenses

Office space

Trained bus drivers continue

working for the public

bus corporation.

Trained mechanics continue

working from the public bus

corporation.

Customs clearance and transport

procedures do not become greatly

delayed.

Pre-conditions

Page 37: INSTRUCTIONAL DESIGNprobased.org/wp-content/uploads/2018/12/Instruction-Design.pdfInstructional designers use these models to guide the creation of engaging learning activities based

37

3-1 Produce a maintenance

manual.

3-2 Specify the technical skills

required for mechanics.

......................

4-1 Draw up a periodic

maintenance

system.

4-2 Conduct periodic

maintenance.

...................

-END-