1 research on the design of technology-based training richard e. clark rossier school of education...
TRANSCRIPT
1
Research on the Design of Technology-Based
Training
Richard E. ClarkRossier School of Education
Center for Cognitive TechnologyUniversity of Southern California
ICT Immersive Training WorkshopMay 19, 2005
Marina del Rey California
2
Topics for Today’s WorkshopYou will be able to answer five questions:1. How does our cognitive architecture
influence learning from training?2. Do media influence learning? What are the
six principles for effective media-based training?
3. What new and promising training design models are out there right now?
4. What motivates us to learn and how can we influence it during training?
5. How can we decide whether training should be delivered in the classroom or by multi media?
3
Have you ever?
• Spaced out when you were studying a chapter in a book?
• Had difficulty memorizing lists • Noticed that you stopped listening to
someone who was giving you information and missed some of it?
• Had trouble remembering when you were under pressure?
4
I. How does our cognitive architecture influence learning from training?
1. We have a very limited thinking space, easily overloaded to protect us from scrambling our brain ( 3 +/- 1 information chunks/thoughts at one time)
– When overloaded, our brain short circuits and interferes with the mental resources needed to learn
– Novices most vulnerable – experts insulated– Overload is not noticed – may be enjoyable?
To overcome this limit we have two different but interacting knowledge systems
– Declarative (what, why, where, when) – Procedural (how)
5
I. Cognitive Architecture
2. Declarative system is information rich and helps us handle novelty and develop new procedures to achieve goals
– All new knowledge is declarative – about 10%– Knowledge stays flexible while it is being used to
achieve our goals– Once it is successful it gradually becomes fixed,
procedural, automated and unconscious – 90%• Types: concepts, processes and principles• Formats: Sensory Images, symbols
6
I. Cognitive Architecture
3. Procedural system operates outside of consciousness and so avoids 3 chunk limit on thinking
– Goal of nearly all training is to teach procedures – Consist of conditions (when, where) actions (what to
do) and decisions (alternatives and criteria for deciding)
– Expertise 90% automated & unconscious– Experts 50% inaccurate so “telling them how it is
for an expert” is not enough
7
I. Cognitive Architecture
4. Once we accept a new goal, our “learning system” is structured to select and/or develop a procedure for achieving it.
– General learning procedures include:• Plans to achieve the goal• Connections to relevant prior knowledge, • Ways to select the important and ignore the rest• Self monitoring for accuracy and results feedback• Procedures for using feedback to tune the plan
8
I. Cognitive Architecture
5. Our “learning system” requires external help to learn when information is relatively new
– Effective help procedures include:• Plans to achieve the goal (and clear goals)• Connections to relevant prior knowledge (examples)• Ways to select the important and ignore the rest• Self monitoring for accuracy and results feedback• Procedures for using feedback to tune the plan
9
I. Cognitive Architecture
5. Successful training provides clear goals and supports the learning strategies that trainees can’t or won’t provide for themselves.
– Provide a clear “3C” goal (concrete, current, challenging), motivate them to accept it and persist at achieving it
– Show them how to reach it with a “worked example”– Immerse them in a variety of realistic examples – Support their learning strategies as they develop
their own version of a procedure
10
II. Do media influence learning?1. Media influence the efficiency (cost, speed)
of learning but not whether someone learns.– Learning is due to instructional strategies and
the relevance of the information we provide.– More than one medium will result in comparable
learning of any task
2. Instructional strategies influence learning. They are external representations of internal cognitive strategies needed to learn
– Examples: Goals/objectives, learning plans, connections to prior experience (examples, analogies), practice exercises, progress evaluation, feedback
11
II. Do media influence learning?
3. Multi-media presentations often cause cognitive overload due to media features that split attention and require integration
– Caused by background music, irrelevant sounds, animated “help” characters, separated video and print explanations, narration over print etc.
4. Goal is to lower cognitive load and focus attention only on essential, goal-relevant information in multi media programs
How? Six Research Based Principles Next
12
Six Cognitive Principles about Media
1 Contiguity principle.– Integrate all visual and aural information -
use both together at the same time and in the same instructional space
2 Multiple modalities principle.– Visual and sound modalities better than
either alone – use both where possible
13
Spatial Contiguity PrinciplePeople learn better when corresponding words and graphics are placed near rather than far from each other on the screen.As a possible modification to the annotated illustrations, you might be tempted to separate the text and graphics. Your rationale might be that the learner can concentrate on the text and then concentrate on the graphics, and thus gain two exposures to the material. This strategy, however, violates the spatial contiguity principle.
When the driver steps on the car’s brake pedal, a piston moves forward inside the master cylinder. The piston forces brake fluid out of the master cylinder and through the tubes to the wheel cylinders. In the wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move. These smaller pistons activate the brake shoes. When the brake shoes press against the drum, both the drum and the wheel stop or slow down.
This is a screenful of printed text describing how brakes work.
See graphics
Return
When you click on “See graphics” you see these graphics depicting how brakes work.
Problem: When graphics are separated from the text, learners have difficulty in making connections between corresponding words and graphics.
14
Modality PrinciplePeople learn better from animation and narration than from animation and on-screen text.
A solution to the split attention problem is to present spoken words rather than printed words, thus reducing the load on the visual system.
“When the driver steps on the car’s brake pedal, a piston moves forward inside the master cylinder.”
“In the wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move.”
“The piston forces brake fluid out of the master cylinder and through the tubes to the wheel cylinders.”
Each sentence is spoken at a slow pace as the animation shows what happens when the driver steps on the brake petal.
The words are spoken rather than printed on the screen.
15
Six Cognitive Principles about Media
3 Coherence Principle– Eliminate all extraneous words, sounds
(e.g. music) and pictures to reduce load– If information is not essential to achieve
the goal, drop it or risk distracting people
4 Narration principle– Narration is better than text in media
displays – people listen more effectively than they read. Natural voice narration is better than machine voice.
16
Coherence Principle People learn better from multimedia messages when extra words, pictures, and sounds are excluded rather than included.It may be tempting to embellish a presentation by adding background sounds or music. However, the added materials can distract the learner and disrupt the learner’s building of a causal chain.
“When you step on your car’s brake pedal, a piston moves forward inside your master cylinder.”
“In your wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move.”
“The piston forces brake fluid out of your master cylinder and through the tubes to your wheel cylinders.”
To add interest, you are tempted to add background music such as aninstrumentalloop.
To add interest, you are tempted to add background sounds such as as swooshing sound of the piston
To add interest, you add background sounds such as as sound of flowing liquid.
To add interest, you add background sounds such as as mechanical sound of the piston moving.
The music continues.
The music continues.
17
“When you step on your car’s brake pedal, a piston moves forward inside your master cylinder.”
“In your wheel cylinders, the increase in fluid pressure makes a smaller set of pistons move.”
“The piston forces brake fluid out of your master cylinder and through the tubes to your wheel cylinders.”
To create a sense of social interaction, the narration is spoken in a non-accented human voice that conveys friendliness, intelligence, and energy.
The voice principle can be applied to conversational style as in this example, or to formal style.
Narration Principle People learn better from narration than text and better yet when the narration has a human voice
with a standard accent than a machine voice or an accented voice.Another possible way to create a sense of learning partnership is to use a standard accented human voice that the learners interpret as friendly, intelligent, and dynamic. The reason is that learners may try harder to understand when they accept the speaker as a social partner.
18
Six Cognitive Principles about Media
5 Reverse expertise principle– Most load principles needed for novice
and/or low spatial ability learners, not for experts (but lower load does not hurt experts)
6 Worked-example principle– Discovery problem-based learning
imposes high cognitive load with little benefit – use worked examples based on cognitive task analysis
19
Worked example: How to fix instructional materials that are classified as split attention risks.
1.Read the text material, and separate the text into segments where each segment describes or explains one area or element in the visual.
2. Identify the location in the visual that each segment of the text is discussing.
3. Decide wither the visual can be reformatted so that you can insert each segment of text immediately next to the part that it is discussing or whether you must draw arrows from the part being discussed to a text fragment that is placed as close as possible to the part?
4. Revise the visual so that text and visual are fully integrated.
20
Worked example: How to fix instructional materials that are split
attention risks.Test : To test Insulation Resistance from conductors to earth. How conducted : i )
Disconnect appliances and busways during these tests. Make sure main switch is "on" and all fuses are "in". Remove main earth from neutral bar and set meter to read insulation. Connect one lead to earth wire at MEN bar and take first measure by connecting the other lead to the active. Take next measure by connecting the lead to the neutral.ii) If resistance is not high enough in either of the two tests in i) then measure each circuit separately.Results required :i) At least One Mega ohm ii) Same result as i) above
21
Corrected example:
22
IV. What new and promising training design models are available?
Reminder: • Subject matter experts design most training• Expert’s knowledge is automated and
unconscious • When asked “how do you do that” expert
explanations are approximately 50% wrong• Less than 20% of what is learned in training
transfers back to performance• ISD models are suited to less complex
knowledge
23
New complex knowledge training design models: • Use Cognitive Task Analysis for accuracy• Mix declarative and procedural knowledge• Provide cognitive learning support• Provide “Part” and “Whole” task practice• Advanced assessment possibilities• Examples: vanMerrieboer’s 4C/ID and Clark’s
Guided Experiential Learning (GEL)
IV. What new and promising training design models are available?
24
How Is CTA Conducted?
Three or more experts interviewed separately:• Sequence of activities (before, during, after)• Conditions (when, cues, indications, contra
indications)• Actions (How to complete each step)• Decisions (When, Alternatives, Criteria)• Concepts, processes, principles operating
in this task – case study examples• Standards of performance• Equipment, materials
25
New 10 Stage Design System:(vanMerrienboer' s 4C/ID system)
1 Decompose the skill2 Sequence task
classes3 Design learning tasks4 Analyze mental
models - process5 Analyze cognitive
strategies6 Design supportive
information
7 List rules or procedures
8 Prerequisite knowledge
9 Design JIT information
10 Design part-task and whole task practice
26
A
B
C
analyze non -recurrent aspects
analyze recurrent aspects
analyze rules
analyze prerequisiteknowledge
analyzecognitivestrategies
analyzementalmodels
decomposethe complex skill
sequence skill clustersand/or task classes
A
B
C
analyze non -recurrent aspects
analyze recurrent aspects
analyze rules
analyze prerequisiteknowledge
analyzecognitivestrategies
analyzementalmodels
decomposethe complex skill
sequence skill clustersand/or task classes
27
Overview of GEL Design
Select Course& SME’s
Identify Many Job and Mission Problems
CognitiveTask Analysis
Identify• How to act and decide• New concepts, processes• Equipment and materials• Performance standards
Design• Lessons • Immersive demonstrations• Checklists for practice• Job aids for transfer• Guided practice exercises• Feedback for practice
Select delivery media
Design• Four level evaluation• Test of prior knowledge• Transfer letters
28
Using Guided Experience to Design Courses
Problem Selection and Examples
• Identify prior experience of trainees• Identify five large, authentic field problems
• Easy to very complex• Create “worked examples” of the problems
• Use cognitive task analysis• What are conditions, input and output•Solve for prior experience of trainees
Select Course& SME’s
Identify Many Job and Mission Problems
Cognitive Task Analysis
29
Conceptual Knowledge About Field Problems
• Collect information about• New concepts (definition and example)• Process (how it works – big picture)• Procedure (How to do it, conditions and
consequences)
Identify• How to act and decide• New concepts, processes• Equipment and materials• Performance standards
30
Design Lessons
• Sequence groups of problems into lessons• First performed in field are first taught• If no fixed sequence, easy before difficult
• Develop goals for each lesson• Remember definitions of concepts• Remember description of processes• Practice and perform procedure• Remember conditions and consequences
Design• Lessons • Immersive demonstrations• Checklists for practice• Job aids for transfer• Guided practice exercises• Feedback for practice
31
6 Instructional Methods for Each Lesson
Lesson Sequence:1. Goals –
• You will learn how to (REMEMBER, DO…)2. Reasons
• Value of learning - consequences of not learning
• What you already know that you should use3. What You Need to Know to Perform
• Teach new concepts and processes needed to learn procedure
Design• Lessons • Immersive demonstrations• Checklists for practice• Job aids for transfer• Guided practice exercises• Feedback for practice
32
Lesson Sequence:
4. Demonstrate procedure• Worked example – authentic setting• Instructor or Model should be credible, similar
5. Practice with incomplete problems• Trainees use partly worked example or demo• Practice more complex/difficult problems that
integrate previous learning• Gradually fade support – training wheels come
off!6. Feedback focused on strategy improvement
Design• Lessons • Immersive demonstrations• Checklists for practice• Job aids for transfer• Guided practice exercises• Feedback for practice
33
Select delivery media Design• Four level evaluation• Test of prior knowledge• Transfer letters
Media Selection,Evaluation and Transfer
• Select media based on context, practice and cost• Evaluate on four levels
1. Reactions (motivation - confidence, value)2. Learning (use practice exercises to evaluate)3. Transfer (check with supervisor/commander)4. Impact (did it make a difference to bottom
line?)• Send letter to commander asking for transfer
help
34
Assessment Possibilities
Stages in the development of expertise– Strategy assembly, gradual automation– Links between sub goals and explanation– Increasing speed with more practice– Reliable, valid measurement of expertise– Clear targets for corrective feedback– Connections between expertise and
measured performance outcomes
35
Evidence for Effectiveness?
Trainees learn faster, make fewer mistakes:• No differences on routine, directly
observable actions• Large differences on implicit, complex
mental strategies – Novices learn 25% faster (range 20-80%)– Made 70% fewer mistakes on cognitive
process– Mistakes of control groups are more serious
36
Evidence for Effectiveness?• Strategy sequence learned more quickly• Fewer and less important decision errors• More accurate explanations of decisions• Speed of automation increased• Better assessment of developing expertise • Coaching and feedback targets obvious• Learning and transfer increased but not
motivation
37
IV. What motivates us to learn and how can we influence it during training?
• Learning is 50% motivation• Three motivation outcomes:
– active choice, – persistence and – mental effort
• Active choice may be increased by multimedia
• Persistence may increase initially but it often decreases quickly
• Evidence that new media may decrease mental effort invested in learning
38
3. Design to support learner motivation
• Web persistence rates 50% less than classroom
• Problem with value for course content, social connections, poor planning and/or misperceived demands
• Apply motivation research (provide value, clear learning and performance goals, support confidence and planning)
39
V. Deciding whether training should be delivered live or by multi media?
1) Can application context be simulated on web?
2) Can timing, complexity of feedback be accommodated on web?
3) If “no” to 1) or 2), offer classroom instruction.
4) If 1) and 2) are “yes” Use ingredients method of cost/utility analysis – if web is cost beneficial, use it.
40
In Conclusion: Avoiding Misconceptions
• Start by analyzing problems to be solved with web-based courses
• Separate design AND development for learning AND motivation
• Use cognitive task analysis to capture expertise www.ctaresources.com
• Use worked examples and scaffolding to support constant, incremental practice
• Reduce cognitive load to the minimum when training complex knowledge
41
Research on the Design of Technology-Based
Training
Richard E. ClarkRossier School of Education
Center for Cognitive TechnologyUniversity of Southern California
ICT Immersive Training WorkshopMay 19 2005
Marina del Rey California