muri: training knowledge and skills for the networked battlefield aro award no. w9112nf-05-1-0153...
TRANSCRIPT
MURI: Training Knowledge and Skills for the Networked BattlefieldARO Award No. W9112NF-05-1-0153
Alice Healy and Lyle Bourne, Principal Investigators
Benjamin Clegg, Bengt Fornberg, Cleotilde Gonzalez, Eric Heggestad, Ronald Laughery, Robert Proctor,
Co-Investigators
Project Mission(As Defined by the BAA)
Objectives
“Develop and evaluate models that predict performance
improvement or decrement for a range of militarily
significant individual and collective tasks that can be
linked to various types and amounts of training while
considering the effects of aptitude and experience.”
Proposed Project(As Defined in Executive Summary)
Goals
• Construct a theoretical & empirical framework for training
• Predict the outcomes of different training methods on particular tasks
• Point to ways to optimize training
Statement of Work
The work to be performed falls into 3 interrelated categories:
(1) Experiments(a) Development & testing of training principles(b) Acquisition & retention of basic skill components(c) Levels of automation, individual differences, &
team performance
(2) Taxonomic analysis (a) Training methods(b) Task types (c) Performance measures(d) Training principles
(3) Predictive computational models(a) Formulated from experimental data (b) Applied to military tasks
Parts of Project(1) Experiments
(a) Development & Testing of Training Principles
(b) Acquisition & Retention of Basic Components of Skill
(c) Levels of Automation, Individual Differences, & Team Performance
(2) Taxonomy
(3) Models
(a) ACT-R
(b) IMPRINT
(c) Model Assessment
Three Major Parts of Present Meeting
(I) Introduction
(II) Plans For Project and Progress So Far
(A) Experiments
(B) Taxonomy
(C) Models
(III) Summary and Reactions
Introduction to MURI Personnel
(1) University of Colorado (CU)Alice Healy, Principal InvestigatorLyle Bourne, Co-Principal InvestigatorBengt Fornberg, Co-InvestigatorRon Laughery, Co-InvestigatorBill Raymond, Research Associate
(2) Carnegie Mellon University (CMU)Cleotilde Gonzalez, Co-Investigator
(3) Colorado State University (CSU)Ben Clegg, Co-InvestigatorEric Heggestad, Co-Investigator
(4) Purdue University (Purdue)Robert Proctor, Co-Investigator
Roles in Project
(1) Overview and CoordinateCU, Healy & Bourne
(2) Experiments(a) Development & Testing of Training Principles
CU, Healy & Bourne(b) Acquisition & Retention of Basic Components of Skill
Purdue, Proctor(c) Levels of Automation, Individual Differences, & Team
Performance, CSU, Clegg & Heggestad (3) Taxonomy
CU, Raymond(4) Models
(a) ACT-RCMU, Gonzalez
(b) IMPRINTCU, Laughery
(c) Model AssessmentCU, Fornberg
Key Comments from Review(1) Tighter integration of the modeling effort with the learning research and experimentation is needed and should take place during the first few months of the project.
(2) Data-tractability (how much data on the training and the subjects are needed to make reasonable evaluations) and computational tractability need to be addressed in greater depth.
(3) Training in a complex networked environment could be addressed at greater depth, but, since even training for more elementary tasks is not yet understood, the proposed work is reasonable.
(4) More emphasis on software and less emphasis on papers published in professional journals and books is needed in the deliverables.
(5) There is a question about how the obligations of one senior MURI team member to a company and to the Advanced Decision Architectures Collaborative Technology Alliance will be coordinated with that member’s obligation to the MURI.
Outline of Plans for Project and Progress So Far
(I) Preliminary Investigator’s Meeting, Boulder, May 25, 2005(II) Preparation of Investigators’ WIKI and Public MURI website(III) Experiments
(A) Development & Testing of Training PrinciplesHealy & Bourne
(B) Acquisition & Retention of Basic Components of SkillProctor
(C) Levels of Automation, Individual Differences, & Team Performance Clegg & Heggestad
(IV) TaxonomyRaymond
(V) Models(A) ACT-R
Gonzalez(B) IMPRINT
Laughery(C) Model Assessment
Fornberg
Development and Testing of Training Principles
• Summary of 30 Training Principles: Prepared for NASA cooperative agreement
• Two Examples of Training Principles
Strategic-Use-of-Knowledge Principle When a large amount of new factual information must be learned and retained, that information should be related to the learner’s existing knowledge in any way possible.
Principle of Contextual InterferenceIntroduce sources of interference into training material. Interference may weaken performance during training but should strengthen retention and transfer.
Development and Testing of Training Principles: Proposed and
In-Progress Experiments
(1) Tests of the generality across tasks of individual principles -- 1 in-progress on strategic use of knowledge
(2) Tests of multiple principles in a single task -- 1 in-progress on serial position, list length, and chunking effects
(3) Tests of principles in complex, dynamic environments -- 1 in-progress on contextual interference
CU Experiments: Communication with Modelers to Date
(1) Data Entry: Fatigue Effects; Speed-Accuracy Tradeoffs
Sent to Gonzalez & Laughery data from 2 previously published experiments
(2) Hand-Eye Coordination: Specificity of Training; Retention and Transfer Effects
Sent to Gonzalez & Laughery data from 1 unpublished experiment
(3) Further Work on Data Entry: Multiple Principles in a Single Task
Sent to Gonzalez & Laughery data from 8 previously published experiments examining (a) specificity of training, (b) procedural reinstatement, (c) depth of processing, (d) phonological coding
Sent to Gonzalez & Laughery data from 2 newly completed experiments examining (a) cognitive and motoric fatigue, (b) feedback and cognitive load
Data Entry Experiments
Task: Subjects see a 4-digit number, and they type it on a
computer keypad
Design: In each session half, subjects see and type 5 blocks of
64 numbers
Measures: Both typing accuracy (proportion correct) and
typing speed (total response time) are measured
1 2 3 4 50.86
0.87
0.88
0.89
0.90
0.91
2.58
2.60
2.62
2.64
2.66
2.68
2.70
Proportion Correct
Total Response Time
Healy, Kole, Buck-Gengler, & Bourne (2004) Experiment 1
Block
Proportion CorrectTotal Response Time (in s)
1 2 3 4 50.82
0.84
0.86
0.88
0.90
0.92
Suppression
SilentSuppression
Silent
Kole, Healy, and Bourne (2005) Experiment 1
Block
Proportion Correct
No Weight
Weight
1 2 3 4 50.82
0.84
0.86
0.88
0.90
0.92
Feedback
No FeedbackFeedback
No Feedback
Kole, Healy, and Bourne (2005) Experimernt 2
Block
Proportion Correct
Data Entry
Multiplication
CU Experiments: Communication with Modelers Planned for Next Year
(1) Data Entry: Mental Rehearsal
2 experiments on repetition priming and motor imagery
(2) Hand-Eye Coordination: Further Work on Specificity of Training
1 experiment assessing relative merits of specificity and variability of training
1 experiment on strategy instructions and gender effects
1 experiment on immediate testing and transfer
(3) Duration Estimation: Functional Task Principle
1 experiment varying presence of secondary task
2 experiments varying features of secondary and primary tasks
2 experiments varying difficulty and modality of secondary task
CU Experiments: Expanded Work on Complex Tasks
(1) RADAR Task from CMU
Test of Training Difficulty Principle
(2) First Responder Navigation Task with Emergencies from NSF SGER Grant
Test of Memory Constriction Hypothesis
Test of Look-Up Speed in Emergency Check-List
Summary of Response to Comments from Review
(1) Tighter integration of modeling effort and experimentationExperimenter-taxonomist-modeler interactions are on-going, facilitated by
meetings and WIKI
(2) Data and computational tractabilityAssessments of data-computation compatibility are on-going, facilitated by
meetings between Fornberg and research personnel
(3) Training in a complex networked environmentExperiments underway using complex and more naturalistic tasks, such as
RADAR tracking, emergency response teams, flight simulation
(4) More emphasis on software in the deliverablesACT-R and IMPRINT software products will be available at various points
in the future
(5) The multiple obligations of one senior MURI team memberPeriodic meetings between senior member and research associate, enabling
use of IMPRINT by other team members to model existing data
Present and Future Activities
(1) Activities
(a) Experiments
(b) Taxonomy
(c) Modeling
(2) How do we propose to get from the current state of knowledge to the final goal of predicting performance as a function of training