![Page 1: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/1.jpg)
Machine Learning Approaches to Cognitive Parameter Acquisition
Terran LaneUniversity of New
Chris Forsythe, Patrick Xavier
Sandia National Labs{jcforsy,pgxavie}@sandia.gov
![Page 2: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/2.jpg)
Sandia’s Cognitive Modeling Framework
Computational models of human decision-makers
Models attention, perceptual cues, situational awareness, decision making
Based on oscillatory models of activation Spreading activation networks and feedback
loops between functional elements Applications -- data analysis, security,
tutoring… Bottleneck: models hand-built/tuned
Expensive and slow!
![Page 3: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/3.jpg)
The Big PictureW
orl
d
Cue0 0
Cue1 1
CueN N
Situation0
Situation1
SituationM
Act
ion
s/D
eci
sion
s
01
10
N1
NM
![Page 4: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/4.jpg)
Automated Model Acquisition High predictive accuracy
87% correct prediction of operator’s interpretation of scenario (incl. relevance)
91% correct in recognizing situation only Insights into operator decision-making
process Models are task & user specific
Only 26% overlap between users Large effort in building and tuning models
Project goal: (semi-)automate acquisition of parameters, network topologies, etc.
Prediction accuracy secondary concern
![Page 5: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/5.jpg)
Roles for Machine Learning
Parameter acquisition Interconnection weights Activation levels Oscillator frequencies
Network topologies Inter-cue spreading activation network Cue <-> situation relations Feedbacks
Cues and situation identification
![Page 6: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/6.jpg)
Parameter AcquisitionW
orl
d
Cue0 0
Cue1 1
CueN N
Situation0
Situation1
SituationM
Act
ion
s/D
eci
sion
s
01
10
N1
NM
![Page 7: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/7.jpg)
Parameter Acquisition: Issues
Superficially supervised learning Observe features/cues and operator actions;
induce params (find s.t. f:CA) Similar to ANN backprop, EM, etc. Many effective, well understood techniques
Problem: not just high-likelihood params Actually want params used by human operator Much harder – observable stimuli don’t
directly reflect operator’s internal state Cognitive plausibility constraint
![Page 8: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/8.jpg)
Parameter Acquisition: Approaches
Additional instrumentation Measure characteristics of operator Biometrics – eye tracking, MEG, etc. Expensive, not widespread Maybe not informative to params anyway
Utility elicitation techniques Software queries user about why
decisions were made / state of attention Picks questions to maximally improve
model Emulates expert knowledge engineer
![Page 9: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/9.jpg)
Network Topology InductionW
orl
d
Cue0 0
Cue1 1
CueN N
Situation0
Situation1
SituationM
Act
ion
s/D
eci
sion
s
01
10
N1
NM
![Page 10: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/10.jpg)
Topology Induction: Issues
Find structure of interconnections between variables (I.e., cues, situations)
Much harder than parameter acquisition Formally, maximum likelihood/MAP search
through all possible networks
![Page 11: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/11.jpg)
Topology Induction: Issues
Find structure of interconnections between variables (I.e., cues, situations)
Much harder than parameter acquisition Formally, maximum likelihood/MAP search
through all possible networks
L=137
![Page 12: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/12.jpg)
Topology Induction: Issues
Find structure of interconnections between variables (I.e., cues, situations)
Much harder than parameter acquisition Formally, maximum likelihood/MAP search
through all possible networks
L=137 L=238
![Page 13: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/13.jpg)
Topology Induction: Issues
Find structure of interconnections between variables (I.e., cues, situations)
Much harder than parameter acquisition Formally, maximum likelihood/MAP search
through all possible networks
L=137 L=238 L=493
![Page 14: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/14.jpg)
Topology Induction: Issues
Find structure of interconnections between variables (I.e., cues, situations)
Much harder than parameter acquisition Formally, maximum likelihood/MAP search
through all possible networks
L=137 L=238 L=493 L=318
![Page 15: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/15.jpg)
Topology Induction: Issues
Find structure of interconnections between variables (I.e., cues, situations)
Much harder than parameter acquisition Formally, maximum likelihood/MAP search
through all possible networks
L=137 L=238 L=493 L=318
![Page 16: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/16.jpg)
Topology Induction: Approaches
Principles of structure search well understood
Gradient ascent, annealing, genetic search, constrained search, etc.
Difficult in practice Computationally intractable Resulting models very sensitive to data Spurious likelihood spikes low
confidence models Compounded by cognitive plausibility
constraint Can get leverage from cognitive plausibility,
though
![Page 17: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/17.jpg)
Cue and Situation IdentificationW
orl
d
Cue0 0
Cue1 1
CueN N
Situation0
Situation1
SituationM
Act
ion
s/D
eci
sion
s
01
10
N1
NM
![Page 18: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/18.jpg)
Cue and Situation Identification: Issues
Discern cues and whole environmental situations employed by user
Related to constructive feature induction, nonlinear projection identification, relational learning, etc.
Search across all possible nodes/relations
N=2 N=3
![Page 19: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/19.jpg)
Cue and Situations: Approaches
Cutting-edge ML problem Direct elicitation is probably most
promising approach Formulating search space/uncertainty
reduction not straightforward Even user interface is difficult (naming
synthetic nodes/relations)
![Page 20: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/20.jpg)
Conclusions
Decrease time/effort/cost to construct and tune cognitive model
Constrained to correspond to human’s internal model Both bane and boon to automated
model construction Insights into operator’s mental
state/decision-making process Requires/drives novel ML algorithms Future work: all of it…
![Page 21: Machine Learning Approaches to Cognitive Parameter Acquisition Terran Lane University of New Mexico terran@cs.unm.edu Chris Forsythe, Patrick Xavier Sandia](https://reader035.vdocuments.net/reader035/viewer/2022070306/5517fbd3550346a2228b4a5b/html5/thumbnails/21.jpg)
Questions?