neurological modeling & cooperation: automatic acquisition of triggered reactions, a...
Post on 21-Dec-2015
226 views
TRANSCRIPT
Neurological Modeling & Cooperation:Automatic Acquisition of Triggered
Reactions, a Physiological Approach
Cooperative Control of Distributed Autonomous Vehicles in Adversarial Environments2001 MURI: UCLA, CalTech, Cornell, MIT
Mao/Massaquoi/Dahleh/Feron
May 14, 2001
UCLA
• Impression– Useful, complex group behavior is based on a
combination of relatively simple, perhaps identical Triggered Programmed-Reactions existing within a collection of nominally autonomous agents
• Hypothesis– The physiological basis for general behavioral
TPRs is the same as that for TPRs used for elemental body movement control/postural regulation
Basic Route
Examples
ss
s
d
s
gX
d
–Maintenance of upright standing and herding are functionally equivalent in body reference frame
– Both postural defense, herd containment and dancing via triggered reactions require• Assessment of continuous (though perhaps only piecewise,
intermittent) sensory information
• Selection of stereotyped movements (motion primitives) that are appropriately scaled and timed to project beyond the anticipated motion of the target
– Learning based on goals and reinforcement as dictated by environment and higher control levels
Selection,Timing,Scaling
Assessment,Prediction
Multichannel sensory information
Partially pre-programmedbehavior
Goals, constraints,Reward/failure
results
Observations
Observations (ct'd)
– Presumably, scaling, timing and selection also automatically learn to take into account supportive or obstructive features of environment, e.g.• Traction/motion characteristics of floor
• responsiveness of target
Or• Presence or absence of multiple actuators (e.g. ankles and hips
when falling forward, hips only when falling backward)
• Presence or absence of other herders on one side vs. another
– General sensitivity to environment may be physiological substrate for functionally useful group-aware behavior
Modeling Assumptions– Natural motor control system can be represented
as a hierarchy consisting of a high level, largely conscious, discrete state-machine-type ‘computer’ and a low/intermediate level, largely unconscious, continuous signal processing controller.
– In between are structures enabling the development of flexible, simple, semi-conscious ‘motor programs’ (behaviors) that address/adhere to the goals and constraints provided by the high level computer
– That is, our Interest:• Understand Control, Assessment and Learning at the
interface between higher and intermediate/low functional levels of natural sensorimotor system
Action Production
Action Monitoring
Continuous Action Control, Assessment & Adaptation (subconscious?)
Environment
Discrete Behavior Control, Assessment & Adaptation (conscious/preconscious?)
MURI
Natural Sensorimotor Control
• More specifically,– Natural Sensorimotor control Hierarchy
• High level Goals (conscious)
– e.g. win point vs. conserve energy
• Strategic Planning/Decisions (conscious) – e.g. return to right rear baseline
• Tactical Objectives (preconscious/”overlearned’?)
– e.g. contact ball with racket face having particular orientation and velocity
• Tactical Assessment/Planning/Decisions (preconscious/’overlearned’?/development of “motor program”)
– assess/predict ball trajectory, spin, body location in court
– use forehand, assume particular posture, generate specific trajectory
Natural Sensorimotor Control
MURI
– Natural Control Hierarchy (cont’d)• Action (force, position) generation& on-line
control (subconscious)
• Action (continuous trajectory) improvement (optimization?) with practice (subconscious motor learning)
• Behavior (discrete program, trajectory) improvement (optimization?) with practice (conscious--> preconscious: ‘tactical motor learning’, ‘motor programming’)
• Behavior improvement (optimization?) with practice (conscious: ‘strategic motor learning’, ‘gamesmanship’)
Natural Sensorimotor Control
MURI
• Natural Sensorimotor Control System
Natural Sensorimotor Control
ST
MTR SENS
MTBGInterface
between highand intermediate/lowcontrol levels involvessensorimotor and association cortices (especially frontal)and the Basal Ganglia.These link ‘automatic’ behavior and reward.Cerebellum likely contributes optimization
(frontal)ASSOC
(parietal)ASSOC
Cbl
Neural signals------------------ executive
sensory
consciousnessgradient
Mtr CtxBrainstem orSpinal Cord
Segment
Im AntCblL Ant
Cbl
Putamen& GP
Caudate& GP
Frontal & ParietalAssoc Ctx
BodyForce/Motion
Muscle & tendon,Joints, skin
“highest level”PLANS (strategy)
“middle level” (“high” and “intermediate”)
PROGRAMS (tactics)
“lower level”ACTION
(force, velocity)
“Motor Servo”
Vestib
Visual
M. Cbl Flocc Cbl
Human motor control principal information flow (adapted from V. Brooks, 1986)
Motor Servo(proprioceptive)
Mtr Ctx
Im AntCblL Ant
Cbl
Putamen & GP, SN (Basal Ganglia)
M. Cbl Flocc Cbl
“high level”PROGRAMS
(discrete control) (tactics:trajectories,
cues)
Frontal & ParietalAssoc Ctx
“highest level”PLANS,
ALGORITHMS (free assoc, strategy)
“intermediate level”CONTROL
(continuous control)(stability, tracking, stiffness,
scaling, movement time)
• MURI to specifically study ‘Programming’ of Triggered Reaction Loops
L Post Cerebellum
Caudate & GP (Basal Ganglia)
TPR LoopCircuitry
Frontal & Parietal Peri-Sensorimotor Ctxs
MURI Goals
Proposed MURI project (Year 1)
– Acquisition of triggered motor reactions
Robot armImplementing virtual targetsandenvironment
Video monitorshowing virtual targetsandenvironment
Proposed MURI project questions: with respect to physiological structures known
or suspected to be involved in TPRs
(Year 1)– What are the motion primitives?
– How are they generated, scaled, timed, triggered?
– What and how is continuous sensory information used?
– How is prediction performed … evidence for internal models?
– How is reinforcement/suppression mediated?
– What is the statistical nature of the learning and programming?
Background studies and resources
– Existing models for intermediate and low/level motor control based on cerebellar and sensorimotor cortical physiology
– Robot arm laboratory– Access to human subjects including those with diseases of
the basal ganglia and cerebellum
Beyond Year 1
– Useful, complex group behavior may emerge from relatively simple, perhaps identical Triggered Programmed-Reactions existing within a collection of nominally autonomous agents
– Link to emergent group behavior possible via experimental observations / prior and similar approaches in Air Traffic Control (eg Mao, Feron and Bilimoria, IEEE ITS, 06/01)
Research funded by NASA, ONR