laurent itti: cs564 - brain theory and artificial intelligence. saccades 1 1 l. itti: cs564 - brain...
DESCRIPTION
Filling in the Schemas: Neural Network Models Based on Monkey Neurophysiology Peter Dominey & Michael Arbib: Cerebral Cortex, 2: Develop hypotheses on Neural Networks that yield an equivalent functionality: mapping schemas (functions) to the cooperative cooperation of sets of brain regions (structures)TRANSCRIPT
![Page 1: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/1.jpg)
1Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
L. Itti: CS564 - Brain Theory and Artificial IntelligenceUniversity of Southern California
Lecture 15. Saccades 1
Reading Assignments: TMB 6.2The Modular Design of the Oculomotor System in Monkeys
Peter Dominey, Michael Arbib, and Amanda AlexanderSections: Description, Architecture
Supplementary Reading:Crowley-Arbib Saccade ModelM. Crowley, E. Oztop, and S. Marmol
![Page 2: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/2.jpg)
2Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
From Schemas to Neural Networks - A Case Study: Modeling the Control of Saccades
Classical Model of Reflex Saccades
Expanding the Model to Bring in
Cortical Functions
Brainstem Saccade Burst Generator
Eye Plant RETINA Superior
Colliculus
Retinotopic Mapper
Brainstem Saccade Burst Generator
Eye Plant RETINA Superior
Colliculus
Retinotopic Mapper
Target Memory Remapping
![Page 3: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/3.jpg)
PP
SC
vmsm
ms
FOn
FEF
vm
sm
vs
CD
sm
vsSNR
vsmsqv
TH
qv
LGN
VisCx
wta
Brainstem Saccade Generator
eye movement
RetinaVisInput
delay
FEFvsFEFms
FEFvsFEFms
swit
chPPctr
delay
FOn
Filling in the Schemas: Neural Network Models Based on Monkey NeurophysiologyPeter Dominey & Michael Arbib: Cerebral Cortex, 2:153-175
Develop hypotheses on Neural Networks that yield an equivalent functionality: mapping schemas (functions) to the cooperative cooperation of sets of brain regions (structures)
FEF
CAUDATESC
SNr
SG
VisCx
PPMD
![Page 4: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/4.jpg)
4Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Experimental Protocols and Simulation Interfaces
An experimental protocol defines a class of experiments based on:
Preparation used External stimuli Experimental manipulations : electrical stimulation, drug application, etc. Measurements and observations
The protocol can be translated into a simulation interface for “stimulating” a simulation and displaying the results.
![Page 5: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/5.jpg)
5Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Experiment - Double Saccade
![Page 6: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/6.jpg)
6Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Double Saccade -- Neural Array Activity Snapshot
![Page 7: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/7.jpg)
7Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Brain Stem Saccade Burst Generator
LLBN-Long Lead Burst Neurons, MLBN-Medium Lead Burst Neurons, EBN-Excitatory Burst Neurons, PN-Omni-Pause Neurons, RI-Resettable Integrator, TN-Tonic Neurons, MN-Oculomotor Neurons.
In the present Chapter/Lecture, this is treated as an unanalyzed module (available from the NSL Library) so that attention may focus on the other brain regions, which are described next.
![Page 8: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/8.jpg)
8Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Bringing in SC and FEF
SC - Superior Colliculus: The midbrain’s path from vision (and other senses) to eye movement.
FEF - Frontal Eye Field: The cortex’s path from vision to eye movement.
Data point: A cat or monkey can still make saccades if one of SC or FEF is lesioned.
![Page 9: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/9.jpg)
9Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Simple Saccade Task
time
+
![Page 10: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/10.jpg)
10Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Experimental Setup
![Page 11: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/11.jpg)
11Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Timing Diagram for Simple Saccade Experiment
"visinP3M3" is the stimulus for the first target , "fixation" is the fixation timing, "verticalTheta" is the vertical eye movement response, and "horizontalTheta" is the horizontal eye movement response. Notice that the eyes do not move until the "posteriorParietalCenter", a specific neuron in the model of LIP, goes low.
![Page 12: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/12.jpg)
12Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Memory Saccade Experiment
time
+
+
+
![Page 13: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/13.jpg)
13Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Timing Diagram for Memory Saccade Experiment
"visinM2P2" is the stimulus for the target, "fixation" is the fixation timing, "verticalTheta" is the vertical eye movement response, and "horizontalTheta" is the horizontal eye movement response
![Page 14: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/14.jpg)
14Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Double Saccade Experiment
time
+
![Page 15: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/15.jpg)
15Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Timing Diagram for the Double Saccade Experiment
"visinM3P0" is the stimulus for the first target and "visinM3P3" is the stimulus for the second target. "fixation" is the fixation timing, "verticalTheta" is the vertical eye movement response, and "horizontalTheta" is the horizontal eye movement response.
![Page 16: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/16.jpg)
16Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Timing Diagram for Lesioning of SC Experiment
Output of SC is disabled.
"visinP3M3" is the stimulus for the first target , "fixation" is the fixation timing, "verticalTheta" is the vertical eye movement response, and "horizontalTheta" is the horizontal eye movement response.
![Page 17: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/17.jpg)
17Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
FEF Stimulation with SC Lesioned
time
+
FEF Stimulation
![Page 18: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/18.jpg)
18Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
Compensatory Saccade: Stimulation of FEF with Lesioning of SC
A visual target is briefly presented and removed before a saccade can begin. Before the visual saccade can begin, an electrical stimulus is applied to the FEF. The monkey will first saccade to the stimulated location and then to the real target even though timewise the real target appeared first. This is due to the fact that the the visual signal takes time to get from the retina to the FEF.
![Page 19: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/19.jpg)
19Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
The Full Dominey Model
DCEP-Damped Change in Eye Position7a/LIP-Oculomotor Region of Posterior Parietal Cortex
Retina,VisCxDelay(d1)
Mechanism A
7a/LIP/PP
FEF
SC
LLBN,MLBN
EBN MNTN
PN RI
Brain Stem – Saccade Burst Generator
trig
DCEP
velocity
dimension
![Page 20: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/20.jpg)
20Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
![Page 21: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/21.jpg)
21Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1
![Page 22: Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1 1 L. Itti: CS564 - Brain Theory and Artificial Intelligence University of Southern](https://reader035.vdocuments.net/reader035/viewer/2022082210/5a4d1b277f8b9ab059997b3d/html5/thumbnails/22.jpg)
22Laurent Itti: CS564 - Brain Theory and Artificial Intelligence. Saccades 1