fundação para a ciência e a tecnologia; phd scholarship – srfh/bd/24628/2005 contacts:
DESCRIPTION
Bayesian Cognitive Models for 3D Structure and Motion Multimodal Perception Multimodal Perception Systems 01/01/2006 - 31/12/2009. Goals - PowerPoint PPT PresentationTRANSCRIPT
Mobile Robotics LaboratoryInstitute of Systems and Robotics
ISR – Coimbra
Bayesian Cognitive Models for3D Structure and Motion Multimodal Perception
Multimodal Perception Systems 01/01/2006 - 31/12/2009
Fundação para a Ciência e a Tecnologia; PhD Scholarship – SRFH/BD/24628/2005
Contacts:João Filipe Ferreira, Jorge Dias {jfilipe,jorge}@isr.uc.pt
• Goals– To research generic Bayesian models to deal with fusion, multimodality, conflicts, and ambiguities in perception and apply them in artificial cognitive systems.
– To answer questions such as:
• Where are the limits on optimal sensory integration behaviour?
• What are the temporal aspects of sensory integration?
• How do top-down influences such as learning, memory and attention affect sensory integration?
• How do we solve the “correspondence problem” for sensory integration? How to answer the combination versus integration debate?
• How to answer the switching versus weighing controversy?
• What are the limits of crossmodal plasticity?
• Motivations
– A moving observer is presented with a non-static 3D scene – how does this observer perceive:• his own motion (egomotion);• the 3D structure of all objects in the scene;• the 3D trajectory and velocity of moving objects (independent motion)?
• Challenges– Perceptual uncertainties:
– Perceptual ambiguities:
Biological Perception,Bayesian Model
Artificial Perception,Bayesian Model
Artificial Observer
SensorReadings
Artificial Perception
Human/BiologicalObserver
Perception Psychophysical Study Model Analysis
Artificial & Biological
Model Output Comparison
EgomotionIllusions, Conflicts & Ambiguities
Model Re-
evaluation
Model Re-
evaluation
Model Re-
evaluation
Model Re-
evaluation
Model
SynthesisM
odel
SynthesisM
odel
SynthesisM
odel
Synthesis
Sensation
3D Scene
& Moving Objectsw/ Static Objects
3D Scene
& Moving Objectsw/ Static Objects
• Expected Outputs
– Development of novel perceptual computational models:
1. based on vision, audition and vestibular sensing;
2. which mimic biological multimodal perceptual fusion processes;
3. which perform perceptual fusion within a Bayesian framework.
Ideal Observer Bayesian Framework
Perceptual Module
Prior Knowledge
Sensory Processing
Posterior
Gain/ Loss Function
Bayes’ RuleDecision Rule
Response
Keypress
Audio
Vinput Voutput
Egomotion
Haptics
Goal
EEG
fMRI
…
Video
Timeline (s)
Psychophysics
Physiology
Stimulus Onset
Stimulus Offset
KeypressTrigger
Perception Action/Response
ChoiceSensation
Triggers
SynchTriggers
Stimulus
Vestibular/InertialSystem
Z
Y
X
Vestibular/InertialSystem
Z
Y
X
Visual SystemVisual SystemBayesian Framework
for MultimodalPromotion → Integration
Moving Objects/Background
Segmentation
3D Occupancy+
3D MotionMap
Ancillary Informationfor
Promotion3D Occupancy
+3D Motion
Map
3D Sound-Source+
3D MotionMap
Egomotion estimate
Auditory SystemAuditory System
3D Scene
& Moving Objectsw/ Static Objects
3D Scene
& Moving Objectsw/ Static Objects
Artificial systems: sensor accuracy and precision discretisation (analogue-to-digital) noise not accounted by artificial perception models round-off effects and data representation limitations
Biological systems: physical constraints on sensors discretisation (analogue-to-spike train) neural noise (firing apparently not due to stimuli) structural constraints on neural representations and computations
??