Slide 1

16.899A: Physiology (contd) Lavanya Sharan January 24 th, 2011

Slide 2

Before we start, a few caveats A lot is not known about how the human visual system works. We (Alyosha + Lavanya) dont know a lot about physiology. But, before you worry, a few lines from Marr

Slide 3

Slide source: Nancy Kanwisher & Jim DiCarlo

Slide 4

We care about big picture In this class, we are interested in the underlying software/algorithm/computations Not in specifics of the `particular hardware Want back pocket models for various components of the human visual system Very few of these exist. Our closest cousins: computational neuroscientists/cognitive scientists/psychophysicists

Slide 5

Overview of `particular hardware Retina, LGN What are visual areas? Tools for studying human visual system Area V1 Beyond area V1 What and where pathways Summary of what (and how little) we know

Slide 6

Primary Visual Pathway 1.Retina 2.Thalamus Lateral Geniculate Nucleus (LGN) divided into magno and parvo layers 3.Primary visual cortex (V1) 4.Extrastriate visual areas Each visual hemifield projects to the opposite hemisphere Slide source: Jody Culham

Slide 7

7 Slide source: Nancy Kanwisher & Jim DiCarlo

Slide 8

Slide 9

Primary Visual Pathway 1.Retina 2.Thalamus Lateral Geniculate Nucleus (LGN) divided into magno and parvo layers 3.Primary visual cortex (V1) 4.Extrastriate visual areas Each visual hemifield projects to the opposite hemisphere Slide source: Jody Culham

Slide 10

What is a Visual Area? 1.Function an area has a unique pattern of responses to different stimuli 2.Architecture different brain areas show differences between cortical properties (e.g., thickness of different layers, sensitivity to various dyes) 3.Connectivity Different areas have different patterns of connections with other areas 4.Topography many sensory areas show topography (retinotopy, somatotopy, tonotopy) boundaries between topographic maps can indicate boundaries between areas (e.g., separate maps of visual space in visual areas V1 and V2 Slide source: Jody Culham

Slide 11

Why are there so many visual areas? Source: Felleman & Van Essen, 1991Source: Mapping the MInd cover image MAGNO quick and dirty PARVO slow and detailed Slide source: Jody Culham

Slide 12

More brain, more visual areas Slide source: Jody Culham

Slide 13

Why not one really big visual area? V1 Slide source: Jody Culham

Slide 14

Why not a really big visual area? As areas become larger, longer interconnections are required Limits on cortical thickness and connections may constrain max area size Slide source: Jody Culham

Slide 15

Parallel processing is more efficient Teach neural network to identify what and where One neural network with 18 nodes (~neurons) devoted to both tasks versus One neural networks with two streams of 9 nodes each (total = 18) After 300 training trials, the two stream model outperformed the single-system model Rueckl, Cave & Kosslyn, 1989 Slide source: Jody Culham

Slide 16

Different Tasks Require Different Information different regions may need to use different coding systems ventral stream: object-centred dorsal stream: viewer-centred Slide source: Jody Culham

Slide 17

Wiring Constraints Source: Van Essen, 1997 David Van Essen proposes that as the brain develops, areas that are richly interconnected will be pulled together to form a gyrus (and those that are weakly interconnected form sulci). Slide source: Jody Culham

Slide 18

Sulcal Formation: V1-V2 Source: Van Essen, 1997 The V1/V2 border provides one example of two richly interconnected areas that form a gyrus. This arrangement also explains why maps in V1 and V2 are mirror images of each other! calcarine sulcus Slide source: Jody Culham

Slide 19

Optimized Connections Multidimensional Scaling strength of connections can be used to infer spatial layout expected layout of visual areas matches anatomy amazingly well Occipital Parietal Temporal Malcolm Young Slide source: Jody Culham

Slide 20

Tools for mapping human areas Neuropsychological Lesions Temporary Disruption transcranial magnetic stimulation (TMS) Electrical and magnetic signals electroencephalography (EEG) magnetoencephalography (MEG) Brain Imaging positron emission tomography (PET) functional magnetic resonance imaging (fMRI) Slide source: Jody Culham

Slide 21

Slide source: Nancy Kanwisher & Jim DiCarlo

Slide 22

Overview of `particular hardware Retina, LGN What are visual areas? Tools for studying human visual system Area V1 Beyond area V1 What and where pathways Summary of what (and how little) we know

Slide 23

Cortical Receptive Fields Single-cell recording from visual cortex David Hubel & Thorston Wiesel Stephen E. Palmer, 2002

Slide 24

Cortical Receptive Fields Single-cell recording from visual cortex Stephen E. Palmer, 2002

Slide 25

Cortical Receptive Fields Three classes of cells in V1 Simple cells Complex cells Hypercomplex cells Stephen E. Palmer, 2002

Slide 26

Cortical Receptive Fields Simple Cells: Line Detectors Stephen E. Palmer, 2002

Slide 27

Cortical Receptive Fields Simple Cells: Edge Detectors Stephen E. Palmer, 2002

Slide 28

Cortical Receptive Fields Constructing a line detector Stephen E. Palmer, 2002

Slide 29

Cortical Receptive Fields Complex Cells 0o0o Stephen E. Palmer, 2002

Slide 30

Cortical Receptive Fields Complex Cells 60 o Stephen E. Palmer, 2002

Slide 31

Cortical Receptive Fields Complex Cells 90 o Stephen E. Palmer, 2002

Slide 32

Cortical Receptive Fields Complex Cells 120 o Stephen E. Palmer, 2002

Slide 33

Cortical Receptive Fields Constructing a Complex Cell Stephen E. Palmer, 2002

Slide 34

Cortical Receptive Fields Hypercomplex Cells Stephen E. Palmer, 2002

Slide 35

Cortical Receptive Fields Hypercomplex Cells Stephen E. Palmer, 2002

Slide 36

Cortical Receptive Fields Hypercomplex Cells Stephen E. Palmer, 2002

Slide 37

Cortical Receptive Fields Hypercomplex Cells End-stopped Cells Stephen E. Palmer, 2002

Slide 38

Cortical Receptive Fields End-stopped Simple Cells Stephen E. Palmer, 2002

Slide 39

Cortical Receptive Fields Constructing a Hypercomplex Cell Stephen E. Palmer, 2002

Slide 40

Overview of `particular hardware Retina, LGN What are visual areas? Tools for studying human visual system Area V1 Beyond area V1 What and where pathways Summary of what (and how little) we know

Slide 41

Logothetis 1999; from http://psychology.uwo.ca/fMRI4Newbies/RetinotopicandEarlyVisualAreas.html Overview of visual areas

Slide 42

Macaque & human visual areas are similar Tootell et al. 2003; from http://psychology.uwo.ca/fMRI4Newbies/RetinotopicandEarlyVisualAreas.html

Slide 43

Slide source: Nancy Kanwisher & Jim DiCarlo

Slide 44

Retinotopy (Tootell et al. 1982) Adjacent parts of visual field are mapped to adjacent parts of cortex. Not all visual areas have retinotopy, may be graded. Slide source: Nancy Kanwisher & Jim DiCarlo

Slide 45

Slide source: Nancy Kanwisher, Jim DiCarlo, David Heeger

Slide 46

Slide 47

Why edges? So, why edge-like structures in the Plenoptic Function?

Slide 48

Two visual pathways The two visual processing streams for different visual percepts: What (ventral stream)- object recognition main input from slow and detailed parvo system Where or How (dorsal stream) - spatial perception, motor planning main input from quick and dirty magno system Source: Mishkin & Ungerleider, 1982 Slide source: Jody Culham

Slide 49

Two visual pathways The two visual processing streams for different visual percepts: What (ventral stream)- object recognition main input from slow and detailed parvo system Where or How (dorsal stream) - spatial perception, motor planning main input from quick and dirty magno system Source: Mishkin & Ungerleider, 1982 Slide source: Jody Culham

Slide 50

The What Pathway body motionfacesplacesbodiesobjects Other Visual Areas contain more complex receptive fields Temporal Lobe contains many specialized areas for recognizing various things Slide source: Jody Culham

Slide 51

The Where or How Pathway eye movements grasping and reaching motion perception Parietal Lobe contains many specialized areas for using vision to guide actions in space head movements attention Slide source: Jody Culham

Slide 52

Slide source: Nancy Kanwisher & Jim DiCarlo

Slide 53

Summary Low-level areas Filter banks, SIFT, HOG for color, orientation, spatial frequencies, motion High-level areas Desired output from computer vision systems e.g., segmentation, robust object/scene/texture recognition, motion understanding and planning Middle-level area Where the magic happens No one (neuroscientists, psychologists, computer scientists, etc.) really understands this stage of processing. For more, come find us for pointers to papers/books/readings and people to talk to.