group 4 neurovision
DESCRIPTION
visionTRANSCRIPT
NEURO VISION
What is so Great about it?
Deceptively simple anatomical appearance. Incredibly complicated structure. Developed through millennia of evolution. Hard-wired to prefer certain objects right at
birth. Invariant to position, scale and rotation of
the object. “Tuned” to quickly recognize objects.
IS HUMANVISION
PERFECT ?
PENROSE’S TRIANGLE
Tricks of the brain
I’d love to see someone try to get to the top
PENROSE’S STAIRS
Why is it so? Our vision system neural network has been tuned
to perform recognition, processing and classification of phenomena that was vital to our survival and progress. In this context, every species has a different vision system.
Hence, we are not very good in dealing with artificially generated images, as these phenomena rarely occurred in nature during our evolution.
However, we are the best for natural images!
How human visual perception works.
How human visual perception works
Perceptions of static scenes are inadequate to describe motion
Gibson’s theory of affordances Vision evolved in organisms embedded in a
dynamically changing environment What is important to an organism is a collection of
processes, not a single unique one These processes are at different levels of abstraction.
E.g. We see waves on a shore, and also the innumerable molecules in it moving
How human visual perception works(contd.)
“Seeing involves multi-level process simulations in partial registration using different ontologies, with rich (but changing) structural relations between levels”
Use of structures of various sorts Agglomeration/grouping: Structures of different sizes at
same level of abstraction Interpretation: Structures at different levels of abstraction-
mapping to a new ontology Fragments recognized in parallel, assembled into larger
wholes-may trigger higher level fragments, or redirect processing at lower levels to resolve ambiguities, etc.
Functions of vision
Segment the image (or scene) and recognize the objects distinguished
Compute distance to contact in every direction Provide feedback and triggers for action Provide a low-level summary of the 2-D and/or 3-
D features of the image, leaving it to the central non-visual processes to draw conclusions
Is something left out?
Visual/spatial reasoning Our ability to use diagrams and visual images to
reason about very abstract mathematical problems, like thinking about the complexity of a search strategy
“Seeing” that 7+5=12 by a rearrangement of dots “Seeing” that angles of a triangle add up to a
straight line Visualize infinitely thin and long lines of
Euclidean geometry Many more examples
Visual/spatial reasoning(contd.) Uses of spatial reasoning: Knowing where to search for an object
thrown over a wall, assembling toy crane from a toy set, uses of spatial concepts(notion of search space) in programming design
Reasoning using a grasp of spatial structures requires at least: the ability to see various structures involved in the proof, the possibilities for variatins(rearrangements) in them, the invariant structures during the rearrangements, etc.
In contrast, a reasoning system like logic is completely discrete and all syntactic composition involves function appllication
Specification of the requirements for visual reasoning is very vague, and would not be easy to mechanize
Visual perception involves much more..
Visual perception involves “affordances” Affordances are the possibilities for, and
constraints on action and change in a situation. Seeing the possibility of things that do not exist, but might exist
Example: A person perceiving a chair can immediately see the possibility of sitting on it, that is, the chair "affords" sitting
Visual perception involves much more..(contd.)
POPEYE(1970’s): The Popeye project investigated how it is possible for humans to see structure in very cluttered scenes, where structure exists at different levels of abstraction-it showed that we recognize fll words before individual alphabets
Consider looking at a smiling or a sad face. Does it involve only perceiving the structure of the pattern? We are able to perceive mental states of happiness or sadness
Visual perception involves much more..(contd.)
What may appear to be only one task, might consist of many different tasks in different contexts, e.g. estimating the length of a plank to fit across a ditch
When a number of images are speedily flashed before the eyes in order, the speed with which people can see at least roughly, what sort of scene is depicted by each image, implies that our visual mechanisms are capable of finding low level features, using them to cue in features of the images at various levels of size and abstraction, arriving at percepts involving known types of objects, within 1 or 2 seconds
High level precisions are made in less than 1/2 a second
Artificial Vision systems-what they aim at
recognize objects or people in static images without acquiring or reasoning about the information using 3-D structure
track moving objects represented in simple shapes(points or blobs) often using 2-D representations
explore an environment building a 2-D map of walls, doors, etc. without a possible human understanding of the maps
control a moving robot, regarded as a moving object obtain some 3-D information about the environment, only
to generate new images
What vision systems cannot do After Freddy, the Edinburgh robot built in 1973, there was a
need to move from 2-D to 3-D. Failure due to limitations of computational power, and difficulty of choosing a representation
Consider a cup on a table. Humans can "see" the orientation required for the grasping object at different grasping locations-visual systems cannot. Alignment of grasped surfaces with grasping ones is important
Affordances in the object being grasped, if it has sharp corners, some part of it is more fragile than others-requires a grasp of counterfactual conditionals involving processes that do not actually exist
Why the limitation? To develop a human-like visual system that will
do what a small child/many animals do-need for an adequate analysis of the requirements for such a system
The requirements might seem much simpler than they actually are, if they are not studied in sufficient depth
The failure to achieve set goals is not a fault of the choice of domains, or the representation-it is a problem of overoptimistic predictions
Where is the complexity? Different levels of perception needed. High level of
precision to lift a hair with a pair of tweezers, much lower precision to see something is not graspable
Perception can involve multi-strand relationships requiring much richer forms of representation that just a logical form
Multiple levels of abstraction, affordances, causation-all is needed
Many more subtleties..
Visual Pathway
Hierarchical Neural Network Architecture
Contents
Brain Mechanism of Vision Hubel‘s and Wiesel's hierarchy model
Cerebral Cortex
Evolution of cerebral cortex is one of the great success in the history of living beings.
Insights of cortical organization: Division into different regions having different
functionalities. e. g. , Visual, auditory, somatic sensory, speech
and motor regions
Visual Pathway
Retina to the Visual Cortex
Hubel’s and Wiesel’s Model
Hierarchical model of cortical cells .
The cortical cells are divide into various types Type IV Simple cells Complex cells
Hubel’s and Wiesel’s ModelType IV Cells have circular symmetry. The receptive field of the cell is divided into
on Center. (Excitatory Center and Inhibitory Surrounding) off Center. (Inhibitory Center and Excitatory Surrounding)
Hubel’s and Wiesel’s Model Simple Cells Respond to an optimally oriented line in a
narrowly defined location. Achieved by requiring the centers of layer
Iv cells that lie along the line.
Hubel’s and Wiesel’s Model Complex cells the main feature of complex cell
o They are less particular about the location, Concerned mainly on orientation.
oAquired is from a number of simple cellso Detects motion(direction specific).
Hubel’s and Wiesel’s Model
Biological Visual Systems as Guides
Modelling attempts to imitate primate vision systems
Extended Hubel-Weisel
Hubel-Weisel hierarchical models have been extended to obtain a fine balance between selectivity and invariance.
Simple and complex cells are interleaved at different levels of the inferotemporal (IT) lobes.
Max-like pooling mechanisms have been suggested at certain levels as opposed to a weighted sum of afferents to boost invariancy in scale, position and rotation.
Feedforward Architecture The S cells (simple cells) in the previous figure passed on information
to the C cells (complex cells) by a bell-tuned weighted sum or a max-like operation.
These cells were further arranged in a higher feature-level hierarchy. Some cells bypass a level in propagating information. This model only considers the feedforward architecture model for the
primary visual cortex, V4 and the posterior IT lobe, and a top-level supervised learning mode (coloured regions). [Serre et al. 2007]
Feedforward Architecture Primates have a very advanced level of attention modulation
(fixation) which is a feedback propagation from the IT lobes to the primary visual cortex and lower levels.
This mechanism allows to shift attention from one part of the image to another.
However, crude object recognition is done in a very small duration after stimulus which indicates use of only the feedforward architecture for rapid categorization.
Such a model was attempted at the McGovern Institute for Brain Research at MIT with some simplifications.
The input consisted of 4 different orientations and several scales, densely covering the gray-value input image of 7ºx 7º
Results The model was evaluated against human responses for input
stimulus of 20ms followed by varying inter-stimulus interval. No single model parameter was adjusted to fit the human
data. All unsupervised parts were fixed and constant throughout all the runs.
The supervised mode was tuned differently in different runs using different test images. Humans were also shown these test images.
An evaluation across all such runs for the identification of animal objects was done for both humans and animals. The results were compared.
Results Various categories of
images in different clutter, scale, position, rotation were given.
Maximum similarity was found for ISI until 80ms.
Conclusions
Biologically inspired computation models have shown very promising results. They are versatile and fast learners. Why not learn from nature’s best?
Advances in neuroscience are picking up, allowing us greater understanding. Also, simulations of hypothetical models will help us validate neuroscience findings.
References Talks by Aaron Sloman, Univ of Birmingham, UK
2005 - 2007http://www.cs.bham.ac.uk/~axs/invited-talks.html
http://www.lifesci.sussex.ac.uk/home/George_Mather/Linked%20Pages/Physiol/Cortex.html
Last accessed - 13 April 2008 Brain Mechanism of Vision,
David H. Hubel and Torsten N. WieselScientific American, September 1979
How We See What See - V. Demidov, Mir Publishers, 1986 A feedforward architecture accounts for rapid categorization
Serre et al., PNAS, 2007 Hierarchical Models of Object Recognition in Cortex
Poggio et al., Nature America, 1999 http://www.thebrain.mcgill.ca
Last accessed - 13 April 2008