binocular vision development, depth perception and disorders

EYE AND VISION RESEARCH DEVELOPMENTS

BINOCULAR VISION:

DEVELOPMENT, DEPTH

PERCEPTION AND DISORDERS

EYE AND VISION RESEARCH

DEVELOPMENTS

Eye Cancer Research Progress Edwin B. Bospene (Editor)

2008. ISBN: 978-1-60456-045-9

Non-Age Related Macular Degeneration Enzo B. Mercier

2008. ISBN: 978-1-60456-305-4

Optic Nerve Disease Research Perspectives Benjamin D. Lewis and Charlie James Davies (Editors)

2008. ISBN: 978-1-60456-490-7 2008. ISBN: 978-1-60741-938-9 (E-book)

New Topics in Eye Research

Lauri Korhonen and Elias Laine (Editors) 2009. ISBN: 978-1-60456-510-2

Eye Infections, Blindness and Myopia

Jeffrey Higgins and Dominique Truax (Editors) 2009. ISBN: 978-1-60692-630-7

Eye Research Developments:

Glaucoma, Corneal Transplantation, and Bacterial Eye Infections Alan N. Westerhouse (Editor)

2009. ISBN: 978-1-60741-1772

Retinal Degeneration: Causes, Diagnosis and Treatment Robert B. Catlin (Editor)

2009. ISBN: 978-1-60741-007-2 2009. ISBN: 978-1-60876-442-6 (E-book)

Binocular Vision: Development, Depth Perception and Disorders

Jacques McCoun and Lucien Reeves (Editors)

2010. ISBN: 978-1-60876-547-8

Understanding Corneal Biomechanics through Experimental

Assessment and Numerical Simulation

Ahmed Elsheikh

2010. ISBN: 978-1-60876-694-9

Retinitis Pigmentosa: Causes, Diagnosis and Treatment

Michaël Baert and Cédric Peeters (Editors)

2010. ISBN: 978-1-60876-884-4

Color: Ontological Status and Epistemic Role Anna Storozhuk

2010. ISBN: 978-1-61668-201-9 2010. ISBN: 978-1-61668-608-6 (E-book)

Coherent Effects in Primary Visual Perception

V.D. Svet and A.M. Khazen

2010. ISBN: 978-1-61668-143-2 2010. ISBN: ISBN: 978-1-61668-496-9 (E-book)

Conjunctivitis: Symptoms, Treatment and Prevention

Anna R. Sallinger

2010. ISBN: 978-1-61668-321-4 2010. ISBN: 978-1-61668-443-3 (E-book)

Novel Drug Delivery Approaches in Dry Eye Syndrome Therapy

Slavomira Doktorovová, Eliana B. Souto, Joana R. Araújo,

Maria A. Egea and Marisa L. Garcia

2010. ISBN: 978-1-61668-768-7 2010. ISBN: 978-1-61728-449-6 (E-book)

Pharmacological Treatment of Ocular Inflammatory Diseases

Tais Gratieri, Renata F. V. Lopez, Elisabet Gonzalez-Mira,

Maria A. Egea and Marisa L. Garcia

2010. ISBN: 978-1-61668-772-4 2010. ISBN: 978-1-61728-470-0 (E-book)

Cataracts: Causes, Symptoms, and Surgery

Camila M. Hernandez (Editor)

2010. ISBN: 978-1-61668-955-1 2010. ISBN: 978-1-61728-312-3 (E-book)

EYE AND VISION RESEARCH DEVELOPMENTS

BINOCULAR VISION:

DEVELOPMENT, DEPTH

PERCEPTION AND DISORDERS

JACQUES MCCOUN

AND

LUCIEN REEVES

EDITORS

Nova Science Publishers, Inc.

New York

No part of this digital document may be reproduced, stored in a retrieval system or transmitted in any form orby any means. The publisher has taken reasonable care in the preparation of this digital document, but makes noexpressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. Noliability is assumed for incidental or consequential damages in connection with or arising out of informationcontained herein. This digital document is sold with the clear understanding that the publisher is not engaged inrendering legal, medical or any other professional services.

Copyright © 2010 by Nova Science Publishers, Inc. All rights reserved. No part of this book may be reproduced, stored in a retrieval system or transmitted in any form or by any means: electronic, electrostatic, magnetic, tape, mechanical photocopying, recording or otherwise without the written permission of the Publisher. For permission to use material from this book please contact us: Telephone 631-231-7269; Fax 631-231-8175 Web Site: http://www.novapublishers.com

NOTICE TO THE READER

The Publisher has taken reasonable care in the preparation of this book, but makes no expressed or implied warranty of any kind and assumes no responsibility for any errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of information contained in this book. The Publisher shall not be liable for any special, consequential, or exemplary damages resulting, in whole or in part, from the readers’ use of, or reliance upon, this material. Any

parts of this book based on government reports are so indicated and copyright is claimed for those parts to the extent applicable to compilations of such works. Independent verification should be sought for any data, advice or recommendations contained in this book. In addition, no responsibility is assumed by the publisher for any injury and/or damage to persons or property arising from any methods, products, instructions, ideas or otherwise contained in this publication. This publication is designed to provide accurate and authoritative information with regard to the subject matter covered herein. It is sold with the clear understanding that the Publisher is not engaged in rendering legal or any other professional services. If legal or any other expert assistance is required, the services of a competent person should be sought. FROM A DECLARATION OF PARTICIPANTS JOINTLY ADOPTED BY A COMMITTEE OF THE AMERICAN BAR ASSOCIATION AND A COMMITTEE OF PUBLISHERS. LIBRARY OF CONGRESS CATALOGING-IN-PUBLICATION DATA

Binocular vision : development, depth perception, and disorders / editors, Jacques McCoun and Lucien Reeves. p. ; cm. Includes bibliographical references and index. ISBN 978-1-61761-957-1 (eBook) 1. Binocular vision. 2. Binocular vision disorders. 3. Computer vision. 4. Depth perception. I. McCoun, Jacques. II. Reeves, Lucien. [DNLM: 1. Vision, Binocular--physiology. 2. Dominance, Ocular--physiology. 3. Pattern Recognition, Visual--physiology. 4. Vision Disparity--physiology. WW 400 B6145 2009] QP487.B56 2009 612.8'4--dc22 2009038663

Published by Nova Science Publishers, Inc. New York

CONTENTS

Preface ix

Chapter 1 New Trends in Surface Reconstruction Using Space-Time Cameras: Fusing Structure from Motion, Silhouette, and Stereo

1

Hossein Ebrahimnezhad and Hassan Ghassemian

Chapter 2 Ocular Dominance within Binocular Vision 63

Jonathan S. Pointer

Chapter 3 Three-Dimensional Vision Based on Binocular Imaging and Approximation Networks of a Laser Line

81

J. Apolinar Muñoz-Rodríguez

Chapter 4 Eye Movement Analysis in Congenital Nystagmus: Concise Parameters Estimation

107

Pasquariello Giulio, Cesarelli Mario,

La Gatta Antonio, Bifulco Paolo and Fratini Antonio

Chapter 5 Evolution of Computer Vision Systems 125

Vladimir Grishin

Chapter 6 Binocular Vision and Depth Perception: Development and Disorders

139

Ken Asakawa and Hitoshi Ishikawa

Contents viii

Chapter 7 Repeatability of Prism Dissociation and Tangent Scale Near Heterophoria Measurements in Straightforward Gaze and in Downgaze

155

David A. Goss, Douglas K. Penisten, Kirby K. Pitts

and Denise A. Burns

Chapter 8 Temporarily Blind in One Eye: Emotional Pictures Predominate in Binocular Rivalry

161

Georg W. Alpers and Antje B.M. Gerdes

Chapter 9 Stereo-Based Candidate Generation for Pedestrian Protection Systems

189

David Geronimo, Angel D. Sappa

and Antonio M. López

Chapter 10 Development of Saccade Control 209

Burkhart Fischer

Short Commentary 247

Ocular Dominance

Jonathan S. Pointer

Index 249

PREFACE "Binocular vision" literally means vision with two eyes, and refers to the

special attributes of vision with both eyes open, rather than one eye only. Our perception under binocular conditions represents a highly complex coordination of motor and sensory processes and is markedly different from and more sophisticated than vision with one eye alone. This book reviews our ability to use both eyes, while also providing basic information on the development of binocular vision and on the clinical disorders that interfere with our depth perception, such as strabismus and amblyopia. This book also describes the development of eye movement control, particularly those that are important for reading. In addition, the authors of this book review the phenomenon of ocular dominance (OD) in the light of the types of test used to identify it; question whether inter-test agreement of OD in an individual might be anticipated, and address some practical implications of OD as demonstrated in healthy eyes and in cases where there is compromised binocular function. Other chapters in this book disclose new methodologies in congenital nystagmus eye movements analysis and evaluate heterophoria as an important element of assessment of binocular vision disorders.

Three dimensional model reconstruction from image sequences has been extensively used in recent years. The most popular method is known as structure from motion, which employs feature and dense points matching to compute the motion and depth. Chapter 1 is intended to present an overview of new trends in three dimensional model reconstruction using multiple views of object, which has been developed by the authors. Robust curve matching method in stereo cameras for extraction of unique space curves is explained. Unique space curves are constructed from plane curves in stereo images based on curvature and torsion consistency. The shortcoming of outliers in motion estimation is extremely

Jacques McCoun and Lucien Reeves x

reduced by employing the space curves. Besides, curve matching method deals with pixel range information and does not require the sub-pixel accuracy to compute structure and motion. Furthermore, it finds the correspondence based on curve shape and does not use any photometric information. This property makes the matching process very robust against the color and intensity maladjustment of stereo rigs. The recovered space curves are employed to estimate robust motion by minimizing the curve distance in the next sequence of stereo images. An efficient structure of stereo rigs – perpendicular double stereo – is presented to increase accuracy of motion estimation. Using the robust motion information, a set of exactly calibrated virtual cameras is constructed, which the authors call space-time cameras. Then, the visual hull of object is extracted from intersection of silhouette cones of all virtual cameras. Finally, color information is mapped to the reconstructed surface by inverse projection from two dimensional image sets to three-dimensional space. All together, the authors introduce a complete automatic and practical system of three-dimensional model reconstruction from raw images of arbitrarily moving object captured by fixed calibrated perpendicular double stereo rigs to surface representation. While, the simple methods of motion estimation suffer from the statistical bias due to quantization noise, measurement error, and outliers in the input data set; the complicated system overcomes the bias problem, by fusing several constraints, even in pixel-level information. Experimental results demonstrate the privileged performance of the complicated system for a variety of object shapes and textures.

Ocular dominance (OD) can be defined and identified in a variety of ways. It might be the eye used to sight or aim, or whose input is favoured when there is competing information presented to the two eyes, or the eye whose functional vision appears superior on a given task or under certain conditions. The concept, which has been the subject of much discussion and revision over the past four centuries, continues to excite controversy today. What is becoming evident is that even in its most direct and behaviourally significant manifestation – sighting preference – it must be regarded as a flexible laterality within binocular vision, influenced by the physical circumstances and viewing constraints prevailing at the point of testing.

Chapter 2 will review the phenomenon of OD in the light of the types of test used to identify it; question whether inter-test agreement of OD in an individual might be anticipated; briefly consider the possibility of any relationship between OD and limb or cortical laterality; and speculate whether OD is essentially the product of forced monocular viewing conditions and habitual use of one or other eye. The chapter will conclude with remarks addressing some practical

Preface xi

implications of OD as demonstrated in healthy eyes and in cases where there is compromised binocular function.

The authors present a review of their computer vision algorithms and binocular imaging for shape detection optical metrology. The study of Chapter 3 involves: laser metrology, binocular image processing, neural networks, and computer vision parameters. In this technique, the object shape is recovered by means of laser scanning and binocular imaging. The binocular imaging avoids occlusions, which appear due to the variation to the object surface. A Bezier approximation network computes the object surface based on the behavior of the laser line. By means of this network, the measurements of the binocular geometry are avoided. The parameters of the binocular imaging are computed based on the Bezier approximation network. Thus, the binocular images of the laser line are processed by the network to compute the object topography. By applying Bezier approximation networks, the performance of the binocular imaging and the accuracy are improved. It is because the errors of the measurement are not added to the computational procedure, which performs the shape reconstruction. This procedure represents a contribution for the stripe projection methods and the binocular imaging. To describe the accuracy a mean square error is calculated. This technique is tested with real objects and its experimental results are presented. Also, the time processing is described.

Along with other diseases that can affect binocular vision, reducing the visual quality of a subject, Congenital Nystagmus (CN) is of peculiar interest. CN is an ocular-motor disorder characterized by involuntary, conjugated ocular oscillations and, while identified more than forty years ago, its pathogenesis is still under investigation. This kind of nystagmus is termed congenital (or infantile) since it could be present at birth or it can arise in the first months of life. The majority of CN patients show a considerable decrease of their visual acuity: image fixation on the retina is disturbed by nystagmus continuous oscillations, mainly horizontal. However, the image of a given target can still be stable during short periods in which eye velocity slows down while the target image is placed onto the fovea (called foveation intervals). To quantify the extent of nystagmus, eye movement recordings are routinely employed, allowing physicians to extract and analyze nystagmus main features such as waveform shape, amplitude and frequency. Use of eye movement recording, opportunely processed, allows computing “estimated

visual acuity” predictors, which are analytical functions that estimate expected

visual acuity using signal features such as foveation time and foveation position variability. Hence, it is fundamental to develop robust and accurate methods to measure both those parameters in order to obtain reliable values from the predictors. In this chapter the current methods to record eye movements in

Jacques McCoun and Lucien Reeves xii

subjects with congenital nystagmus will be discussed and the present techniques to accurately compute foveation time and eye position will be presented.

Chapter 4 aims to disclose new methodologies in congenital nystagmus eye movements analysis, in order to identify nystagmus cycles and to evaluate foveation time, reducing the influence of repositioning saccades and data noise on the critical parameters of the estimation functions. Use of those functions extends the information acquired with typical visual acuity measurement (e.g., Landolt C test) and could be a support for treatment planning or therapy monitoring.

In Chapter 5, applications of computer vision systems (CVS) in the flight control of unmanned aerial vehicles (UAV) are considered. In many projects, CVS are used for precision navigation, angular and linear UAV motion measurement, landing (in particular shipboard landing), homing guidance and others. All these tasks have been successfully solved separately in various projects. The development of perspective CVS can be divided into two stages. The first stage of perspective CVS development is the realization of all the above tasks in a single full-scale universal CVS with acceptable size, weight and power consumption. Therefore, all UAV flight control tasks can be performed in automatic mode on the base of information that is delivered by CVS. All necessary technologies exist and the degree of its maturity is high. The second stage of CVS development is integration of CVS and control systems with artificial intelligence (AI). This integration will bring two great benefits. Firstly it will allow considerable improvement of CVS performance and reliability due to accumulation of additional information about the environment. Secondly, the AI control system will obtain a high degree of awareness about the state of the environment. This allows the realization of a high degree of control effectiveness of the autonomous AI system in a fast changing and hostile environment.

“Binocular vision” literally means vision with two eyes, and refers to the

special attributes of vision with both eyes open, rather than one eye only. Our perception under binocular conditions represents a highly complex coordination of motor and sensory processes and is markedly different from and more sophisticated than vision with one eye alone. However, the use of a pair of eyes can be disrupted by a variety of visual disorders, e.g., incorrect coordination between the two eyes can produce strabismus with its associated sensory problems, amblyopia, suppression and diplopia. What, then, is the reason for-and the advantage of-having two eyes? From our visual information input, we can perceive the world in three dimensions even though the images falling on our two retinas are only two-dimensional. How is this accomplished? Chapter 6 is a review of our ability to use both eyes, while also providing basic information on

Preface xiii

the development of binocular vision and on the clinical disorders that interfere with our depth perception, such as strabismus and amblyopia.

The evaluation of heterophoria is an important element of assessment of binocular vision disorders. Chapter 7 examined the interexaminer repeatability of two heterophoria measurement methods in a gaze position with no vertical deviation from straightforward position and in 20 degrees downgaze. The two procedures were von Graefe prism dissociation method (VG) and the tangent scale method commonly known as the modified Thorington test (MT). Serving as subjects were 47 young adults, 22 to 35 years of age. Testing distance was 40 cm. A coefficient of repeatability was calculated by multiplying the standard deviation of the difference between the results from two examiners by 1.96. Coefficients of repeatability in prism diopter units were: VG, straightforward, 6.6; VG, downgaze, 6.2; MT, straightforward, 2.8; MT, downgaze, 3.6. The results show a better repeatability for the tangent scale procedure than for the von Graefe prism dissociation method.

As explained in Chapter 8, preferential perception of emotional cues may help an individual to respond quickly and effectively to relevant events. Existing data supports this hypothesis by demonstrating that emotional cues are more quickly detected among neutral distractors. Little data is available to demonstrate that emotional stimuli are also preferentially processed during prolonged viewing. The preferential perception of visual emotional cues is apparent under conditions where different cues compete for perceptual dominance. When two incompatible pictures are presented to one eye each, this results in a perceptual alternation between the pictures, such that only one picture is visible while the other is suppressed. This so called binocular rivalry involves different stages of early visual processing and is thought to be relatively independent from intentional control. Several studies from our laboratory showed that emotional stimuli predominate over neutral stimuli in binocular rivalry. These findings can be interpreted as evidence for preferential processing of emotional cues within the visual system, which extends beyond initial attentional capture. Taken together, data from this paradigm demonstrates that emotional pictures are perceived more intensively.

Chapter 9 describes a stereo-based algorithm that provides candidate image windows to a latter 2D classification stage in an on-board pedestrian detection system. The proposed algorithm, which consists of three stages, is based on the use of both stereo imaging and scene prior knowledge (i.e., pedestrians are on the ground) to reduce the candidate searching space. First, a successful road surface fitting algorithm provides estimates on the relative ground-camera pose. This stage directs the search toward the road area thus avoiding irrelevant regions like

Jacques McCoun and Lucien Reeves xiv

the sky. Then, three different schemes are used to scan the estimated road surface with pedestrian-sized windows: (a) uniformly distributed through the road surface (3D); (b) uniformly distributed through the image (2D); (c) not uniformly distributed but according to a quadratic function (combined 2D- 3D). Finally, the set of candidate windows is reduced by analyzing their 3D content. Experimental results of the proposed algorithm, together with statistics of searching space reduction are provided.

Chapter 10 describes the development of eye movement control. The authors will consider, however, only those aspects of eye movements that are important for reading: stability of fixation and control of saccades (fast eye movements from one object of interest to another). The saccadic reflex and the control of saccades by voluntary conscious decision and their role in the optomotor cycle will be explained on the basis of the reaction times and neurophysiological evidence. The diagnostic methods used in the next part of the book will be explained in this chapter. The age curves of the different variables show that the development of the voluntary component of saccade control lasts until adulthood.

The Short Commentary discusses ocular dominance and the rationale behind this phenomenon.

In: Binocular Vision ISBN: 978-1-60876-547-8Editors: J. McCoun et al, pp. 1-62 © 2010 Nova Science Publishers, Inc.

Chapter 1

NEW TRENDS IN SURFACE RECONSTRUCTIONUSING SPACE-TIME CAMERAS:

FUSING STRUCTURE FROM MOTION,SILHOUETTE, AND STEREO

Hossein Ebrahimnezhad1,a and Hassan Ghassemian2,b

1 Sahand University of Technology, Dept. of Electrical Engineering, Computer Vision Research Lab, Tabriz, Iran

2 Tarbiat Modaress University,Dept. of Electrical and Computer Engineering, Tehran, Iran

Abstract

Three dimensional model reconstruction from image sequences has beenextensively used in recent years. The most popular method is known as structurefrom motion, which employs feature and dense points matching to compute themotion and depth. This chapter is intended to present an overview of new trendsin three dimensional model reconstruction using multiple views of object, whichhas been developed by the authors [43]. Robust curve matching method in stereocameras for extraction of unique space curves is explained. Unique space curvesare constructed from plane curves in stereo images based on curvature andtorsion consistency. The shortcoming of outliers in motion estimation is

a E-mail address: [email protected],.b E-mail address: [email protected] address: http://ee.sut.ac.ir/ showcvdetail.aspx?id=5

Hossein Ebrahimnezhad and Hassan Ghassemian2

extremely reduced by employing the space curves. Besides, curve matchingmethod deals with pixel range information and does not require the sub-pixelaccuracy to compute structure and motion. Furthermore, it finds thecorrespondence based on curve shape and does not use any photometricinformation. This property makes the matching process very robust against thecolor and intensity maladjustment of stereo rigs. The recovered space curves areemployed to estimate robust motion by minimizing the curve distance in the nextsequence of stereo images. An efficient structure of stereo rigs – perpendiculardouble stereo – is presented to increase accuracy of motion estimation. Using therobust motion information, a set of exactly calibrated virtual cameras isconstructed, which we call them space-time cameras. Then, the visual hull ofobject is extracted from intersection of silhouette cones of all virtual cameras.Finally, color information is mapped to the reconstructed surface by inverseprojection from two dimensional image sets to three-dimensional space. Alltogether, we introduce a complete automatic and practical system of three-dimensional model reconstruction from raw images of arbitrarily moving objectcaptured by fixed calibrated perpendicular double stereo rigs to surfacerepresentation. While, the simple methods of motion estimation suffer from thestatistical bias due to quantization noise, measurement error, and outliers in theinput data set; the complicated system overcomes the bias problem, by fusingseveral constraints, even in pixel-level information. Experimental resultsdemonstrate the privileged performance of the complicated system for a varietyof object shapes and textures.

Keywords: 3D model reconstruction; space-time cameras; perpendicular doublestereo; structure from silhouette; structure from motion; space curves; uniquepoints; visual hull.

1. Introduction

Reconstruction of surface model for a moving rigid object, through asequence of photo images, is a challenging problem and an active research topicin computer vision. In recent years, there has been extensive focus in literature torecover three-dimensional structure and motion from image sequences [1-6].Different types of algorithms are used because of the wide range of options, e.g.,the image projection model, number of cameras and available views, availabilityof camera calibration, feature types and model of the scene. For a fixed objectwith a moving camera (or a moving rigid object with a fixed camera) setup, theshape and motion recovery problem can be formulated as trying to find out the 6motion parameters of the object, e.g. its position and orientation displacementtogether with the accurate 3D world coordinates for each point. This problem isalso known as bundle adjustment [7]. The standard method of rigid motion

New Trends in Surface Reconstruction Using Space-Time Cameras 3

recovery has been developed in the last decade based on sparse feature points [8-9]. The sparse method typically assumes that correspondences between scenefeatures such as corners or surface creases have been established by trackingtechnique. It can compute only the traveling camera positions, and is not sufficient formodeling the object as it only reconstructs sparsely distributed 3D points. Typically,motion estimation methods suffer from instability due to quantization noise,measurement errors, and outliers in the input datasets. Outliers occur in thefeature-matching process due mostly to occlusions. Different robust estimationtechniques have been proposed to handle outliers. RANdom SAmple Consensus(RANSAC) is known as a successful technique to deal with outliers [10]. M-Estimators reduce the effects of outliers by applying the weighted leastsquares[11]. Many other similar methods also are available [12-13]. Another standardmethod of shape recovery from motion has been developed in the last decadebased on optical flow [1, 2, and 14].

The process of structure and motion recovery usually consists of theminimization of some cost function. There are two dominant approaches tochoose the cost function. The first approach is based on epipolar geometry leadingto a decoupling of the shape and motion recovery. In epipolar constraint approach,the cost function reflects the amount of deviation of the epipolar constraint asmade happen by noise and other measurement errors. In this method, the motioninformation can be achieved as the solution to a linear problem. Presence ofstatistical bias in estimating the translation [15-17] as well as the sensitivity tonoise and pixel quantization is the conventional drawback, which makesadditional error in linear solution. Even small pixel-level perturbations can makethe image plane information ineffective and cause the wrong motion recovery. Toimprove the solution, some methods minimize the cost function using thenonlinear iterative methods like Levenberg-Marquardt algorithm [8]. Suchmethods are initialized with the output of the linear algorithms. The secondapproach directly minimizes the difference between observed and predictedfeature coordinates using Levenberg-Marquardt algorithm [18]. This method ismarked by a high-dimensional search space (typically n+6 for n imagecorrespondences) and, unlike the epipolar constraint-based approach, it does notexplicitly account for the fact that a one-parameter family of solutions exists.

In general, the structure and motion from monocular view image sequences isinherently a knotty problem and has its own restrictions, as the computations arevery sensitive to noise and quantization of image points. Actually, the motion andstructure computations are highly dependent to each other and any ambiguity instructure computation propagates to motion computation and vise versa. On theother hand, calibrated stereo vision directly computes the structure of feature


points. Therefore, integrating stereo and motion can reasonably improve thestructure and motion procedure. Some works in literature fuse stereo and motionfor rigid scene to get better results. Young et al. [19] computed the rigid motionparameters assuming that depth information had been computed already by stereovision. Weng et al. [20] derived a closed form approximate matrix weighted leastsquares solution for motion parameters from three-dimensional pointcorrespondences in two stereo image pairs. Li et al. [21] proposed a two-stepfusing procedure: first, translational motion parameters were found from opticalflows in binocular images, then the stereo correspondences were estimated withthe knowledge of translational motion parameters. Dornaika et al. [22] recoveredthe stereo correspondence using motion of a stereo rig in two consecutive steps.The first step uses metric data associated with the stereo rig while the second stepemploys feature correspondences only. Ho et al. [23] combined stereo and motionanalyses for three-dimensional reconstruction when a mobile platform wascaptured with two fixed cameras. Park et. al [24] estimated the object motiondirectly through the calibrated stereo image sequences. Although, the combinationform of motion and stereo enhances the computations [25], presence of statisticalbias in estimating the motion parameters still has destructive effect in structureand motion estimation procedure.

In this chapter, a constructive method is presented to moderate the bias problemusing curve based stereo matching and robust motion estimation by tracking theprojection of space curves in perpendicular double stereo images. We provemathematically and demonstrate experimentally that the presented method canincrease motion estimation accuracy and reduce the problem of statistical bias.Moreover, the perpendicular double stereo setup appears to be more robust against theperturbation of edge points. Any large error in depth direction of stereo rig 1 isrestricted by minimizing the error in parallel direction of stereo rig 2 and vice versa.In addition, the curve-matching scheme is very robust against the color maladjustmentof cameras and shading problem during object motion.

In section 2, a robust edge point correspondence with match propagationalong the curves is presented to extract unique space curves with extremelyreduced number of outliers. In section 3, two sets of space curves, which havebeen extracted from two distinct stereo rigs, are used to estimate object motion insequential frames. A space curve-tracking algorithm is presented by minimizingthe geometric distance of moving curves in camera planes. The proposed curve-tracking method works as well with pixel accuracy information and does notrequire the complicated process of position computation in sub-pixel accuracy. Anefficient structure of stereo setup - the perpendicular double stereo - is presentedto get as much accuracy as possible in motion estimation process. Its properties


are discussed and proven mathematically. In section 4, a set of calibrated virtualcameras are constructed from motion information. This goal is achieved byassuming the moving object as a fixed object and the fixed camera as a movingcamera in opposite direction. To utilize the benefits of multiview reconstruction,the moving object is supposed to be fixed and the camera is moved in the oppositedirection. So, the new virtual cameras are constructed as the real calibratedcameras around the object. In section 5, object's visual hull is recovered as fine aspossible by intersecting the large number of cones established by silhouettes ofmultiple views. A hierarchical method is presented to extract the visual hull of theobject as bounding edges. In section 6, experimental results with both syntheticand real objects are presented. We conclude in section 7 with a brief discussion.The total procedure of three dimensional model reconstruction from raw imagesto fine 3d model is done in the following steps:

Step1- Extraction of unique space curves on the surface of rigid object fromcalibrated stereo image information based on curvature and torsionconsistency of established space curves during rigid motion.

Step2- Object tracking and rigid motion estimation by curve distanceminimization in projection of space curves to the image planes ofperpendicular double stereo rigs

Step3- Making virtual calibrated cameras as many as required from fixed realcamera information and rigid motion information

Step4- Reconstruction of the object's visual hull by intersecting the conesoriginated from silhouettes of object in virtual cameras across time(Space-Time Cameras)

Step5- Texture mapping to any point of visual hull through the visible virtualcameras

2. Reconstruction of Space Curves on the Surface ofObject

The problem of decision on the correspondence is the main obscurity ofstereo vision. Components in the left image should be matched to those of theright one to compute disparity and thus depth. Several constraints such asintensity correlation, epipolar geometry, ordering, depth continuity and localorientation differences have been proposed to improve the matching process, buttheir success has been limited. There are many situations, where it is not possibleto find point-like features as corners or wedges. Then there is need to deal e.g.


with silhouette of the object instead of sparse local features. Besides, there existobjects, which cannot be represented adequately by primitive object features aspoints, lines, or circles. Moreover, pose estimations of global object descriptionsare, statistically, more accurate and robust than those from a sparse set of localfeatures. Whereas point features reveal little about surface topology, space curvesprovide such geometrical cues. Therefore, we focus on the space curves todevelop an inverse problem approach.

Robert and Faugeras presented an edge-based trinocular stereo algorithmusing geometric matching principles [26]. They showed that given the imagecurvature of corresponding curve points in two views, it is possible to predict thecurvature in the third one and use that as a matching criterion. Schmid andZisserman offered an extension to Robert method by fusing the photometricinformation and edge information to reduce the outlier matches [27]. Both theseapproaches apply many heuristics. Han and Park developed a curve-matchingalgorithm based on geometric constraints [28]. They apply epipolar constraintbetween two sets of curves and compute corresponding points on the curves.From the initial epipolar constraints obtained from corner point matching,candidate curves are selected according to the epipolar geometry, curve-endconstraints, and curve distance measures. Assuming that the corresponding curvesin stereo images are rather similar, they apply curve distance measure as aconstraint of curve matching. In general, this assumption is not true, as it will bediscussed in section 2.2. Kahl and August developed an inverse method to extractspace curves from multiple view images [29]. Instead of first seeking acorrespondence of image structure and then computing 3D structure, they seek thespace curve that is consistent with the observed image curves. By minimizing thepotential associated to prior knowledge of space curves, i.e. average curvature,and potential associated to the image formation model, they look for the candidatespace curves. The main deficiency of this method is that the relative motion of thecameras is assumed to be known.

2.1. Differential Geometry of Space Curves

Let ( ), ,P X Y Z= be a point whose position in space is given by the equations( ), ( ) and ( )X f s Y g s Z h s= = = where f, g, and h are differentiable functions of s.

As s varies continuously, P traces a curve in space. The differential geometry ofcurves traditionally begins with a vector ( ) ( ) ( ) ( )s X s Y s Z s= + +R i j k that describesthe curve parametrically as a function of s that is at least thrice differentiable.


Then the tangent vector T(s) is well-defined at every point R(s) and we maychoose two additional orthogonal vectors in the plane perpendicular to T(s) toform a complete local orientation frame (see figure 1). We can choose this localcoordinate system to be the Frenet frame consisting of the tangent T(s), theprincipal normal N(s) and the binormal B(s), which are given in terms of thecurve itself:

( ) ( )( )

( ) ( ) ( )( ) ( )

( ) ( ) ( ); ; s s s

s s s s ss s s

′ ′ ′′×= = = ×

′ ′ ′′×

R R RT B N B T

R R R(1)

Differentiating the Frenet frame yields the classic Frenet equations:

( )( )( )

( )( )

( ) ( )( )

( )( )( )

0 00

0 0

s s ss s s s ss s s

κκ τ

τ

′⎡ ⎤ ⎡ ⎤⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥′ ′= −⎢ ⎥ ⎢ ⎥⎢ ⎥⎢ ⎥ ⎢ ⎥⎢ ⎥′ −⎣ ⎦ ⎣ ⎦⎣ ⎦

T TN R NB B

(2)

Here, ( )sκ and ( )sτ are curvature and torsion of the curve, respectively, whichmay be written in terms of the curve itself :

( )( ) ( )

( ) 3

s sdsds s

κ′ ′′×

= =′

R RT

R(3)

( )( ) ( )( ) ( )( ) ( ) 2

s s sdsds s s

τ′ ′′ ′′′× ⋅

= =′ ′′×

R R RB

R R(4)

Considering the vector ( ) ( ) ( ) ( )s X s Y s Z s= + +R i j k , Eq.4 and Eq.5 can be modifiedas:

( ) ( ) ( )( )

2 2 2

3 22 2 2

Y Z Y Z Z X Z X X Y X Y

X Y Zκ

− + − + −=

+ +(5)

( ) ( ) ( )( ) ( ) ( )2 2 2

X Y Z Y Z Y Z X Z X Z X Y X Y

Y Z Y Z Z X Z X X Y X Yτ

− + − + −=

− + − + −(6)


Figure 1. Space curve geometry in Frenet frame.

2.2. Inverse Problem Formulation

Given a sequence of edge curves of a moving object in the calibrated stereorig, the problem is to extract the space curves on the surface of the object. It isobvious that the plane curve is established by projection of the space curve tocamera plane. As it is shown in figure 2, projections of space curve in twodifferent camera planes do not necessarily have the same shape. In the small baseline stereo setup, the assumption of shape similarity for correspondent curves willbe reasonable because of the small variation of viewpoint. However, in general,for any curve pair in two camera planes we can find one space curve byintersecting the projected rays from plane curves into space through the cameracenters. Therefore, the inverse problem of determining the space curve from planecurves is an ill posed problem. To find a way out to this problem, we consider thefundamental theorem of space curves as:

Theorem 1. If two single-valued continuous functions ( )sκ and ( )sτ aregiven for 0s> , then there will exist exactly one space curve determined except fororientation and position in space, i.e. up to a Euclidean motion, where s is the arclength, κ is the curvature, and τ is the torsion [30].

The fundamental theorem illustrates that the parameters ( )sκ and ( )sτ areintrinsic characteristics of space curve that do not change when the curve moves


in the space. This cue leads us to propose a new method of space curve matchingbased on curvature and torsion similarity through the curve length. The spacecurves that fit in the surface of rigid object must be consistent in curvature andtorsion during the object movement. As illustrated in figure 2, each pair of curvesin stereo cameras can define one space curve. However, for any curve in the leftcamera there is only one true match in the right camera. Hence, only one of theestablished space curves fits to the surface of the object and the others are outliers(or invalid space curves). In the following, we present a new method to determinethe true pair match between different curves.

Figure 2. Space Curve is established by intersection of the rays projected from imagecurves into the space through the camera centers. Any pair of curves in the left and rightcamera images can make one space curve.

Proposition 1. Between different pairs of curves in the left and right imagesof calibrated stereo rig, only the pair is true match that its associated space curveis consistent in curvature and torsion during motion of the curve (or stereo rig).

Proof: For any pair of curves in stereo cameras, we can establish one spacecurve by projecting them to 3D space through the related camera center andintersecting the projected rays by triangulation (figure 2). Based on thefundamental theorem, if the internal characteristics of space curve i.e. ( )sκ and

( )sτ be consistent during movement, all points of curve should have the samemotion parameters. In the other words, we can find a fixed motion matrix i.e. Rand T, which transforms all points of the space curve to their new positions.Therefore, if we show that it is impossible to find such a fixed motion matrix forall points of invalid space curve during motion, the proof will be completed. Tosimplify the proof, we deal with the rectified stereo configuration (figure 3). Any otherconfiguration can be easily converted to this configuration.


Figure 3. Reconstructed 3D points from different pairs of points in left and right cameraimages in rectified stereo configuration. Horizantal scan line is the epipolar line.

Applying the constraint of horizontal scan line, as the epipolar line, thefollowing equations can be easily derived:

( ) ( ) ( ) ( )( ) ( )

( ) ( ), , , ,1t ti i i i i ix

L Li iL R

TX Y Z x y

x x⎡ ⎤ ⎡ ⎤= =⎣ ⎦ ⎣ ⎦−

P (7)

( ) ( ) ( ) ( )( ) ( )

( ) ( ), , , ,1t tij ij ij ij i ix

L Li jL R

TX Y Z x y

x x⎡ ⎤ ⎡ ⎤= =⎣ ⎦ ⎣ ⎦−

P (8)

( )( )

( )

( )

( )( )

( )

( )

( )

( )

( )( )

( )

( )

( )( )

( )

( )

( )

( )

( ) ( ) ( ) ( )( )

( )

( )

( )

( )

( )

( )

( )

;

;

i jii iji i x x

L Ri ij i ji

j ijj jij j x x

L Rj ji j ij

i ij j jii i j j

L R L R i ij j ji

X T X TX Xx xZ Z Z Z

X T X TX Xx xZ Z Z Z

Y Y Y Yy y y yZ Z Z Z

− −= = = =

− −= = = =

= = = = = = =

(9)


Suppose that ( )iP is the valid three-dimensional point, which has beenreconstructed from the true match pair ( )i

Lx , ( )iRx and ( )ijP is the invalid three-

dimensional point, which has been reconstructed from the false match pair ( )iLx ,

( )jRx . Based on the fundamental theorem of space curves, we can find a fixed

matrices R and T, which transform any point ( )iP of the valid space curve to newposition ( )i

mP after movement:

( ) ( ) ( ) ( ) ( ) i ims SpaceCurve i s s∀ ∈ → = ⋅ +P R P T (10)

In the next step, we should inspect whether there are another fixed matrices R'and T', which transform any point ( )ijP of the invalid space curve to its newposition ( )ij

mP after movement, or not. The following equations can be easilyextracted:

( ) ( ) ( ) ( )( ) ( )

( ) ( )( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

, , , ,1 , ,1

, ,1

ti it tij ij ij ij i ix x m mm m m m Lm Lmi j i j i i

Lm Rm m m x m mi j

m mt

x x xi j i i j i j

m m x m m m x m m xi j i i j i j

m m m m m m m

T T X YX Y Z x yx x X X T Z Z

Z Z

T T TZ X T X Z X T X X TX Z Y Y Z Z Z

⎡ ⎤⎡ ⎤ ⎡ ⎤= = = ⎢ ⎥⎣ ⎦ ⎣ ⎦− − ⎢ ⎥⎣ ⎦−

⎡ ⎤⎢ ⎥⎢ ⎥= ⎢ ⎥− − −⎢ ⎥− ⋅ − ⋅ −⎢ ⎥⎣ ⎦

P

(11)

[ ]

( ) ( )

( ) ( )

( ) ( )

1 1t

1 2 3 1 2 3 2 2

3 3

, , ; , ,

i itm

t i it t t tm

i itm

X T

T T T Y T

Z T

= ⋅ +

⎡ ⎤= = → = ⋅ +⎣ ⎦= ⋅ +

R P

R R R R T R P

R P

(12)

Combining Eq.11 and Eq.12 results in:

( )( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )3 3 3 3 1 1 1 11 1 1 1 1 1

3 3 3 31 1 3 3 2 2 2 2 3 3

, ,1

t

ij x x xm i i i jj i jt t t tt t t

xx xi ji j i i j t tt t t t t

T T TT T T T TT T T T T

T TT T T T T

⎡ ⎤⎢ ⎥⎢ ⎥=⎢ ⎥+ + + + −+ − + + −⎢ ⎥−− ⋅ − ⋅

+ ++ + + + +⎢ ⎥⎣ ⎦

PR P R P R P R PR P R P R P

R P R PR P R P R P R P R P

(13)


• and ( )jP can be written as a function of ( )ijP using Eq.9:

( ) ( ) ( ) ( ) ( )( )

( )

( )

( )( )

( )

( )

( )

( )

( )

( )( ) ( ) ( )

( )

( )( ), , , ,1 , ,1 , ,

t ti i ij ij i it ti i i i i i ij ij ij iji i ij ij ij ij

X Y X Y Z ZX Y Z Z Z X Y ZZ Z Z Z Z Z

⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤= = = = =⎢ ⎥ ⎢ ⎥⎣ ⎦ ⎣ ⎦

⎣ ⎦ ⎣ ⎦P P

(14)

( ) ( ) ( ) ( )( )

( )( ) ( ) ( )

( )

( )

( )

( )( )

( )

( ) [ ], , , , 1 ,0,0 1 1,0,0t

j j i jt t tj j j j ij ij ij ijx xij ij ij ij

Z Z Z ZX Y Z X Y Z T TZ Z Z Z

⎡ ⎤⎛ ⎞ ⎛ ⎞⎡ ⎤ ⎡ ⎤= = + − = + −⎢ ⎥⎜ ⎟ ⎜ ⎟⎣ ⎦ ⎣ ⎦ ⎜ ⎟ ⎜ ⎟⎢ ⎥⎝ ⎠ ⎝ ⎠⎣ ⎦

P P

(15)

Substituting Eq.14 and Eq.15 in Eq.13, we obtain:

( )

( )

( )( )

( )

( )( )

( )

( )( )

( )

( )

( )

( )( )

( )

( )

( )

( )( )

( )

( )( )

( )

( )( )

( )

( )( )

( )

( )( )

( )

( )

1 11 13 3

1 1 3 31 3

1 111 1 3 3

2 2 2 2

11

1

1

xi ji ijtijt

x xij ijij

i i jijt ijt

xij ij ij

ij xm i ji i ijtij ijt t

xij ijij ij

i iij ijt t

ij ij

TZ ZZ R T T TT Z ZZ

Z Z ZT R T TZ Z Z

TZ ZZ Z R TT T Z ZZ Z

Z ZT TZ Z

⎛ ⎞+ − + −⎜ ⎟+ ⎜ ⎟

⎝ ⎠− ⋅⎛ ⎞

+ + − +⎜ ⎟⎜ ⎟⎝ ⎠

=⎛ ⎞

+ −⎜+ + ⎜⎝− ⋅

+ +

R PR P

R P R P

PR PR P R P

R P R P( )

( )( )

( )

( )

( )

( )( )

( )

( )( )

( )

( )( )

( )

( )

( )

( )( )

( )

( )

1

3 31 3

1 11 11 1

3 3 3 31 3

1

1

1

x

i jijt

xij ij

xi ji ijtijt

x xij ijij

i i jijt ijt

xij ij ij

T T

Z ZR T TZ Z

TZ ZZ R T T TT Z ZZ

Z Z ZT R T TZ Z Z

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥

+ −⎟⎢ ⎥⎟⎠⎢ ⎥

⎢ ⎥⎛ ⎞⎢ ⎥+ − +⎜ ⎟⎜ ⎟⎢ ⎥⎝ ⎠⎢⎢⎢ ⎛ ⎞

+ − + −⎢ ⎜ ⎟+ ⎜ ⎟⎢ ⎝ ⎠−⎢ ⎛ ⎞⎢ + + − +⎜ ⎟⎜ ⎟⎢ ⎝ ⎠⎣ ⎦

R P

R PR P

R P R P

( )( )

( ), , ,i

ijij

ZZ

⎛ ⎞= ⎜ ⎟⎜ ⎟

⎝ ⎠

⎥⎥⎥⎥⎥⎥⎥⎥

f P R T

(16)

Eq.16 clarifies that ( )ijmP is a nonlinear function of ( )ijP and we cannot find the

fixed rotation and translation matrices that transform all points of ( )ijP to ( )ijmP .

Moreover, the elements of ( )ijmP depend on

( )

( )

i

ij

ZZ

which may vary for any point of


curve. Therefore, we cannot find a fixed motion matrix for all points of invalidspace curve and the proof is completed.

In special situation where ( ) ( )i ijZ Z= , Eq.16 can be modified as:

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )3 3 1 1 1 1

1 1 3 3

3 31 1 1 1

2 2 2 2 3 3

1 1 1 1

3 3 3 3

1 1

x xij ij ijt t t

xij ijt t

ij xm ijij ijtt t

xij ij ijt t t

xij ijt t

xij ijt t

T TT T T T TT T

TTT T T

T T TT

T T TT T

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥+ + − + −⎢ ⎥− ⋅ −

+ +⎢ ⎥⎢ ⎥⎢ ⎥= =⎢ ⎥++ + −

− ⋅⎢ ⎥+ + +⎢ ⎥

⎢ ⎥⎢ ⎥⎢ ⎥+ + −

−⎢ ⎥+ +⎣ ⎦

R P R P R PR P R P

PR PR P R P

R P R P R P

R P R PR P R P

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

( )

1 11 1

2 21 1 1 1

3 32 2 2 2

1 1 1 1

3 3 3 3

xijt

ijt

ij ijtxij ijt t

x ijtij ijt t

xij ijt t

xij ijt t

T TT

TT T T

TT T

TT T TT T

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥

+⎢ ⎥ ⎡ ⎤+⎢ ⎥ ⎢ ⎥⎢ ⎥ = + = +⎢ ⎥⎢ ⎥+ + − ⎢ ⎥−⎢ ⎥ +⎢ ⎥⎣ ⎦+ +⎢ ⎥⎢ ⎥⎢ ⎥

+ + −⎢ ⎥−⎢ ⎥+ +⎣ ⎦

R P R P

R P RP TR P R P

R PR P R P

R P R PR P R P

(17)

Referring to figure 3, this condition can occur if and only if curve i andcurve j in both stereo images be identical (i=j). In the proof, we assumed thatthe space curve is not occluded during movement. This assumption isachievable for the proper length of curves with small amount of motion. Figure4 illustrates the proof in graphical method. Two fixed space curves are capturedby moving stereo rig in 3D-studio max environment. In part (a) and (b), thestereo images are shown before and after motion of the rig. Part (c) and (d)display the established space curves by different pair of curves before and aftermotion. Part (e) illustrates that the valid space curve established by true match isconsistent in shape during the movement, but the invalid space curve establishedby false match is not.

The presented curve matching method, which applies shape consistency ofspace curve during motion, does not consider any shape similarity between planecurves in stereo images. So, it can be used effectively to extract space curves fromwide base line stereo setup, where projections of the space curve in stereo camerasdo not have similar shape. Consequently, we can get more precise depth values asillustrated in figure 5. On the other hand, the wide base line stereo setupintensifies occlusion, which can make the curve matching inefficient. Therefore,there is a tradeoff between occlusion and depth accuracy to choose the properlength of base line.


Figure 4. Stereo curve matching by curvature and torsion consistency of established spacecurve during motion: (a) stereo images befor motion, (b) stereo images after motion, (c)established space curves from different curve pairs before motion, (d) established spacecurves from different curve pairs after motion and (e) determining the true mathes fromconsistent space curve in curvature and torsion during motion.


Figure 5. Stereo triangulation and uncertanity in depth. The true point can lie anywhereinside the shaded uncertanity region. Uncertanity in depth ΔΗ is reduced by increasing thelength of base line b (In the formulation, it is assumed that the maximum error of matchingis one pixel in every camera image). In practice, the right configuration is employed tomake efficient use of camera plane.

2.3. Reconstruction of Unique Space Curves

Here, we describe the different steps involved in the curve matching processto extract 3D position of unique feature points by forcing the constraint of spacecurves as global object descriptions. The unique points are defined as the points inthree-dimensional space that are matchless after forcing all the constraints, i.e.edge positioning, epipolar line, capability of curve forming with proper length byjoining the neighboring points, curvature and torsion consistency of relevant spacecurve in two or more consecutive frames, and the uniqueness of such space curve.The unique space curves are also composed from adequate number of continuousadjacent unique points. As illustrated in figure 6, to check the uniqueness of anyedge point in the left camera, all potential matches are labeled in the right cameraby intersecting all existence curves with the epipolar line. Then, the associatedcurves to test point and each labeled point are projected to space through their


camera centers. Each intersection generates a potential space curve with differentshape, which should be checked for curvature and torsion consistency during theobject motion. This process is done by intersecting the moved version of curves inthe next frame as illustrated in below part of figure 6. The consistent space curvein torsion and curvature is selected as valid space curve if there were only onesolution. The 3D point on this curve that corresponds to the test point is labeled asunique point. Details of the presented algorithm are given at the following:

Algorithm:Step1- At any instance of time, the moving object is captured by two

calibrated fixed cameras. The edge curves of image are extracted and thinned bythe canny method. Edges are then linked into chains, jumping up to one pixel gap.The small size edge regions are removed to reduce the effect of noise and get themore descriptive curves. The extracted image curves are never perfect in practice:there will be missing segments, erroneous additional segments, etc. Therefore, toimprove robustness to these imperfections we begin with one such segment in theleft image and seek confirming evidence in the right one.

Step2- To extract the unique points, the resulted left curve image in step 1 isscanned and all edge points are checked one by one for uniqueness. For eachexamined edge point ( )0L sc , which is considered as initial point, the correspondingepipolar line in the right curve-image is computed. Intersection of the epipolar linewith edge curves are labeled as candidate match points. One or more matchcandidates ( ) ( )0 , 1,2,...i

R s i′ =c may be found in this step (see figure 6). Only one of thecandidate points is the true match and the other points are outliers.

Step3- The points ( )0L sc and ( ) ( )0 , 1,2,...iR s i′ =c are grown n points from two

sides to form the curves. The curves with smaller length than 2n and the branchedcurves are discarded.

Step4- To distinguish the true match from other points, the next sequence ofstereo images is also considered. It is assumed that the frame rate is adjusted as wellto capture consecutive images with small amount of the object motion. Theneighborhood of ( )0L sc is inspected in the next sequence of left camera to find the

shift of the curve ( )L sc as ( )Lm sc . The key property of the curve with smallmovement is its proximity and similarity to the main curve. Therefore, we define acorrelation function as a combination of curve distance and curvature differencealong the curves:


( ) ( )( ) ( ) ( )( ) ( ) ( )( )( ) ( ) ( ) ( )( )0 0

0

1, max , ; ,

L k Lm k j

L Lm L Lm j L Lm j nj

L Lm j s sk n

cor s s cor s s cor s ss s α κ κ

+=−

⎡ ⎤= =⎣ ⎦− + ⋅ −∑ c c

c c c c c cc c

(18)

where: ( ) ( ) ( ) ( )( ) ( ) ( )( )2 2

0 L k L kLm k j Lm k j

n

L Lm j s ss sk n

s s x x y y+ +

=−

− = − + −∑ c cc cc c (19)

( )3 22 2

xy yx

x yκ −=

+ (20)

The shift of ( )L sc is selected as the argument ( )Lm sc which maximizes the

correlation function ( ) ( )( ),L Lmcor s sc c . The center of ( )Lm sc is also determined as an

argument sj , which maximizes the ( ) ( )( )0 ,L Lm jcor s sc c .

Step 5- The epipolar line of ( )Lm jsc is computed in next sequence of the right

image. Intersections of the curves with the epipolar line are labeled as( ) ( ), 1,2,...iRm js i′ =c according to their proximity to ( ) ( )0 , 1,2,...i

R s i′ =c .

Step 6- The Space curves ( ) ( )0 , 1,2,...i s i =SC corresponding to ( ) ( ) ( ){ }0 0, iL Rs s′c c

and the Space curves ( ) ( ), 1,2,...im js i=SC corresponding to ( ) ( ) ( ){ }, i

Lm j Rm js s′c c are

established by projecting the two dimensional curves in to the space andintersecting the rays. For each space curve, the curvature and torsion arecomputed from Eq. 5 and 6. The correlation between two space curves before andafter motion is computed from Eq. 21 for i=1, 2, …

( ) ( )( )( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

1, i i i i

k m k j k m k j

i im n n

s s s sk n k n

corκ κ τ τ

+ +=− =−

=− + −∑ ∑SC SC SC SC

SC SC (21)

The space curve i=q that maximizes the correlation function, is selected as theconsistent space curve and the pair ( )0L sc and ( ) ( )0

qR s′c are selected as the unique

points with determined depth value. If there were more than one solution becauseof having the close values of correlation function, the third sequence is alsoinspected to find the more confident answer. If there were only one solution, the


point ( )0L sc would be selected as known depth value. Otherwise, it would belabeled as non-unique point and rejected.

Figure 6. Curve stereo matching to find the unique points of the object.

Step 7- Going back to step 2 and repeat the procedure for all edge points tofind adequate number of unique points. At the end, the unique space curves arecomposed from continuous adjacent unique points.

Shape descriptivity, branching, and occlusion are considered as three factorsto choose the proper length of curves in matching process. The short curves areless descriptive in shape and result in low confidence matching, as the number ofdetected similar curves will be increased. Hence, the uniqueness-checking processwill fail to find the best match. On the contrary, the long curves are moredescriptive in shape and result in high confidence matching, as the number ofdetected similar curves will be decreased. Occlusion and branching are the otherfactors that restrict lengthening of the curves, so that the number of appropriatecurves reduces by increasing the length of curves.

Our experiments show that the curve length between 20 to 40 points (for480×640 image size) provides good result. Of course, the proposed length is arepresentative value. Depending on texture of the object, number of the similar


curve on surface of the object, and range of depth-variation across the curves, therepresentative curve length may be varied.

3. Rigid Motion Estimation by Tracking the Space Curves

The movement of a rigid object in space, can be expressed as the six rotationand translation parameters. Now, suppose that we have extracted a set of points onthe surface of an object and the goal is to estimate 3D motion of the object acrosstime. To estimate the motion parameters, an error function, which describes thedifference of points before and after motion, should be minimized. To get rid ofphotometric information, we define the error function as a distance of uniquepoints from the nearby curves after movement. To explain the problemmathematically, suppose that Wi is the ith unique point, Nu is the total number ofunique points, ( )k i⋅ +R W TP is projection of Wi in camera plane k after

movement, and ( )mk

contour is the curve number m in camera plane k. To estimate

the motion matrix, i.e. R and T, the error component for each unique point in eachprojected camera image is defined as the minimum distance of that point fromnearby curves in that camera. The total error is also calculated by summing errorcomponents over all unique points and all cameras.

( ){ }( )

1 1

min ( ),

; , arg min{ }

k mi k i km

Nu K kii k

e distance contour

e e e= =

= ⋅ +

= =∑ ∑

R W T

R T

P (22)

Where K is the total number of cameras (K=2, for single stereo rig). To find theminimum distance of each point from nearby curve in the camera image, weuse a circle based search area with increasing radius (figure 7). Therefore, theminimum distance is determined as the radius of the first osculating circle withadjacent curves. R and T are parameterized as , , , , ,x y z x y zt t tϕ ϕ ϕ⎡ ⎤= ⎣ ⎦Θ where

, ,x y zϕ ϕ ϕ are the Euler angles of rotation and , ,x y zt t t are the x, y, zcomponents of translation vector. The total error function defined in Eq.22 canbe minimized by an iterative method similar to the Levenberg-Marquardtalgorithm [8]:


Figure 7. To find the minimum distance of point from the adjacent curves in the cameraimage, a circle based search window, with increasing radius, is considered. The minimumdistance is determined as the radius of the first touching circle with adjacent curves.

1. With an initial estimate Θ , calculate the Hessian matrix H and thedifference vector d as:

k k k k k k k k k k k ki i i i i i i i i i i i

x x x y x z x x x y x z


y x y y y z y x y y y z

k ki i

z xik

e e e e e e e e e e e et t t

e e e e e e e e e e e et t t

e e

φ φ φ φ φ φ φ φ φ

φ φ φ φ φ φ φ φ φ

φ φ

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂⋅ ⋅ ⋅ ⋅ ⋅ ⋅

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂⋅ ⋅ ⋅ ⋅ ⋅ ⋅

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂

∂ ∂ ∂⋅

∂ ∂=H

k k k k k k k k k ki i i i i i i i i i

z y z z z x z y z z


x x x y x z x x x y x z

k k k k ki i i i i

y x y y

e e e e e e e e e et t t

e e e e e e e e e e e et t t t t t t t t

e e e e et t t

φ φ φ φ φ φ φ

φ φ φ

φ φ

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂⋅ ⋅ ⋅ ⋅ ⋅

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂⋅ ⋅ ⋅ ⋅ ⋅ ⋅

∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂

∂ ∂ ∂ ∂ ∂⋅ ⋅

∂ ∂ ∂ ∂ ∂

k k k k k k ki i i i i i i

y z y x y y y z


z x z y z z z x z y z z

e e e e e e et t t t t t

e e e e e e e e e e e et t t t t t t t t

φ

φ φ φ

⎡ ⎤⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥⎢ ⎥∂ ∂ ∂ ∂ ∂ ∂ ∂⎢ ⎥⋅ ⋅ ⋅ ⋅⎢ ⎥∂ ∂ ∂ ∂ ∂ ∂ ∂⎢ ⎥⎢ ⎥∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂

⋅ ⋅ ⋅ ⋅ ⋅ ⋅⎢ ⎥∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂⎣ ⎦


1 1

Nu Kiki k= =

=∑ ∑H H (23)

, , , , ,t

k k k k k kk k k k k ki i i i i i

ik i i i i i ix y z x y z

e e e e e ee e e e e e

t t tφ φ φ

⎡ ⎤∂ ∂ ∂ ∂ ∂ ∂= ⋅ ⋅ ⋅ ⋅ ⋅ ⋅⎢ ⎥

∂ ∂ ∂ ∂ ∂ ∂⎢ ⎥⎣ ⎦d

1 1

2Nu K

iki k= == − ⋅∑ ∑d d (24)

2. Update the parameter Θ by an amount ΔΘ :

( 1) ( ) ( ) 11ˆ ˆ ˆn n n

λ+ −= + Δ = + ⋅Θ Θ Θ Θ H d (25)

Where λ is a time-varying stabilization parameter.

3. Go back to step1 until the estimate of Θ converges.Unless the object has periodic edge curves, the error function in Eq.22 usually

has one minimum and convergence of the algorithm will be guaranteed. Outlierpoints have destructive effect on convergence of the algorithm. Naturally,projection of the outlier point in the camera planes will not be close to thetracking curves. As a result, minimization of the error function cannot beaccomplished accurately. To explain the problem mathematically, consider theunique points in two groups, i.e. inliers and outliers. The error function can be re-arranged as:

1 1 : inlier outlierN N

i j inlier outlier ui je e e where N N N

= == + + =∑ ∑ (26)

In the provision that Noutlier is very small than Ninlier, the error component

1

outlierN

jje

=∑ has negligible effect compared to 1

inlierN

iie

=∑ and estimation of the

motion will go in the true way. However, the unique points will not join to thetracking curves during convergence. To make the algorithm more efficient, theminimum distance of each unique point from nearby curve is checked afteradequate number of iterations. The points that their distance is very greater thanthe average distance (i.e. i ue e N>> ), are distinguished as outliers. Such pointsare excluded in calculation of the error function and hence the closer unique


points to the tracking curves, with more precise motion parameters, can beachieved.

Once the six motion parameters were estimated for two consecutivesequences, the motion matrix can be constructed as:

( ), ,0 0 0 1

x y zϕ ϕ ϕ⎡ ⎤= ⎢ ⎥⎣ ⎦R TM (27)

Where:

( )cos 0 sincos sin 0 1 0 0

, , sin cos 0 0 1 0 0 cos sin ; sin 0 cos0 0 1 0 sin cos

y yz z x

x y z z z x x y

y y x x z

ttt

ϕ ϕϕ ϕϕ ϕ ϕ ϕ ϕ ϕ ϕ

ϕ ϕ ϕ ϕ

−⎡ ⎤ ⎡ ⎤⎡ ⎤ ⎡ ⎤⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥= − =⎢ ⎥ ⎢ ⎥⎢ ⎥ ⎢ ⎥− ⎢ ⎥⎢ ⎥⎣ ⎦⎣ ⎦ ⎣ ⎦⎣ ⎦

R T (28)

The new position of each point, in the next frame, can be calculated bymultiplying the motion matrix to the position vector.

( ) ( ) ( )1 1: , , ,1Tn n w w w

n where X Y Z+ ⎡ ⎤= ⋅ = ⎣ ⎦W M W W (29)

4. Motion Estimation Using Double Stereo Rigs

In this section, we present a double stereo configuration to get as muchaccuracy as possible in estimation of motion parameters. The basic idea to achievethis end is to find an arrangement of stereo cameras in which the sensitivity ofimage pose variation to space pose variation is maximized. At first, the singlestereo setup is investigated and then a perpendicular double stereo configuration ispresented and its dominance to the single stereo is demonstrated.

4.1. Single Stereo Rig

As mentioned in section 2.3, the base line of stereo rig is adjusted neither smallnor wide to compromise between depth uncertainty and occlusion. Moreover, toutilize the linear part of camera lens and to get rid of the complex computations ofnonlinear distortion, the view angle is chosen as small as possible. Hence, the sizeof the object is usually very smaller than its distance from the camera center,i.e. 2 zr t (see figure 8).


Figure 8. Single stereo setup with small view angle ( 2zt r>> ).

Now, we would like to answer the question that how much accuracy isachievable in space motion estimation by tracking the projection of points incamera planes. For the sake of simplicity, we assume that the optical axis ofcamera1 is in the depth direction of world coordinate (i.e. Zw). Projection of anypoint ( , , )w w wX Y Z in the image plane of camera1 is computed as:

( )1 1 1 1, ,w wim im x y

w z w z

X Yx y f f

Z t Z t⎛ ⎞

= − ⋅ − ⋅⎜ ⎟+ +⎝ ⎠

(30)

By differentiating, we can write:

1 1 11

1 1 11

im im imim w w w

w w w

im im imim w w w

w w w

x x xx X Y Z

X Y Zy y y

y X Y ZX Y Z

∂ ∂ ∂⎧Δ = Δ + Δ + Δ⎪ ∂ ∂ ∂⎪⎨ ∂ ∂ ∂⎪Δ = Δ + Δ + Δ⎪ ∂ ∂ ∂⎩

(31)

11

11

wxim w w

w z w z

y wim w w

w z w z

Xfx X Z

Z t Z t

f Yy Y Z

Z t Z t

⎧ ⎛ ⎞Δ = −Δ + Δ⎪ ⎜ ⎟

+ +⎪ ⎝ ⎠⎨

⎛ ⎞⎪Δ = −Δ + Δ⎜ ⎟⎪ + +⎝ ⎠⎩

(32)

For the provision of small view angle (i.e. 2zt r>> ) andassuming , ,w w wX Y Z r≤ , we have:


1, 1w w

w z w z

X YZ t Z t

<< <<+ +

(33)

Therefore, xim1 and yim1 are very sensitive to Xw and Yw compared to Zw. As weexplained in section 3, the six motion parameters are adjusted in which thedistance error in image planes to be minimized. Hence, the assumption of

1 0imxΔ ≈ and 1 0imyΔ ≈ will be reasonable for each tracking point afterconvergence of motion estimation algorithm. Combination of this assumptionwith Eq.32 and Eq.33 can be resulted in:

1

1

, 0 or0

, 0 or0

w w

wimw w w w

w z

w w

wimw w w w

w z

X ZXx X Z Z X

Z t

Y ZYy Y Z Z Y

Z t

Δ Δ ≈⎧⎪Δ ≈ →⎨Δ ≈ Δ →Δ >>Δ⎪ +⎩

Δ Δ ≈⎧⎪Δ ≈ →⎨Δ ≈ Δ →Δ >>Δ⎪ +⎩

(34)

This equation reveals that the inverse problem of 3D motion estimation bytracking the points in camera plain is an ill posed problem and does not have onesolution. Any small estimation error of Xw or Yw (i.e. 0 or 0w wX YΔ ≠ Δ ≠ ) imposesa large estimation error of Zw (i.e. or w w w wZ X Z YΔ >>Δ Δ >>Δ ). Therefore, the

total 3D positional error 2 2 2+ +w w wX Y ZΔ Δ Δ will be notably increased and theinaccurate 3D motion parameters will be estimated.

4.2. Double Stereo Rigs

Due to the limitation of large base line selection in single stereo rig, bothstereo cameras have approximately the same effect in motion estimationprocess. To take the advantages of both small and wide base line stereocameras, we present a combined double stereo setup. This combination iscomposed of two single stereo rigs in which they make angleθ from each other(see figure 9).


Figure 9. Structure of double stereo setup: (a) Double stereo setup with angleθ , (b)Perpendecular double stereo setup.

Similar to single stereo setup and considering the rotation angle of θ forcamera3, it can be easily shown that:

3 3 3

3 3 3

cos sinsin cos

sin cos

w wim x o

w w z

wim y o

w w z

X Zx f x

X Z tY

y f yX Z t

θ θθ θ

θ θ

−⎧ = − ⋅ +⎪ + +⎪⎨⎪ = − ⋅ +⎪ + +⎩

(35)

( ) ( )( )

( )

33

33

cos sin

sin cos

xim w z w w z w

yim w w w w w

fx Z t X X t Z

Af

y Y X A Y Y ZA

θ θ

θ θ

⎧Δ = − + Δ + + Δ⎪⎪⎨⎪Δ = ⋅Δ − ⋅Δ + ⋅Δ⎪⎩

(36)

Where:

( )2sin cosw w zA X Z tθ θ= + + (37)

By choosing a proper amount ofθ , it is possible to increase the sensitivity of xim3

and yim3 to Zw as much as possible. Therefore, we can minimize the 3D motionestimation errors ∆Xw and ∆Yw by minimizing ∆xim1 and ∆yim1, and the estimation


error ∆Zw by ∆xim3 and ∆yim3. It can be verified, by differentiating, that themaximum value of sensitivity is achieved by 90θ = . For 90θ = , the Eq.36 issimplified as:

33

33

x wim w w

w z w z

y wim w w

w z w z

f Zx X Z

X t X t

f Yy X Y

X t X t

⎧ ⎡ ⎤−Δ = Δ + Δ⎪ ⎢ ⎥+ +⎪ ⎣ ⎦⎨

⎡ ⎤⎪Δ = Δ − Δ⎢ ⎥⎪ + +⎣ ⎦⎩

(38)

Similar to single stereo, we can assume ( )1 1, 0im imx yΔ Δ ≈ for each tracking point

in camera1 and ( )3 3, 0im imx yΔ Δ ≈ for each tracking point in camera3 afterconvergence of motion estimation algorithm. Hence:

3

3

, 0 or0

, 0 or0

w w

wimw w w w

w z

w w

wimw w w w

w z

X ZZx X Z X Z

X t

Y ZYy X Y X Y

X t

Δ Δ ≈⎧⎪Δ ≈ →⎨ Δ ≈Δ →Δ >>Δ⎪ +⎩

Δ Δ ≈⎧⎪Δ ≈ →⎨ Δ ≈Δ →Δ >>Δ⎪ +⎩

(39)

Combination form of the Eq.34 and Eq.39 result in , , 0w w wX Y ZΔ Δ Δ ≈ . Therefore,

the total 3D positional error 2 2 2+ +w w wX Y ZΔ Δ Δ will be notably decreased inperpendicular double stereo setup and more precise motion parameters will beresulted.

5. Shape Reconstruction from Object Silhouettes AcrossTime

Three-dimensional model reconstruction by extracting the visual hull of anobject has been extensively used in recent years [31-34] and it has become astandard and popular method of shape estimation. Visual hull is defined as arough model of the object surface, which can be calculated from different views


of the object's silhouette. The silhouette of an object in an image refers to thecurve, which separates the object from background. The visual hull cannotrecover concave regions regardless of the image numbers that are used. Inaddition, it needs a large number of different views for recovering the finedetails. To moderate the first drawback of visual hull, combination with stereomatching can be employed. To get rid of the second drawback, more silhouettesof the object can be captured by the limited number of cameras across time.Cheng et al. presented a method to enhance the shape approximation bycombining multiple silhouette images captured across time [34]. Employing abasic property of visual hull, which affirms that each bounding edge must touchthe object in no less than one point, they use multi-view stereo to extract thesetouching points called Colored Surface Points (CSP) on the surface of theobject. These CSPs are then used in a 3D image alignment algorithm to find thesix rotation and translation parameters of rigid motion between two visual hulls.They utilize the color consistency property of the object to align the CSP points.Once the rigid motion across time is known, all of the silhouette images aretreated as being captured at the same time instant and the shape of the object isrefined.

Motion estimation by CSP method suffers from some drawbacks. Non-accurate color adjustment between cameras is one problem that makes some errorin color-consistency test. Moreover, variation of the light angle while the objectmoves around the light source produces additional error.

Our presented method of motion estimation which uses only the edgeinformation as the space curves, is very robust against the color maladjustment ofcameras and shading during the object motion. Moreover, it can be effectivelyused to extract visual hull of poorly textured objects.

In the remainder of this section, it is assumed that the motion parameters areknown for multiple views of the object and the goal is to reconstruct 3D shape ofthe object from silhouette information across time.

5.1. Space-Time or Virtual Camera Generation

Let P defined in Eq.40, be the projection matrix of camera, which translatesthe 3D point W in the world coordinate to ( ),im imx y in the image coordinate ofthe camera plane:


( ) ( )0

1 1

0

000 0 1

xw w w

y c c c

f xf y

− −−⎡ ⎤⎢ ⎥ ⎡ ⎤= − − ⋅⎢ ⎥ ⎢ ⎥⎣ ⎦⎢ ⎥⎣ ⎦

P R R T (40)

[ ], ,1 , , ,1TT w w w

im imx y X Y Z⎡ ⎤∝ ⎣ ⎦P (41)

Where, wcR and w

cT are the rotation and translation of the camera coordinate systemto the world coordinate system, respectively. fx and fy are the focal length in x andy direction and x0, y0 are the coordinates of principal point in the camera plane.From Eq.29, we can get:

( ) ( ) ( )1 11 1 2 1

n nn n

−− −= ⋅ = ⋅ ⋅ ⋅ ⋅W M W M M M W (42)

By multiplying the projection matrix to the motion matrix, a new projectionmatrix is deduced for any sequence. This matrix defines a calibrated virtualcamera for that sequence as:

( ) ( )1 2 1n

n −= ⋅ ⋅ ⋅ ⋅P P M M M (43)

The matrix ( )nP , which produces a new silhouette of the object, projects any pointin the world coordinate to the image plane of the virtual camera n as:

( ) ( ) ( ), ,1 , , ,1T Tn n n w w w

im imx y X Y Z⎡ ⎤ ⎡ ⎤∝ ⎣ ⎦⎣ ⎦ P (44)

In fact, by generating the virtual cameras, the moving object and fixed camerasystem is substituted by the fixed object and moving camera that moves in theopposite direction.

5.2. Visual Hull Reconstruction from Silhouettes of Multiple Views

In the resent years, shape from silhouette has been used widely to reconstructthree dimensional shape of an object. Each silhouette makes one cone in thespace, with the camera center. Using more cameras from different views of object,more cones are constructed. The visual hull is defined as a shared volume betweenthese cones. There are two conventional approach to extract the visual hull: voxel


carving [35,36] and view ray sampling [37]. In the voxel carving method, adiscrete number of voxels are constructed around the volume of interest. Then,each voxel is checked for all silhouettes and any voxels that project outside thesilhouettes are removed from the volume. Voxel carving can be accelerated usingoctree representation which employs coarse to fine hierarchy. In view raysampling, a sampled representation of the visual hull is constructed. The visualhull is sampled in a view-dependent manner. For each viewing ray in somedesired view, the intersection points with all surfaces of the visual hull arecomputed. Moezzi et al. [38] construct the visual hull using voxels in an off-lineprocessing system. Cheung et al. [39, 40] show that the voxel method can achieveinteractive reconstruction results. The polyhedral visual hull system developed byMatusik et al. [41] also runs at interactive rate.

In this section, two efficient algorithms are presented to improve the speed ofcomputations in visual hull extraction. The first algorithm accelerates the voxelcarving method. This algorithm reduces the number of check-points atintersection test procedure. The octree division method is optimized, byminimizing the number of check-points, to find intersection between cubes andsilhouette images. To accomplish this function, the points are checked on theedges of octree cubes rather than the inside of volume. Furthermore, the points arechecked hierarchically and their number is changed corresponding to the size ofoctree cubes. The second algorithm employs the ray sampling method to extractthe bounding edge model of visual hull. To find the segments of any ray whichlies inside the other silhouette cones, the points of ray are checked hierarchically.

5.2.1. Volume Based Visual Hull

Many algorithms have been developed to construct the volumetric modelsfrom a set of silhouette images [35, 36, 37, 39]. Starting from a bounding volumethat is known to surround the whole scene, the volume is divided into voxels. Thetask is finding which voxels belong to the surface of 3D object, corresponding tothe intersection of back-projected silhouette cones. The most important step inthese algorithms is the intersection test. To make the projection and intersectiontest more efficient, most methods use an octree representation and test voxels in acourse-to-fine hierarchy.

5.2.1.1. Intersection Test in Octree Cubes

The most important and time-consuming part of octree reconstruction is thecubes intersection check with silhouette images. All algorithms use one common


rule to decide whether or not intersection is happened between cube and object.Cube is known as outside if all points inside cube are “1” and known as inside ifall points inside cube are “0”. Also, intersected cube is the cube which has at leasttwo different color points. Different methods of point checking are classified infigure 10. The number of check points may be constant in all size of cubes, orchange dynamically based on the size of cube. In all methods, the 8 corners ofeach cube are checked by projecting them to all the silhouettes. If there were atleast two different color corners, occurrence of intersection will be inferred andthe process for this cube can be terminated, otherwise more points in the cubeshould be checked. If there was color difference during check, the cube will bemarked as intersected cube and the process will be terminated. After checking allpoints, if there was no color difference, the cube will be identified as outside (orinside) according to the color of points "1" (or "0").

To compare the complexity of different types of intersection check in theoctree cubes, the following parameters will be considered: L = level of octreedivision; CL = number of grey (intersected or surface) cubes in level L; NL=maximum number of checking points to identify the mark of cube in level L; S =number of silhouettes. Since each grey cube is divided to 8 sub-cubes in octreedivision, so the number of grey cubes in level L-1 will be equal or greater than 1/8grey cubes in level L according to the number of child grey cubes. The totalnumber of point projections to silhouette images in the worst case will be:

( ) ( )to t L L L -1 L -1 L -2 L -2 L -3 L -3

L -3L -1 L -2L L

N m ax = S N C + N C + N C + N C + · · ·

NN N S C N + + + + · · · 8 6 4 5 1 2

⋅

⎛ ⎞≥ ⋅ ⎜ ⎟⎝ ⎠

(45)

Figure 10. Checking methods in octree cubes. a) Sequential check in volume b) Randomcheck in volume c) Sequential check on edges d) Hierarchical check on edges.


Obviously, the total number of check points will be smaller than Ntot(max),because intersected cubes normally will be recognized in early checking. Thisnumber can be used as a measure to compare different methods of intersectioncheck. It is obvious that the computing time is proportional with Ntot. Edgebased checking is one approach to decrease the number of checking points toidentify the mark of cube without loss of accuracy. Any intersection betweencube and silhouette should be occurred through edges in one-piece objects. Ofcourse there is one exception case when the object is small and posed inside thecube so that there is no intersection between object and cube through the edges.For such cases, the edge base method can not be employed to decide if object isinside the cube or intersects with cube through the face. Therefore checkingsome points inside the volume will be inevitable for this situation. If the size ofbounding cube be selected properly, comparable to that of object, the cube willbe larger than the object only in first level of division. So the ambiguity ofintersection test through the edges will be stay only for the first level. Since theoctree division is done at all times without checking the occurrence ofintersection in first level, the use of edge based intersection test for one pieceobject can be applied with certainty. Another approach to decrease the numberof check points is to change the number of points dynamically in each level. Infact, the large cubes may intersect with small parts of silhouette and it needschecking of more points to identify the intersection. In small cubes, thissituation can not be occurred and there is no need to check more points. Bychoosing NL=8 (checking only corners of cube in last level) and increasingchecked points with the factor of 'k' in lower levels, we can minimize Ntot (max)as below:

( ) 3 31 2 1 2tot L L

8k k8k 8k k kN max S C 8+ + + + · · · 8 S C 1+ + + + ·· ·8 64 512 8 64 512

⎛ ⎞ ⎛ ⎞≥ ⋅ ≥ ⋅ ⋅⎜ ⎟ ⎜ ⎟⎝ ⎠ ⎝ ⎠

(46)

The final approach to increase the speed is to check the edge points hierarchically.In this way the chance to find two different color points in early checks could beincreased.

5.2.1.2. Synthetic Model Results

To determine the capability of presented algorithm and to quantify itsperformance, we have tested it on a synthetically generated image named Bunnyand Horse. Simulation was run on PC Pentium-III 933Mhz using Matlab and C++generated Mex files. In this analysis, 18 silhouettes of bunny from equal space


angle viewpoints have been used. Figure 11 shows the result of simulation fordifferent methods. In this figure 'CN' and 'DN' mean Constant Number orDynamic Number of points should be checked in different size of cubes. 'S', 'H'and 'R' mean Sequential, Hierarchical and Random method to check the points,respectively. The last word 'V' and 'E' mean that the check points are selectedinside the Volume or on the Edges of cube. To compare the efficiency of methods,computing time for a fix number of recovered cubes (voxels) in the last levelcould be evaluated for different types of intersection check. As it is cleared infigure, DNHE method gives the best result and CNRV method gives the worstresult. Computing time for random check method is high, because some checkpoints may be chosen near each other as it is cleared in figure 10-b.

1

10

100

1000

6880 6900 6920 6940 6960 6980 7000

Number of Recovered Voxels of Visual Hull

Com

putin

g Ti

me

(Sec

)

CNSVDNSVCNSECNHEDNHECNRV

Figure 11. Computation time for different types of intersection check.

In figures 12, a synthetic object named Bunny has been applied to reconstructthe 3D shape using DNHE algorithm. The synthetic object has been captured fromdifferent views and 18 silhouettes of object have been prepared. These silhouettesare shown in figure 11-a. The different levels of octree division are illustrated infigure 11-b and the depth-map of reconstructed 3d-model is shown in figure 11-c


from different views. Figure 13 shows the result of shape reconstruction foranother synthetic object named Horse.

Figure 12. Three-dimensional shape Reconstruction of Bunny from 18 silhouettes usingDNHE algorithm. a) different silhouettes of object from 18 view angles b) different levelsof octree division using DNHE algorithm c) depth-map of reconstructed 3d-model indifferent view angles.


Figure 13. Three-dimensional shape Reconstruction of Horse from 18 silhouettes usingDNHE algorithm a) different silhouettes of object from 18 view angles b) different levelsof octree division using DNHE algorithm c) depth-map of reconstructed 3d-model indifferent view angles.


5.2.2. Edge Base Visual Hull

Cheng et al. suggested a representation of visual hull that uses a one-dimensional entity called bounding edge [34]. To reconstruct the bounding edges,a ray is formed by projecting any point on the boundary of each silhouette imageinto space through the camera center. This ray is projected to other silhouetteimages. Those segments of the ray whose projections remain inside the othersilhouettes will be selected as the bounding edges. So, by unifying these boundingedges of all rays of all silhouettes, the visual hull of the object will bereconstructed. Note that the bounding edge is not necessarily a continuous line. Itmay consist of several segments if any of the silhouette images are not convex.This point has been illustrated in figure 14-a where 1

iray consists of two sharedsegments in S2, S3 and S4. To represent each bounding edge, it is enough to findthe start and end position of each segment. We employ a hierarchical checkingmethod to extract the start-end points very fast. The idea has been illustrated infigure 14-b. Instead of checking all the points of the ray, they are checked inhierarchical method. First, the middle point of the ray is projected to all thesilhouette images and its status is checked. If it was inside in all the silhouettes,the quarter point in the left side is checked; otherwise, the quarter points in bothsides must be checked. This procedure is repeated hierarchically to find the startpoint of the bounding edge with the favorite accuracy. Once the start point isfound, hierarchical checking is restarted to find the end of the segment. Still, it isunclear how to find the start and end when the ray consists of more than onesegment (in concave parts). To resolve this ambiguity, the points can be checkedin the coarse-to-fine method. This means that the points are checked in large stepson the ray at first. Upon changing the status, hierarchical checking is applied tofind exact position of the start-end points for each segment. This process isrepeated to reach the end of the ray. Therefore, position of the start-end points inmulti-segment bounding edges can definitely be determined. This concept isillustrated in the right part of figure 14-b.

To have a sense on computation complexity of the hierarchical method andsee its efficiency, suppose that: jm is the number of points on the boundary of

silhouette j; rayN is the number of points on each ray, and silN is the number ofsilhouettes. In the ordinary method, the points on the surface of each cone areprojected to all other silhouettes.

Then, decision is made to see which point is inside in all the silhouettes. Thetotal number of projections is given by:


( ) ( )1

max 1silN

tot j ray silj

N m N N=

= ⋅ ⋅ −∑ (47)

It is possible to decrease this number by removing the points that are identified asoutside in one silhouette and it would not be necessary to project such points toother silhouettes. Therefore, the total number of projections will be:

( )1 1 2 1 2 ( 2)1

1 ... ...sil

sil

N

tot j ray j j j j j N jj

N m N k k k k k k −=

= ⋅ ⋅ + + + +∑ (48)

Where 0 1ijk< < is the percentage of points on cone(j) which project to the insideof silhouette i. The amount of ijk depends on the shape of cone(j) and silhouette i.For hierarchical checking method, consider that each ray is divided into n part atfirst. To find the exact position of each start and end points, ( )2log /rayN n points

should be checked. In convex parts, bounding edges are formed in one segment.Therefore, the number of checking points in each ray is:

( )2 2 log /ray rayNH n N n= + ⋅ (49)

Figure 14. (a) Projection of ray to the silhouette images to extract the bounding edges(b) hierarchical method to find the start and the end of segments.


And total number of projections for the convex object is:

( )1 1 2 1 2 ( 2)1

1 ... ...sil

sil

N

tot j ray j j j j j N jj

NH m NH k k k k k k −=

= ⋅ ⋅ + + + +∑ (50)

In concave parts, the bounding edges are formed in two or more segments. For qsegmented bounding edges, the total number of checking points in the ray is givenby:

( ) ( )2 2 log /ray rayNH q n q N n= + ⋅ ⋅ (51)

5.2.2.1. Synthetic Model Results

To show the capability of the presented algorithm and to quantify itsperformance, we have tested it on a synthetically generated image namedBunny. In this analysis, 18 silhouettes of Bunny from equal space angleviewpoints have been captured and used. Table 1 shows the result of simulationfor different methods. To compare the efficiency of methods, computing timefor a fix number of recovered points on visual hull can be evaluated for differenttypes of intersection check. Table 2 shows the result for voxel based DNHEextraction for the same silhouettes. As it is clear, hierarchical method ofbounding edge extraction gives very good results compared to ordinarybounding edge method and voxel based DNHE method, especially for highnumber of recovered points which is proportion to high accuracy.

Table 1. Computing time to extract bounding edges of bunny

Computing Time (sec)Hierarchical method

Num. of pointson each ray ofbounding cone

Recovered pointson visual hull Ordinary

method n=10 n=3055 7192 0.25 0.1 0.1780 10353 0.33 0.11 0.18

150 19421 0.59 0.112 0.19250 32351 0.95 0.113 0.191500 64764 1.83 0.115 0.1951000 129578 3.52 0.12 0.21


Table 2. Computing time for voxelbase visual hull extraction

Num. of voxels in eachedge of bounding cube

Recovered voxels onvisual hull

Computing Time (sec)(DNHE method)

27=128 6890 2.228=256 29792 12.4

Table 3. Computing time to extract boundingedges of bunny from different

number of silhouettes

Computing Time (sec) (HierarchicalMethod)Number of

silhouettesNumber of points

on each rayn=10 n=30

48 500 0.22 0.3918 500 0.115 0.1958 500 0.05 0.094 500 0.03 0.05

The computing time 0.12 or 0.21 sec to extract object from 18 silhouetteswith the resolution of 1000 is very low and it makes possible to use algorithmfor real-time extraction purposes. Table-3 shows the result of simulation fordifferent number of silhouettes of Bunny.

Figure 15. Continued on next page.


Figure 15. Start-End points, Bounding-Edges and Depth-Map for 3 models named bunny,female and dinosaur which have been extracted through 18 silhouettes from differentviews using hierarchical algorithm.

In figure 15 we have demonstrated the start-end points of edges, boundingedges and depth map of reconstructed 3d models for synthetically objects namedhorse, bunny, female and dinosaur from 18 silhouette by hierarchical method hasbeen shown.

Implementation and Exprimental Results

To evaluate the efficiency of our approach to 3D model reconstruction bytracking the space curves using the perpendicular double stereo rigs, there are


two questions that we need to consider. First, how good is the estimation ofmotion parameters? Second, how efficient is our method in practice or howrobust is our method against the disturbing effects like noise? In this section, wepresent the results of experiments designed to address these two concerns.Experimental results are conducted with both synthetic and real sequences thatcontain different textures of objects. Two synthetic objects named Helga andCow were captured by perpendicular double stereo cameras in 3D-StudioMaxenvironment. The objects were moved with arbitrary motion parameters and 36sequences were captured by each camera. Figure 16 shows the camera setup tocapture Helga and Cow models along with their extracted space curves. It isclear that the number of outlier points is significantly less than the number ofvalid unique points. Figure 17 illustrates the curve tracking and motionestimation process by minimizing the geometric distance of curves in thecamera images. Two sets of space curves, which have been extracted by twodistinct stereo rigs, are projected to the camera planes in the next sequence andmotion parameters are adjusted in which the projection of space curves in therelated camera planes to be as close as possible to nearby curves. Figures 18 and19 tend to demonstrate how the projection of unique points becomes closer andcloser to edge curves, in each iteration, to minimize the geometric distanceerror. To evaluate the estimation error of motion process, variation of the sixmotion parameters across time has been plotted in figures 20 and 21 for Cowand Helga sequences. Comparing diagrams of true motion with diagrams ofestimated motion by single stereo and perpendicular double stereo setup revealsthe superiority of the perpendicular double stereo to the single stereo. Theassessment is also given numerically in table 4 and table 5. Figures 22 and 23show the temporal sequence of Helga and result of implementation for virtualcamera alignment across time. In addition, this figure illustrates how thesilhouette cones of virtual cameras are intersected to construct the object visualhull. Figure 24 compares the quality of reconstructed Helga model using truemotion information and estimated motion information by single andperpendicular double stereo setups. Figures 25 to 27 demonstrate the result ofimplementation for Cow sequences. To evaluate the robustness of motionestimation against the noise and color maladjustment, comparison betweensingle stereo and perpendicular double stereo are given both quantitatively andqualitatively in table 6 and figure 28. To get the qualified edge curves in noisyimage, it will be necessary to smooth the image before edge detection process.At the same time, smoothing the image makes small perturbation in the positionof edge points. The perpendicular double stereo setup appears to be more robustagainst the perturbation of edge points. Figure 29 to 34 demonstrate the result of


implementation for real objects sequences named Buda, Cactus and Head byperpendicular double stereo setup. All objects have been captured through theturntable sequences. Notice the small perturbations from circular path in thealignment of virtual cameras for Head sequence. These perturbations are causedby the none-rigid motion of body in the neck region. In this experiment, theprocess of motion estimation has been accomplished only based on the head(not body) region information. Both synthetic and real experimental resultsdemonstrate the honored performance of the presented method for the variety ofmotions, object shapes and textures.

Figure 16. Reconstructed spaced curves on the surface of synthetic models Helga and Cow(captured in 3D Studio Max).


Figure 17. Motion estimation by tracking the projections of space curves in perpendeculardouble stereo images.


Figure 18. Motion estimation by minimizing the geometric distance of space curves fromadjacent curves in four camera images. Convergence of algorithm is shown for differentnumber of iterations.


Figure 19. Motion estimation by minimizing the geometric distance of space curves fromadjacent curves in the projected camera images. Convergence of algorithm is shown fordifferent number of iterations for (a) Cow and (b) Helga.


Figure 20.True and estimated motion parameters for Cow sequences by single andperpendicular double stereo setup.

Table 4. Estimation error of motion parameters forCow sequences by single and perpendicular double stereo setup

Mean of Abstract Error Maximum of Abstract ErrorMotion

Parameter SingleStereo Rig

DoublePerpendicular

Stereo Rigs

SingleStereo Rig

DoublePerpendicular

Stereo Rigs

xφΔ (deg) 0.46 0.13 1.53 0.50

yφΔ (deg) 0.72 0.17 1.97 0.61

zφΔ (deg) 0.34 0.12 1.07 0.41

totalφΔ (deg) 0.51 0.14 1.97 0.61

xTΔ (mm) 0.27 0.15 1.02 0.39

yTΔ (mm) 0.13 0.10 0.31 0.24

zTΔ (mm) 0.28 0.30 1.08 1.02

totalTΔ (mm) 0.23 0.18 1.08 1.02


Figure 21. True and estimated motion parameters for Helga sequences by single andperpendicular double stereo setup.

Table 5. Estimation error of motion parameters for Helga sequences bysingle and perpendicular double stereo setup



DoublePerpendicular

Stereo Rigs

SingleStereo Rig

DoublePerpendicular

Stereo Rigs

xφΔ (deg) 0.54 0.14 2.92 0.53

yφΔ (deg) 0.96 0.57 2.91 1.20

zφΔ (deg) 0.32 0.14 1.17 0.44

totalφΔ (deg) 0.61 0.28 2.92 1.20

xTΔ (mm) 0.25 0.18 0.75 0.40

yTΔ (mm) 0.19 0.19 0.43 0.34

zTΔ (mm) 0.28 0.23 0.61 0.46

totalTΔ (mm) 0.24 0.20 0.75 0.46


Figure 22. Different views of Helga in 36 sequence of its motion.


Figure 23. Three-dimensional model reconstruction from multivies for Helga sequences.(top) Trajectory of estimated virtual cameras by perpendicular double stereo rigs alongwith two silhouette cones intersection, (middle) extraction of visual hull by all silhouettecones intersection and, (down) color mapping from visible cameras.

Figure 24. Reconstructed model of Helga including the bonding edges visual hull, depth-map and texture mapped 3D model: (a) estimated-motion with single stereo, (b) estimated-motion with perpendicular double stereo, and (c) true-motion.


Figure 25. Different views of Cow in 36 sequence of its motion.


Figure 26. Trajectory of estimated virtual cameras by perpendicular double stereo rigs andextraction of visual hull by silhouette cones intersection (Cow sequences).

Figure 27. Reconstructed model of Cow including the bonding edges visual hull, depth-map and texture mapped 3D model: (a) estimated-motion with single stereo, (b) estimated-motion with perpendicular double stereo and (c) true-motion.


Table 6. Estimation error of motion parameters for noisy sequences of Helgaby single and perpendicular double stereo setup (σn

2 = 0.1)



DoublePerpendicular

Stereo Rigs

SingleStereo Rig

DoublePerpendicular

Stereo Rigs

xφΔ (deg) 1.02 0.29 5.80 1.41

yφΔ (deg) 1.89 1.43 10.30 3.93

zφΔ (deg) 0.71 0.21 5.12 0.61

totalφΔ (deg) 1.21 0.64 10.30 3.93

xTΔ (mm) 0.33 0.24 0.54 0.38

yTΔ (mm) 0.38 0.30 0.94 0.54

zTΔ (mm) 0.31 0.21 0.68 0.51

totalTΔ (mm) 0.34 0.25 0.94 0.54

Figure 28. Effect of noise and color unbalance in 3D reconstruction: (a) noisy images ofdifferent cameras with σ2 = 0.1, (b) reconstructed model with single stereo rig and (c)reconstructed model with perpendicular double stereo rigs.


Figure 29. Different views of Buda in 36 sequence of its motion.


Figure 30. Reconstruction of Buda statue by perpendicular double stereo rigs (circularmotion with turn table).


Figure 31. Different views of Cactus in 36 sequence of its motion.


Figure 32. Reconstruction of Cactus by perpendicular double stereo rigs (circular motionwith turn table).


Figure 33. Different views of Head (the picture of author) in 36 sequence of its motion.


Figure 34. Reconstruction of Head by perpendicular double stereo rigs (non-circularmotion).

Conclusions

In this chapter, an efficient method has been presented to reconstruct the threedimensional model of a moving object by extracting the space curves and tracking


them across time using perpendicular double stereo rigs. A new method of spacecurve extraction on the surface of the object was presented by checking theconsistency of torsion and curvature through motion. The nature of space curvesmakes the extraction very robust against the poor color adjustment betweencameras and changing the light angle during the object motion. Projection of thespace curves in the camera images were employed for tracking the curves and forthe robust motion estimation of the object. The concept of virtual cameras, whichare constructed from motion information, was introduced. Constructing the virtualcameras makes possible to use a large number of silhouette cones in the structure-from-silhouette method. Experimental results show that the presented edge basedmethod can be effectively used in reconstruction of visual hull for poorly texturedobjects. In addition, it does not require the accurate color adjustment duringcamera setup and provides a better result compared to the other methods, whichuse the color consistency property. Moreover, the presented method is not limitedto the circular turntable rotation. Finally, the double perpendicular stereo setuphas been presented as a way to reduce the effect of statistical bias in motionestimation and to enhance the quality of 3D reconstruction. Quantitatively, theaverage of abstract error in totalφΔ is reduced from 0.51 deg in single stereo rig to0.14 deg in perpendicular double stereo rigs and, the average of abstract error in

totalTΔ is reduced from 0.23 mm to 0.18 mm for Cow sequence. The respectivevalues in Helga sequence are totalφΔ 0.61 - 0.28 deg and totalTΔ 0.24-0.20 mm.These values are increased in noisy sequence of Helga to totalφΔ 1.21 - 0.64 degand totalTΔ 0.34-0.25 mm.

Acknowledgment

This research was supported in part by ITRC, the Iran TelecommunicationResearch Center, under grant no. TMU 85-05-33. Main part of this chapter hasbeen published previously in [43] by the authors.

References

[1] Y. Han, Geometric algorithms for least squares estimation of 3-Dinformation from monocular image, IEEE Trans. Circuits and Systems forVideo Technology, (15) (2) (2005), pp.269-282.


[2] T. Papadimitriou, K.I. Diamantaras, M.G. Strintzis, and M. Roumeliotis,Robust Estimation of Rigid-Body 3-D Motion Parameters Based on PointCorrespondences, IEEE Trans. Circuits and Systems for Video Technology,(10) (4) (2000), pp.541-549.

[3] M. Lhuillier, L. Quan, A Quasi-Dense Approach to Surface Reconstructionfrom Uncalibrated Images, IEEE Trans. Pattern Analysis and MachineIntelligence, (27) (3) (2005), pp. 418-433.

[4] Y. Zhang and C. Kambhamettu, On 3-D Scene Flow and StructureRecovery From Multiview Image Sequences, IEEE Trans. Systems, Man,and Cybernetics - Part B, (33) (4) (2003), pp. 592-606.

[5] P. Eisert, E. Steinbach, and B. Girod, Automatic Reconstruction ofStationary 3-D Objects from Multiple Uncalibrated Camera Views, IEEETrans. Circuits and Systems for Video Technology, (10) (2) (2000), pp.261-277.

[6] Y. Furukawa, A. Sethi, J. Ponce, and D.J. Kriegman, Robust Structure andMotion from Outlines of Smooth Curved Surfaces, IEEE Trans. PatternAnalysis and Machine Intelligence, (28) (2) (2006), pp. 302-315.

[7] B. Triggs, P. McLauchlan, R. Hartley and A. Fiztgibbon, Bundleadjustment- a modern synthesis, In vision Algorithms: Theory and Practice,Lecture Notes on Computer Science, Sppringer-Verlag, (1883) (2000), pp298-372.

[8] R.I. Hartley and A. Zisserman, Multiple View Geometry in ComputerVision. Cambridge Univ. Press, Second Edition, 2003.

[9] M. Pollefeys, R. Koch, M. Vergauwen, and L. Van Gool, Metric 3D SurfaceReconstruction from Uncalibrated Image Sequences, Proc. EuropeanWorkshop 3D Structure from Multiple Images of Large-Scale Environments,(1998), pp. 139-154.

[10] M.A. Fischler and R.C. Bolles, Random Sample Consensus: A Paradigm forModel Fitting with Applications to Image Analysis and AutomatedCartography, Comm. ACM, (24) (6) (1981), pp. 381-395.

[11] P. Torr and D. Murray, The Development and Comparison of RobustMethods for Estimating the Fundamental Matrix, Int’l J. Computer Vision,(24) (3) (1997), pp. 271-300.

[12] Z. Zhang, Determining the epiipolar Geometry and its Uncertainity: aReview, Int’l J. Computer Vision, (27) (2) (1998), pp. 161-195.

[13] P. Torr, A. Zisserman, MLESAC: A New robust estimator with applicationto estimating image geometry, Computer Vision and Image Understanding,(78) (2000), pp. 138-156.


[14] A. Calway, Recursive Estimation of 3D Motion and Surface Structure fromLocal Affine Flow Parameters, IEEE Trans. Pattern Analysis and MachineIntelligence, (27) (4) (2005), pp. 562-574.

[15] K. Kanatani, 3D interpretation of optical flow by renormalization, Int’l J.Computer Vision, (11) (3) (1993), pp. 267–282.

[16] A. D. Jepson and D. J. Heeger, Linear subspace methods for recoveringtranslation direction, in Spatial Vision in Humans and Robots, CambridgeUniv. Press, (1993), pp. 39–62.

[17] A.K.R. Chowdhury and R. Chellappa, Statistical Bias in 3-D Reconstructionfrom Monocular Video, IEEE Trans. Image Processing, (14) (8) (2004), pp.1057-1062.

[18] R. Szeliski and S. B. Kang, Recovering 3D shape and motion from imagestreams using nonlinear least squares, J. Visual Commun. ImageRepresentation, (5) (1) (1994), pp. 10–28.

[19] G. Young and R. Chellappa, 3-D motion estimation using a sequence ofnoisy stereo images: Models, estimation, and uniqueness results, IEEETrans. Pattern Anal. Machine Intell., (12) (1990), pp. 735–759.

[20] J. Weng, P. Cohen, and N. Rebibo, "Motion and Structure Estimation fromStereo Image Sequences ," IEEE Trans. Robotics and Automation, (8) (3)(1992), pp. 362-382.

[21] L. Li and J. Duncan, 3-D translational motion and structure from binocularimage flows, IEEE Trans. Pattern Anal. Machine Intell., (15) (1993), pp.657–667.

[22] F. Dornaika and R. Chung, Stereo correspondence from motioncorrespondence, Proc. IEEE Conf. Computer Vision Pattern Recognition,(1999), pp. 70–75.

[23] P.K. Ho and R. Chung, Stereo-Motion with Stereo and Motion inComplement, IEEE Trans. Pattern Anal. Machine Intell., (22) (2) (2000),pp. 215–220.

[24] S.K. Park and I.S. Kweon, Robust and direct estimation of 3-D motion andScene depth from Stereo image sequences, Pattern Recognition, (34) (9)(2001), pp. 1713-1728.

[25] H. Ebrahimnezhad, H. Ghassemian, 3D Shape Reconstruction of MovingObject by Tracking the Sparse Singular Points, IEEE Internationalworkshop on Multimedia Signal Processing, (2006), pp. 192-197.

[26] L. Robert and O.D. Faugeras, Curve-based stereo: figural continuity andcurvature, Proc. IEEE Conf. Computer Vision Pattern Recognition, (1991),pp. 57–62.


[27] C. Schmid and A. Zisserman, The geometry and matching of lines andcurves over multiple views, Int. Journal ComputerVision, (40) (3) (2000),pp. 199–233.

[28] J.H. Han, J. S. Park, Contour Matching Using Epipolar Geometry, IEEETrans. Pattern Analysis and Machine Intelligence, (22) (4) (2000), pp.358-370.

[29] F. Kahl, J. Augut, Multiview Reconstruction of Space Curves, In Int.Conf.Computer Vision, (2003), pp. 181–186.

[30] A.Gray, Modern Differential Geometry of Curves and Surfaces withMathematica, CRC Press, Second Edition, (1997), pp. 219-222.

[31] A. Bottino, A. Laurentini, Introducing a New Problem: Shape-from-Silhouette When the Relative Positions of the Viewpoints is Unknown,IEEE Trans. Pattern Analysis and Machine Intelligence, (25) (11) (2003),pp. 1484-1492.

[32] Yang Liu, George Chen, Nelson Max, Christian Hofsetz, PeterMcGuinness, Visual Hull Rendering with Multi-view Stereo, Journal ofWSCG. (12) (1-3) (2004).

[33] A.Y. Mulayim, U. Yilmaz, V. Atalay, Silhouette-based 3-D modelreconstruction from multiple images, IEEE Trans. on Systems, Man andCybernetics, Part B, (33) (4) (2003), pp. 582-591.

[34] G.K.M. Cheung, S. Baker, T. Kanade, Shape-from-Silhouette Across TimePart I: Theory and Algorithms, Int’l J. Computer Vision, (62) (3) (2005), pp.221-247.

[35] M. Potmesil, "Generating Octree Models of 3D Objects from theirSilhouettes in a Sequence of Images", CVGIP 40, pp. 1-29, 1987.

[36] R.Szeliski, "Rapid Octree Construction from Image Sequences", CVGIP:Image Understanding 58, 1, pp. 23- 32, July 1993.

[37] W. Matusik, C. Buehler, R. Raskar, S. Gortler, "Image- Based VisualHulls", SIGGRAPH 2000, McMillan, pp. 369- 374, July 23- 28, 2000.

[38] S. Moezzi, A. Katkere, D. Y. Kuramura, and R. Jain, "Reality modeling andvisualization from multiple video sequences", IEEE Computer Graphicsand Applications, 16( 6), pp. 58– 63, November 1996.

[39] G.K.M. Cheung, T. Kanade, J.Y. Bouguet, and M. Holler. "A real timesystem for robust 3D voxel reconstruction of human motions", InProceedings of the 2000 IEEE Conference on Computer Vision and PatternRecognition (CVPR ’00), Vol. 2, pp. 714–720, June 2000.

[40] G.K.M. Cheung, et all. "Visual hull alignment and refinement across time: a3D reconstruction algorithm combining shape-frame-silhouette with stereo."


In Proceedings of IEEE Conference on Computer Vision and PatternRecognition 2003, Vol. 2, pp. 375-382, June 2003.

[41] W. Matusik, et all, "Polyhedral visual hulls for real-time rendering", InProceedings of 12th Euro graphics Workshop on Rendering, pp. 115–125,June 2001.

[42] S. Lazebnik, E. Boyer, and J. Ponce. "On computing exact visual hulls ofsolids bounded by smooth surfaces", In Proceedings of IEEE Conference onComputer Vision and Pattern Recognition (CVPR'01), Kauai HI, December2001.

[43] H. Ebrahimnezhad, H. Ghassemian, “Robust Motion from Space Curvesand 3D Reconstruction from Multiviews Using Perpendicular DoubleStereo Rigs, Journal of Image and Vision Computing, Elsevier,Vol 26,No.10, pp. 1397-1420, Oct. 2008.


Chapter 2

OCULAR DOMINANCE WITHIN BINOCULARVISION

Jonathan S. PointerOptometric Research; 4A Market Square, Higham Ferrers,

Northamptonshire NN10 8BP, UK

Abstract

Ocular dominance (OD) can be defined and identified in a variety of ways. Itmight be the eye used to sight or aim, or whose input is favoured when there iscompeting information presented to the two eyes, or the eye whose functionalvision appears superior on a given task or under certain conditions. The concept,which has been the subject of much discussion and revision over the past fourcenturies, continues to excite controversy today. What is becoming evident is thateven in its most direct and behaviourally significant manifestation – sightingpreference – it must be regarded as a flexible laterality within binocular vision,influenced by the physical circumstances and viewing constraints prevailing atthe point of testing.

This chapter will review the phenomenon of OD in the light of the types oftest used to identify it; question whether inter-test agreement of OD in anindividual might be anticipated; briefly consider the possibility of anyrelationship between OD and limb or cortical laterality; and speculate whetherOD is essentially the product of forced monocular viewing conditions andhabitual use of one or other eye. The chapter will conclude with remarksaddressing some practical implications of OD as demonstrated in healthy eyesand in cases where there is compromised binocular function.

Jonathan S. Pointer64

Introduction: Ocular Dominance

Walls (1951: p. 394) has observed: “… dominance is a phenomenon ofbinocular vision: an eye does not become dominant only when the other eye is outof action”.

So what is ‘ocular dominance’? And what conceivable benefit might such apreferent facility – apparently demonstrable in the majority of binocularly sightedpersons (Ehrenstein et al., 2005; Miles, 1930) – bestow upon the individual?

This chapter will describe and review the phenomenon of ocular dominance.Historically, discussion of eye dominance has occurred in the context of theoriesof binocular visual function (Wade, 1998). But while its existence has beenacknowledged for over 400 years, are we any nearer to understanding this putativelateral oculo-visual preference?

The human eyes, although naturally paired, not infrequently manifest functionalasymmetries. The apparent behavioural performance superiority of one eye isrecognised by a variety of terms: these include ocular dominance, eye preference,sighting dominance or, in terminology analogous to ‘handedness’ or ‘footedness’(motor preference demonstrated by an upper or lower limb, respectively), eyedness(Porac & Coren, 1981). Ocular dominance (OD) is the term that will be usedpreferentially throughout this chapter to embrace this concept, although it should benoted that this choice is not intended to imply that any dominance is of ‘ocular’origin or even a unitary concept (Warren & Clark, 1938).

OD means different things to different people. The lay person might perhapsencounter the phenomenon when aligning objects in space during DIY tasks orwhen threading a needle; when participating in aiming sports activities (eg, claypigeon shooting); or if engaged in specific occupations or pastimes that requiremonocular use of an optical (usually magnification) aid (eg, microscopy,astronomy). Under these circumstances one eye is unconsciously chosen (whenviewing in binocular free space) or consciously selected (when using a gun ormonocular instrument) to undertake the task (Miles, 1929; Porac & Coren, 1976):unconscious and conscious sighting choices as regards right or left eye use arereportedly in agreement approximately 92% of the time (Coren et al., 1979). Allsighting tasks determine OD on the basis of the alignment of two objectspresented at a stereo-disparity sufficiently far outside Panum’s area such thatfusion is denied (Kommerell et al., 2003): the subject is forced to choose betweenone or the other image (ie, eye).

The clinical research scientist might consider criteria other than a sighting(motor) preference as providing a more appropriate indication of ocular lateralitypreference under particular circumstances. These alternatives are likely to be

Ocular Dominance within Binocular Vision 65

performance-related measures of sensory origin; two longstanding popularexamples are the eye with apparently better visual acuity (van Biervlet, 1901;Miles, 1930), or that eye which shows greater facility to suppress a rivalrousretinal image under conditions of binocular viewing (Washburn et al., 1934).

The first recorded description of what we now call OD is usually attributed toGiovanni Battista della Porta (ca1535-1615) [figure 1] in Book 6 of his treatiseDe Refractione (Porta, 1593: Wade, 1998). Porta (1593, pp. 142-143) described apointing test to determine the preferred eye: a rod is held directly in front of thebody and, with both eyes open, the viewer aligns the tip of the rod with a definedobject in the mid-distance – the eye which retains an aligned view of the rod andthe fixation object when the eyes are alternately closed is the preferred (sighting)eye. Translations of Porta’s text, originally published in Latin, have been providedby Durand & Gould (1910) and Wade (1998). It has also been suggested (Wade,1987, p. 793) that the Flemish artist Peter Paul Rubens (1577-1640) might havemade an engraving depicting Porta’s sighting test.

Figure 1. Giovanni Battista (sometimes Giambattista) della Porta (born probably late 1535,deceased 4 February 1615), Neapolitan scholar, polymath and playright. Portrait in profile:frontispiece engraving to expanded 20-volume edition of Magiae Naturalis (Naples,1589).


It might also be noted that around 330BC Aristotle (ca384-322BC) described(without appreciating the significance of his observation) the ability of mostindividuals to wink or temporarily close one eye despite both eyes havingsimilarly acute vision (Ross, 1927, p. 959).

For the interested reader Wade (1998) provides a wide historical survey ofstudies of eye dominances: his narrative places discussion of reported oculo-visualpreferences in the context of evolving theories of human binocular visualfunction.

Over the years numerous techniques have been devised to define OD (Crider,1944). Walls (1951) compiled a list of 25 tests (and indicated that his inventorycould not be regarded as exhaustive). Gronwall & Sampson (1971) compared 18techniques, and Coren & Kaplan (1973) analysed 13 measures. The issue of testappropriateness and comparative inter-technique agreement will be addressedbelow. Suffice to say here that on the grounds of the results of Coren & Kaplan(1973) if one were to choose a single technique to predict eye laterality it couldreasonably – as four hundred years ago – be that of sighting alignment.Furthermore, subsequent research has suggested that the sighting eye thusdetermined seems to extract and process visual spatial information moreefficiently than its fellow (Porac & Coren, 1977; Shneor & Hochstein, 2006 and2008).

Demography of Ocular Dominance

The majority of individuals can record a sighting (motor) dominant eye. Thislaterality is apparently established in early life, possibly around four years of age(Barbeito, 1983; Dengis et al., 1996), and becomes stable by the middle of thehuman development period (Dengis et al., 1993).

Any contribution of genetic factors to an individual’s OD has been littleexplored. Porac & Coren (1981) have concluded that familial traits are absent.Reiss (1997) has expressed equivocation, although admitting that in anexamination of fresh family data OD failed to align with any direct recessive ordominant Mendelian model of genetic transfer.

A number of studies throughout the twentieth century have explored thelaterality distribution of OD in large samples of normally sighted humanpopulations. Porac & Coren ( 1976: Table 1, p. 884) have surveyed much of thiswork, drawing on studies published between 1929-1974, and undertaken in NorthAmerica, the UK, Japan, Australia, and Africa. The broad conclusion was thatapproximately 65% of persons sighted with their right eye, 32% with the left and


only 3% demonstrated no consistent preference. Chronological age and country oforigin appeared to be negligible influences in this compilation.

Subsequent to this summary, the same authors (Porac & Coren, 1981)published the results of a substantial questionnaire-based population study of eye(also ear and limb) laterality. The survey was completed by 5147 individuals(representing an approximately 25% response rate to the mailing) across NorthAmerica. Results indicated that 71.1% of respondents were right sightingpreferent. The balance (28.9%) of respondents were left sighting preferent; thisapparently included a tiny proportion of persons who were unable to indicate apreference.

Replicating the outcome of a previous study (Porac et al., 1980), males (at72.9%) were revealed in this survey as being statistically significantly more righteyed than females (69.1%): this gender imbalance has recently been reportedagain in an independent study (Eser et al., 2008). Male subjects have also beenshown (Porac & Coren, 1975) to be statistically significantly more consistent thanfemales (81% versus 63%, respectively) in their sighting preferences (regardlessof whether laterality was dextral or sinistral).

Adults in the North American postal survey of Porac & Coren (1981) werepossibly more dextral than children, but the trend with advancing chronologicalage was weak and not statistically significant. Suggestions that refractive errorand OD might be associated have not been substantiated in a recent largepopulation study of adult subjects (Eser et al., 2008). Furthermore, over a two-year longitudinal study (Yang et al., 2008) the development of childhood myopiahas been shown to be free of the influence of OD.

A Taxonomy of Ocular Dominance

Over the four centuries subsequent to Porta’s (1593) description of sightingpreference the bibliography of the topic covering theoretical, practical andconjectural issues has expanded to perhaps 600 or more articles (Coren & Porac,1975; updated by Mapp et al., 2003). Unfortunately this burgeoning literature hasnot produced a consistent or unifying theory of OD. As others have voicedpreviously (Flax, 1966; Warren & Clark, 1938) we can still legitimately ask:“What is the purpose of ocular dominance?” Controversially, does it have apurpose or – given that the eye that is considered dominant in a given personmight vary with the task and circumstances (see below) – might the phenomenonbe considered an artefact resulting from a particular test format or approach?


A way forward might be provided by considering a taxonomy of eyedominance based on whether OD might be regarded as a unitary concept or amulti-factorial phenomenon. There have been those who claim a generalisedlaterality (examined by Porac & Coren, 1975), with OD matching hand/footpreferences: an early supporter of absolute unilateral superiority of sensori-motorfunction was the anatomist G. M. Humphrey (1861; as discussed by Woo &Pearson, 1927, pp. 167-169). Others (Berner & Berner, 1953; Cohen, 1952)considered OD to be composed of two factors, namely sighting dominance andrivalry dominance. Walls (1951) regarded OD as a composite of sensory andmotor (eye movement) dominance. Jasper & Raney (1937) added acuitydominance and a dominant cerebral hemisphere to motor control. Lederer (1961)contemplated five varieties of OD (a proposal examined further by Gilchrist,1976): monocular sighting dominance, motor dominance of one eye in binocularcircumstances, orientational dominance, sensory dominance of one eye, anddominance of the lateral (right or left) hemi-field.

A weakness of many of these claims is that essentially they are based onindividual observation or founded on theoretical considerations associated withthe extant literature. More recently Gronwall & Sampson (1971) and Coren &Kaplan (1973) have both brought some enlightenment to a reappraisal of the issueby actually undertaking fresh examinations and applying modern comparativestatistical analyses to the results. The outcome of the study by Coren & Kaplan(1973) in particular provides us with an entry into our probing of the form andfunction of OD.

Is Ocular Dominance Test Specific?

Coren & Kaplan (1973) assessed a group of fifty-seven normally sightedsubjects using a battery of thirteen OD tests that intentionally covered the broadrange of sensori-motor approaches suggested by other investigators over theyears. The test scoring methodology indicated the strength as well as the lateralityof dominance. A factor analysis of the results identified three orthogonaldeterminants of OD: (i) sensory – as revealed by (form) rivalry of stereoscopicallypresented images; (ii) acuity – a visual functional superiority indicated directly bythe comparative inter-eye level of Snellen visual acuity; and (iii) sighting – ocularpreference indicated by an aiming, viewing or alignment type of task.

These three alternative bases for a dominant eye will each be consideredfurther.


I. Tests of Rivalry

Suppression of rivalrous stimuli has been a consistently recognised feature ofeye dominance since the earliest writing on the phenomenon (Porta, 1593: Wade,1998).

The viewing of superficially similar but subtly different stimulus pairs,separated using different coloured optical filters or cross-polarised lenses, can beused to quantify the proportion of time that the view of one or the other eye holdssway in the binocular percept (Washburn et al., 1934). Suppression of competingstimuli is demonstrably not limited to one eye, being usually in a state of flux. Butthis inter-eye rivalry is only possible when the stimuli are small and discrete(Levelt, 1968), immediately questioning the ability of such a rivalry technique toprovide a wider ‘global’ indication of OD.

II. Tests of Asymmetry

Historically (as summarised by Wade, 1998) Aristotle in the third century BCcontended that, although both eyes possessed equal visual acuity, superioraccuracy was achieved using one eye; this opinion prevailed for nearly twomillennia. Only in the eighteenth century was the possibility of an inter-ocularacuity difference given wider credence. Nowadays, it is recognised that innormally sighted (non-amblyopic) individuals, the level of visual acuity isdemonstrably similar in the two eyes (Lam et al., 1996) but not atypically one eyemight perform very slightly better than its fellow. Unfortunately manyinvestigators have been tempted to claim the better-sighted eye as the dominantone (Duke-Elder, 1952; Mallett, 1988a; Miles, 1930).

The problem is that the evidence base for such a supposition is weak (Pointer,2001 and 2007). In addition, laterality correlations with other recordable ocularasymmetries (eg, vergence and version saccades: Barbeito et al., 1986; Pickwell,1972) are absent or unexplored, throwing into question reliance upon oculo-visualfunctional asymmetries as indicators of OD either in the individual or on auniversal scale.

III. Sighting Tests

Probably the most direct and intuitive demonstration of OD originates withthe pointing/alignment test described by Porta (1593). Test variations are manybut include viewing a discrete distant target either through a circular hole cut in a


piece of card or through a short tube (Durand & Gould, 1910), or through thecircular aperture created at the narrow end of a truncated cone when the wider endis held up to the face (A-B-C Test: Miles, 1929).

Prima facie OD thus determined is usually clearly defined and consistentwithin (Miles, 1928 and 1929; Porac & Coren, 1976) and – importantly (seebelow) – between tests (Coren & Kaplan, 1973; Gronwall & Sampson, 1971).Sighting dominance in the study by Coren & Kaplan (1973) accounted for thegreatest proportion (67%) of the variance among all of the tests analysed.

Unfortunately, it has come to be realised that even sighting dominance is notentirely robust, being subject to possible corrupting influences which include:observer (subjective) expectation and test knowledge (Miles, 1929); aspects ofoperational detail for even this simplest of tests (Ono & Barbeito, 1982); the factthat dominance, possibly as a result of relative retinal image size changes (Bankset al., 2004), has been shown to cross between eyes at horizontal viewing anglesas small as 15 degrees eccentricity (Khan & Crawford, 2001; also Henriques etal., 2002; Quartley & Firth, 2004); and not least (for sighting tests which have tobe held), the potentially modulating influence of hand use (Carey, 2001).

The dilemma that arises when OD appears to be test specific is simply whereto place one’s reliance: has dominance switched between eyes or is the outcomean artefact of the testing technique? This rather begs the question: what is thepurpose of OD?

Some Misconceptions

It remains a belief in some quarters that OD is a fixed characteristic in a givenindividual (a claim disputed by Mapp et al., 2003) or even, in an unwarranted leapof reasoning, displays the same laterality as limb (hand/foot) preferences(Delacato, 1959; Humphrey, 1861; Porta, 1593). Apparently one has only to applyone’s choice of test(s) as summarised in the previous section (or identify thewriting hand or the ball-kicking foot) and the laterality of the dominant eye isestablished. But as we have just discussed, these several tests unfortunatelyindicate that more often than not OD appears to vary with test selection orcircumstances. Also, the majority of studies investigating OD in tandem withhand (and rarely, foot) preference have failed to show a congruent relationship:selected examples include Annett (1999), Coren & Kaplan (1973), Gronwall &Sampson (1971), McManus et al. (1999), Merrell (1957), Pointer (2001), Porac &Coren (1975), and Snyder & Snyder (1928).


The content of the previous section of this chapter was based on the results ofthe modern analytical comparative study of OD tests undertaken by Coren &Kaplan (1973). Three statistically significant but independent features werecommon in analysis: viz, the eye which most dominated during binocular rivalrytests, the eye with the better visual acuity and (most significantly) the eye used forsighting. This outcome formalised the several results reported by manyinvestigators before and since: in fact Mapp et al., (2003) have listed 21 suchrelevant studies published between 1925 (Mills) and 2001 (Pointer). Putsuccinctly, OD measured in the individual with one test format does notnecessarily agree with that determined using an alternative approach.

Undaunted, a neuro-anatomical explanation for this functional inconsistencyhas been essayed. Hemispheric cortical specialisation has frequently been claimedas the causal factor underlying all dominances of paired sensory organs or motorlimbs. However as long as seventy years ago Warren & Clark (1938) disputed anyrelation between OD and cortical laterality, but still speculation has not beenentirely silenced.

Suggestions continue to be made that a greater proportion of the primaryvisual cortex is activated by unilateral stimulation of the dominant eye than bythat of the companion eye (eg, Menon et al., 1997; Rombouts et al., 1996).However, what must be remembered in the specific case of human ocular neuro-anatomy is that there is semi-decussation of the optic nerve fibres at the opticchiasma (Wolff, 1968), which results in the unique bi-cortical representation ofeach eye (Duke-Elder, 1952; Flax, 1966). This situation is quite unlike thestraightforward contra-lateral cortical representation pertaining for the upper orlower limbs. Quite simply, ocular neuro-anatomy denies any unifying concept oflaterality.

With an equal longevity to misunderstandings surrounding a claimed corticalbasis for OD is the suggestion that sighting laterality provides the reference framefor the spatial perception of visual direction (Khan & Crawford, 2001; Porac &Coren, 1981; Sheard, 1926; Walls, 1951). The justification for this assertion,linking a monocular task with a binocular system, is doubtful (Gilchrist, 1976). Ithas been convincingly argued by Mapp et al. (2003; pp. 313-314), drawing onboth their own research and independent evidence in the literature, that both eyesparticipate in determining visual direction (Barbeito, 1981; Porac & Coren, 1986)and not the sighting dominant eye alone (eg, Khan & Crawford, 2001). Thispaired influence is also of course in accord with Hering’s (1868/1977) concept ofbinocular vision, wherein the two eyes are composite halves of a single organ.


Resolving the Paradox of Ocular Dominance

From the foregoing discussion of the phenomenon of OD and attempts todefine and measure it, we have apparently arrived at a paradoxical position. Onthe one hand the majority of binocularly sighted persons would quite reasonablyclaim to have experienced a demonstration of OD; on the other hand we areunable to confirm uniformity of laterality in the individual, or identify a clearlydefined oculo-visual role for the phenomenon.

Of the non-human species, only primates have been considered to showcharacteristics consistent with having a sighting dominant eye (Cole, 1957; Smith,1970). This has led Porac & Coren (1976) to speculate that perhaps OD isimportant to animals (including man) where the two eyes are located in the frontalplane of the head: this anatomical arrangement means that the left and rightmonocular visual fields display a substantial binocular overlap, with thefunctional consequence of enhanced depth perception through binocular disparity.However, given that there is fusion only for objects stimulating correspondingretinal points, perhaps suppression of the image in the non-dominant eye removesthe interference arising from disparate and non-fusible left and right images thatwould otherwise confuse the visual percept. Thus the result when undertaking asighting task, for example, is that the image in the dominant eye prevails. Theapparent consistency of right and left OD proportions across the humanpopulation, and the almost universal demonstration of OD from an early (pre-school) age, have inclined Porac & Coren (1976, p. 892) to the opinion that: “…monocular viewing via the dominance mechanism is as natural and adaptive tothe organism as binocular fusion”.

But how reasonable is it to suggest that the highly evolved human visualsystem, which normally seeks to maintain binocular single vision, will ‘naturally’occasionally resort to monocularity? And in this regard, as we have touched on inthe previous section of this chapter and also stated when discussing tests ofrivalry, in the individual with normal (ie, non-amblyopic) sight suppression ofcompeting or rivalrous stimuli is not limited to one eye and one eye alone butrather is fluid from one moment to the next.

It is evident that the sighting (syn. aiming or alignment) test format is the onlytechnique that apparently has the potential to identify OD consistently in abinocularly sighted individual (Coren & Kaplan, 1973). This form of taskspecifically allows the selection of only one eye to accomplish its demands(Miles, 1928); whether by ease or habit most individuals usually perform reliablyand repeatedly in their choice of eye under these constrained circumstances (Porac& Coren, 1976).


It is precisely when the restricted condition of monocular sighting is deniedthat the consistency of eye choice deteriorates. For example, Barbeito (1981)reported that when performing the hole-in-the-card test with the hole covered (orthe Porta pointing test with the view of one’s hand obscured) the imagined hole(or unseen finger or rod) is located on a notional line joining the target and a pointbetween the eyes. Interestingly, a similar effect is observed in infants: when askedto grasp and look through a tube, they will invariably place the tube betweenrather than in front of one or other eye (the Cyclops effect: Barbeito, 1983).Dengis et al., (1998) have replicated this latter outcome in binocular adultsubjects: an electronic shutter obscured the view of the target as a viewing tubewas brought up to the face, resulting in failure to choose a sighting eye but ratherplace the tube at a point on or either side of the bridge of the nose.

Gathering these strands together, perhaps it is possible to reconcile theconflicting views of OD by considering it to be (in older children and adults) aphenomenon demonstrated under constrained viewing conditions (Mapp et al.,2003), with convenience or personal habit (Miles, 1930) forcing the (likelyconsistent) choice of one or the other eye. The elaboration of convoluted,parsimonious or test-specific explanations of OD in an attempt to reconcileobserved phenomena with the known facts regarding binocular vision thenbecomes unnecessary. In summary, the functional significance of OD simplyextends to identifying which of a pair of eyes will be used for monocular sightingtasks (Mapp et al., 2003).

Some Clinical Implications of Ocular Dominance

The concept of OD as a phenomenon identified under circumstances wheremonocular forced-choice viewing conditions prevail does not exist in a void.Consequently, this chapter will conclude with a consideration of the clinicalimplications of OD in patients for whom unilateral refractive, pathological orphysiological (usually age-related) changes might impact on their binocular status.

While the normally sighted binocular individual may not substantially dependon a dominant eye for the majority of daily activities, the identification of apreferred (sighting) eye could become functionally beneficial under specificcircumstances. In an optometric context, for example, for the maximum relief ofsymptoms (including blurred vision and headaches) associated withuncompensated heterophoria (a tendency for the two eyes to deviate from theintended point of fixation), Mallett (1988b) advised that the greater part of anyprescribed prism should be incorporated before the non-dominant eye; clinical


experience suggests that inattention to this point might fail to resolve the patient’sasthenopia or visual discomfort.

A specialist clinical area that has come to the fore in recent years is that ofmonovision, a novel method of correcting presbyopia (the physiological age-related deterioration in near vision as a consequence of impaired accommodativeability). In this approach (Evans, 2007), one eye is optically corrected for distanceviewing and its companion focused for near. Prospective candidates for thisprocedure include middle-aged (usually longstanding) contact lens wearers, andpersons electing to undergo laser refractive surgery.

Typically in monovision it is the dominant eye (usually identified by asighting test) that is provided with the distance refractive correction; near focustasks are allotted to the companion eye. This allocation recognises (Erickson &Schor, 1990) that performance with the dominant eye is superior for spatio-locomotor tasks (including ambulatory activities and driving a vehicle), suchactions relying on an accurate sense of absolute visual direction. In addition, it isclaimed that this ‘dominant/distance’ clinical approach produces better binocularsummation at middle distance and reasonable stereo-acuity at near (Nitta et al.,2007). Others have disputed the necessity to adhere to such a rigid rule, not onlywhen fitting contact lenses (Erickson & McGill, 1992) but also when undertakingrefractive surgery (Jain et al., 2001). However, it might be remarked that afundamental area of concern remains centred on the identification or theprocedural choice of the ‘dominant’ eye, ie, that eye which is likely to take on thedistance-seeing role.

While the visual sensori-motor system shows great adaptability and, as wehave seen, OD can switch between eyes depending upon circumstances, great careshould be taken when practitioners choose to prescribe monovision. Given thewealth of stimuli and changeable viewing circumstances in the ‘real world’, evensighting tests may not reliably indicate which eye should have the distancecorrection; furthermore, such tests cannot accurately predict or guarantee thesuccess of monovision in an individual.

Medico-legal, occupational and vocational caveats surround such a specificapproach to visual correction. Appropriate patient selection and screening areessential features (Jain et al., 1996). Subjects usually require a variable period ofspatial adaptation due to compromised binocular function and the evidentnecessity for visual suppression as viewing circumstances demand. These andrelated features associated with this slightly controversial clinical modus operandihave been well reviewed as experience with the procedure has evolved (Erickson& Schor, 1990; Evans, 2007; McMonnies, 1974) so will not be considered furtherhere.


A naturally occurring version of monovision might temporarily appear inpatients affected by physiological age-related lens opacities (cataracts). The visualdeterioration usually occurs in both eyes, but often to differing degrees andprogresses at different rates, eventually requiring first then frequently second eyesurgery with artificial lens implantation to restore reasonable visual acuity.However, it has been reported (Talbot & Perkins, 1998) that following theimprovement in binocular function after second eye treatment the laterality of ODmay change.

It has also been recorded (Waheed & Laidlaw, 2003) that the debilitatingeffects of monocular injury or pathology may more markedly impair mobility orquality of life if it is the performance of the hitherto sighting dominant eye that isprimarily compromised. However, again, the possibility of sighting dominanceswitching in patients with unilaterally acquired macular disease cannot bediscounted (Akaza et al., 2007).

Conclusion

It might be that a conclusion drawn by Warren & Clark (1938: p. 302)seventy years ago remains pertinent today: “eye dominance as a single unitaryfactor does not exist”.

Perhaps OD is no more than a “demonstrable habit” (Miles, 1930) inbinocular vision, adopted when viewing circumstances demand that only one eyecan conveniently be used (Porac & Coren, 1976). Since classical times theparadoxical question of why, under binocular conditions, unilateral sighting mightbe considered an advantage compared to the continued use of two eyes hascontinued to be asked (Wade, 1998). Allied to this, what could be the oculo-visualpurpose or benefit to the individual of a dominant eye whose laterality canapparently be modified by test conditions (Carey, 2001), by vision training(Berner & Berner, 1953), and by attentional factors (Ooi & He, 1999)?

While the functional basis of OD remains uncertain in a species with a highlyevolved binocular visual system, its demonstrable existence in the majority ofnormally sighted individuals has been linked to a number of perceptual andclinical phenomena. Unfortunately, the years of burgeoning knowledge haveperhaps tended to obscure rather than clarify many issues surrounding OD and itsrelation to oculo-visual performance. What can be stated is that OD must berecognised as a dynamic concept, fluid and deformable in the context of specificviewing conditions and with regard to the methods used to identify it.


References

Akaza, E., Fujita, K., Shimada, H. & Yuzawa, M. (2007). Sighting dominance inpatients with macular disease. Nippon Ganka Gakkai Zasshi, 111, 322-325 [inJapanese].

Annett, M. (1999). Eye dominance in families predicted by the right shift theory.Laterality, 4, 167-172.

Banks, M. S., Ghose, T. & Hillis, J. M. (2004). Relative image size, not eyeposition, determines eye dominance switches. Vision Res., 44, 229-234.

Barbeito, R. (1981). Sighting dominance: an explanation based on the processingof visual direction in tests of sighting dominance. Vision Res., 21, 855-860.

Barbeito, R. (1983). Sighting from the cyclopean eye: the Cyclops effect inpreschool children. Percept. Psychophys., 33, 561-564.

Barbeito, R., Tam, W. J. & Ono, H. (1986). Two factors affecting saccadicamplitude during vergence: the location of the cyclopean eye and a left-rightbias. Ophthalmic Physiol.Opt., 6, 201-205.

Berner, G. E. & Berner, D. E. (1953). Relation of ocular dominance, handednessand the controlling eye in binocular vision. A. M. A. Arch. Ophthalmol. 50,603-608.

van Biervlet, J. J. (1901). Nouvelle contribution a l’étude de l’asymetriesensorielle. Bull. l’Acad. Roy. Sci. Belgique, 3, 679-694.

Carey, D. P. (2001). Vision research: Losing sight of eye dominance. Curr. Biol.,11, R828-R830.

Cohen, J. (1952). Eye dominance. Am. J. Psychol., 65, 634-636.Cole, J. (1957). Laterality in the use of the hand, foot, and eye in monkeys. J.

Comp. Physiol. Psychol., 50, 296-299.Coren, S. & Kaplan, C. P. (1973). Patterns of ocular dominance. Am. J. Optom.

Arch. Am. Acad. Optom., 50, 283-292.Coren, S. & Porac, C. (1975). Ocular dominance: an annotated bibliography. JSAS

Catalogue of Selected Documents in Psychology, 5, 229-230. (Ms. No. 922).Coren, S., Porac, C. & Duncan, P. (1979). A behaviourally validated self-report

inventory to assess four types of lateral preference. J. Clin. Neuropsychol., 1,55-64.

Crider, B. (1944). A battery of tests for the dominant eye. J. Gen. Psychol., 31,179-190.

Delacato, C. H. (1959). The Treatment and Prevention of Reading Problems (TheNeuropsychological Approach). Springfield, Illinois: Charles C. Thomas.

Dengis, C. A., Simpson, T. L., Steinbach, M. J. & Ono, H. (1998). The Cyclopseffect in adults: sighting without visual feedback. Vision Res., 38, 327-331.


Dengis, C. A., Steinbach, M. J., Goltz, H. C. & Stager, C. (1993). Visualalignment from the midline: a declining developmental trend in normal,strabismic, and monocularly enucleated children. J. Pediatr. Ophthalmol.Strabismus, 30, 323-326.

Dengis, C. A., Steinbach, M. J., Ono, H., Gunther, L. N., Fanfarillo, R., Steeves,J. K. & Postiglione, S. (1996). Learning to look with one eye: the use of headturn by normals and strabismics. Vision Res., 36, 3237-3242.

Duke-Elder, W. S. (1952). Textbook of Ophthalmology, Vol. 4. London: HenryKimpton.

Durand, A. C. & Gould, G. M. (1910). A method of determining oculardominance. J. Am. Med. Assoc., 55, 369-370.

Ehrenstein, W. H., Arnold-Schulz-Gahmen, B. E. & Jaschinski, W. (2005). Eyepreference within the context of binocular functions. Graefes Arch. Clin. Exp.Ophthalmol., 243, 926-932.

Erickson, P. & McGill, E.C. (1992). Role of visual acuity, stereoacuity, andocular dominance in monovision patient success. Optom. Vis. Sci., 69, 761-764.

Erickson, P. & Schor, C. (1990). Visual function with presbyopic contact lenscorrection. Optom. Vis. Sci., 67, 22-28.

Eser, I., Durrie, D. S., Schwendeman, F. & Stahl, J. E. (2008). Associationbetween ocular dominance and refraction. J. Refract. Surg., 24, 685-689.

Evans, B. J. (2007). Monovision: a review. Ophthalmic Physiol. Opt., 27, 417-439.

Flax, N. (1966). The clinical significance of dominance. Am. J. Optom. Arch. Am.Acad. Optom., 43, 566-581.

Gilchrist, J. M. (1976). Dominance in the visual system. Br. J. Physiol. Opt., 31,32-39.

Gronwall, D. M. & Sampson, H. (1971). Ocular dominance: a test of twohypotheses. Br. J. Psychol., 62, 175-185.

Henriques, D. Y., Medendorp, W. P., Khan, A. Z. & Crawford, J. D. (2002).Visuomotor transformation for eye-hand coordination. Prog. Brain Res., 140,329-340.

Hering, E. (1977). The Theory of Binocular Vision (B. Bridgeman & L. Stark,Eds. and trans.). New York: Plenum. (Originally published 1868).

Humphrey, G. M. (1861). The Human Foot and the Human Hand. Cambridge:Cambridge University Press, pp. 201 et seq.

Jain, S., Arora, I. & Azar, D. T. (1996). Success of monovision in presbyopes:review of the literature and potential applications to refractive surgery. Surv.Ophthalmol., 40, 491-499.


Jain, S., Ou, R. & Azar, D. T. (2001). Monovision outcomes in presbyopicindividuals after refractive surgery. Ophthalmology, 108, 1430-1433.

Jasper, H. H. & Raney, E. T. (1937). The phi test of lateral dominance. Am. J.Psychol., 49, 450-457.

Khan, A. Z. & Crawford, J. D. (2001). Ocular dominance reverses as a function ofhorizontal gaze angle. Vision Res., 41, 1743-1748.

Kommerell, G., Schmitt, C., Kromeier, M. & Bach, M. (2003). Ocular prevalenceversus ocular dominance. Vision Res., 43, 1397-1403.

Lam, A. K. C., Chau, A. S. Y., Lam, W. Y., Leung, G. Y. O. & Man, B. S. H.(1996). Effect of naturally occurring visual acuity differences between twoeyes in stereoacuity. Ophthalmic Physiol. Opt., 16, 189-195.

Lederer, J. (1961). Ocular dominance. Aust. J. Optom., 44, 531-539; 570-574.Levelt, W. J. M. (1968). On Binocular Rivalry. Paris: Mouton.Mallett, R. F. J. (1988a). Techniques of investigation of binocular vision

anomalies. In K. Edwards & R. Llewellyn (Eds.), Optometry (p. 266).London: Butterworths.

Mallett, R. F. J. (1988b). The management of binocular vision anomalies. In K.Edwards & R. Llewellyn (Eds.), Optometry (pp. 281-282). London:Butterworths.

Mapp, A. P., Ono, H. & Barbeito, R. (2003). What does the dominant eyedominate? A brief and somewhat contentious review. Percept. Psychophys.,65, 310-317.

McManus, I. C., Porac, C., Bryden, M. P. & Boucher, R. (1999). Eye dominance,writing hand, and throwing hand. Laterality, 4, 173-192.

McMonnies, C. W. (1974). Monocular fogging in contact lens practice. Aust. J.Optom., 57, 28-32.

Menon, R. S., Ogawa, S., Strupp, J. P. & Ugurbil, K. (1997). Ocular dominance inhuman V1 demonstrated by functional magnetic resonance imaging. J.Neurophysiol., 77, 2780-2787.

Merrell, D. J. (1957). Dominance of eye and hand. Human Biology, 29, 314-328.Miles, W. R. (1928). Ocular dominance: methods and results. Psychol. Bull., 25,

155-156.Miles, W. R. (1929). Ocular dominance demonstrated by unconscious sighting. J.

Exp. Psychol., 12, 113-126.Miles, W. R. (1930). Ocular dominance in human adults. J. Gen. Psychol., 3, 412-

420.Mills, L. (1925). Eyedness and handedness. Am. J. Ophthalmol., 8 (Series 3), 933-

941.


Nitta, M., Shimizu, K. & Niida, T. (2007). The influence of ocular dominance onmonovision – the interaction between binocular visual functions and the stateof dominant eye’s correction. Nippon Ganka Gakkai Zasshi, 111, 434-440 [inJapanese].

Ono, H. & Barbeito, R. (1982). The cyclopean eye vs. the sighting-dominant eyeas the centre of visual direction. Percept. Psychophys., 32, 201-210.

Ooi, T. L. & He, Z. J. (1999). Binocular rivalry and visual awareness: the role ofattention. Perception, 28, 551-574.

Pickwell, L. D. (1972). Hering’s law of equal innervation and the position of thebinoculus. Vision Res., 12, 1499-1507.

Pointer, J. S. (2001). Sighting dominance, handedness, and visual acuitypreference: three mutually exclusive modalities? Ophthalmic Physiol. Opt.,21, 117-126.

Pointer, J. S. (2007). The absence of lateral congruency between sightingdominance and the eye with better visual acuity. Ophthalmic Physiol. Opt.,27, 106-110.

Porac, C. & Coren, S. (1975). Is eye dominance a part of generalized laterality?Percept. Mot. Skills, 40, 763-769.

Porac, C. & Coren, S. (1976). The dominant eye. Psychol. Bull., 83, 880-897.Porac, C. & Coren, S. (1977). The assessment of motor control in sighting

dominance using an illusion decrement procedure. Percept. Psychophys. 21,341-346.

Porac, C. & Coren, S. (1981). Lateral Preferences and Human Behaviour. NewYork: Springer-Verlag.

Porac, C. & Coren, S. (1986). Sighting dominance and egocentric localization.Vision Res., 26, 1709-1713.

Porac, C., Coren, S. & Duncan, P. (1980). Life-span age trends in laterality. J.Gerontol., 35, 715-721.

Porta, G. B. (1593). De Refractione. Optices Parte. Libri Novem. Naples:Salviani.

Quartley, J. & Firth, A. Y. (2004). Binocular sighting ocular dominance changeswith different angles of horizontal gaze. Binocul. Vis. Strabismus Q., 19, 25-30.

Reiss, M. R. (1997). Ocular dominance: some family data. Laterality, 2, 7-16.Rombouts, S. A., Barkhof, F., Sprenger, M., Valk, J. & Scheltens, P. (1996). The

functional basis of ocular dominance: functional MRI (fMRI) findings.Neurosci. Lett., 221, 1-4.

Ross, W. D. (1927). The Works of Aristotle: Volume 3 (Ed.). Oxford: ClarendonPress.


Sheard, C. (1926). Unilateral sighting and ocular dominance. Am. J. Optom. Arch.Am. Acad. Optom., 7, 558-567.

Shneor, E. & Hochstein, S. (2006). Eye dominance effects in feature search.Vision Res., 46, 4258-4269.

Shneor, E. & Hochstein, S. (2008). Eye dominance effects in conjunction search.Vision Res., 48, 1592-1602.

Smith, L. (1970). Eye dominance in a monkey. Percept. Mot. Skills, 31, 657-658.Snyder, A. M. & Snyder, M. A. (1928). Eye preference tendencies. J. Ed.

Psychol., 19, 431-433.Talbot, E. M. & Perkins, A. (1998). The benefit of second eye cataract surgery.

Eye, 12, 983-989.Wade, N. J. (1987). On the late invention of the stereoscope. Perception, 16, 785-

818.Wade, N. J. (1998). Early studies of eye dominances. Laterality, 3, 97-108.Waheed, L. & Laidlaw, D. A. H. (2003). Disease laterality, eye dominance, and

visual handicap in patients with unilateral full thickness macular holes. Br. J.Ophthalmol., 87, 626-628.

Walls, G. L. (1951). A theory of ocular dominance. A. M. A. Arch. Ophthalmol.,45, 387-412.

Warren, N. & Clark, B. (1938). A consideration of the use of the term oculardominance. Psychol. Bull., 35, 298-304.

Washburn, M. F., Faison, C. & Scott, R. (1934). A comparison between the MilesA-B-C method and retinal rivalry as tests of ocular dominance. Am. J.Psychol., 46, 633-636.

Wolff, E. (1968). The Visual Pathway. In R. J. Last (Ed.), Anatomy of the Eye andOrbit (6th edition, pp. 341-344). London: H. K. Lewis.

Woo, T. L. & Pearson, K. (1927). Dextrality and sinistrality of hand and eye.Biometrika, 19, 165-199.

Yang, Z., Lan, W., Liu, W., Chen, X., Nie, H., Yu, M. & Ge, J. (2008).Association of ocular dominance and myopia development: a 2-yearlongitudinal study. Invest. Ophthalmol. Vis. Sci., 49, 4779-4783.


Chapter 3

THREE-DIMENSIONAL VISION BASED ONBINOCULAR IMAGING AND APPROXIMATION

NETWORKS OF A LASER LINE

J. Apolinar Muñoz-Rodríguez*

Centro de Investigaciones en Optica, A. C. Leon, Gto, 37150 Mexico.

Abstract

We present a review of our computer vision algorithms and binocular imagingfor shape detection optical metrology. The study of this chapter involves: lasermetrology, binocular image processing, neural networks, and computer visionparameters. In this technique, the object shape is recovered by means of laserscanning and binocular imaging. The binocular imaging avoids occlusions, whichappear due to the variation to the object surface. A Bezier approximation networkcomputes the object surface based on the behavior of the laser line. By means ofthis network, the measurements of the binocular geometry are avoided. Theparameters of the binocular imaging are computed based on the Bezierapproximation network. Thus, the binocular images of the laser line areprocessed by the network to compute the object topography. By applying Bezierapproximation networks, the performance of the binocular imaging and theaccuracy are improved. It is because the errors of the measurement are not addedto the computational procedure, which performs the shape reconstruction. Thisprocedure represents a contribution for the stripe projection methods and thebinocular imaging. To describe the accuracy a mean square error is calculated.

* E-mail address: [email protected]. Tel: (477) 441 42 00.

J. Apolinar Muñoz-Rodríguez82

This technique is tested with real objects and its experimental results arepresented. Also, the time processing is described.

Keywords: Binocular imaging, laser line, Bezier approximation networks.

1. Introduction

In computer vision, optical systems have been applied for shape detection.The use of structured illumination makes the system more reliable and theacquired data are easier to interpret. A particular technique is the laser lineprojection [1-3]. When a laser line is projected on an object, the line position isshifted in the image due to the surface variation and the camera position. From theline position and the geometry of the optical setup, the object contour is deduced.The main aim of the line projection is the detection of the line behavior in theimage [4-6]. Also, the geometry of the setup is measured to obtain the objecttopography. When the surface variation produces an occlusion, the line positioncan not be detected. Therefore, in the area of the line occlusion the topographicdata are not retrieved [7-8]. In the proposed technique here, occluded areas areretrieved by binocular imaging. To extract the object surface, the object is movedin the x-axis and scanned by a laser line. Based on the line behavior in the image,the object shape is retrieved. In each step of the scanning, a pair of binocularimages of the line is captured. When the surface produced a line occlusion, theoccluded area appears only one of the two images. Thus, the data missed in eachimage can be retrieved from each other. In this manner, the line occlusion isavoided by the binocular imaging. Typically, the disparity is needed to retrieve theobject depth in a binocular system [9]. In the proposed technique, the disparity isdetermined by detecting the line position in each pair of the binocular images. Inthe proposed setup, the position of the line disparity is proportional to the objectdepth. By means of a Bezier network, a mathematical relationship between theline position and the object depth is generated. In this manner, the measurementsof the focal length, and the distance between the two cameras are avoided. TheBezier network is constructed using the disparity of the line, which is projected onthe objects with known dimensions. The disparity is detected measuring theposition of the line displacement in the image. Thus, the network calculates thedepth dimension for a position of a stripe displacement given. The line position isdetected by means of Bezier curves with a resolution of a fraction of pixel. Thisposition is processed by the network to determine the object shape. In thismanner, all steps of the proposed technique are performed automatically by

Three-Dimensional Vision Based on Binocular Imaging… 83

computational algorithms. Thus, the physical measurements on the setup areavoided. This kind of computational process improves the performance and theaccuracy. Thus, a contribution is provided in the binocular imaging methods. Inthe proposed technique, the object is moved in the x-axis and scanned by a laserline. From the scanning, a set of binocular images of the line is captured. Each oneof these images is processed by the Bezier curves to determine the position of theline disparity. This position is processed by the network to determine the objectdepth. The structure of the network consists of an input vector, a hidden layer andan output layer. The input vector includes: object dimensions, line position andparametric data. The hidden layer is constructed by neurons of Bezier Basisfunctions. The output layer is formed by the summation of the neurons, which aremultiplied by a weight. The produced information by a pair of the binocularimages corresponds to a transverse section of the object. The data of transversesections are stored in an array memory to obtain the complete object shape. Theresults obtained in this technique are achieved with very good repeatability.

2. Basic Teory

The proposed setup figure 1 consists of a line projector, two CCD cameras,an electromechanical device and a computer. In this arrangement, the object isfixed on a platform of the electromechanical device. The platform moves theobject in the x-axis. A laser stripe is projected onto the object surface by a laserdiode to perform the scanning. In each step of the scanning, the laser line isdigitized by two CCD cameras. In each image, the line is deformed in the x-axisaccording to the object surface. The profilometric method is based on the linedeformation. By detecting the position of the line deformation, the object depthis determined. Therefore, the line position is main parameter to perform theobject contouring. The object contour is broken when an occlusion appears. Toretrieve the complete object contour, the binocular imaging is applied. In thissystem, the occluded line in the first camera can be observed by the secondcamera. Also, the occluded line in the second camera can be observed by thefirst camera. Thus, the binocular system overcomes the occlusion in the lineprojection. In this manner, the object contour can be computed completely. Thecontouring is described based on the geometry shown in figure 2. On thereference plane are located the x-axis, y-axis and the object depth is indicated byh(x, y) in the z-axis. The points o and p correspond to line projected on thereference plane and object surface, respectively. The focal length is indicated byF, d is the distance between the two cameras and the image center of each


camera is indicated by ac and ec, respectively. When a laser line is projected onthe surface, the line position is moved from ao to ap in the image plane of thecamera a. At the same time, the line position is moved from eo to ep in thecamera e. The line displacement in the image plane for each camera isrepresented by

as(x,y)= ao - ap. (1)

es(x,y)= ep - e0. (2)

By means of the positions ep and ap, the line displacement respect to the referencepoint “o” is given by as(x,y) for the camera a and es(x,y) for the camera e. Basedon the pinhole camera model [10], the surface zi in a binocular system is deducedby

εδ +=

Fdzi , (3)

from this equation, δ+ε is the disparity. The distances of the disparity are deducedby δ = ac- ap and ε = ec – ep, respectively. To compute the surface zi by means ofEq.(3), the constants F, d, ac, ec and camera orientation should be known.Typically, the parameters d and F are obtained by an external procedure to thecontouring. Then, these parameters are given to the computer system toreconstruct the object shape. This means that Eq.(3), can not be computed in theimage processing of the laser line. For the proposed technique, the object depthh(x,y) is computed directly in the image processing of line position. In thisprocedure, a Bezier network provides a function that computes the object depthbased on the line displacement. Also, this network provides information of thecamera orientation, the focal lens, the center coordinates, the distance betweentow cameras, disparity and center coordinates. Thus, the performance for objectcontouring is improved. In our contouring system, the object depth is computedby the network based on the line position using only one camera of the binocularsystem. This means that the network produces the depth h(x,y) using as(x,y) ores(x,y). The image processing provides a least one of the two displacements.When a line occlusion appears in the camera a, as(x,y) is missing. Then, the ep ofthe line disparity is used to compute the object depth h(x,y). In this manner, theocclusion problem of the laser line is solved by the binocular imaging. To detectthe disparity, the line position is measured in every row of the binocular images.This position is computed by detecting the maximum of the line intensity in each


row of the image. The intensity projected by a laser diode is a Gaussiandistribution in the x-axis [11]. The intensity in every row of the image isrepresented by (x0, z0), (x1, z1), (x , z2),.......,(xn, zn), where xi is the pixel positionand zi is the pixel intensity. To detect the line position, the maximum intensitymeasured in the image. To carry it out, Bezier curves and peak detection are used.The nth-degree Bezier function is determined by n+1pixels [12]. The nth-degreeBezier function is determined by two parametric equations, which are describedby

x(u) = ⎟⎟⎠

⎞⎜⎜⎝

⎛0n

(1 - u)n u0 x0 + ⎟⎟⎠

⎞⎜⎜⎝

⎛1n

(1 - u) n-1u x1 +

+ ⎟⎟⎠

⎞⎜⎜⎝

⎛nn

(1 - u) 0 un xn, 0 ≤ u ≤ 1. (4)

z(u) = ⎟⎟⎠

⎞⎜⎜⎝

⎛0n

(1 - u)n u0 z0 + ⎟⎟⎠

⎞⎜⎜⎝

⎛1n

(1 - u) n-1u z1 +

+ ⎟⎟⎠

⎞⎜⎜⎝

⎛nn

(1 - u) 0 un zn, 0 ≤ u ≤ 1. (5)

Eq.(4) represents the pixel position and Eq.(5) represents the pixel intensity. To fitthe Bezier curve shown in figure 3, x0, x1, x2,......,xn, are substituted into Eq. (10)and z0, z1, z2,....,zn, are substituted into Eq. (5). Then, these equations areevaluated in the interval 0≤ u≤1. In this case, the second derivative z”(u) > 0 inthe interval 0≤ u≤1. Therefore, the maximum is detected by the first derivativeequal to zero z´(u)=0 [13] via bisection method. Beginning with a pair of values ui

= 0 and us =1, because z(u) is defined for the interval 0 ≤ u ≤ 1, u* is halfwaybetween ui and us. If z´(u) evaluated at u = u* is positive, then ui = u*. If z´(u)evaluated at u = u* is negative, then us=u*. Next, u* is taken as the mid point ofthe last pair values that converges to the root. The value u* where z´(u) = 0 issubstituted into Eq.(5) to obtain maximum position x*. The result is x* = 34.274and the stripe position is ap = 34.274 pixels, which is shown in figure 3. Theprocedure of stripe detection is applied to all rows of the image. Then, the lineposition is processed by the network to obtain the object contour.


Figure 1. Experimental setup.

Figure 2. Geometry of the experimental setup.0.


Figure 3. Maximum position from a set of pixels fitted to a Bezier curves.

3. Bezier Networks for Surface Contouring

From the binocular images, the line displacement is proportional to the objectdepth. A Bezier network is built to compute the object depth h(x,y) based on thedisplacement as(x,y) or es(x,y). This network is constructed based on a lineprojected on objects with known dimensions. The network structure consists of aninput vector, a parametric input, a hidden layer and an output layer. This networkis shown in figure 4. Each layer of the network is deduced as follow. The inputincludes: the object dimensions hi, the stripe displacements as, es and theparametric values u and v. The input data as0, as1, as2,….,asn and es0, es1,es2,….,esn are the stripe displacements obtained by image processing described insection 2. By means of these displacements, a linear combination LCa and LCe aredetermined [14] to compute u and v. The relationship between the displacementand the parametric values is described by

u= b0 +b1as, (6)

v= c0 +c1es, (7)


where bi and ci are the unknown constants. The Bezier curves are defined in theinterval 0≤ u ≤ 1 and 0≤v≤ 1 [17]. For the depth h0, the displacements as0 and es0

are produced. Therefore, u=0 for as0 and v=0 for es0. For hn the displacements asn

and esn are produced. Therefore, the u=1 for asn and v=1 for esn is. Substitutingthe values (as0, u=0) and (asn, u=1) in Eq.(6), two equations with two unknownconstants are obtained. Solving these equations b0 and b1 are determined andEq.(6) is completed. Substituting the values (es0, v=0) and (esn, v=1) in Eq.(7),two equations with two unknown constants are obtained. Solving these equationsc0 and c1 are determined and Eq.(7) is completed. Thus, for the displacement asand es, the parametric values u and v are computed via Eq.(6) and Eq.(7),respectively. The input h0, h1, h2,...,hn are obtained from the pattern objects, whosedimensions are known. The hidden layer is constructed by Bezier basis function[16], which is described by

inii uu

in

uB −−⎟⎟⎠

⎞⎜⎜⎝

⎛= )1()( ,

)!(!!

inin

in

−=⎟⎟

⎠

⎞⎜⎜⎝

⎛ (8)

denotes the binomial distribution from statistics. The output layer is thesummation of the neurons of the hidden layer, which are multiplied by a weight.The output is the depth h(x,y), which is represented by ah(u) and eh(v). These twooutputs are described by the next equations

a ∑=

=n

iiii huBwuh

0)()( , 0 ≤ u ≤ 1, (9)

e ∑=

=n

iiii hvBrvh

0)()( , 0 ≤ v ≤ 1, (10)

where wi and ri are the weights, hi is the known dimension of the pattern objectsand Bi is the Bezier basis function Eq.(8). To obtain the network Eq.(9) andEq.(10), the suitable weights wi and ri should be determined. To obtain theweights w0, w1, w2,…..,wn, the network is being forced to produce the outputsh0, h1, h2,.....,hn by means of an adjustment mechanism. To carry it out, thevalues hi and its u are substituted in Eq.(9). The value u is computed via Eq.(6)based on the displacement as, which corresponds to the known dimension hi.


For each input u, an output ah is produced. Thus, the next equation system isobtained

ah0 = h0 = w0 ⎟⎟⎠

⎞⎜⎜⎝

⎛0n

(1 - u)n u0 h0 + w1 ⎟⎟⎠

⎞⎜⎜⎝

⎛1n

(1 - u) n-1uh1 +,…..,+ wn ⎟⎟⎠

⎞⎜⎜⎝

⎛nn

(1 - u) 0 un hn, 0 ≤ u ≤ 1.

ah1 =h1 = w0 ⎟⎟⎠

⎞⎜⎜⎝

⎛0n

(1 - u)n u0 h0 + w1 ⎟⎟⎠

⎞⎜⎜⎝

⎛1n

(1 - u) n-1uh1 +,…..,+ wn ⎟⎟⎠

⎞⎜⎜⎝

⎛nn

(1 - u) 0 un hn, 0 ≤ u ≤ 1. (11)

ahn =hn = w0 ⎟⎟⎠

⎞⎜⎜⎝

⎛0n

(1 - u)n u0 h0 + w1 ⎟⎟⎠

⎞⎜⎜⎝

⎛1n

(1 - u) n-1uh1 +,…..,+ wn ⎟⎟⎠

⎞⎜⎜⎝

⎛nn

(1 - u) 0 un hn, 0 ≤ u ≤ 1.

This linear system of Eq.(11) can be represented as

h0 = w0β0,0 + w1β0,1+..........+wnβ0,n

h1 = w0β1,0 + w1β1,1+..........+wnβ1,n (12)

hn = w0βn,0 + w1βn,1+..........+wnβn,n

This equation can be rewritten as the product between the matrix of the input dataand the matrix of the corresponding output values: βW = H. The linear systemrepresented by the next matrix

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

nnnnnnn

n

n

h

hh

w

ww

1

0

1

0

,2,1,0,

,12,11,1101

,02,01,00,0

....

....

....

ββββ

ββββββββ

(13)

This system Eq.(13) is solved by the Chelosky method [17]. Thus the weights w0,w1, w2,….,wn are calculated and Eq.(15) has been completed.


To determine the weights r0, r1, r2,…..,rn, again, the network is being forcedto produce the outputs h0, h1, h2,.....,hn. ri. To carry it out, the values v and hi aresubstituted in Eq.(10). The value v is calculated via Eq.(7) using the displacementes, which corresponds to the dimension hi. For each input v, an output eh isproduced and the next equation system is obtained

eh0 = h0 = r0 ⎟⎟⎠

⎞⎜⎜⎝

⎛0n

(1 - v)n v0 h0 + r1 ⎟⎟⎠

⎞⎜⎜⎝

⎛1n

(1 - v) n-1vh1 +,…..,+ rn ⎟⎟⎠

⎞⎜⎜⎝

⎛nn

(1 - v) 0 vn hn, 0 ≤ v ≤ 1.

eh1 =h1 = r0 ⎟⎟⎠

⎞⎜⎜⎝

⎛0n

(1 - v)n v0 h0 + r1 ⎟⎟⎠

⎞⎜⎜⎝

⎛1n

(1 - v) n-1vh1 +,…..,+ rn ⎟⎟⎠

⎞⎜⎜⎝

⎛nn

(1 - v) 0 vn hn, 0 ≤ v ≤ 1. (14)

ehn =hn = r0 ⎟⎟⎠

⎞⎜⎜⎝

⎛0n

(1 - v)n v0 h0 + r1 ⎟⎟⎠

⎞⎜⎜⎝

⎛1n

(1 - v) n-1vh1 +,…..,+ rn ⎟⎟⎠

⎞⎜⎜⎝

⎛nn

(1 - v) 0 vn hn, 0 ≤ v ≤ 1.

The linear system Eq.(14) can be represented as the product between the inputmatrix and the matrix of the corresponding output: ℑ R = H. The linear systemrepresented by the next matrix

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

=

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

⎥⎥⎥⎥

⎦

⎤

⎢⎢⎢⎢

⎣

⎡

ℑℑℑℑ

ℑℑℑℑℑℑℑℑ

nnnnnnn

n

n

h

hh

r

rr

1

0

1

0

,2,1,0,

,12,11,1101

,02,01,00,0

....

....

....

(15)

This linear system Eq.(15) is solved and the weights r0, r1, r2,….,rn aredetermined. In this manner, Eq.(16) has been completed. Thus, the networkproduces the shape dimension via Eq.(9) and Eq.(10) based on the linedisplacement as and es respectively. In this manner, the BAN provides the objectdepth by means of h(x,y)= ah(u) and h(x,y)=eh(v). This network is applied to thebinocular images shown in figure 5(a) and figure 5(b), respectively. Thebinocular images correspond to a line projected on a dummy face. From theseimages, the stripe position ap and ep are computed along the y-axis. Then, thedisplacement as(x,y) and es(x,y) are computed via Eq.(1) and Eq.(2), respectively.


The contours provided by the network are shown figure 6(a) and figure 6(b),respectively. The contour of figure 6(a) is not completed due to the occlusion infigure 5(a). But, figure 5(b) does not contain line occlusions and the completecontour is achieved in figure 6(b).

Figure 4. Structure of the proposed Bezier network.


Figure 5 (a). First line captured by the binocular system.

Figure 5 (b). Second line captured by the binocular system.


Figure 6 (a). Surface profile computed by the network from figure 5(a).

Figure 6 (b). Surface profile computed by the network from figure 5(b).

4. Parameter of the Vision System

In optical metrology, the object shape is reconstructed based on theparameters of the camera and the setup. Usually, these parameters are computedby external procedure to the reconstruction system. The camera parametersinclude focal distance, image center coordinates, pixel dimension, distortion andcamera orientation. In the proposed binocular system, the camera parameters aredetermined based on the data provided by the network and image processing.

The camera parameters are determined based on the pinhole camera model,which is shown in figure 7. In the binocular system, the optical axis of thecameras is perpendicular to the reference plane. The camera orientation in the x-axis is determined by means of the geometry figure 8(a). In this geometry, the line


projected on the reference plane and the object is indicated by ao and ap at theimage plane, respectively. The distance between the image center and the laserstripe in the x-axis is indicated by a. The object dimension is indicated by hi and

D = zi + hi. For this geometry, the camera orientation is performed based on si andhi from the network. According to the perpendicular optical axis, the object depthhi has a projection ki in the reference plane. From figure 8(a), the displacement isdefined as si = (xc - ap) - (xc - ao). Thus, the projection ki at the reference plane iscomputed by

oci

ii axs

hFk

−+= (16)

From Eq.(16) F, xc, ao are constants and hi is computed by the network based onsi. In this case, ki is a linear function. Therefore, the derivative ki respect to si dk/dsis a constant. Other configuration is an optical axis not perpendicular to thereference plane. In this case, si does not produce a linear function ki. Also, thederivative dk/ds is not a constant. The orientation of the camera in y-axis isperformed based on the geometry of figure 8(b). In this case, the object is movedin y-axis over the line stripe. When the object is moved, the pattern positionchanges from ayp to ayi in the laser stripe. In this case, t = (ayc-ayp) - (ayc-ayi). Foran optical axis perpendicular to the reference plane y-axis, a linear q produces alinear t at the image plane. Therefore, the derivative dt/dq is a constant. Thus, theorientation camera is performed by means of dk/ds = constant for the x-axis anddt/dq = constant for the y-axis. Based on these criterions, the optical axis isaligned perpendicular to x-axis and y-axis. For the orientation in x-axis, ki iscomputed from hi provided by the network. Due to the distortion, the derivativedk/ds is slightly different to a constant. But, this derivative is the more similar to aconstant, which is shown in figure 8(c). In this figure, the dash line is dk/ds for βminor than 90° and the dot line is dk/ds for β major than 90°. Thus, the generatednetwork corresponds to an optical axis aligned perpendicularly to the x-axis. Forthe orientation in y-axis, qi is provided by the electromechanical device and t isobtained by image processing. In this process, the object position is detected inthe line in each movement. Due to the distortion, the derivative dt/dq is notexactly a constant. But, this derivative is the more similar to a constant. Thus, theoptical axis is aligned perpendicular to the y-axis. In this manner, the network andimage processing provide an optical axis aligned perpendicularly to the referenceplane. Based on the optical axis perpendicular to reference plane, the camera


parameters are obtained. To carry it out, the network produces the depth hi basedon si for the calibration. The geometry of the setup figure 8(a) is described by

apc

i

a

i

axFzz+−

+=

)(η , (17)

From this equation η is scale factor to convert the pixels to millimeters. Using D= zi + hi and ηsi = η (xc - ap) - η (xc - ao), Eq.(17) is rewritten as

aoci

i

a

i

axsFhDhD+−+

+−=

−)(η

(18)

Where D is the distance from the lens to the reference plane. From Eq.(18) theconstants D, a, F, η, xc and ao should be determined. To carry it out, Eq.(18) is

rewritten as equation system

)( 11

oc

a

axsF

Dh−+

−=η

)( 22

oc

a

axsF

Dh−+

−=η

)( 33

oc

a

axsF

Dh−+

−=η

(19)

)( 44

oc

a

axsF

Dh−+

−=η

)( 55

oc

a

axsF

Dh−+

−=η

)( 66

oc

a

axsF

Dh−+

−=η

The values h1, h2,…., h6, are computed by the network according to s1, s2,…., s6.These values are substituted in Eq.(19) and the equation system is solved. Thus,


the constants D, a, F, η, xc and ao are determined. The coordinate ayc is computed

from the geometry figure 8(c) described by

)()()(1−−

−−−=

ib

ipci q

hDFayaytη

, (20)

From Eq.(20) the constants D, F, η, qi, ti, hi are known and ayc, b, ayp should be

determined. To carry it out, Eq.(20) is rewritten as equation system for an hi

constant by

)()()(

0

11 qg

hDFayayt pc −−

−−=η

)()()(

1

12 qg

hDFayayt pc −−

−−=η

(21)

)()()(

2

13 qg

hDFayayt pc −−

−−=η

The values t1, t2, t3, are taken from the orientation in y-axis, q0= 0 and the valuesq1, q2, are provided by the electromechanical device. These values are substitutedin Eq.(21) and the equation system is solved. Thus, the constants ayc, ayp and b

are determined. In this manner the camera parameters are calibrated based on thenetwork and image processing of the laser line.

The distortion is observed by means of the line position ap in the image plane,which is described by

ci

ap x

hDF

a +−

= (22)

Based on Eq.(22), the behavior of ap respect to hi is a linear function. However,due to the distortion, the real data ap are not linear. The network is constructed bymeans of the real data using the displacement si =(ac- ap) - (xc- ao). Therefore, thenetwork produces a non linear data h, which is shown in figure 8(d). Thus, thedistortion is included in the network, which computes the object depth in theimaging system.


Figure 7. Geometry of the pinhole camera model.

Figure 8 (a). Geometry of an optical axis perpendicular to x-axis.


Figure 8 (b). Geometry of an optical axis perpendicular to y-axis.

Figure 8 (c). Derivative dk/ds for an optical axis perpendicular to x-axis and for an opticalaxis not perpendicular to x-axis.


5. Experimental Results

The approach of the Binocular imaging in this technique is to avoid the lineocclusions. When an occlusion appears, the object contour is not completed.However, in a binocular imaging one of two images provides the occluded lineand the object contour is completed. Based on the network, the parameters of thebinocular setup are computed and physical measurements are avoided. In thismanner, the computational performance provides all parameters to reconstruct theobject shape. Figure 1 shows the experimental setup. The object is moved in the x-axis by means of the electromechanical device in steps of 1.77 mm. A laser line isprojected on the target by a 15 mW laser diode to perform the scanning. The lineis captured by two CCD cameras and digitized by a frame grabber of 256 graylevels. By means of image processing, the displacement as and es are computed.Then, the object depth h(x,y) is computed via Eq.(9) or Eq.(10). From each pair ofthe binocular images, a transverse section of the object is produced. Theinformation of all transverse sections is stored in array memory to construct thecomplete object shape.

The experiment is performed with three objects. The first object to be profiledis a dummy face see figure 9(a) and 9(b). The object surface produces stripeocclusions shown in figure 9(a). In this case, the second image figure 9(b)provides the area of the occluded stripe. Therefore, the object reconstruction canbe done by means of the binocular imaging. To perform the contouring, thedummy face is scanned in x-axis in steps of 1.27 mm and binocular images of theline are captured. From each pair of images, the first image is used to detect thestripe displacement as. If the first image contains line occlusions, the secondimage will be used to detect the stripe displacement es. The line occlusion isdetected based pixel of the stripe. If the pixels of high intensity is minor than threein a row, a line occlusion is detected in the image. By image processing, the stripedisplacement as or es is calculated via Eq.(1) or Eq(2) respectively. Then, thevalues u and v are deduced via Eq.(6) and Eq.(7). By means of the network Eq.(9)or Eq.(10), the object depth is computed via u or v, respectively. Thus, thenetwork produces a transverse section based on the binocular images. To knowthe accuracy of the data provided by the network, the root mean squared error(rms) is calculated [18] based on a contact method. To carry it out, the object ismeasured by a coordinate measure machine (CMM). The rms is described by thenext equation


2

1)(1

ii

n

ihcho

nrms −= ∑

=, (30)

where hoi is the data measured by the CMM, hci is the calculated data h(x,y) bynetwork and n is the number of data. The rms was computed using n=1122 dataand the result is a rms = 0.155 mm for the dummy face. In this case, sixty eightylines were processed to determine the complete object shape shown in figure 9(c).The scale of this figure is mm.

The second object to be profiled is a metallic piece figure 10(a). Thecontouring is performed by scanning the metallic piece in steps of 1.27 mm. Inthis case, Fifty eight lines were processed to determine the complete object shapeshown in figure 10(b). The metallic piece was measured by the CMM todetermine the accuracy provided by the network. The rms was computed usingn=480 data, which were provided by the network and by the CMM as reference.The rms is calculated for this object is a rms = 0.114 mm.

Figure 9(a). First image of the dummy face whit line occlusion.


Figure 9(b). Second image of the dummy face with out line occlusion.

Figure 9(c). Three-dimensional shape of the dummy face.


Figure 10(a). Metallic piece with to be profiled.

Figure 10(b). Three-dimensional shape of the metallic piece.

The value n has an influence in the confidence level respect to the precisionof the error calculated. To determine if the value n is according to the desiredprecision, the confidence level is calculated by the next relation [19]

2

⎟⎠⎞

⎜⎝⎛=

ezn xσα , (31)


where zα is the confidence desired, e is the error expressed in percentage, and σx isstandard deviation. Therefore, the confidence level according to the data n can bedescribed by

nezxσα = . (32)

To know if the value n chosen is according with the confidence of level desiredEq.(31) is applied. The confidence level desired is 95 %, which corresponds tozα=1.96 according to the confidence table [26]. The average of the height of theface surface is 19.50 mm, therefore using the rms the error is 0.0079, whichrepresents a 0.79 % of error. To determine the precision of this error, theconfidence level is calculated for the n=1122, e = 0.79 and standard deviation is7.14. Substituting the values in Eq.(32), the result is zα =3.7061. It indicates aconfidence level greater than the 95%. Also, the confidence level is greater than95% for metallic piece.

The employed computer in this process is a PC to 1 GHz. Each stripe image isprocessed in 0.011 sec. This time processing is given because the data of theimage is extracted with few operations via Eq.(4) and Eq.(5). The capture velocityof the camera used in this camera is 34 fps. The electromechanical device ismoved also at 34 steps per second. The complete shape of the dummy face isprofiled in 4.18 sec, and the metallic piece is profiled in 3.22 sec. In thisprocedure, distances of the geometry of the setup to obtain the object shape arenot used. Therefore, the procedure is easier than those techniques that usedistances of the components of optical setup. In this manner, the technique isperformed by computational process and measurements on optical step areavoided. Therefore, a good repeatability achieved in experiment of a standarddeviation +/- 0.01 mm.

Conclusions

A technique for shape detection performed by means of line projection,binocular imaging and approximation networks has been presented. The describedtechnique here provides a valuable tool for inspection industrial and reverseengineering. The automatic technique avoids the physical measurements of thesetup, as is common in the methods of laser stripe projection. In this technique,the parameters of the setup are obtained automatically by computational processusing a Bezier network. It improves the accuracy of the results, because


measurement errors are not introduced to the system for the shape detection. Inthis technique, the ability to measure the stripe behavior with a sub-pixelresolution has been achieved by Bezier curves. It is achieved with few operations.By using this computational-optical setup a good repeatability is achieved inevery measurement. Therefore, this technique is performed in good manner.

References

[1] L. Zagorchev and A. Goshtasby, “A paintbrush laser range scanner”,Computer vision and image understating, Vol. 10, p. 65-86 (2006).

[2] Z. Wei, G. Zhang and Y. Xu, “Calibration approach for structured–light-stripe vision sensor based on invariance of double cross-ratio”, OpticalEngineering, Vol. 42 No. 10, p. 2956-2966 (2003).

[3] A. M. Mclvor, “Nonlinear calibration of a laser profiler”, OpticalEngineering, Vol. 42 No.1, p. 205-212, (2002).

[4] W. Ch. Tai and M. Chang, “Non contact profilometric measurement of largeform parts”, Optical Engineering, Vol. 35 No. 9, p. 2730-2735 (1996).

[5] M. Baba, T. Konishi and N. Kobayashi, “A novel fast rangefinder with non-mechanical operation”, Journal of Optics, Vol. 29 No. 3, p. 241-249 (1998).

[6] J. A. Muñoz Rodríguez and R. Rodríguez-Vera, “Evaluation of the light linedisplacement location for object shape detection”, Journal of ModernOptics, Vol. 50 No.1, p. 137-154 (2003).

[7] Q. Luo, J. Zhou, S. Yu and D. Xiao, “Stereo matching and occlusiondetection with integrity and illusion sensitivity”, Pattern recognition letters,Vol. 24, p. 1143-1149, (2003).

[8] H. Mitsudo, S. Nakamizo and H. Ono, “A long-distance stereoscopicdetector for partially occluding surfaces”, Vision Research, Vol. 46, p.1180-1186, (2006).

[9] H. J. Andersen, L. Reng and K. Kirk, “Geometric plant properties byrelaxed stereo vision using simulated annealing”, Computers andElectronics in Agriculture, Vol. 49, p. 219-232, (2005).

[10] R. Klette, K. Schluns and A. Koschan, Computer vision: Three-dimensionaldata from images, Springer, Singapore 1998.

[11] F. Causa and J. Sarma, “Realistic model for the output beam profile ofstripe and tapered superluminescent light–emitting diodes”, Applied Optics,Vol. 42 No.21, p. 4341-4348 (2003).

[12] Peter C. Gasson, Geometry of spatial forms, U.S.A. John Wiley and Sons,1989.


[13] H. Frederick and G. J. Lieberman, Introduction to operations research,McGraw- Hill, U.S.A. 1982.

[14] Robert J. Schalkoff, Artificial Neural Networks, Mc Graw Hill, U.S.A.1997.

[15] Y. J. Ahn, Y. S. Kim and Y. Shin, “Approximation of circular arcs and o setcurves by Bezier curves of high degree”, Journal of Computational andApplied Mathematics, Vol. 167, p. 405–416, (2004).

[16] G. D. Chen and G. J. Wang, “Optimal multi-degree reduction of Béziercurves with constraints of endpoints continuity”, Computer AidedGeometric Design Vol. 19, p. 365–377, (2002).

[17] W.H. Press, B.P.Flannery, S.A.Teukolsky, W.T.Vetterling, NumericalRecipes in C, Cambridge Press, U.S.A. 1993.

[18] T. Masters, Practical Neural Networks Recipes in C++, Academic Press,U.S.A 1993.

[19] J. E. Freund, Modern Elementary Statistics, Prentice Hall, U.S.A. (1979).


Chapter 4

EYE MOVEMENT ANALYSIS IN CONGENITALNYSTAGMUS: CONCISE PARAMETERS

ESTIMATION

Pasquariello Giulio1, Cesarelli Mario1, La Gatta Antonio2,Bifulco Paolo1 and Fratini Antonio1

1 Dept. of Biomedical, Electronic and Telecommunication Engineering,University “Federico II” of Naples, Via Claudio, 21, 80125, Napoli, Italy

2 Math4Tech Center, University of Ferrara,via Saragat, 1, 44100, Ferrara, Italy

Abstract

Along with other diseases that can affect binocular vision, reducing the visualquality of a subject, Congenital Nystagmus (CN) is of peculiar interest. CN is anocular-motor disorder characterized by involuntary, conjugated ocularoscillations and, while identified more than forty years ago, its pathogenesis isstill under investigation. This kind of nystagmus is termed congenital (orinfantile) since it could be present at birth or it can arise in the first months oflife. The majority of CN patients show a considerable decrease of their visualacuity: image fixation on the retina is disturbed by nystagmus continuousoscillations, mainly horizontal. However, the image of a given target can still bestable during short periods in which eye velocity slows down while the targetimage is placed onto the fovea (called foveation intervals). To quantify the extentof nystagmus, eye movement recordings are routinely employed, allowingphysicians to extract and analyze nystagmus main features such as waveform

Pasquariello Giulio, Cesarelli Mario, La Gatta Antonio et al.108

shape, amplitude and frequency. Use of eye movement recording, opportunelyprocessed, allows computing “estimated visual acuity” predictors, which areanalytical functions that estimate expected visual acuity using signal featuressuch as foveation time and foveation position variability. Hence, it isfundamental to develop robust and accurate methods to measure both thoseparameters in order to obtain reliable values from the predictors. In this chapterthe current methods to record eye movements in subjects with congenitalnystagmus will be discussed and the present techniques to accurately computefoveation time and eye position will be presented.

This study aims to disclose new methodologies in congenital nystagmus eyemovements analysis, in order to identify nystagmus cycles and to evaluatefoveation time, reducing the influence of repositioning saccades and data noiseon the critical parameters of the estimation functions. Use of those functionsextends the information acquired with typical visual acuity measurement (e.g.,Landolt C test) and could be a support for treatment planning or therapymonitoring.

Introduction

Congenital nystagmus (CN) is an ocular–motor disorder that appears at birthor during the first few months of life, characterized by involuntary, conjugated,bilateral to and fro ocular oscillations. Clinical descriptions of nystagmus areusually based on the direction of the fast phase and are termed horizontal, vertical,or rotary, or any combination of these. CN is predominantly horizontal, with sometorsional and, rarely, vertical motion [1]. Nystagmus oscillations can persist alsoclosing eyes, moreover they tend to damp in absence of visual activity. Invertebrates, eye movements are controlled by the oculomotor system in a complexmanner, depending on the stimuli and viewing conditions. In the human eye, thelittle portion of the retina which allows the maximal acuity of vision is called thefovea. An attempt to bring the image of a target onto the fovea can involve up tofive oculomotor subsystems: the saccadic, smooth pursuit, vestibular, optokineticand vergence systems. The vestibular system is driven by non-visual signals fromthe semicircular canals, while the other systems are mainly driven by visualsignals encoding target information. Pathogenesis of the congenital nystagmus isstill unknown; dysfunctions of at least one of the ocular stabilization systems havebeen hypothesized, but no clear evidence was reported.

Nystagmus can be idiopathic or associated to alteration of the central nervoussystem and/or ocular system such as achromathopsia, aniridia and congenitalcataract. Both nystagmus and associated ocular alterations can be geneticallytransmitted, with different modalities; estimates of the prevalence of infantilenystagmus range from 1 in 1000 to 1 in 6000 [23,26,34,38].

Eye Movement Analysis in Congenital Nystagmus 109

CN occurrence associated with total bilateral congenital cataract is of 50–75%, while this percentage decreases in case of partial or monolateral congenitalcataract. CN is present in most cases of albinism.

The cause or causes and pathophysiological mechanisms of CN have not beenclarified. Children with this condition frequently present with a head turn, whichis used to maintain the eyes in the position of gaze in which the nystagmus isminimum. This happens more often when the child is concentrating on a distantobject, as this form of nystagmus tends to worsen with attempted fixation. Thehead turn is an attempt to stabilize the image under these conditions.

CN may result from a primary defect (e.g., familial X-linked) of ocular motorcalibration. Some authors (e.g., Hertle, 2006) hypothesized that CN may alsoresult from abnormal cross-talk from a defective sensory system to the developingmotor system at any time during the motor system’s sensitive period; this canoccur from conception due to a primary defect (e.g., retinal dystrophy), duringembryogenesis due to a developmental abnormality (e.g., optic nerve hypoplasia),or after birth during infancy (e.g., congenital cataracts). This theory of the genesisof CN incorporates a pathophysiological role for the sensory system in its genesisand modification. Although the set of physiological circumstances may differ, thefinal common pathway is abnormal calibration of the ocular motor system duringits sensitive period.

Terminology (Definitions)

Efforts are being made to add precision and uniformity to nystagmusterminology. The terms congenital nystagmus (CN), infantile nystagmus andidiopathic motor nystagmus have become synonymous with the most commonform of neonatal nystagmus [4,17,30,31,42]. However, the term infantilenystagmus syndrome (INS) is a broader and more inclusive term that we prefernot to use since it refers to the broad range of neonatal nystagmus types, includingthose with identifiable causes. According to the bibliography, idiopathicnystagmus can be classified in different categories depending on thecharacteristics of the oscillations [2]; typically in CN eye movement recordingsare possible to identify, for each nystagmus cycle, the slow phase, taking thetarget away from the fovea, the fast (or slow) return phase. According to thenystagmus waveform characterization by Dell’Osso [20], in case the return phaseis slow then the nystagmus cycle is pendular or pseudo-cycloid; if the return phaseis fast then the waveform is defined as jerk (unidirectional or bidirectional). Ingeneral, CN waveform has an increasing velocity exponential slow phase [2].


A schematic illustration of a unidirectional jerk nystagmus waveform(pointing to the left) is presented in figure 1.

Figure 1. A schematic illustration of a jerk nystagmus waveform (bold line) with fast phasepointing to the left; on the picture are depicted various nystagmus features, such as: fastand slow phase components, nystagmus period and amplitude; the grey box on each cyclerepresents the foveation window. The baseline oscillation is shown as a dashed line, and itsamplitude is also shown.

In general, CN patients show a considerable decrease of the visual acuity,since image fixation on the fovea is reduced by nystagmus continuousoscillations. CN patient visual acuity reach a maximum when eyes are in theposition of least ocular instability, hence, in many cases, a compensatory headmalposition is commonly achieved, in order to bring the zone of best vision intothe straight-ahead position. Such so-called ‘null zones’ (or null positions)correspond to a particular gaze angle, in which a smaller nystagmus amplitudeand a longer foveation time can be obtained, thus reaching a better fixation of thevisual target onto the retina. Abnormal head posture could be alleviated bysurgery (mainly translating the null zone to straight-ahead position).

Other clinical characteristics, not always present, include increased intensitywith fixation and decreased intensity with sleep or inattention; variable intensityin different positions of gaze; decreased intensity (damping) with convergence;changing direction in different positions of gaze (about a so-called neutralposition); strabismus and an increased incidence of significant refractive errors.

In normal subjects, i.e., not affected by nystagmus, when the velocity of theimage projected on the retina increases by a few degrees per second, visual acuityand contrast sensitivity decrease. In CN patients, fixation is disrupted bynystagmus rhythmical oscillations, which result in rapid movements of the targetimage onto the retina [6]. Ocular stabilization is achieved during foveation periods


[2] in which eye velocity slows down (less than 4 degrees/s) while the visualtarget crosses the foveal region (± 0.5 degree); in this short time interval called‘foveation window’, it is said that the subject ‘foveates’.

Visual acuity was found to be mainly dependent on the duration of thefoveation periods [2,20,22], but the exact repeatability of eye position from cycleto cycle and the retinal image velocities also contribute to visual acuity [1, 35].

Numerous studies of CN in infants and children confirm an age-dependentevolution of waveforms during infancy from pendular to jerk [4,17,31,42]. Thisconcept is consistent with the theory that jerk waveforms reflect modification ofthe nystagmus by growth and development of the visual system [28,29].

Accurate, uniform, and repeatable classification and diagnosis of nystagmusin infancy as CN is best accomplished by a combination of clinical investigationsand motility analysis; in some cases, eye movement recording and analysis areindispensable for diagnosis. If a subject is diagnosed with CN, ocular motilitystudy can also be helpful in determining visual status. Analysis of binocular ormonocular differences in waveforms and foveation periods could be an importantinformation in therapy planning or can be used to measure outcome of (surgical)treatment. Presence of pure pendular or jerk waveforms without foveation periodsare indicators of poorer vision whereas waveforms of either type with extendedperiods of foveation are associated with good vision; moreover significantinterocular differences in a patient reflect similar differences in vision between thetwo eyes. Ocular motility analysis in CN subjects is also the most accuratemethod to determine nystagmus changes with gaze (null and neutral zones).

Clinical Assessment

The clinical examination of a subject affected by congenital nystagmus is acomplex task; eye movement recording is often one of the necessary steps, but ageneral physical examination and the assessment of vision are usually preliminaryperformed.

During the first examination, the physicians can assess the most importantfeatures of nystagmus, such as direction of eyes’ beating, presence of anomaloushead positions while viewing distant or near objects. In addition an adequatefundus examination is often carried out, in order to asses eventual prechiasmalvisual disorders.

Complete clinical evaluation of the ocular oscillation also includesidentification of fast-phase direction, movement intensity, conjugacy, gaze effects,


convergence effects, and effect of monocular cover. Changes in the character ofthe nystagmus with convergence or monocular viewing are often evaluated.

Visual acuity of the patient is habitually tested with both eyes open (binocularviewing) and with one eye covered (monocular). It is important not to forget thatbinocular acuity is the “person’s” acuity and monocular acuity is the “eye’s”acuity. These two are often very different in patients with nystagmus and both hasto be tested in CN subjects. Among the various available tests, the best choice toassess visual acuity in an older child and cooperative adult is the ETDRS chart,since it provides LogMar evaluation of all acuities, especially those between20/400 and 20/100 [29].

Examination Techniques: Motility

However, it is well documented that differentiating true nystagmus fromsaccadic oscillations and intrusions is sometimes impossible clinically. Recentadvances in eye movement recording technology have increased its application ininfants and children who have disturbances of the ocular motor system [1,4].

As stated above, nystagmus is caused by disorders of the mechanismsresponsible in holding gaze steady: the vestibular system, the gaze-holdingmechanism, the visual stabilization system, and the smooth pursuit system. Thus,evaluation of a patient’s nystagmus requires a systematic examination of eachfunctional class of eye movements. Measurement of the nystagmus waveform,using reliable methodology, is often helpful in securing a diagnosis. Suchmeasurements help differentiate acquired nystagmus from congenital forms ofnystagmus and from other saccadic disorders that lead to instability of gaze [36].

Ocular Motility Recordings

Qualitative or quantitative analysis of eye movements was attempted since theearly twentieth century, with primitive electronic technology available at thattime. Nowadays more complex and less invasive methods are available, from bio-potential recording up to high-speed photographic methods. Various techniquesare currently in use to record eye movements: electro-oculography (EOG),infrared oculography (IROG), magneto-oculography (MOG) also known asscleral search coil system (SSCS) and video-oculography (VOG). The firsttechnique relies on the fact that the eye has a standing electrical potential betweenthe front and the back. Horizontal EOG is measured by placing electrodes on thenasal and temporal boundaries of the eyelids; as the eye turns a proportional


change in electrodes potential is measured. The IROG approach relies onmeasuring the intensity of an infrared light reflected back from the subject’s eye.Infrared emitters and detectors are located in fixed positions around the eye. Theamount of light reflected back to a fixed detector varies with eye position. TheVOG approach relies on recording eye position using a video camera, often aninfrared device coupled with an infrared illuminator (in order to avoid disturbingthe subject), and applying image processing techniques. The scleral search coilmethod is based on electromagnetic interaction at radio frequencies between twocoils, one (embedded in a contact lens) fixed on the eye sclera and the otherexternal.

Bilateral temporal and nasal electrode placement is useful for gross separationof fast and slow phases but is limited by nonlinearity, drift, and noise. Infraredreflectance solves these problems and can be used in infants and children but it islimited by difficulty in calibration. IR video systems have become increasinglypopular in research laboratories and in the clinical setting, hence the comparisonbetween IR and the scleral search coil method has become an actual issue.

Different studies analyzed this subject reporting a good performance of videooculography compared with scleral search coils. Van der Geest and Frens [43]compared the performance of a 2D video-based eye tracker (Eyelink I; SRResearch Ltd., Mississauga, Ontario, Canada) with 2D scleral search coils. Theyfound a very good correspondence between the video and the coil output, with ahigh correlation of fixation positions (average discrepancy, +/-1° over a testedrange of 40 by 40° of visual angle) and linear fits near one (range, 0.994 to 1.096)for saccadic properties. However, Houben, Goumans, and van der Steen, [33]found that lower time resolution, possible instability of the head device of thevideo system, and inherent small instabilities of pupil tracking algorithms stillmake the coil system the best choice when measuring eye movement responseswith high precision or when high-frequency head motion is involved. For lessdemanding and for static tests and measurements longer than a half an hour, thelatest generation infrared video system is a good alternative to scleral search coils;in addition video oculography is not at all invasive, making it suitable for childrenyounger than 10 years old. However, the quality of torsion of the infrared videosystem is less compared with scleral search coils and needs further technologicalimprovement..

CN eye movement recordings are often carried out only on the horizontal axisand display the data, by convention, during continuous periods of time. Positionand velocity traces are clearly marked, with up being rightward eye movementsand down being leftward eye movements. Figure 2 reports, as example, a signaltracts recorded from actual CN patients; computed eye velocity is shown


underneath the eye movement signal. It is possible to identify some nystagmuscharacteristics, such as nystagmus amplitude, frequency and the fast and slowphases.

Figure 2. An example of eye movement recording, jerk left The eye velocity is alsodepicted with a 0 °/s threshold; the figure also shows (between 26.5 s and 27.6 s) a saccadeof about ten degrees corresponding to a gaze angle voluntary shift.

Semiautomatic Analysis of Eye Movement Recordings

Congenital nystagmus is a rhythmic phenomenon and researchers have triedto analyze eye movements signals using methodologies specific for frequencyanalysis such as spectral and wavelet analysis (Reccia et al., 1989-1990, Clementet al., 2002; Miura et al., 2003). Fewer authors (e.g. Hosokawa, 2004) applied theShort Time Fourier Transform (STFT) to congenital nystagmus recordings, inorder to highlight modifications in the principal component and in the harmonicsduring time. However resolution of this technique is limited by the duration of thewindows in which the signal is divided [3]. Wavelet analysis seems more usefulsince it is able to assess how much the signal in study differs in time from aspecific template adopted as a reference and it is able to localize a brief transientintrusion into a periodic waveform [37]. It has been used with success to separate


fast and slow phases in caloric nystagmus. However, as stated by Abel [3], theoutcome of this analysis is a time-frequency plot or a coefficient sequence, whichare difficult to relate to a subject visual ability. Moreover, the foveation timemodification in each cycle and the variability of position between successivefoveations can hardly be highlighted using STFT and wavelet analysis [3].

On the contrary, time domain analysis techniques, such as velocity thresholds,region-based foveation identification, syntactic recognition or time series analysis,have been routinely employed in the last decades to analyse nystagmus, eithercongenital or vestibular.

Usually, visual acuity increases in people suffering from congenitalnystagmus if the foveation time increases and the signal variability decreases. Ananalysis of the signal characteristics near the desired position (target position) caneasily take place in the time domain, as demonstrated by Dell’Osso et al. whodefined an analytic function to predict visual acuity (NAFX) [15] and by Cesarelliet al. who defined a similar function (NAEF) [10].

Figure 3. An example of slow phase and fast phase (bold) separation in CN eye movementrecordings.


Figure 4. The local foveation windows identified for each nystagmus cycle.

Time domain analysis of congenital nystagmus is the most used techniqueand, in our opinion, best option so far and it is able to estimate the visual ability ofa subject at different gaze angles; however its application with semi-automaticmethods still needs improvement both in performance and reliability.

The first step of each algorithm for the time analysis of rhythmic eyemovements is the cycles identification: in congenital nystagmus, the mostcommon waveforms are jerk, jerk with extended foveation followed by pendularand pseudo-cycloid [1]; however only the first waveform allows foveation timewhich ensure a good visual ability. The CN jerk waveforms can be described as acombination of two different actions: the slow phase taking the eye away from thedesired target, followed by the fast, corrective phase; the foveation takes placewhen eye velocity is small, which happens after the fast phase. Hence, a localfoveation window can be defined, located at the end of each fast phase and at thebeginning of the slow phase, which allows to separate the effects of changes infoveation time and alteration in eye position on visual acuity [10]. The analysis ohthese two separate effects is of strong importance due to the presence of a slow‘periodic’ component in the eye movement signal, which we called baselineoscillation (BLO) [7,39].


Slow Eye Movements in Congenital Nystagmus

The role of the standard deviation of eye position (SDp) during foveationswith respect to visual acuity has been discussed in the past ten years [10,16].Fostered also by a remarkable increase in some CN patients’ visual acuity,obtained with botulinum toxin treatment, which didn’t correspond to largeextensions n foveation time, pointed us to characterize in more details suchfoveation variability. A slow sinusoidal-like oscillation of the baseline (baselineoscillation or BLO) was found superimposed to nystagmus waveforms [9,11,12]and its relation with the SDp was estimated [7].

Presence of similar slow pendular waveforms, superimposed to nystagmus,was also reported by Gottlob et al. [27]. In addition, in eye movementrecordings presented by Dell’Osso et al. [15,16] it is possible to recognize slowoscillations superimposed to the nystagmus. Akman et al. [5], using dynamicalsystems analysis to quantify the dynamics of the nystagmus in the region offoveation, found that the state-space fixed point, or steady state, is not unique.Physiologically this means that the control system does not appear to maintain aunique gaze position at the end of each fast phase. Similarly, Evans [24]reported that some of the analyzed patients fail to coordinate target with foveaposition (approximately 50% of patients). Kommerell [35] noticed that in CNpatients, tracking moving targets, the eye recording presented a slow eyemovement superimposed to the stimulus trajectory in addition to nystagmiccycles.

Nystagmus and the slow oscillation could modify visual acuity. Currie et al.[14] evaluated acuity for optotypes in healthy subjects using moving lightsources to simulate retinal image motion that occurs in nystagmus. Their resultsare that acuity depends on both foveation duration and position variability,although the presence of other sensory defects (e.g. astigmatism) must be takeninto account. Moreover, they found that an addition of low-frequency (1.22 Hz)waves to the light stimuli, i.e. slow oscillation, caused a worsening of visualacuity.

In order to estimate the slow sinusoidal oscillations, a common least meansquare (LMS) fitting technique could be used. For each signal block the highestpeak of the power spectrum of the eye movement signal in the range 0.1–1.5 Hzcan be considered as an estimator of the BLO frequency. The high frequency limitresult from the lowest frequency commonly associated to nystagmus (accordinglyto Bedell and Loshin, 1991, and Abadi and Dickinson, 1986), while the lowfrequency limit depends on the signal length corresponding to each gaze position(in our tests approximately 10 s).


Figure 5a and 5b. Examples of acquired signals showing the presence of the slow eyemovement added up to the nystagmus oscillations.


Conclusion

Eye movement recording methodology is most commonly used as a researchtool by neurologists, neurophysiologists, psychophysicists, psychologists/psychiatrists, ophthalmologists, and optometrists [18,21,25]. Eye movementrecording and estimation of concise parameters, such as waveform shape,nystagmus amplitude, frequency, direction of beating, foveation periods and eyeposition variability, are a strong support for an accurate diagnosis, for patientfollow-up and for therapy evaluation [8].

Regarding the last parameter, the slow eye movement, described as baselineoscillation, explains most of the eye position variability during foveations (SDp)[7], which in turn was found exponentially well related to visual acuity [10].According to the procedure described above, baseline oscillation parameters canbe estimated for any CN eye movement recordings. In a case study byPasquariello et al. carried out on 96 recordings, almost 70% of the recordings hadBLO amplitude greater than 1° (appreciatively the fovea angular size); in theremaining 30% the amplitude of the BLO was smaller and didn’t affectsignificantly visual acuity.

In that study a high correlation coefficient (R2 = 0.78) was also found in thelinear regression analysis of BLO and nystagmus amplitude, suggesting a stronglevel of interdependence between the two. The regression line slope coefficientwas about 0.5, which implies that BLO amplitude on average is one half of thecorrespondent nystagmus amplitude.

Specifically, since BLO amplitude resulted directly related to nystagmusamplitude, its presence is particularly evident in the signal tracts away from thenull zone (i.e., not in the position in which nystagmus amplitude is lesser).

The origin of such baseline oscillation is unknown. Some authors assert thatslow movement can be recorded only in subjects with severely reduced visualexperience from birth (like CN patients) [27]. However, the high value of thecorrelation coefficient between BLO and nystagmus amplitude found in this studysuggests that the two phenomena are somewhat linked together. Therefore theorigin of the BLO could be searched analyzing within the same ocular motorsubsystems considered for nystagmus.

The baseline oscillation highlights the presence of a slow ‘periodic’component in the eye movement signal. The sine function is a rather goodestimator of this slow periodic component added to nystagmus; the basic shape ofthe baseline is indeed a sinusoid, sometimes and randomly disrupted by phaseinversions, interruptions (as short as hundreds of milliseconds, lasting to even 1second) and other non linear components. To the periodic component represented


by BLO the small, additional random movements should be added, in order toassess the whole variability of eye position during fixation [7].

Figure 6. The relationship between Baseline Oscillation and Nystagmus amplitude.

References

[1] Abadi RV; Bjerre A. Motor and sensory characteristics of infantilenystagmus. Br. J. Ophthalmol. 2002, 86, 1152-1160

[2] Abadi RV; Dickinson CM. Waveform Characteristics in CongenitalNystagmus. Doc. Ophthalmol. 1986, 64, 153-67.

[3] Abel LA; Wang ZI; Dell'Osso LF. Wavelet analysis in infantile nystagmussyndrome: limitations and abilities. Invest. Ophthalmol. Vis. Sci. 2008 Aug,49(8), 3413-23

[4] Abel LA. Ocular oscillations. Congenital and acquired. Bull. Soc. BelgeOphthalmol. 1989, 237, 163–189.


[5] Akman OE; Broomhead DS; Clement RA, Abadi RV. Nonlinear time seriesanalysis of jerk congenital nystagmus. J. Comput. Neurosci. 2006, 21(2),153-70.

[6] Bedell HE; Loshin DS. Interrelations between Measures of Visual Acuityand Parameters of Eye Movement in Congenital Nystagmus. Invest.Ophthalmol. Vis. Sci. 1991, 32, 416-21.

[7] Bifulco P; Cesarelli M; Loffredo L; Sansone M; Bracale M. Eye MovementBaseline Oscillation and Variability of Eye Position During Foveation inCongenital Nystagmus. Doc. Ophthalmol. 2003, 107, 131-136.

[8] Cesarelli M., et al. Analysis of foveation duration and repeatability atdifferent gaze positions in patients affected by congenital nystagmus.IFMBE Proceedings. 2007, 16 (12), 426-429.

[9] Cesarelli M; Bifulco P; Loffredo L; Magli A; Sansone M; Bracale M. EyeMovement Baseline Oscillation in Congenital Nystagmus. Proceedings ofthe WC2003.

[10] Cesarelli M; Bifulco P; Loffredo L; Bracale M. Relationship between visualacuity and eye position variability during foveation in congenitalnystagmus. Doc. Ophthalmol. 2000, 101, 59-72.

[11] Cesarelli M; Bifulco P; Loffredo L. EOG Baseline Oscillation in CongenitalNystagmus. VIII Mediterranean Conference on Medical BiologicalEngineering and Computing - MEDICON '98, Lemesos - Cyprus, June 14-17, 1998 - CD-ROM 19.3

[12] Cesarelli M; Loffredo L; Bifulco P. Relationship between Visual Acuity andOculogram Baseline Oscillations In Congenital Nystagmus. Proceedings ofthe 4th European Conference on Engineering and Medicine, Warsaw 1997,301-2.

[13] Clement RA; Whittle JP; Muldoon MR; Abadi RV; Broomhead DS; AkmanO. Characterisation of congenital nystagmus waveforms in terms of periodicorbits. Vision Research, 2002, 42, 2123–2130

[14] Currie DC; Bedell HE; Song S. Visual Acuity for Optotypes with ImageMotions Simulating Congenital Nystagmus. Clin. Vision Sci. 1993, 8, 73-84.

[15] Dell’Osso LF; Jacobs JB. An Expanded Nystagmus Acuity Function: Intra-and Intersubject Prediction of Best-Corrected Visual Acuity. Doc.Ophthalmol. 2002, 104, 249-276.

[16] Dell'Osso LF; Van Der Steen J; Steinman RM; Collewijn H. FoveationDynamics in Congenital Nystagmus. I: Fixation. Doc. Ophthalmol. 1992,79, 1-23.


[17] Dell’Osso LF. Congenital, latent and manifest latent nystagmus-similarities,differences and relation to strabismus. Jpn. J. Ophthalmol. 1985, 29(4),351-368.

[18] Dell’Osso LF; Flynn JT. Congenital Nystagmus surgery. A quantitativeevaluation of the effects. Arch. Ophthalmol. 1979, 97(3), 462–469.

[19] Dell’Osso LF; Schmidt D; Daroff RB. Latent, manifest latent, andCongenital Nystagmus. Arch. Ophthalmol. 1979, 97(10), 1877–1885.

[20] Dell'Osso LF; Darof RB. Congenital Nystagmus waveform and foveationstrategy. Doc. Ophthalmol. 1975, 39, 155-82.

[21] Dell’Osso LF; Gauthier G; Liberman G; Stark L. Eye movement recordingsas a diagnostic tool in a case of Congenital Nystagmus. Am. J. Optom. Arch.Am. Acad. Optom. 1972, 49(1), 3-13.

[22] Dickinson CM; Abadi RV. The Influence of the nystagmoid oscillation oncontrast sensitivity in normal observers. Vision Res. 1985, 25, 1089-96.

[23] Duke-Elder S. Systems of ophthalmology. Vol III, Part 2. London: HenryKimpton, 1973.

[24] Evans N. The significance of the nystagmus. Eye. 1989; 3, 816-832.[25] Flynn JT; Dell’Osso LF. The effects of Congenital Nystagmus surgery.

Ophthalmology. 1979, 86(8), 1414–1427.[26] Forssman B; Ringer B. Prevalence and inheritance of congenital nystagmus

in a Swedish population. Ann. Hum. Genet. 1971, 35, 139-47.[27] Gottlob I; Wizov SS; Reinecke RD. Head and eye movements in children

with low vision. Graefes Arch. Clin. Exp. Ophthalmol. 1996, 234, 369-77.[28] Harris C; Berry D. A developmental model of infantile nystagmus. Semin.

Ophthalmol. 2006 Apr-Jun, 21(2), 63-9.[29] Hertle, RW, Nystagmus and Ocular Oscillations in Childhood and Infancy.

In: Wright KW, Spiegel PH, Thompson LH editors. Handbook of pediatricneuro-ophthalmology. New York: Springer; 2006; 289-323

[30] Hertle RW; Zhu X. Oculographic and clinical characterization of thirty-seven children with anomalous head postures, nystagmus, and strabismus:the basis of a clinical algorithm. J. Am. Assoc. Pediatr. Ophthalmol.Strabismus. 2000, 4(1), 25–32.

[31] Hertle RW; Dell’Osso LF. Clinical and ocular motor analysis of congenitalnystagmus in infancy (see also comments). J. Am. Assoc. Pediatr.Ophthalmol. Strabismus. 1999, 3(2), 70–9.

[32] Hosokawa M; Hasebe S; Ohtsuki H; Tsuchida Y. Time-Frequency Analysisof Electro-nystagmogram Signals in Patients with Congenital Nystagmus.Jpn. J. Ophthalmol. 2004, 48, 262-7


[33] Houben, MMJ; Goumans J and van der Steen J Recording Three-Dimensional Eye Movements: Scleral Search Coils versus VideoOculography. Invest. Ophthalmol. Vis. Sci. 2006, 47(1):179-187

[34] Hu DN. Prevalence and mode of inheritance of major genetic eye disease inChina. J. Med. Genet. 1987, 24, 584–8.

[35] Kommerell G. Congenital nystagmus: control of slow tracking movementsby target offset from the fovea. Graefes Arch. Clin. Exp. Ophthalmol. 1986,224(3), 295-8.

[36] Leigh RJ. Clinical features and pathogenesis of acquired forms ofnystagmus. Baillieres Clin. Neurol. 1992, 1(2), 393–416.

[37] Miura K, Hertle RW, FitzGibbon EJ, Optican LM. Effects of tenotomysurgery on congenital nystagmus waveforms in adult patients. Part I.Wavelet spectral analysis. Vision Research. 2003, 43, 2345–2356

[38] Norn MS. Congenital idiopathic nystagmus. Incidence and occupationalprognosis. Acta Ophthalmol. 1964, 42, 889–96.

[39] Pasquariello G; Cesarelli M; Bifulco P; Fratini A; La Gatta A, Romano M.Characterisation of baseline oscillation in congenital nystagmus eyemovement recordings. Biomedical Signal Processing and Control, 2009Apr, 4, 102–107.

[40] Reccia R; Roberti G; Russo P. Computer analysis of ENG spectral featuresfrom patients with congenital nystagmus. Journal of BiomedicalEngineering. 1990, 12, 39–45.

[41] Reccia R; Roberti G; Russo P. Spectral analysis of pendular waveforms incongenital nystagmus. Ophthalmic Research. 1989, 21, 83–92.

[42] Reinecke RD. Idiopathic infantile nystagmus: diagnosis and treatment. J.Am. Assoc. Pediatr. Ophthalmol. Strabismus. June 1997; 1(2), 67–82.

[43] van der Geest JN, Frens MA, Recording eye movements with video-oculography and scleral search coils: a direct comparison of two methods. J.Neurosci. Methods. 2002, 114, 185–195.


Chapter 5

EVOLUTION OF COMPUTER VISION SYSTEMS

Vladimir Grishin*

Space Research Institute (IKI) of the Russian Academy of Sciences117997, 84/32 Profsoyuznaya Str, Moscow, Russia

Abstract

Applications of computer vision systems (CVS) in the flight control of unmannedaerial vehicles (UAV) are considered. In many projects, CVS are used forprecision navigation, angular and linear UAV motion measurement, landing (inparticular shipboard landing), homing guidance and others. All these tasks havebeen successfully solved separately in various projects. The development ofperspective CVS can be divided into two stages. The first stage of perspectiveCVS development is the realization of all the above tasks in a single full-scaleuniversal CVS with acceptable size, weight and power consumption. Therefore,all UAV flight control tasks can be performed in automatic mode on the base ofinformation that is delivered by CVS. All necessary technologies exist and thedegree of its maturity is high. The second stage of CVS development isintegration of CVS and control systems with artificial intelligence (AI). Thisintegration will bring two great benefits. Firstly it will allow considerableimprovement of CVS performance and reliability due to accumulation ofadditional information about the environment. Secondly, the AI control systemwill obtain a high degree of awareness about the state of the environment. Thisallows the realization of a high degree of control effectiveness of the autonomousAI system in a fast changing and hostile environment.

* E-mail address: [email protected]

Vladimir Grishin126

Introduction

The computer vision systems (CVS) revealed a great evolution during the lastdecades. This chapter attempts to estimate the nearest perspective for itsdevelopment. Further analysis will be dedicated to usage of CVS in mobile robotcontrol systems, mainly in unmanned aerial vehicles (UAV). This problem is oneof the most challenging tasks of control theory and practice. However, the mainprinciples of this analysis are applicable to most of the different kinds of mobilerobots.

Present-Day Level of CVS Development

A large number of publications is devoted to the application of CVS todifferent tasks of UAV flight control. The enumeration of these publications maytake many pages; so here we refer to a few arbitrarily chosen papers. Let’s list thekey tasks of such CVS.

• High precision navigation [1–6]. This task can be solved by means ofrecognition (matching) beforehand specified objects (landmarks) whosecoordinates are known [1]. Another demand which is imposed on theselandmarks is reliability of detection and recognition process. Reliabilityhad to be guaranteed in conditions of possible imitation and masking.Since these landmarks are selected in advance and their reference patternscan be carefully prepared, the process of recognition (matching) can bereliably performed. Reference patterns are prepared with the account ofdifferent distances, perspective aspect angles of observation andobservation conditions. Reliability of landmark recognition can besubsequently increased by joint usage of landmark images and their 3Dprofiles. 3D profiles are widely used for navigation of missiles ofdifferent kinds (Tomahawk cruise missiles and others). The technologiesfor 3D profile reconstruction are well known. For instance, the complexof the Israeli firm VisionMap [2] can be referred. The complex allowsreconstructing of a 3D profile with precision about 0.2–0.3 m from thealtitude of 3250 m. This complex is heavy enough and has considerablesize. Some weakening of the precision requirement will allow significantdecrease in weight and size. Further increasing of navigation reliabilitycan be achieved by selection of a redundant number of landmarks in thewhole area of observation. Information from CVS is usually integrated

Evolution of Computer Vision Systems 127

with information from inertial navigation system. Such integration allowsserious decreasing of accumulated errors of the inertial navigation system[3]. The 3D profile of observed surface can be calculated simultaneously[4, 5]. In the presence of the on-board high precision inertial navigationsystem, it is possible to make a 3D reconstruction in the monocular mode(with longitudinal stereo base) [6]. In this case it is possible to make a 3Dreconstruction of very distant objects and surfaces.

• Flight stabilization and control of angular orientation [7–12]. This task issolved by the measurement of angular and linear UAV motion relative tothe observed surface or objects [7, 8]. The set of features (points in framewith good localization) are selected and traced in subsequent frames.Observed shifts of a selected set of points are used for calculation ofrelative angular and linear motion. In this aspect, the star trackers shouldbe mentioned. These devices are the specialized CVS which are used inautomatic spacecrafts for measurement of angular position with highprecision and reliability [9, 10]. For flight control, it is important toestimate the UAV orientations with regard to local vertical (pitch and rollangles). These estimations are used for attitude stabilization and control.CVS sensors of local vertical use algorithms of horizon line detection,recognition and tracing [11–12]. Joint usage of CVS and inertialnavigation systems allows significant improvement of precision andreliability of angular orientation [7, 8]. The pose estimation and 3Dreconstruction are frequently realized in single algorithm, and thisprocess is called SLAM (simultaneous localization and mapping) [4, 5,7].

• Near-obstacle flight [13–17]. It is a very complicated flight control task.The control system had to guarantee high precision control with a shorttime delay. A good example of such control is the flight control of anairplane or helicopter between buildings in an urban environment inaltitude about 10–15 m [15]. Another example is the ground huggingflight or on-the-deck flight. The CVS is able to provide all necessaryinformation for solving such control tasks. In particular, the optical flowcalculation allows us to evaluate the risk of collision and to correct thedirection of flight to avoid collision (obstacle avoidance). The distanceand 3D profile of observed objects can be calculated by stereo pairs. Theoptical flow is used for estimation of flight altitude, too.

• Landing [18–26]. Landing is the most dangerous stage of flight. Duringthis stage, UAV can crash. Moreover, some persons can be injured orproperty can be damaged. CVS can provide all necessary information for

Vladimir Grishin128

the control system which should realize this maneuver. This task includesthe recognition of landing stripe or landing site and flight motion controlrelative to the landing stripe. CVS is used also for landing site selectionin conditions of forced landing; see for example [19, 21 and 25]. Inparticular, 3D reconstruction of observed surface allows selection of themost proper landing area. In other words, CVS supports landing onunprepared (non-cooperative) sites. CVS are used for autonomouslanding on the surface of Solar system planets and small bodies [22, 24].The most complicated task is the shipboard landing [26]. There are twomain difficulties. The first – the landing stripe is small. The second – theship deck is moving.

• Detection and tracking of selected moving targets [27–30]. A target canbe a pedestrian, car, ship or other moving object. The selected target canmake evolutions, attempts to hide from the observation and so on. In sucha complicated condition, CVS should guarantee reliable tracing ofspecified target. In the case of automatic tracking collapse, the CVSshould use effective search algorithms for target tracking restoration.

• Homing guidance to selected objects [31]. The homing task is similar tothe task of navigation. It includes search, detection, recognition andtracking on the aim object. Significant problem is the multiple changingof distance during the homing process. During this process the observedsize of the tracking object changes very significantly. In such case thecorrelation matching technologies are used. These technologies are usedin smart weapons (precision-attack bombs). Another example is theTomahawk cruise missile which is equipped with so-called DSMAC(Digital Scene Matching Area Corellator) system. The other scenematching terminal guidance systems exist.

All these tasks have been successfully solved separately in different projects.The larger part of these projects belongs to the on-board real-time systems. Theremaining part will be realized in the form of on-board real-time systems in nearfuture.

During the last decades, great attention had been paid to the patternrecognition problem. From the flight control CVS view, the pattern recognitionproblem can be divided into two problems.

• Recognition of preliminary specified objects. This problem in fact isbeing solved in the high precision navigation task, tracking of selectedobjects and homing guidance.


• Recognition as a classification of observing objects [32-36]. Thisproblem is being successfully solved during the development ofautomatic systems intended for the processing of visual information –space images of high resolution. The huge volume of such informationmakes the manual processing impossible. Such tasks should be realizedin work stations as they require a great deal of calculation resources. Butsubsequent development of such systems and some simplification of taskwill allow realizing it in the form of on-board real-time systems. Thepossibility to recognize wide set of objects creates the necessaryprerequisites for development artificial intelligence (AI) control systems.

Full-Scale Universal CVS

The development of perspective CVS can be divided into two stages. The firststage of perspective CVS development is realization of all listed tasks in singlefull-scale system. In other words the task is to combine (to join) thesetechnologies into the whole interrelated system. Mention should be made that allthese technologies have many common methods and algorithms. All necessarytechnologies exist and the degree of its maturity is high. Many algorithms havebeen suggested for solution any of the listed tasks. One of the main aims is theselection of the most effective and reliable algorithms. Then it is necessary todevelop appropriate hardware and software. The serious attention should bedrawn to the development of CVS architecture [40] and special processors formost calculation consuming algorithms. It is highly probable that developmentand production of specialized chipsets for image processing in CVS will berequired. Some attempts to move in that direction of the multifunctional CVSelaboration are currently appearing [37-42]. The cooperation of many groups ofresearches, engineers and industry for realization this complicated CVS inacceptable size, weight and power consumption will be needed. These parametersshould be acceptable for wide area of practical applications (robots of differentkind). Another area of application is the safety systems which should be designedfor preventing pilot errors in flight control or driver errors in car control. Theaccessibility of this task by means of present-day technologies is undoubted.Realization of the first stage will allow realizing complete automatic vision-basedcontrol of UAV and other mobile robots. Small size and cheap CVS will havevery wide area of application which will be comparable with the area of GPSnavigation receivers application.

Vladimir Grishin130

Integration of CVS and AI Control System

One can say with confidence that the second stage of CVS development is theintegration of CVS with AI control systems which realize the functions of taskanalysis, estimation of current situation, operation planning, realization, andcorrection of plans in real time. On the one hand such integration will allows theessential improvement of CVS performance and reliability due to accumulation ofadditional information about objects of environment, the possibility to undertakespecial action for investigation of the environment and the aggregation with thehuge volume of non-visual information of AI control systems. On the other handthe AI control system will obtains high degree of awareness about the currentstate of environment. This allows to realize high degree of control effectiveness inuncertain, fast changing and hostile environment. The possibility of gaining andaccumulation of AI system individual experience allows improving CVSreliability and effectiveness autonomously during the whole time of CVS and AIsystem operation. It stands to reason that CVS and AI control system have to betightly integrated with the conventional flight control system.

It should be emphasized once more that CVS is the base for AI controlsystems development due to their high informativity and high awareness. Fromthe other side, the requirements of AI system in solution of a task will stimulatethe build-up of CVS functions and opportunities. On the certain stage of AIsystem development these processes can occur autonomously. In other words, thesynergetic effect should takes place. In that way the close interaction between AIcontrol system and CVS should be established.

There are other high informative sensors, such as radar sets and laserscanners. The modern radar set with electronically scanned array can provide thehuge information flow. Laser scanner is capable to provide millionsmeasurements per second. But its size, weight and power consumption are muchlarger then similar parameters of TV camera. The TV camera cost is much smallerthen cost of a radar set or laser scanner. Moreover, a radar set and laser scannerproduce electromagnetic wave emission what in some circumstances is highlyundesirable. Thus the application area of CVS will be much wider then theapplication area of radar sets and laser scanners.

Some activities in the direction of integration some elements of AI and CVSare described in [43-49]. However, the realization of full-scale AI control systemis still too complicated task. The effective methods for developing such combinedCVS-AI systems are debatable and rather complicated.

The second stage is characterized by the considerable degree of uncertainty.But during the last 20-25 years the quite acceptable approaches to the


development of AI control systems and its integration with CVS have beenformulated. The most popular introduction to principles of artificial and naturalintelligence structure and function is presented in [50]. This book is dedicatedmainly to a brain functioning. A more complicated and advanced conception ispresented in [51]. This book is dedicated to development of autonomous AIsystems and their function. These AI conceptions are suitable for development ofcontrol system for mobile objects which are equipped with CVS. Mention shouldbe made that these authors traditionally paid great attention to the artificial andnatural neuron networks functioning. It seems that more effective specialprocessors architectures should be used in developing of AI processors. Blindimitation of biological structures on the completely different technological base isineffective. For instance, wheels allow such speed of motion, which is absolutelyinaccessible for legs or its mechanical imitations. The present-day aircraft wingspermit flying with the speed, which is absolutely inaccessible for birds withflapping wings or their mechanical imitations.

One can state that precedent thinking is the basis of natural and artificialintelligence. The development of the effective methods of permanent precedentsaccumulation and processing is required. These methods should includeprecedents data bases development, associative search, information andknowledge extraction methods from precedents data bases. Very significantprecedent based methods of the situation evolution forecast should be developed.These methods should function in real time. Principles of precedent accumulationand processing are used by all living creature which have nervous system of anykind. Memory capacity and effectiveness of precedent processing determines thestage of the creature evolution. We describe here over-simplified constructiveconception which is suitable for embedded autonomous AI system design.

CVS can be considered as a basis for gradual wide practical implementationof AI. It will require the wide integration of different commands (groups) ofresearches, engineers and industry. The second stage will require much more time,money and other resources then the first stage. But even during the developmentprocess it is possible to obtain practically useful and commercially applicableresults. For instance vision system of such species as frog or crocodile are veryprimitive as well as their intelligence. However starting from survival task,admissible size, weight and power consumption such vision and intelligencesystems are quite effective and useful. This fact is confirmed by wide spreadingand quantity of frogs and crocodiles species. So even not very advanced andperfect CVS and AI systems can find their useful applications and becommercially successful.

Vladimir Grishin132

Returning back to the UAV control, we see that the overwhelming majority ofUAV operate under the remote control or preliminary prepared mission planwhich is based on waypoints navigation. Possibilities of remote control are verylimited. Remote control requires high reliability of information exchange with theUAV. Such information exchange is possible in limited range from the controlstation. Another drawback of remote control is the vulnerability tocountermeasures. Remote control requires a highly skilled operator. Neverthelessthe crash ratio of remote-controlled UAV is relatively high. In the case of acomplicated environment, the preparation of a flight plan for automatic UAV is acomplicated and time consuming task. The initial information which is used forflight plan preparation can grow old rather fast. Any unmapped obstacle can causethe UAV to crash . Flight in a physically cluttered environment (such as streets ofa city) on a preliminarily prepared mission plan is impossible. Mention should bemade about the high degree of vulnerability of such popular GPS navigation tocountermeasures.

Conclusion

One of the most difficult and attractive aims is the development of fullyautonomous unmanned aerial vehicles and other robots. These vehicles should becapable effectively to solve the required tasks which should be carried-out incomplicated unpredictable varying and hostile environments without remotecontrol in any form. High awareness of computer vision systems is the necessarycondition for the development of such advanced control systems.

References

[1] Xie, S.-R.; Luo, J.; Rao, J.-J.; Gong, Z.-B. Computer Vision-basedNavigation and Predefined Track Following Control of a Small RoboticAirship. Acta Automatica Sinica. 2007, vol. 33, issue 3, pp. 286-291.

[2] Pechatnikov, M.; Shor, E.; Raizman, Y. Visionmap A3 - Super Wide AngleMapping System Basic Principles and Workflow. The InternationalArchives of the Photogrammetry, Remote Sensing and Spatial InformationSciences. 2008, vol. XXXVII, part B4.

[3] Veth, M. J.; Raquet, J. F. Two-Dimensional Stochastic Projections for TightIntegration of Optical and Inertial Sensors for Navigation. Proceedings of


the 2006 National Technical Meeting of the Institute of Navigation. 2007,pp. 587–596.

[4] Karlsson, R.; Schon, T. B.; Tornqvist, D.; Conte, G.; Gustafsson F. UtilizingModel Structure for Efficient Simultaneous Localization and Mapping for aUAV Application. Proceedings of 2008 IEEE Aerospace Conference. 2008,pp. 1-10.

[5] Hygounenc, E.; Jung, I.-K.; Soueres, P.; Lacroix, S. The AutonomousBlimp Project of LAAS-CNRS: Achievements in Flight Control and TerrainMapping. The International Journal of Robotics Research. 2004, vol. 23,issue 4-5, pp. 473-511.

[6] Borodovskiy, V. N.; Grishin, V. A. Estimation of Limitations in Precisionfor 3-D Surface Reconstruction. Transactions on IEEE InternationalConference on Systems, Man and Cybernetics. 1998, vol. 5, pp. 4387-4391.

[7] Baker, C.; Debrunner, C.; Gooding, S.; Hoff W.; Severson, W. AutonomousVehicle Video Aided Navigation – Coupling INS and Video Approaches.Advances in visual computing: Proceedings of the Second InternationalSymposium on Visual Computing (ISVC 2006). 2006, vol. 4292, issue 2, pp.534-543.

[8] Roumeliotis, S. I.; Johnson, A. E.; Montgomery, J. F. Augmenting InertialNavigation with Image-Based Motion Estimation. Proceedings of the IEEEInternational Conference on Robotics and Automation (ICRA ‘02). 2002,vol. 4, pp. 4326–4333.

[9] Avanesov, G. A.; Forsh, A. A.; Bessonov, R. V.; Ziman, Ya. L.; Kudelin,M. I. BOKZ-M star tracker and its evolution. Proceedings of the 14th SaintPetersburg International Conference on Integrated Navigation Systems.2007, pp. 219-224.

[10] SED 26 Star Tracker. (2006). http://www.sodern.fr/site/docs_wsw/fichiers_sodern/SPACE%20EQUIPMENT/FICHES%20DOCUMENTS/SED26.pdf

[11] Dusha, D.; Boles, W.; Walker, R. Fixed-Wing Attitude Estimation UsingComputer Vision Based Horizon Detection. Proceedings of the 12thAustralian International Aerospace Congress. 2007, pp. 1-19.

[12] Ettinger, S. M.; Nechyba, M. C.; Ifju, P. G.; Waszak, M. Vision-GuidedFlight Stability and Control for Micro Air Vehicles. Advanced Robotics.2003, vol. 17, issue 7, pp. 617-640.

[13] Oh, P. Y.; Green, W. E.; Barrows, G. Neural Nets and Optic Flow forAutonomous Micro-Air-Vehicle Navigation. Proceedings of the ASMEInternational Mechanical Engineering Congress and Exposition (IMECE04). 2004, vol. 2, pp. 1-7.

Vladimir Grishin134

[14] Kellogg, J.; Bovais, C.; Dahlburg, J.; Foch, R.; Gardner, J.; Gordon, D.;Hartley, R.; Kamgar-Parsi, B.; McFarlane, H.; Pipitone, F.; Ramamurti, R.;Sciambi, A.; Spears, W.; Srull, D.; Sullivan, C. The NRL MITE AirVehicle. Proceedings of the Bristol RPV/AUV Systems Conference. 2001.

[15] Muratet, L.; Doncieux, S.; Briere, Y.; Meyer, J.-A. A Contribution toVision-Based Autonomous Helicopter Flight in Urban Environments.Robotics and Autonomous Systems. 2005, vol. 50, issue 4, pp. 195-209.

[16] Ortiz, A. E.; Neogi, N. N. Object Detection and Avoidance Using OpticalTechniques in Uninhabited Aerial Vehicles. Proceedings of the AIAAGuidance, Navigation, and Control Conference and Exhibit. 2007.

[17] Zufferey, J.-C.; Floreano, D. Toward 30-gram Autonomous Indoor Aircraft:Vision-based Obstacle Avoidance and Altitude Control. Proceedings of the2005 IEEE International Conference on Robotics and Automation(ICRA'2005). 2005, pp. 2594-2599.

[18] Saripalli, S.; Montgomery, J. F.; Sukhatme, G. S. Visually-Guided Landingof an Unmanned Aerial Vehicle. IEEE Transactions on Robotics andAutomation. 2003, vol. 19, issue 3, pp. 371-380.

[19] Fitzgerald, D.; Walker, R.; Campbell, D. A Vision Based EmergencyForced Landing System for an Autonomous UAV. Proceedings ofAustralian International Aerospace Congress Conference. 2005.

[20] Sharp, C. S.; Shakernia, O.; Sastry, S. S. A Vision System for Landing anUnmanned Aerial Vehicle. Proceedings of the IEEE InternationalConference on Robotics and Automation. (2001 ICRA). 2001, vol. 2, p.1720-1727.

[21] Theodore, C.; Rowley, D.; Ansar, A.; Matthies, L.; Goldberg, S.; Hubbard,D.; Whalley, M. Flight Trials of a Rotorcraft Unmanned Aerial VehicleLanding Autonomously at Unprepared Sites. Proceedings of the AmericanHelicopter Society 62nd Annual Forum. 2006, pp. 1250-1264.

[22] Zhukov, B.; Avanesov, G.; Grishin, V.; Krasnopevtseva, E. On-Board Real-Time Image Processing to Support Landing on Phobos. Proceedings of the7th International Symposium on Reducing Costs of Spacecraft GroundSystems and Operations (RCSGSO). 2007, pp. 423-428.

[23] Grishin, V. A. A Cramer–Rao Bound for the Measurement Accuracy ofMotion Parameters and the Accuracy of Reconstruction of a Surface ProfileObserved by a Binocular Vision System. Pattern Recognition and ImageAnalysis. 2008, vol. 18, issue 3, pp. 507–513.

[24] Trawny, N.; Mourikis, A. I.; Roumeliotis, S. I.; Johnson, A. E.;Montgomery, J. F.; Ansar, A.; Matthies, L. H. Coupled Vision and Inertial


Navigation for Pin-Point Landing. Proceedings of the NASA Science andTechnology Conference (NSTC’07). 2007, paper B2P2.

[25] Fitzgerald, D. L.; Walker, R. A; Campbell, D. A Vision Based ForcedLanding Site Selection System for an Autonomous UAV. Proceedings ofthe 2005 International Conference on Intelligent Sensors, Sensor Networksand Information Processing (ISSNIP). 2005, pp. 397- 402.

[26] Saripalli, S.; Sukhatme, G. S. Landing on a Moving Target using anAutonomous Helicopter. Proceedings of the International Conference onField and Service Robotics. 2003, vol. 24, pp. 277-286.

[27] Helble, H.; Cameron, S. OATS: Oxford Aerial Tracking System. Roboticsand Autonomous Systems. 2007, vol. 55, issue 9, pp. 661-666.

[28] Lenhart, D.; Hinz, S. Automatic Vehicle Tracking in Low Frame RateAerial Image Sequences. International Archives of Photogrammetry,Remote Sensing, and Spatial Information Sciences. 2006, vol. 36-3, pp. 203-208.

[29] Nguyen, T. T.; Grabner, H.; Bischof, H.; Gruber, B. On-line Boosting forCar Detection from Aerial Images. Proceedings of the IEEE InternationalConference on Research Information and Vision for the Future (RIVF’07).2007, pp. 87-95.

[30] Zhao, T.; Nevatia, R. Car Detection in Low Resolution Aerial Image.Proceedings of the Eighth IEEE International Conference on ComputerVision (ICCV 2001). 2001, vol. 1, pp. 710-717.

[31] Min-Shou T. (1996). The Application of Correlation Matching Technique inImage Guidance. http://handle.dtic.mil/100.2/ADA316752

[32] Rekik, A.; Zribi, M.; Hamida, A. B.; Benjelloun, M. An OptimalUnsupervised Satellite image Segmentation Approach Based on PearsonSystem and k-Means Clustering Algorithm Initialization. InternationalJournal of Signal Processing. 2009, vol. 5, issue 1, pp. 38-45.

[33] Lari, Z.; Ebadi, H. Automatic Extraction of Building Features from HighResolution Satellite Images Using Artificial Neural Networks. Proceedingof the Conference on Information Extraction from SAR and Optical data,with Emphasis on Developing Countries. 2007.

[34] Kampouraki, M.; Wood, G. A.; Brewer, T. R. The Suitability of Object-Based Image Segmentation to Replace Manual Aerial Photo Interpretationfor Mapping Impermeable Land Cover. Proceedings of the 2007 AnnualConference of the Remote Sensing & Photogrammetry Society(RSPSoc2007). 2007.

[35] Samadzadegan, F.; Abbaspour, R. A.; Hahn, M. Automatic ChangeDetection of Geospatial Databases Based on a Decision-Level Fusion

Vladimir Grishin136

Technique. Proceedings of the XXth International Society forPhotogrammetry and Remote Sensing Congress (ISPRS 2004). 2004, pp.489-491.

[36] Li, Y.; Atmosukarto, I.; Kobashi, M.; Yuen, J.; Shapiro, L. G. Object andEvent Recognition for Aerial Surveillance. Proceedings of the SPIEConference on Optics and Photonics in Global Homeland Security. 2005,vol. 5781, pp. 139-149.

[37] Grishin, V. A; Aust, S. A. Architecture of Computer Vision SystemsIntended for Control of Aircraft. Proceedings of the Fourth InternationalConference Parallel Computations and Control Problems (PACO ‘2008).2008, pp. 1071-1086. (in Russian).

[38] Wagter, C. D.; Proctor, A. A.; Johnson, E. N. Vision-Only Aircraft FlightControl. Proceedings of the 22nd AIAA Digital Avionics Systems Conference(DASC '03). 2003, vol. 2, pp. 8.B.2-81-11.

[39] Montgomery, J. F.; Johnson, A. E.; Roumeliotis, S. I.; Matthies, L. H. TheJet Propulsion Laboratory Autonomous Helicopter Testbed: A Platform forPlanetary Exploration Technology Research and Development. Journal ofField Robotics. 2006, vol. 23, issue 3-4, pp. 245-267.

[40] Wagter, C. D.; Bijnens, B.; Mulder J. A. Vision-Only Control of a FlappingMAV on Mars. Proceedings of AIAA Guidance, Navigation and ControlConference and Exhibit. 2007, pp. 1-7.

[41] Nordberg, K.; Doherty, P.; Farneback, G.; Forssen, P.-E.; Granlund, G.;Moe, A.; Wiklund, J. Vision for a UAV helicopter. Proceedings of theIEEE/RSJ International Conference on Intelligent Robots and Systems(IROS'02). Workshop WS6 on aerial robotics. 2002, pp. 29-34.

[42] Kurdila, A.; Nechyba, M.; Prazenica, R.; Dahmen, W.; Binev. P.; DeVore,R.; Sharpley, R. Vision-Based Control of Micro–Air–Vehicles: Progressand Problems in Estimation. Proceedings of the 43rd IEEE Conference onDecision and Control. 2004, vol. 2, pp. 1635– 1642.

[43] Kim, H. J.; Shim, D. H. A flight control system for aerial robots: algorithmsand experiments. Control engineering practice. 2003, vol. 11, issue 12, pp.1389-1400.

[44] Smith, N. W. (2006). Artificially Intelligent Autonomous AircraftNavigation System Using a Distance Transform on an FPGA.www.altera.com/literature/dc/2006/a1.pdf

[45] Granlund, G.; Nordberg, K.; Wiklund, J.; Doherty, P.; Skarman, E.;Sandewall E. WITAS: An Intelligent Autonomous Aircraft Using ActiveVision. Proceedings of the UAV 2000 International Technical Conferenceand Exhibition. 2000.


[46] Dickmanns, E. D. Vehicles Capable of Dynamic Vision. Proceedings of 15th

International Joint Conference on Artificial Intelligence (IJCAI-97). 1997,vol. 2, pp. 1577-1592.

[47] Heintz, F.; Doherty, P. Managing Dynamic Object Structures usingHypothesis Generation and Validation. Proceedings of the AAAI Workshopon Anchoring Symbols to Sensor Data. 2004, pp. 54-62.

[48] Cliff, D.; Noble, J. Knowledge-based vision and simple visual machines.Philosophical Transactions of the Royal Society. Biological Sciences. 1997,vol. 352, issue 1358, pp. 1165-1175.

[49] Johnson, E. N.; Proctor, A. A.; Ha, J.; Tannenbaum, A. R. Development andTest of Highly Autonomous Unmanned Aerial Vehicles. AIAA Journal ofAerospace Computing, Information, and Communication. 2004, vol. 1, issue12, pp. 485-501.

[50] Hawkins, J.; Blakeslee, S. On Intelligence; Times Books; Henry Holt andCompany: New York, NY, 2004.

[51] Zhdanov, A. A. Autonomous artificial intelligence; Adaptive andIntellectual systems; Binom. Laboratory of Knowledge: Moscow, Russia,2008. (in Russian).


Chapter 6

BINOCULAR VISION AND DEPTH PERCEPTION:DEVELOPMENT AND DISORDERS

Ken Asakawa* and Hitoshi IshikawaDepartment of Ophthalmology and Visual Science, Kitasato University

Graduate School, Doctors Program of Medical Science.

Introduction

“Binocular vision” literally means vision with two eyes, and refers to thespecial attributes of vision with both eyes open, rather than one eye only. Ourperception under binocular conditions represents a highly complex coordinationof motor and sensory processes and is markedly different from and moresophisticated than vision with one eye alone. However, the use of a pair of eyescan be disrupted by a variety of visual disorders, e.g., incorrect coordinationbetween the two eyes can produce strabismus with its associated sensoryproblems, amblyopia, suppression and diplopia. What, then, is the reason for-and the advantage of-having two eyes? From our visual information input, wecan perceive the world in three dimensions even though the images falling onour two retinas are only two-dimensional. How is this accomplished? Thisarticle is a review of our ability to use both eyes, while also providing basic

* E-mail address: [email protected]. Correspondence to Ken Asakawa, CO (Orthoptist),

Department of Ophthalmology and Visual Science, Kitasato University Graduate School,Doctors Program of Medical Science, 1-15-1, Kitasato, Sagamihara, Kanagawa, 228-8555,Japan

Ken Asakawa and Hitoshi Ishikawa140

information on the development of binocular vision and on the clinical disordersthat interfere with our depth perception, such as strabismus and amblyopia.

1. Advantages of Binocular Vision

“Two eyes are better than one,” it is said; and, indeed, two eyes do offer anumber of advantages over just one. It is reported that some 80% of the neuronsin the visual cortex receive input from both eyes, which offers anatomicalsupport for the view that binocular vision is an attribute of considerable valueand importance. Clearly, binocular vision has a number of functionaladvantages, the main ones being: 1) Binocular summation, in which many visualthresholds are lower than with monocular vision[16]. Binocular visual acuity,for example, is typically better than monocular visual acuity; and two eyes offerbetter contrast detection thresholds than one does. 2) The binocular field of viewis larger than either monocular field alone. We have a horizontal field ofapproximately 200 degrees, in which the two visual fields overlap by about 120degrees when both eyes are used together[29]. We can see objects whose imagesare formed on both foveas as if their images fell on a single point midwaybetween the two eyes, like an imaginary single eye in the middle of ourforehead, named a “cyclopean eye” [7,45]. 3) If one looks at the fingertip infront of the eyes, noticing what can be seen behind it, and one first closes oneeye, and then the other, the objects behind the fingertip should appear to move.This positional difference results from the fact that the two eyes are arrangedlaterally, and are a certain distance-the interocular distance (60 to 65 mm)-apart.They therefore see the world from two slightly different points. The subtledifferences between the images entering each eye make possible the binocularform of depth perception, which is the true advantage of binocular vision, and isknown as “stereopsis”,[15,55,56]. The large designed studies of binocular visionand stereopsis by Howard, Rogers[32], Saladin[51] and Watt[63] areinvestigated.

2. Foundations of Binocular Vision

Images of a single object that do not stimulate corresponding retinal pointsin both eyes are said to be disparate[22,37]; binocular disparity is defined as thedifference in position of corresponding points between images in the two eyes[48,49,50] (figure 1). Binocular disparity can be classified as crossed or

Binocular Vision and Depth Perception: Development and Disorders 141

uncrossed in relation to the point at which the two eyes converge (the fixationpoint)[44]. Points perceived to be nearer than the fixation point (within theVieth-Müller circle, a theoretical prediction of objects in space that stimulatecorresponding points in the two eyes) generally have lines of sight that cross infront of the fixation point; these points are said to have crossed disparity. Pointsfarther away than the fixation point have lines of sight that meet behind thefixation point; this is called uncrossed disparity. The Vieth-Müller circleintersects the fixation point and the entrance pupils of each eye. Diplopia is theresult of a large binocular disparity; however, the visual system is able tocombine two images into a single percept with smaller disparities. In binoculardisparities associated with normal binocular vision, the relationship betweenmotor and sensory fusion is more complex[25]. Panum’s area determines theupper limit of disparities that can produce single vision[41,54]. Smalldifferences in the perception of the two eyes give rise to stereopsis—three-dimensional depth perception. When a distant object is fixated bifoveally, nearerobjects in front of it will be imaged on the temporal retina of each eye onnoncorresponding points, resulting in a double image; crossed diplopia. Incontrast, when a near object is fixated and a distant object is seen double, this iscalled uncrossed diplopia. In this case, each image is formed on the nasal retinaof the eye. These phenomenons called physiological diplopia. The double imagearises from visual corresponding or non-corresponding retinal areas underbinocular vision. Binocular retinal correspondence is defined by the set ofretinal image locations that produces identical visual directions when viewingwith both eyes at the same time. These object locations, imaged ontocorresponding retinal points, can be imagined as a cylinder with an infiniteradius of curvature. This surface of points, called the horopter, stimulates theperception of identical visual directions for the two eyes[57,58]. However, thehoropter will not precisely intersect only the fixation target. Because singlebinocular vision only requires the retinal image to fall within Panum’s area, asmall residual misalignment of the visual axis (vergence error) may occur,causing a constant retinal disparity of a fixated object without diplopia. Fixationdisparity is used by the vergence eye movement system to maintain itsinnervational level and compensate for a heterophoria. In the United States,testing has primarily followed a motor approach, whereas a strong sensory-based analysis has been used in Germany[39].


Figure 1. Crossed and uncrossed disparities result when objects produce images that areformed on closely separated retinal points. Any point within Panum’s area yields a perceptof a single image, while points outside Panum’s area produce diplopia.

3. Stereopsis as the Highest Level of Binocular Vision

The first descriptions of binocular vision were described in detail by Worth(1921), who classified binocular vision as three grades. The first degree consistsof the simultaneous perception of each eye’s image at once. The second degreeconsists of the combination of the two images into a single percept and fusion,


which include motor and sensory fusion. The third degree and highest level ofbinocular visual function is stereopsis—binocular, three-dimensional depthperception resulting from the neural processing of horizontal binocular disparities(figure 2). However, stereopsis is not the only way to obtain depth information;even after closing one eye, we can still determine the relative positions of objectsaround us and estimate our spatial relationships with them. The clues that permitthe interpretation of depth with one eye alone are called monocular clues. Theyinclude pictorial clues, such as the size of the retinal image, linear perspective,texture gradients, aerial perspective, and shading, as well as non-stereoscopicclues, such as accommodation of the crystalline lens, motion parallax, andstructure from motion[62].

Figure 2. The classical model of binocular visual function is composed of threehierarchical degrees.

4. Binocular Viewing Conditions on Pupil Near Responses

Here, the effect of binocular clues on near pupil response as our preliminaryresearch is introduced. When changing visual fixation from a distant to a closeobject, accommodation, convergence, and pupil constriction occur, three feedbackresponses those constitute the near reflex[42]. We investigated the amplitudes ofvergence eye movements associated with pupil near responses for subjects of pre-presbyopia and presbyopia under binocular and monocular viewing conditions indynamics of step change in real target position from far to near (figure 3). The


findings of these experiments were that the convergence response with pupilmiosis was induced in all cases under binocular viewing conditions (figure 4A,C),whereas only presbyopic subjects showed version eye movement without pupilconstriction under monocular conditions (figure 4D).

Our findings imply that accommodation, which is high in younger subjects,but becomes progressively restricted with age, is a most important factor in theinduction of the pupil near response. However, the results of presbyopia subjectsunder binocular conditions suggested that binocular visual function such as fusionof the real target, depth perception, and proximity induces pupil constriction inpresbyopia resulting from the inability to accommodate[27,31]. When both eyesare oriented toward a target, a fused perception of the target is formed, andthrough the processing of retinal disparity, depth perception can be achieved. Asobject distances from the plane of fixation increase, retinal image disparitiesbecome large and an object appears to be in two separate directions i.e., viewing anearby target binocularly yields proximal and disparity clues[20,43].

Consequently, in young subjects, accommodation is active, thus, the pupilnear response with convergence by blur-driven is well induced despite themonocular viewing condition. On the other hand, in presbyopic subjects, since thechange in real target position was performed in real space and binocular viewingconditions, proximity and disparity clues were all available and were inconjunction with each other[47].

Infrared CCDcamera

Target

(near)

Figure 3. We measure and record the dynamics of pupil and convergence simultaneouslywith the step stimuli of a real target in real space.


Figure 4. Measured data of binocular viewing conditions. The upper trace is from a youngsubject (A), and the lower, from a subject with presbyopia (B). The young subject’s typicalresults under monocular (non-dominant eye occluded) visual conditions (C). Typical traceof a subject with presbyopia showed conjugate eye movement without pupil constriction(D).


A

B-1 B-2

C-1

C-2

OA

SA

Uncrosseddiplopia

Visual axis

SA = OA

Visual axis

Confusion

Monofixation Uncrossed diplopia

SAAA

OA

SA

SA < OA

Cyclopean eye

Target

Nasal Temporal

SA = Subjective angle

OA = Objective angle

AA = Angle of anomaly (OA - SA)

Fovea

Zero point (Yoke area)

Anomalous associated point

Suppression

C-1

Figure 5. Suppression and retinal correspondence in strabismus with esodeviataion. (A)Normal subject; (B) Strabismic patient with normal retinal correspondence and withoutsuppression would have diplopia (B-1) and visual confusion (B-2), a common visualdirection for two separate objects. (C) Elimination of diplopia and confusion bysuppression of retinal image (C-1) and anomalous retinal correspondence (C-2): adaptationof visual directions of deviating eye.


5. Development of Binocular Vision

A major question is whether binocularity and stereopsis are present at birth orwhether infants must be learn to see binocularly and three-dimensionally. Thevisual system takes approximately 6 weeks to become sensitive to visual stimulusdeprivation, and binocular vision first appears at about 3 months of age. Althoughit never tapers off completely[52], visual experience has its greatest effects atabout 6 months of age, with effects diminishing rapidly until about 6 years of age[6,11,19]. During the critical period of rapid visual change between 6 weeks and 3months after birth, infants are at a greater risk of developing visual abnormalitiesthan at any other life stage. Therefore, infants are extremely susceptible to severevisual disorders arising from inadequate visual experience during the criticalperiod.

Since Wheatstone (1838), stereopsis has been one of the most popular fieldsof vision research, and it is routinely measured in clinical practice[40]. Disordersaffecting stereopsis include blur, strabismus, and amblyopia, and the clinicalmeasurement of stereopsis is of value as a means of indirect screening. The typeand extent of sensory adaptation are important factors in the re-establishment offunctional binocular vision for disorders such as strabismus and amblyopia inchildren[8,24].

Infantile esotropia, a stable, cross-fixational large-angle esotropia with onsetbefore 6 months of age, is the most common form of strabismus. Generally,cycloplegic refraction reveals less than 3D of hyperopia, with no refractive andaccommodative component responsible for deviation. Accommodative esotropia,in contrast, usually occurs between 6 months and 7 years of age, with an averageage of onset of 3 years[17]. The amount of hyperopic refractive error inaccommodative esotropia averages +4D; esodeviation is restored to orthophoriaby optical correction of the underlying hyperopia[1]. In the non-refractive form,hyperopia averages +2D; esodeviation (not related to uncorrected refractive error)is caused by a high AC/A ratio.

The normal sensory organization of binocular vision can be altered ininfantile strabismus by suppression or anomalous retinal correspondence (figure5). Therefore, most strabismic patients do not experience diplopia and visualconfusion[2,62,64]. Single vision is achieved by suppression, which causeselimination of the perception of objects normally visible to the deviating eyeduring simultaneous binocular viewing [28,34,65]. Anomalous retinalcorrespondence is an adapted shift in the visual directions of the deviated eyerelative to the normal visual directions of the fixating eye [4,21,35,46]. The netresult is that the deviating eye acquires a common visual direction to that of the


fovea of the fixating eye during binocular viewing of a peripheral retinal area[5,14]. According to the recent study, early abnormal binocular visual inputcontributes to poor outcomes in both infantile and accommodative esotropia[33].The accepted strabismus treatment is wearing appropriate glasses and eye musclesurgery. These treatments may prevent the development of sensory and motordysfunctions[60]. However, several factors, including patient age at surgicalalignment and duration of misalignment, influence the outcome of treatment.Future studies should establish critical factors for achieving stable binocularvision.

Abnormal development of spatial vision causes amblyopia, decreased visualacuity that cannot be attributed to suppression scotoma, uncorrected refractiveerror, and visual stimulus deprivation. Clinically, amblyopia is defined as areduction in visual function caused by abnormal visual experience duringdevelopment[30,61]. Strabismic amblyopia refers to amblyopia that is associatedwith the presence of strabismus, typically either esotropia or exotropia. Thestrabismic eye also shows a pronounced suppression of the central and peripheralvisual field[26,53]. In addition, there is a contrast-dependent that is stronglydependent on spatial frequency and a contrast-independent deficit for position oftargets[18,38,59]. Therefore, for infantile esotropia with significant fixationpreference, occlusion therapy and surgery are associated with normal acuitydevelopment and a potential for at least gross stereopsis[10,40]. Anisometropicamblyopia is caused by significant, unequal refractive errors, exceeding +2D,between the eyes. Ametropic amblyopia may have equal refractive errors that areeither extremely myopic (more than -6D) or hyperopic (more than +4D). Yetanother kind of amblyopia, meridional amblyopia, is caused by astigmaticrefractive errors for long periods (more than 2 years)[3]. Moreover, form visiondeprivation amblyopia occurs in patients with a constant obstruction in the imageformation mechanism of the eye, such as congenital ptosis, congenital ortraumatic cataracts and corneal opacities that remain untreated for sometime[9,23]. Pediatric cataract treatment is now undergoing rapid development, andvisual prognosis for children with cataracts is improving due to earlier surgery,increased frequency of intraocular lens (IOL) implantation, and improvedamblyopia therapy. Traditional amblyopia treatment consists of full-timeocclusion of the sound eye, using an adhesive patch. However, recent trendsinclude prescribing fewer hours and using atropine as an alternative or adjunct topatching or even as a first-line treatment[36].


Conclusion

Binocular vision requires a high level of coordination between motor andsensory processes—binocular vision and stereopsis will be compromised if anycomponent in this system fails. Visual inputs from both eyes are combined in theprimary visual cortex (V1), where cells are tuned for binocular vision. Theobservation of the cells tuning in V1, together with psychophysical evidence thatstereopsis occurs in visual processing, suggests that V1 was the neural correlate ofstereoscopic depth perception; however, more recent work has indicated that thisoccurs in higher visual areas (in particular, MT area). In the future, we would liketo review the neural integration of depth perception and binocular vision. Thepresent review provides the basic information on normal and abnormal binocularvision that forms the foundation for the clinical disorder of binocular vision. Welook forward to new ideas and research on binocular vision[12].

References

[1] Asakawa K, Ishikawa H, Shoji N. New methods for the assessment ofaccommodative convergence. J. Pediatr. Ophthalmol. Strabismus. 2009;46:273-277.

[2] Asher H. Suppression theory of binocular vision. Br. J. Ophthalmol. 1953;37:37-49.

[3] Atkinson J. Infant vision screening: Prediction and prevention of strabismusand amblyopia from refractive screening in the Cambridge photorefractionprogram, Oxford University Press, 1993

[4] Awaya S, von Noorden GK, Romano PE. Symposium: Sensory Adaptationsin Strabismus. Anomalous retinal correspondence in different positions ofgaze. Am. Orthopt. J. 1970; 20:28-35.

[5] Bagolini B. Anomalous correspondence: definition and diagnostic methods.Doc. Ophthalmol. 1967; 23:346-98.

[6] Banks MS, Aslin RN, Letson RD. Sensitive period for the development ofhuman binocular vision. Science. 1975; 190:675-677.

[7] Barbeito R. Sighting from the cyclopean eye: the cyclops effect in preschoolchildren. Percept. Psychophys. 1983; 33:561-564.

[8] Birch EE, Gwiazda J, Held R. Stereoacuity development for crossed anduncrossed disparities in human infants. Vision Res. 1982; 22:507-513.

[9] Birch EE, Stager DR. Prevalence of good visual acuity following surgeryfor congenital unilateral cataract. Arch. Ophthalmol. 1988; 106:40-43.


[10] Birch EE, Stager DR, Berry P, Everett ME. Prospective assessment ofacuity and stereopsis in amblyopic infantile esotropes following earlysurgery. Invest. Ophthalmol. Vis. Sci. 1990; 31:758-765.

[11] Birch EE: Stereopsis in infants and its developmental relation to visualacuity, Oxford University Press, 1993

[12] Brodsky MC. Visuo-vestibular eye movements: infantile strabismus in 3dimensions. Arch. Ophthalmol. 2005; 123:837-842.

[13] Brown AM, Lindsey DT, Satgunam P, Miracle JA. Critical immaturitieslimiting infant binocular stereopsis. Invest. Ophthalmol. Vis. Sci. 2007;48:1424-1434.

[14] Burian HM. Anomalous retinal correspondence. Its essence and itssignificance in diagnosis and treatment. Am. J. Ophthalmol. 1951; 34:237-253.

[15] Burian HM. Stereopsis. Doc. Ophthalmol. 1951; 5-6:169-83.[16] Campbell FW, Green DG. Monocular versus binocular visual acuity.

Nature. 1965 9; 208:191-192.[17] Coats DL, Avilla CW, Paysse EA, Sprunger D, Steinkuller, PG, Somaiya

M. Early onset refractive accommodative esotropia. J. AAPOS. 1998; 2:275-278.

[18] Demanins R, Wang YZ, Hess RF. The neural deficit in strabismicamblyopia: sampling considerations. Vision Res. 1999; 39:3575-3585.

[19] Dobson V, Teller DY. Visual acuity in human infants: a review andcomparison of behavioral and electrophysiological studies. Vision Res.1978; 18:1469-1483.

[20] Erkelens CJ, Regan D. Human ocular vergence movements induced bychanging size and disparity. J. Physiol. 1986; 379:145-169.

[21] Flom MC. Corresponding and disparate retinal points in normal andanomalous correspondence. Am. J. Optom. Physiol. Opt. 1980; 57:656-665.

[22] Foley JM, Applebaum TH, Richards WA. Stereopsis with large disparities:discrimination and depth magnitude. Vision Res. 1975; 15:417-21.

[23] Forbes BJ, Guo S. Update on the surgical management of pediatriccataracts. J. Pediatr. Ophthalmol. Strabismus. 2006; 43:143-151.

[24] Fox R, Aslin RN, Shea SL, Dumais ST. Stereopsis in human infants.Science. 1980; 207:323-324.

[25] Fredenburg P, Harwerth RS. The relative sensitivities of sensory and motorfusion to small binocular disparities. Vision Res. 2001; 41:1969-1979.

[26] Gunton KB, Nelson BA. Evidence-based medicine in congenital esotropia.J. Pediatr. Ophthalmol. Strabismus. 2003; 40:70-73.


[27] Gwiazda J, Thorn F, Bauer J, et al. Myopic children show insufficientaccommodative response to blur. Invest. Ophthalmol. Vis. Sci. 1993;34:690-694.

[28] Harrad R. Psychophysics of suppression. Eye. 1996; 10:270-273.[29] Harrington DO. The visual fields, St Louis, Mosby, 1964[30] Harwerth RS, Smith EL 3rd, Duncan GC, Crawford ML, von Noorden GK.

Multiple sensitive periods in the development of the primate visual system.Science. 1986; 232:235-238.

[31] Hokoda SC, Rosenfield M, Ciuffreda, KJ. Proximal vergence and age.Optom. Vis. Sci. 1991; 68:168-172.

[32] Howard IP, Rogers BJ. Binocular vision and stereopsis, Oxford UniversityPress.

[33] Hutcheson KA. Childhood esotropia. Curr. Opin. Ophthalmol. 2004;15:444-448.

[34] Jampolsky A. Characteristics of suppression in strabismus. AMA Arch.Ophthalmol. 1955; 54:683-696.

[35] Kerr KE. Anomalous correspondence--the cause or consequence ofstrabismus? Optom. Vis. Sci. 1998; 75:17-22.

[36] Khazaeni L, Quinn GE, Davidson SL, Forbes BJ. Amblyopia treatment:1998 versus 2004. J. Pediatr. Ophthalmol. Strabismus. 2009; 46:19-22.

[37] Lappin JS, Craft WD. Definition and detection of binocular disparity. VisionRes. 1997; 37:2953-2974.

[38] Levi DM, Harwerth RS, Smith EL. Binocular interactions in normal andanomalous binocular vision. Doc. Ophthalmol. 1980; 49:303-324.

[39] London R, Crelier RS. Fixation disparity analysis: sensory and motorapproaches. Optometry. 2006; 77:590-608.

[40] Maeda M, Sato M, Ohmura T, Miyazaki Y, Wang AH, Awaya S. Binoculardepth-from-motion in infantile and late-onset esotropia patients with poorstereopsis. Invest. Ophthalmol. Vis. Sci. 1999; 40:3031-3036.

[41] Mitchell DE. A review of the concept of "Panum's fusional areas". Am. J.Optom. Arch. Am. Acad. Optom. 1966; 43:387-401.

[42] Myers GA, Stark L. Topology of the near response triad. OphthalmicPhysiol. Opt. 1990; 10:175-181.

[43] North RV, Henson DB, Smith TJ. Influence of proximal, accommodativeand disparity stimuli upon the vergence system. Ophthalmic. Physiol. Opt.1993; 13:239-243.

[44] Ogle KN. Disparity limits of stereopsis. AMA Arch. Ophthalmol. 1952;48:50-60.


[45] Ono H, Barbeito R. The cyclopean eye vs. the sighting-dominant eye as thecenter of visual direction. Percept. Psychophys. 1982; 32:201-210.

[46] Pasino L, Maraini G. Area of binocular vision in anomalous retinalcorrespondence. Br. J. Ophthalmol. 1966; 50:646-650.

[47] Phillips NJ, Winn B, Gilmartin B. Absence of pupil response to blur-drivenaccommodation. Vision Res. 1992; 32:1775-1779.

[48] Regan D. Binocular correlates of the direction of motion in depth. VisionRes. 1993; 33:2359-2360.

[49] Richards W. Stereopsis and stereoblindness. Exp. Brain Res. 1970; 10:380-388.

[50] Richards W. Anomalous stereoscopic depth perception. J. Opt. Soc. Am.1971; 61:410-414.

[51] Saladin JJ. Stereopsis from a performance perspective. Optom. Vis. Sci.2005; 82:186-205.

[52] Scheiman MM, Hertle RW, Kraker RT, Beck RW, Birch EE, Felius J,Holmes JM, Kundart J, Morrison DG, Repka MX, Tamkins SM. Patchingvs atropine to treat amblyopia in children aged 7 to 12 years: a randomizedtrial. Arch. Ophthalmol. 2008; 126:1634-1642.

[53] Schor CM. Visual stimuli for strabismic suppression. Perception. 1977;6:583-593

[54] Schor CM, Tyler CW. Spatio-temporal properties of Panum's fusional area.Vision Res. 1981; 21:683-692.

[55] Sekular R, Blake R: Perception, Knopf, New York, 1985[56] Sheedy JE, Fry GA. The perceived direction of the binocular image. Vision

Res. 1979; 19:201-211.[57] Shipley T, Rawlings SC. The nonius horopter. I. History and theory. Vision

Res. 1970; 10:1225-1262.[58] Shipley T, Rawlings SC. The nonius horopter. II. An experimental report.

Vision Res. 1970; 10:1263-1299.[59] Tychsen L, Burkhalter A. Neuroanatomic abnormalities of primary visual

cortex in macaque monkeys with infantile esotropia: preliminary results. J.Pediatr. Ophthalmol. Strabismus. 1995; 32:323-328.

[60] Uretmen O, Civan BB, Kose S, Yuce B, Egrilmez S. Accommodativeesotropia following surgical treatment of infantile esotropia: frequency andrisk factors. Acta Ophthalmol. 2008; 86:279-283.

[61] von Noorden GK. Amblyopia: a multidisciplinary approach. Proctor lecture.Invest. Ophthalmol. Vis. Sci. 1985;26:1704-1716.

[62] von Noorden GK: Binocular vision and ocular motility, St Louis, Mosby,1990


[63] Watt SJ, Akeley K, Ernst MO, et al. Focus clues affect perceived depth. J.Vis. 2005; 5:834-862.

[64] Wong AM, Lueder GT, Burkhalter A, Tychsen L. Anomalous retinalcorrespondence: neuroanatomic mechanism in strabismic monkeys andclinical findings in strabismic children. J. AAPOS. 2000; 4:168-174.

[65] Wong AM, Burkhalter A, Tychsen L. Suppression of metabolic activitycaused by infantile strabismus and strabismic amblyopia in striate visualcortex of macaque monkeysi 2005; 9:37-47.


Chapter 7

REPEATABILITY OF PRISM DISSOCIATION ANDTANGENT SCALE NEAR HETEROPHORIAMEASUREMENTS IN STRAIGHTFORWARD

GAZE AND IN DOWNGAZE

David A. Goss1, Douglas K. Penisten2,Kirby K. Pitts2 and Denise A. Burns2

1 School of Optometry, Indiana University, Bloomington, IN 474052 College of Optometry, Northeastern State University, Tahlequah, OK 74464

Abstract

The evaluation of heterophoria is an important element of assessment ofbinocular vision disorders. This study examined the interexaminer repeatabilityof two heterophoria measurement methods in a gaze position with no verticaldeviation from straightforward position and in 20 degrees downgaze. The twoprocedures were von Graefe prism dissociation method (VG) and the tangentscale method commonly known as the modified Thorington test (MT). Serving assubjects were 47 young adults, 22 to 35 years of age. Testing distance was 40 cm.A coefficient of repeatability was calculated by multiplying the standarddeviation of the difference between the results from two examiners by 1.96.Coefficients of repeatability in prism diopter units were: VG, straightforward,6.6; VG, downgaze, 6.2; MT, straightforward, 2.8; MT, downgaze, 3.6. Theresults show a better repeatability for the tangent scale procedure than for the vonGraefe prism dissociation method.

David A. Goss, Douglas K. Penisten, Kirby K. Pitts et al.156

Introduction

Vergence disorders can be a source of eyestrain and uncomfortable vision.Measurement of heterophoria is an important component for the clinicalexamination for vergence disorders. Measurement of heterophoria requires amethod for the elimination of binocular fusion and a method for determining theangle between the lines of sight of the eyes and the position the lines of sightwould assume if they intersected at the object of regard.

Two common clinical methods of heterophoria measurement are the vonGraefe prism dissociation method (VG) and the tangent scale method that iscommonly known as the modified Thorington test (MT). The VG test uses prismdissociation to prevent binocular fusion. Alignment of the diplopic images by arotary prism or a prism bar provides measurement of the heterophoria. The MTtest employs a Maddox rod to prevent binocular fusion. A penlight is pointedtoward the patient through a hole in the center of the test card. A line is seen bythe eye covered by the Maddox rod and a tangent scale is seen by the other eye.The position of the line on the tangent scale is reported by the patient.

One of the factors that can be considered in the evaluation of a clinical test isthe repeatability of the results obtained with it. A metric often used in theevaluation of repeatability of clinical tests is found by multiplying the standarddeviation of the differences between pairs of measurements on a series of subjectsby 1.96 to get a 95% limits of agreement between the repeat measurements. Thisvalue is sometimes referred to as a coefficient of repeatability.

Previous studies have reported better repeatability for MT testing than for VGtesting. Morris [1] tested adult subjects, ages 22 to 31 years, on separate days.Near point VG and MT tests were performed on separate days by one examinerwith subjects viewing through a phoropter at a test target. Coefficients ofrepeatability for the VG test were 3.3 prism diopters (Δ) for 20 trained observersand 2.9Δ for 20 untrained observers. On the MT, coefficients of repeatability were2.0Δ for the trained observers and 1.6Δ for the untrained observers. Slightly lowercoefficients of repeatability were obtained when the subjects had kinesthetic inputfrom holding some part of the test target or the target support.

Rainey et al. [2] reported on repeatability of test results between twoexaminers for 72 subjects between the ages of 22 and 40 years. Test distance forboth the VG test and the MT test was 40 cm. VG testing was done usingphoropter rotary prisms, while the MT testing was performed without a phoropter.Coefficients of repeatability for the VG test were 6.7Δ using a flash presentationof the target in which subjects viewed the target intermittently between

Repeatability of Prism Dissociation and Tangent Scale… 157

adjustments of the prism and 8.2Δ using continuous viewing of the target. Thecoefficient of repeatability for the MT was 2.3Δ.

Wong et al. [3] presented repeatability data based on the agreement of resultsfrom two examiners. Seventy-two students, ranging in age from 18 to 35 years,served as subjects. All testing was done without a phoropter. For the VG test, aloose prism was used for prism dissociation and a prism bar was used for thealignment measurement. Test distance for both tests was 40 cm. The coefficient ofrepeatability for the VG test, using continuous presentation, was 3.8Δ.Coefficients of repeatability for the MT were 2.3Δ for continuous presentationand 2.1Δ for a flashed presentation of the test card.

Escalante and Rosenfield [4] examined the repeatability of gradient (lenschange) accommodative convergence to accommodation (AC/A) ratios. Repeatmeasurements, at least 24 hours apart, were performed by one examiner on 60subjects ranging in age from 20 to 25 years. Testing was done with subjectviewing through a phoropter, which was used for the gradient AC/A ratio lenschanges. Viewing distance was 40 cm. Coefficients of repeatability of the AC/Aratios, using various lens combinations, ranged from 2.2 to 3.5 prism diopters perdiopter (Δ/D) for the VG test and from 1.2 to 2.0 Δ/D for the MT test.

A consistent finding of the previous studies was better repeatability on theMT than on the VG. Only one of the studies involved doing both tests outside thephoropter. The present study reports results with both tests done without aphoropter. While not explicitly stated in each of the papers, it may be presumedthat test targets were placed in a position without any vertical deviation fromstraightforward position. The present study reports results for straightforwardposition and for 20 degrees of downgaze. Thus the purpose of the present studywas to test for confirmation of better repeatability on the MT test than on the VGtest and to examine their repeatabilities for a position of downgaze.

Methods

Forty-seven subjects ranging in age from 22 to 35 years served as subjects.Inclusion criteria were no ocular disease, no history of ocular surgery, nostrabismus, no amblyopia, and best corrected distance visual acuity of at least20/25 in each eye. Subjects wore habitual contact lens or spectacle prescriptionsduring testing. Testing protocols were approved by the human subjects committeeat Northeastern State University, Tahlequah, Oklahoma.

Testing was done at 40 cm. VG and MT tests were performed with no verticaldeviation from straightforward gaze and with 20 degrees of downgaze. For 20


degrees of downgaze, test cards were moved down from their position forstraightforward gaze by about 13.7 cm and they were tilted back about 20 degreesso that the subjects’ lines of sight would be approximately perpendicular to thecards. Four recording forms, each with a different test sequence, were used tocounter-balance test order. The recording form used was changed with eachconsecutive subject number. All tests were done by two examiners.

For the VG test, dissociation of the eyes was induced with an 8Δ base-downloose prism held over the subject’s right eye. Subjects viewed a vertical column ofletters on a test card at 40 cm from them. They were instructed to keep the lettersclear to control accommodation. A rotary prism was placed over the subject’s lefteye in a clamp mounted on a table. Subjects were instructed to report when thecolumns of letters were aligned so that the upper column of letters was directlyabove the lower one, and the reading on the rotary prism was noted when thesubjects reported alignment. Two values, one with the prism starting from thebase-in side and one starting from the base-out side, were averaged, and the resultwas recorded. Exo findings (Base-in for alignment) were recorded as negativevalues, and eso findings (base-out for alignment) were treated as positivenumbers. For downgaze measurements, subjects were instructed to turn their eyesdown rather than their heads, and the prisms were tilted forward by approximately20 degrees.

A Bernell Muscle Imbalance Measure (MIM) near test card was used for theMT test. This is calibrated for a 40 cm test distance. For lateral phoria testing,there is a horizontal row of numbered dots separated by 1Δ when the card is at 40cm and a hole in the center of the row. Light from a penlight was directed throughthe hole in the center of the card toward the subject. A red Maddox rod wasplaced over the subject’s right eye oriented so that the subject saw a vertical line.Subjects were then instructed to close their eyes. As soon as they opened theireyes, they reported the number through which the vertical red line passed andwhether the line was to the left or right of the white light. The number indicatedthe phoria magnitude. The position of the red line relative to the white lightindicated the direction of the phoria, to the left for exo (recorded as a negativenumber) and to the right for eso (recorded as a positive number).

Results

The mean findings on both tests were a small amount of exophoria (Table 1).A coefficient of repeatability was determined by finding the mean and standarddeviation of the differences between the two examiners and then multiplying the

Repeatability of Prism Dissociation and Tangent Scale… 159

standard deviation of the differences by 1.96. The coefficients of repeatability forthe VG test were 6.6Δ for no elevation or depression of gaze and 6.2Δ fordowngaze. The respective coefficients of repeatability for the MT test were 2.8Δand 3.6Δ (Table 1).

Table 1. Means (and standard deviations in parentheses) for both examinersand the coefficients of repeatability for each of the tests. Units are prism

diopters. VG, von Graefe test; MT, modified Thorington test; S,straightforward position; D, downgaze

Test Examiner 1Mean (SD)

Examiner 2Mean (SD)

Coefficient ofRepeatability

VG, S -1.9 (4.4) -2.4 (5.1) 6.6VG, D -1.0 (4.6) -0.9 (4.5) 6.2MT, S -2.2 (3.6) -2.3 (4.1) 2.8MT, D -1.9 (4.0) -2.0 (3.7) 3.6

Discussion

The results of the present study agree with previous studies [1-5] in findingbetter repeatability with the MT test than the VG test. This better repeatability ispresent under a variety of test conditions, including with or without the phoropterand in downgaze as well as with no vertical gaze deviation from straightforwardposition.

The VG test is the most commonly used subjective dissociated phoria inoptometric practice. Some clinicians have observed that the MT test is generallysimpler than the VG test in terms of instrumentation and patient instructions. [2,6]The fact that the MT also offers better repeatability than the VG suggests that itsmore widespread adoption as a routine phoria test may be advisable.

Repeatability of other less commonly used phoria tests has also been studied.One investigation reported the repeatability of the VG test to be better than thatfor the Maddox rod test. [1] Another study found repeatability for the near Howellcard phoria to be better than that for the VG test and nearly as good as that for theMT test. [3]

The means found in the present study are similar to previous studiescomparing VG and MT tests, in which means for the VG ranged from -2.2 to -5.0Δ and means for the MT ranged from -2.1 to -3.4Δ. [2,3,5-7] In the presentstudy, as in the previous studies, standard deviations for the VG were higher thanfor the MT. It has been reported that for midrange phorias, there is quite good


agreement of VG and MT findings, but for higher magnitude phorias, either exoor eso, VG tends to yield higher values. [7]

Conclusion

Based on the results of the present study as well as those of previous studies,one can conclude that the MT tangent scale test has better repeatability than theVG prism dissociation test under a variety of test conditions.

References

[1] Morris, FM. The influence of kinesthesis upon near heterophoriameasurments. Am. J. Optom. Arch. Am. Acad. Optom. 1960;37,327-51.

[2] Rainey, BB; Schroeder, TL; Goss, DA; Grosvenor, TP. Inter-examinerrepeatability of heterophoria tests. Optom. Vis. Sci. 1998;75,719-26.

[3] Wong, EPF; Fricke, TR; Dinardo, C. Interexaminer repeatability of a new,modified Prentice card compared with established phoria tests. Optom. Vis.Sci. 2002;79,370-5.

[4] Escalante, JB; Rosenfield, M. Effect of hetereophoria measurementtechnique on the clinical accommodative convergence to accommodationratio. Optom – J. Am. Optom. Assoc. 2006;77,229-34.

[5] Hirsch, MJ; Bing, LB. The effect of testing method on values obtained forphoria at forty centimeters. Am. J. Optom. Arch. Am. Acad. Optom.1948;25,407-16.

[6] Hirsch, MJ. Clinical investigation of a method of testing phoria at fortycentimeters. Am. J. Optom. Arch. Am. Acad. Optom. 1948;25,492-5.

[7] Goss, DA; Moyer, BJ; Teske, MC. A comparison of dissociated phoria testfindings with von Graefe phorometry and modified Thorington testing. J.Behav. Optom. 2008;19,145-9.

Reviewed by Douglas G. Horner, O.D., Ph.D., School of Optometry, IndianaUniversity, Bloomington, IN 47405.


Chapter 8

TEMPORARILY BLIND IN ONE EYE:EMOTIONAL PICTURES PREDOMINATE

IN BINOCULAR RIVALRY

Georg W. Alpers1 and Antje B.M. GerdesUniversity of Würzburg and University of Bielefeld, Germany

Abstract

Preferential perception of emotional cues may help an individual to respondquickly and effectively to relevant events. Existing data supports this hypothesisby demonstrating that emotional cues are more quickly detected among neutraldistractors. Little data is available to demonstrate that emotional stimuli are alsopreferentially processed during prolonged viewing. The preferential perception ofvisual emotional cues is apparent under conditions where different cues competefor perceptual dominance. When two incompatible pictures are presented to oneeye each, this results in a perceptual alternation between the pictures, such thatonly one picture is visible while the other is suppressed. This so called binocularrivalry involves different stages of early visual processing and is thought to berelatively independent from intentional control. Several studies from ourlaboratory showed that emotional stimuli predominate over neutral stimuli inbinocular rivalry. These findings can be interpreted as evidence for preferentialprocessing of emotional cues within the visual system, which extends beyond

1 E-mail address: [email protected]. Address for correspondence: PD Dr. Georg

W. Alpers Department of Psychology, (Biological Psychology, Clinical Psychology, andPsychotherapy), University of Würzburg, Marcusstraße 9-11 , D- 97070 Würzburg, Germany,Tel.: 0049-931-31-2840, Fax: 0049-931-31-2733.

Georg W. Alpers and Antje B.M. Gerdes162

initial attentional capture. Taken together, data from this paradigm demonstratesthat emotional pictures are perceived more intensively.

Keywords: binocular rivalry, emotional pictures, visual perception, preferentialprocessing

1. Preferential Processing of Emotional Stimuli

We are constantly exposed to a multitude of visual stimuli. Evaluating thisinformation and responding with adequate behavior where necessary, helps us tosurvive. Identifying threatening stimuli seems to be especially important as weneed to repel danger or protect ourselves from it as quickly as possible. Evidencefor a privileged role of aversive stimuli in perception and attention processes canbe found in a number of convincing research paradigms.

For example, visual search tasks show that angry faces can be detected veryquickly and pop out among a variety of neutral faces (Hansen and Hansen, 1988;Öhman, Lundqvist, and Esteves, 2001). Similarly, fear relevant stimuli likespiders and snakes surrounded by neutral objects can be found more quickly(Öhman, Flykt, and Esteves, 2001). In the so called dot-probe paradigm,participants respond faster to test probes (letters, for example) that appear on thespot where a fear relevant, as opposed to a neutral, stimulus was presentedbeforehand (Bradley, Mogg, Millar, Bonham-Carter, Fergussoon, Jenkins et al.,1997; Mogg and Bradley, 2002). Many scientists assume that this allocation ofattention is based on automatic processes which operate independently fromconscious processing. Convincing evidence for this assumption comes fromexperiments which reveal psychophysiological reactions to emotional stimuli evenwhen they were not consciously perceived, for example when they were presentedsubliminally and masked by another picture (Dimberg, Thunberg, and Elmehed,2000; Öhman and Soares, 1994).

1.1. Two Pathways for the Processing of Emotional Stimuli

Fast and automatic perception of emotionally relevant visual stimuli calls forrapid neuronal processing. The amygdala is central for the processing ofemotional cues and especially for fear relevant information (LeDoux, 1996;Morris, Friston, Buchel, Frith, Young, Calder et al., 1998). Based on a number ofanimal studies, LeDoux (1996) demonstrated that sensory information of externalstimuli can reach the amygdala via two relatively independent pathways. On the

Temporarily Blind in One Eye 163

one hand, information from the retina projects to the sensory cortex, via thesensory thalamus, for a higher-level analysis. This pathway is rather slow butencompasses detailed information and LeDoux therefore calls it the “high road”of emotional processing. Cortical areas process this input before it reaches theamygdala, from where emotional reactions can be elicited and modulated. Then, amore time consuming and more elaborate processing of emotional stimuli can beguaranteed (Pessoa, Kastner, and Ungerleider, 2002). Even more complexreciprocal influences of emotion and attention can further modulate processing ofvisual perception within a network of frontal and parietal brain regions (Pessoa, etal., 2002; Windmann, Wehrmann, Calabrese, and Güntürkün, 2006).

Figure 1. Two pathways of processing of visual emotional stimuli (after LeDoux, 1996).

In addition to this cortical pathway, a subcortical pathway for immediateprocessing of emotional information exists, where sensory information reaches theamygdala via direct thalamic projections. This is thought to be independent fromthe cortical analysis mentioned above. This direct projection from the thalamus tothe amygdala only provides for a crude analysis but it is much faster than the onefrom the thalamus to the amygdala via the sensory cortex. Thus, this so called“low road” represents a shortcut which bypasses cortical areas. Along thispathway, information about emotional relevance can reach the amygdala muchmore rapidly (see Figure 1). This allows for a direct and fast response topotentially dangerous stimuli before the analysis is complete. Thus, emotionalreactions that are initiated by the amygdala enable for effective fight or flight


responses (e.g., defensive reflexes or an increased respiration rate and heart rate –see Alpers, Mühlberger, and Pauli, 2005).

Figure 2. Amygdala projections to the visual cortex of the macaque brain (Amaral, et al.,2003). L - lateral nucleus, Bmc - magnocellular division, BI - intermediate division, TEO –optic tectum , TE – telencephalon.

Support for the independence of the "low road" from cortical processingcomes from animal studies and studies with cortically blind patients. Theydemonstrate that processing of visual emotional information is indeed possiblewithout involvement of intact cortical circuitry. For example, cortically blindpatients show physiological responses to emotional stimuli even if they are notable to consciously perceive them (Anders, Birbaumer, Sadowski, Erb, Mader,Grodd et al., 2004; Hamm, Weike, Schupp, Treig, Dressel, and Kessler, 2003).

The “low road” has an additional function: It can modulate corticalprocessing on the “high road”.

Once again, it was shown in animal studies that there are direct neuronalprojections from the amygdala to cortical visual areas (V1 or V2, for example)(Amaral, Behniea, and Kelly, 2003; Amaral and Price, 1984) (see figure 2).Evidence supporting that such neuronal circuits also exist in humans has beenprovided since (Catani, Jones, Donato, and Ffytche, 2003). These projectionsmake it possible that "quick and dirty" subcortical processing can influence more


elaborate processing in cortical areas which would allow for enhanced consciousperception and processing (Pessoa, Kastner, and Ungerleider, 2003).

Electrophysiological (Schupp, Junghöfer, Weike, and Hamm, 2003;Stolarova, Keil, and Moratti, 2006), as well as hemodynamic evidence(Herrmann, Huter, Plichta, Ehlis, Alpers, Mühlberger et al., 2008; Lang, Bradley,Fitzsimmons, Cuthbert, Scott, Moulder et al., 1998) shows that emotionallyrelevant stimuli are accompanied by increased activation of visual areas in theoccipital lobe. It is very plausible that this may be partly initiated by input fromthe “low road” in addition to top-down input from higher cortical areas such asdirected attention.

1.2. Intensive Processing of Negative Valence or of Arousal?

In additon to multiple findings documenting a preferential processing ofnegative stimuli in the amygdala there is growing evidence for an equallyintensive processing of arousing positive stimuli. According to the EmotionalityHypothesis, all emotional stimuli are selectively processed, independent of theirspecific valence. And indeed, processing stimuli which are associated with rewardinvolves similar brain circuits as the processing of cues for danger (Berridge andWinkielman, 2003; Davis and Whalen, 2001). As it can be seen in functionalmagnetic resonance imaging (fMRI) and positron-emission-tomography (PET)studies, processing of positive as well as negative words is associated with higheramygdala activation (Hamann and Mao, 2002). Furthermore, higher amygdalaactivation in response to positive and negative as opposed to neutral pictures hasbeen observed (Garavan, Pendergrass, Ross, Stein, and Risinger, 2001; Hamann,Ely, Hoffman, and Kilts, 2002). Electroencephalography (EEG) findings alsosupport the notion that strongly activating affective pictures are processed fasterand more intensely in the visual cortex (Cuthbert, Schupp, Bradley, Birbaumer,and Lang, 2000; Schupp, et al., 2003).

Thus, the intensity of emotional arousal seems to be more crucial than thespecific affective valence of a stimulus. Also, these findings suggest that theamygdala may be involved in the processing of positive as well as negativeemotional stimuli. In conclusion, it could be expected that positive as well asnegative pictures boost visual perception.


2. "Blind" in One Eye: Binocular Rivalry

Information from the environment reaches the two eyes independently. Undernormal circumstances this information is combined into a meaningful spatialrepresentation. However, conscious perception does not inevitably represent thephysical environment. Instead, perception is the end product of several steps ofselective processing. While many stimulus properties are still processed by theeyes’ sensory cells, only a small fraction of them reaches conscious awareness.This is especially obvious when ambivalent information is presented to the twoeyes and information cannot be combined to a meaningful impression at laterstages of processing in the brain. When competing pictures are presented to thetwo eyes and a distinct impression cannot be evoked, this results in a fascinatingperceptual phenomenon called binocular rivalry. For a given period of time, oneof the pictures is perceived dominantly while the other is suppressed and thusremoved from conscious awareness. An unpredictable alternation between the twoimpressions ensues. At times rivalry can also result in percepts which consist ofmixtures of both pictures combined of parts from both. During extended periodsof time, input from one eye is completely suppressed from conscious awareness.Thus, perceptual changes occur while visual input remains constant.

Binocular rivalry is a remarkable phenomenon and offers the possibility tofurther investigate visual perception. Importantly, conscious control over what isperceived during binocular rivalry is very limited (Meng and Tong, 2004; Tong,2001). In general, binocular rivalry enables the researcher to investigate featuresof conscious perception, and processes underlying perception, in detail.

2.1. What Helmholtz Knew Already

Recently, binocular rivalry receives growing attention in research focused onconsciousness, both from the psychological and the neurological point of view,but it is not a newly discovered phenomenon. Binocular rivalry has been a wellknown phenomenon for a very long time (see Humphrey and Blake, 2001). In1593 already, Porta reported a perceptual phenomenon which occurred when heheld the two pages of a book right in front of his two eyes (cited by Wade andOno, 1985). He reported being able to temporarily read one of the pages, whilethe other was invisible.

Early systematic investigations of binocular rivalry trace back to Wheatstone(1838). For his experimental investigations he developed the mirror stereoscope –


an optic apparatus which makes it possible to present different pictures to the twoeyes (see a modern version in Figure 3).

Ever since binocular rivalry has been studied scientifically, the underlyingneuronal processes have been discussed controversially. In spite of the multitudeof interesting findings, the neuronal mechanisms have not yet beenunambiguously clarified. The main disagreement that prevails is whether thecompetition for conscious perception takes place at very early or later stages ofvisual processing. One of the earliest theories was introduced by Helmholtz(1924) and many other studies are based on it.

He assumed that the eyes’ visual fields are completely independent of eachother and that under normal circumstances the consistent perceptual impressiondoes not occur until higher mental processes have taken place. Consequentially,he called the phenomenon “retinal rivalry”. According to this theory, the decisionabout dominance or suppression would only occur after both pictures have beenprocessed independently and higher mental processes such as attention wouldselect among the two.

Figure 3. The mirror stereoscope used in our laboratory (sample pictures on the computerscreen).


Hering (1886), on the other hand, favoured another model. He assumed thatearly inhibitory interactions in visual processing account for the occurrence ofrivalry. This has also been labelled the “low-level” theory. Thus, according to thistheory, the decision about dominance or suppression takes place before the twopictures are completely processed.

2.2. Competition between Input from the Eyes or between thePercepts?

This 19th century controversy continues to the present day. According to the“low-level” theory (advocated for example by Blake, 1989), binocular rivalryexhibits typical properties of early visual processes. Therefore, this theory is alsocalled “eye-rivalry” theory. Rivalry thus occurs by means of inhibitoryconnections between the monocular processing channels. Evidence for this theorycomes from imaging studies showing that rivalry alters activity early in the visualstream, e.g., in V1 (Polonsky, Blake, Braun, and Heeger, 2000; Tong and Engel,2001) and the lateral geniculate nucleus (Haynes, Deichmann, and Rees, 2005).Importantly, processing in these circuits is generally thought to be mostlyindependent from input from the two eyes. Binocular rivalry suppression of onechannel would thus mean that input from one eye is not thoroughly processedbefore being suppressed.

In contrast, the “high-level” theory postulates that binocular rivalry arisesbecause of a competition between stimulus information, independent from thesource of this input. Thus, this perspective is also called the “stimulus-rivalry”perspective. It assumes that rivalry is decided after primary processing inmonocular channels is integrated in binocular channels, that is, the crucialprocesses are thought to be independent from the fact that this input originatedfrom one eye or the other (Logothetis, Leopold, and Sheinberg, 1996). Thus,according to this theory, rivalry takes place between integrated stimulusrepresentations, rather than information from the two eyes.

Support for this perspective comes from single cell recordings in animalswhich show little evidence for rivalry-correlated activation in V1, inconsistenteffects for visual areas V4 and MT, but clear effects in the inferior temporalcortex. These neurophysiological findings favour the theory that rivalry is theresult of a competition between incompatible stimulus representations in higherareas of visual processing, i.e. after information from the two eyes has beenintegrated in V1 (Sheinberg and Logothetis, 1997).


Some findings from human studies also account for the fact that a certainamount of processing has taken place before the competition is decided. Forexample, suppressed stimuli can produce after-images(e.g., O'Shea and Crassini,1981). However, the strongest argument for the “high-level” theory is the findingthat fragments of a picture presented to one eye each can be reassembled to aconsistent percept in conscious perception (Kovacs, Papathomas, Yang, andFeher, 1996). Rivalry then occurs between combined percepts and not betweeninput from one eye. Interestingly, such rivalry between percepts even occurs whenparts of the pictures are intermittently projected to one or to the other eye(Logothetis, et al., 1996).

Because convincing support has been found for both theories, many authorsconclude that binocular rivalry may involve stages of visual processing (Nguyen,Freeman, and Alais, 2003; Wilson, 2003). From this point of view, the theoriesare not mutually exclusive, but rather characterize two extremes on a continuum.Rivalry considerably suppresses activation in the primary processing channels, butenough information of the suppressed picture can proceed to brain circuits whichprocess integrated information from both monocular channels.

2.3. Possible Influences from Non-visual Neuronal Circuits

Taking into account the projections from the amygdala to primary sensoryareas of the visual cortex (V1, V2) which we mentioned above (Amaral, et al.,2003), it becomes apparent that this may be an avenue for a picture's emotionalsignificance to influence processing within visual circuitry. If the relevance of astimulus has been detected within subcortical circuits (“low road”) this may leadto more intense visual processing at several later stages of visual perception.

Indeed, it has been shown that the amygdala is more strongly activated byfearful faces, even when they are suppressed in binocular rivalry. Thus, althoughthe emotional material was not available to conscious perception (confirmed by anabsence of activation in specialized face-sensitive regions of the ventral temporalcortex), emotional circuitry was (Pasley, Mayes, and Schiltz, 2004; Williams,Morris, McGlone, Abbott, and Mattingley, 2004).

Thus, we proposed that such preferential processing of emotional picturesmay result in their predominanceover neutral pictures in binocular rivalry.Because conscious control over binocular rivalry is probably not possible (Mengand Tong, 2004) we argued that demonstrating such predominance would provideparticularly convincing evidence for preferential processing during prolongedperception.


3. Previous Investigations of Emotional Picturesin Binocular Rivalry

3.1. Significance and Predominance

Until recently, very few studies have examined the influence of a pictures’significance on predominance and suppression in binocular rivalry. An early studyby Engel (1956) demonstrated that upright pictures of faces predominate overinverted faces, indicating that greater salience of the familiar upright faces boosttheir competitive strength in binocular rivalry.

Bagby (1957) investigated the possible influence of personal relevance byshowing pairs of pictures containing culturally relevant material to participantswith different cultural backgrounds. In North American participants the typicallyAmerican scene (a baseball player) predominated over less relevant material (amatador) while the latter predominated in participants from Mexican descent.Personal relevance and familiarity thus seem to have an influence on perception inbinocular rivalry. However, it has to be acknowledged that response biases (forexample whether a picture is easier to specify) have not been taken into account inthese studies.

Additional studies further support that perception in binocular rivalry ismodulated by personal significance of the stimuli. Kohn (1960) as well as Shelleyand Toch (1962) showed that certain personality traits (e.g., aggressiveness) caninfluence the predominance of related stimuli (violent scenes). It was alsoreported that depressed participants predominantly perceive pictures with sadcontent compared to healthy control participants (Gilson, Brown, and Daves,1982).

Enthusiasm for this paradigm was hampered by other disappointing results.Blake (1988) did not find predominance of meaningful texts over meaninglessstrings of letters. However, it is important to note that the text elements were notshown at once but as a stream of letters, thus, holistic processing was not possible.Rivalry would have had to occur between non-salient letters of salient words.

Interest in this research paradigm waned for many years after acomprehensive review by Walker (1978) highlighted the methodologicalproblems with early studies. Walker’s main critique was that studies investigatingthe influence of picture content did not apply stringent definitions ofpredominance. He concluded that response biases might have been the main causeof the observed results.


Today, interest in this paradigm has been renewed and a number of wellcontrolled experiments have been conducted since. Results convincinglydemonstrate that pictures with a good Gestalt predominate over meaninglesspictures (de Weert, Snoeren, and Koning, 2005), and that a meaningful context ofpictures promotes predominance (Andrews and Lotto, 2004). Taken together,these investigations more clearly than former studies, support the hypothesis thatsemantic contents of pictures can affect binocular rivalry.

3.2. Emotional Discrepancy and Binocular Rivalry

Although an influence of semantic content of pictures on binocular rivalrythus seems very likely, there is little evidence that emotional salience can alsopromote dominance in rivalry. There are few studies in which different emotionalpictures were presented stereoscopically. A recent study investigated the effect ofdifferent emotional facial expressions on their relative dominance (Coren andRussell, 1992). Here, pairs of pictures showing different facial expressions werepresented to one eye each, and after a presentation time of 350 msec participantswere asked what their predominant percept was. Facial expressions with strongpositive or strong negative valence and high subjective arousal were morefrequently perceived as predominant than faces with less extreme ratings. Overall,valence seemed to have had the strongest impact while arousal mainly influencesdominance when the rivalling pictures had the same valence. It is problematic thatonly emotional faces were presented together and the presentation time was veryshort, because binocular rivalry takes time to built up. Moreover, asking theparticipants about their percept after the presentation introduced sources of errorsuch as memory effects and response biases.

Another study investigated perception of different facial expressions inbinocular rivalry but its aim was to document the two-dimensional structure ofemotions (on scales of valence and arousal) and predominance of emotionalpictures was not the center of its design (Ogawa, Takehara, Monchi, Fukui, andSuzuki, 1999). Pairs of pictures with different emotional facial expressions werepresented and the perceived impression was rated for valence and arousal. Asecond and very similar study by the same group measured specific perceptualimpressions during the presentation in addition to the dimensional structure of theevaluations (Ogawa and Suzuki, 2000). The authors conclude that there is morerivalry between pictures which are evaluated similarly on valence and arousal,while pictures with stronger valence and superior arousal are more unambiguouslyperceived as dominant.


Taken together, there is some evidence that emotional meaning influencesbinocular rivalry, but until now, only few studies investigated this influencesystematically.

4. Binocular Rivalry Experiments at Our Lab

4.1. Predominance of Emotional Scenes

We argue that binocular rivalry provides for very interesting experiments inemotion research, but our review of the literature demonstrated that this questionhas not been thoroughly investigated. There is a general lack of recent andmethodically convincing studies. The first study on binocular rivalry from our labinvestigated whether complex emotional pictures predominate over neutralpictures (Alpers and Pauli, 2006). Sixty-four healthy participants werestereoscopically shown ten pairs of pictures from the International AffectivePicture System (IAPS, Lang, Bradley, and Cuthbert, 2005). These pictures depictdifferent emotional scenes and are frequently used as stimuli in emotion research.Using this widespread picture material allows for a direct comparison of theresults with data from other established paradigms.

Because positive as well as negative pictures elicit similar patterns ofactivation in areas responsible for emotional processing (see above), we chosepictures of negative, neutral and positive valence for this study. If pictures withemotionally activating content predominated over neutral pictures in binocularrivalry, this would suggest that emotional input is preferentially processed at anearly stage of visual processing. Thus, pairs of pictures were composed with oneemotional (positive or negative) and one neutral picture each. These pairs werepresented for 30 sec in a randomized order. Participants looked through a mirrorstereoscope that was mounted about 30 cm in front of the computer monitor. Thusonly one picture was projected to each eye. In contrast to earlier studies,participants had to continuously verbalize what their perceptual impression wasthroughout each trial.

Participants were not explicitly instructed to categorize emotional and neutralpercepts, but were simply asked to report the content they saw. A trained researchassistant coded the participants’ comments as emotional or neutral with buttonpresses. A ratio of the cumulative time for emotional versus neutral percepts wasused as the dominance index. We assessed the initial perceptual impression ineach trial as another dependent variable because this is thought to be less stronglyaffected by habituation and less error-prone with regard to verbalization.


First of all, our results confirmed that emotional pictures are considerablylonger perceived as dominant than neutral pictures. Also, the initial perceptualimpression was significantly more often that of the emotional as opposed to theneutral picture. Differences in predominance were not found between positive andnegative pictures, either in duration of the percept nor in the frequency of theinitial percept. This experiment clearly shows that pictures of emotional scenespresented to one eye predominate over neutral pictures presented to the other eye.This confirms our hypothesis that emotional content can boost visual perception.

As clear cut as the results of this experiment may be, a number of seriouslimitations have to be considered. First, verbal coding is certainly prone toresponse biases. The tendency to more often mention emotional picture contentscould have had an influence on the results, as well as a tendency to avoidverbalizing specific picture content, because some contents may have beenembarrassing for the participant, for example (erotic pictures). Verbalizingunpleasant issues (repellent and disgusting details) could also pose a problem.Furthermore, relatively large pictures were used in this study, and this often leadsto the perception of mixed pictures, so called piecemeal rivalry (O'Shea, Sims,and Govan, 1997). Although it was also possible to report mixed pictures, itremains unclear whether participants applied different decision criteria forreporting mixed or unambiguous percepts.

In conclusion, these results support the hypothesis that binocular rivalry isinfluenced by the emotional contents of two competing pictures.

4.1.1. Possible Confounds

Among the undesirable confounding factors which can potentially influenceperceptual dominance, potential physical differences in the pictures’ complexityand color are certainly most important. However, to a large extent suchdifferences are closely associated with the emotional content (Lang et al., 1998).

Binocular rivalry is strongly influenced by certain physical characteristics.For example, larger pictures tend to fuse with each other more strongly thansmaller pictures (i.e. the rate at which rivalry occurs, is diminished) (O'Shea, etal., 1997). Pictures with high-contrast are perceived more dominantly comparedwith low-contrasted pictures (Blake, 1989). Brighter pictures dominate morefrequently than darker pictures (Kaplan and Metlay, 1964) and moving picturesdominate over stable pictures (Blake and Logothetis, 2002). The studiessummarized below are aimed at controlling for physical characteristics and atcontrolling for possible problems with self report.


4.2. Dominance of Emotional Facial Expressions

In order to further pursue the question of preferential perception of emotionalstimuli with the aid of binocular rivalry, and stimuli which are better controlledfor physical differences, we conducted an experiment with pictures of differentfacial expressions (Alpers and Gerdes, 2007). Presenting emotional faces isespecially useful in emotion research because these stimuli are evolutionaryrelevant (Dimberg, et al., 2000), largely independent from culture (Ekman,Sorenson, and Friesen, 1969) and because they provoke emotional reactions inevery day life (Dimberg, 1982).

Emotional facial expressions are processed very rapidly (Jeffreys, 1996) andholistically (Farah, Wilson, Drain, and Tanaka, 1998) via a specialized subcorticalroute (Johnson, 2005). Emotional faces attract attention as can be seen in visualsearch and Dot-Probe paradigms (Mogg and Bradley, 2002). They elicitsubcortical activation as well as peripheral physiological and behavioral reactions(Whalen, Kagan, Cook, Davis, Kim, Polis et al., 2004; Whalen, Rauch, Etcoff,McInerney, Lee, and Jenike, 1998) even when they are masked and thus notconsciously perceived.

For our study on binocular rivalry we chose pictures of eight actresses, eachof them showing angry, frightened, happy, surprised and neutral facialexpressions, from a standardized set of pictures (Karolinska Directed EmotionalFaces, KDEF, Lundqvist, Flykt, and Öhman, 1998). These pictures are very wellstandardized regarding physical characteristics such as background andbrightness. Pair wise presentation of one neutral and one emotional facialexpression of the same actress made it possible for us to adequately control forinter-individual differences of the actresses.

During the stereoscopic presentation the 30 participants continuously codedtheir perception of emotional, neutral or mixed pictures by button presses. Again,there was clear evidence for predominance of emotional stimuli compared toneutral stimuli for both, the cumulative duration with which each percept wasseen as well as the initial percept seen during each trial (see Figure 4).

Emotional faces were consciously perceived by the participants significantlylonger throughout the trial and they were significantly more often perceived as thefirst clear percept of a trial. Different from our expectation, there were nodifferences between positive and negative facial expressions. These findingssupport our earlier findings of predominance of emotional pictures over neutralpictures in binocular rivalry. In this study we were able to take into accountpotential limitations of the first study. Predominance of emotional faces is equally


strong as that of emotional IAPS pictures, although the latter normally induce ahigher arousal than emotional faces (see Adolph, Alpers, and Pauli, 2006).

Figure 4. Results from the experiment with emotional faces (Alpers and Gerdes, 2007;with permission from APA). Left panel: Cumulative duration of dominant perception (inseconds) and standard errors of the mean, for emotional, neutral and mixed pictures. Rightpanel: Average frequency of initial perception and standard errors of emotional, neutraland mixed pictures.

As participants did not have to verbalize what they perceived and thecategorical classification of perceived facial expressions was easy (emotional vs.neutral), the likelihood of response biases was clearly reduced in this study.Nonetheless, coding of participants' perception was still based on self-report.Thus, biases can not be completely ruled out.

4.3. Inter-Individual Differences: Phobic Stimuli

After having demonstrated that emotional pictures are in fact dominant overneutral pictures, we addressed another early hypothesis of the binocular rivalryliterature: Are inter-individual differences reflected in what people perceive inbinocular rivalry?

With the help of a further improved experimental design, we investigatedwhether fear relevant stimuli are perceived as more dominant by individuals who


are characterized by high levels of high fear. We recruited patients with a specificphobia of spiders because they are especially well suited for this investigation. Thepresentation of phobia-related stimuli elicits strong emotional reactions (Globisch,Hamm, Esteves, and Öhman, 1999). That phobic material activates subcorticalnetworks which may in turn prime or boost the activity of the visual cortex, hasbeen documented in a stronger amygdala activation in response to phobic cues (e.g.,Alpers, Gerdes, Lagarie, Tabbert, Vaitl, and Stark, submitted). Remarkably,possible influences of mere physical differences of pictures (emotional versusneutral) are not problematic in this endeavor because they should affect phobic andnon-phobic participants in equal measure.

Different degrees of dominance between patient and control participantswould also support the theory that phobia-related cues are processed moreintensely in phobic participants. A group of 23 patients who met diagnosticcriteria for spider phobia (DSM-IV, American Psychiatric Association, 1994) and20 non-phobic control participants were recruited for this investigation (Alpersand Gerdes, 2006).

Twenty different pictures of spiders and flowers were presentedstereoscopically. Different from previous studies all of these pictures were pairedwith an abstract pattern,. We hoped that this would minimize the problems relatedto response biases and to interindividual differences in decision criteria. Thesepicture-pattern pairs were presented for 8 sec each (see Figure 5).

Participants were asked to continuously code their perceptual impression bypressing one of three different buttons. There was one button each for dominantperceptual impression of a picture (spider and flower), one for the abstract pattern,and and one for mixed percepts. The advantages of this approach were that thespecific content of a picture did not have to be identified and that no decisionbetween two semantic pictures was needed.

Figure 5. Stimulus material for the experiment with phobic patients (Alpers and Gerdes,2006): examples of a spider picture, the abstract pattern, and a flower.


Figure 6. Results from the experiment with phobic patients; Left panel: mean duration ofdominant perception (with standard errors) of pictures of spiders, patterns and mixedpictures (crossed bars: phobic; empty bars: control participants). Right panel: meanduration of dominant perception (with standard errors) of pictures of flowers, patterns andmixed pictures (crossed bars: phobic; empty bars: control participants).

Spider phobic participants perceived pictures of spiders as dominant forlonger periods of time during the trials and they also reported that they perceivedphobic pictures as the first clear percept of a trial more often than controlparticipants. At the same time, there are no group differences for pictures offlowers versus the pattern; groups did not differ in the duration with which theyperceived pictures of flowers as dominant or in how often they reported seeingflowers as the first percept in a trial (see Figure 6).

This study replicates our previous findings in showing that emotional picturesmodulate competition in binocular rivalry. In addition, we were able todemonstrate that personal relevance is reflected in dominance of specific phobia-related cues. We were also able to support the theoretically founded assumptionthat phobia-related cues are preferentially processed by individuals with spiderphobia.

When we compared negatively valenced and positively valenced pictures inearlier studies, no differences were apparent. With respect to interindividualdifferences, we have no data concerning personal relevance of positive pictures atthis point.


4.4. Controlling for Physical Properties of Stimuli

Although the findings we reported above clearly support the hypothesis thatemotional picture content results in more predominant perception, it cannot becompletely ruled out that physical properties of pictures, as well as self-report ofperception, could have influenced the results in these studies. To account for thisand to validate our previous findings, we conducted four more experiments withthe objective to largely eliminate the influence of physical properties of stimuliand minimize response biases in self-report of perception.

In the first experiment, we presented two physically identical geometricgratings which only differed in spatial orientation. In order to introducedifferences in emotional valence we used a differential fear conditioningparadigm. Thus, the two stimuli which had neutral valence at the outset weregiven different emotional valence by our experimental manipulation (Alpers,Ruhleder, Walz, Mühlberger, and Pauli, 2005). Interestingly, we were able tointerleave conditioning and binocular rivalry trials. This helped us to documentthat more aversive experience with a given grating with a given orientationchanged its influence on predominance across trials. As a result, the aversivelyconditioned pattern was perceived as the dominant percept for a longer period oftime during the trials when compared to the perception before the fearconditioning, and it was reported more and more frequently as the first percept ofa trial. However, these effects were rather small compared with the effects instudies using emotional IAPS pictures and emotional facial expressions. This canprobably be best explained by the fact that geometric patterns are not evolutionaryrelevant, even if they aquire emotional relevance after fear conditioning. Thus,underlying neuronal processes of emotional processing are probably lesspronounced here than in experiments with stimuli that are evolutionary preparedor are naturally relevant.

In a second study we presented schematic emotional faces which are morebiologically relevant but differ in physical characteristics. We designed neutral,positive, and negative facial expressions by arranging nearly identical pictureelements (also see Lundqvist, Esteves, and Öhman, 2004). Although those facesare rather simple, several studies have demonstrate that schematic faces can elicitsimilar emotional reactions as photographs of faces (Bentin and Gollanda, 2002;Eastwood, Smilek, and Merikle, 2003; Lundqvist and Öhman, 2005).

In this experiment schematic emotional faces clearly predominantedcompared with neutral faces (Experiment 2, Alpers and Gerdes, 2007). Thepattern of results was very similar to the pattern reported above for photographicemotional faces. Taken together, both control-experiments demonstrate that


dominance of emotional stimuli can not be exclusively attributed to covaryingphysical differences between neutral and emotional pictures.

4.5. Validation of Self-report

Two further studies were aimed at verifying the participants' self report ofperception during binocular rivalry. Similar to the conditioning experimentintroduced above, two geometric patterns were shown stereoscopically. In order toobtain an objective measure of the participants' perception we coded eachstimulus with one of two flicker frequencies. When perceived dominantly, eachfrequency resulted in a distinguishable EEG signals (steady-state visually evokedpotentials) (Brown and Norcia, 1997). With this approach, we were able todemonstrate that the participants' self-report of what they saw corresponded withthe respective objective EEG signal of the dominant pattern over occipital regions(Alpers, et al., 2005).

Another experiment was based on the finding that changes in the suppressedpicture are harder to detect than changes in the dominant picture (Fox and Check,1968; Freeman and Jolly, 1994; Nguyen, Freeman, and Wenderoth, 2001). In thisexperiment, we again presented emotional and neutral faces from the KDEFpicture set (Alpers and Gerdes, 2007). In addition, in the course of a trial, weoccasionally presented a small dot in either the emotional or the neutral picture.Participants were asked to press a button when they detected a probe. More dotswere identified in the more emotional pictures which were dominant more oftenand reaction times were shorterwhen dots were identified in emotional picturescompared to neutral ones. Taken together, these two experiments may providesupport for the validity our participants' self-reportof their perception in binocularrivalry.

4.6. Summary

The series of studies presented here documents that emotional visual stimuliclearly predominate in binocular rivalry. The findings are consistent across avariety of stimuli such as emotional scenes, emotional facial expressions oraversively conditioned stimuli. In addition, differences in perception betweendifferentially affected groups of people were documented using phobia-relatedpictures. Moreover, we can largely rule out effects of physical differences orresponse biases on our results.


These results are consistent with other findings showing that meaningfulpictures dominate over meaningless ones in binocular rivalry (Yu and Blake,1992). However, as to the mechanisms, we have not yet shown that dominancewas mediated by activation of emotional neuronal circuits (the amygdala, forexample). Nonetheless, predominance of emotional stimuli in binocular rivalry isanother piece of evidence for preferential processing in the visual stream. Whetherthis is based on automatic processes (Öhman, 2005), higher order (cortical)attentional processes (Pessoa, et al., 2003) or an interaction of both is achallenging problem for future research.

5. Conclusion: "Blind" in One Eye - But not When ItComes to Emotion

The preferential perception of emotional pictures in binocular rivalry isclearly consistent with results from other experimental paradigms, such as thefaster detection of emotional stimuli in search tasks (Hansen and Hansen, 1988;Öhman, et al., 2001). Furthermore, the results are consistent with findings formpsychophysiological studies which show stronger activation of the visual cortexwhen looking at emotional pictures (Alpers, et al., 2005; Herrmann, et al., 2008;Schupp, et al., 2003).

An evolutionary advantage of faster detection of potentially meaningfulstimuli accounts for an easier processing of emotional material (Öhman andMineka, 2001). However, with the paradigm we described here we were not ableto verify whether activation of emotional neuronal circuits is in fact responsiblefor the competitive strength of emotional pictures in binocular rivalry.

With regard to neuronal substrates in which binocular rivalry is processed, itcan be hypothesized that feedback from subcortical circuity such as the amygdala(Amaral, Price, Pitkanen, and Carmichael, 1992) and the anterior cingulate cortex(Posner and Raichle, 1995) may be involved in processes leading to consciousperception. As we explained in the introduction, emotional material activates theamygdala in binocular rivalry even when it is suppressed. Because different stagesof processing in primary and extrastriatal areas are involved in binocular rivalry(Kovacs, et al., 1996; Logothetis, et al., 1996; Sheinberg and Logothetis, 1997), itis apparent that influences from emotional processing centers could take effect.

While some experiments with different paradigms found emotion specificeffects which suggest that preferential processing of negative pictures is specific(Hansen and Hansen, 1988; Öhman, Lundqvist, et al., 2001), our experimentswith binocular rivalry point to a dominance of both positive and negative pictures.


Although it seems to be particularly reasonable, from an evolutionary perspective,to preferentially process negative stimuli, there are several findings, which showthat positive stimuli are also processed preferentially (Garavan, et al., 2001;Hamann and Mao, 2002). Effects of arousal irrespective of valence are alsoevident in event related potentials of the EEG (Cuthbert, et al., 2000).Furthermore, automatic allocation of attention was found for both positive andnegative pictures (Alpers, 2008; Chen, Ehlers, Clark, and Mansell, 2002). It mightbe left to the higher cortical circuits to precisely analyze the specific valence andto control appropriate approach or avoidance responses.

In conclusion, positive and negative picture content seems to influenceperception in binocular rivalry, largely independent of physical differences. Thisyields more evidence for a preferential processing of emotional stimuli in thevisual system. This series of experiments provides some of the first evidence thatemotional stimuli are also preferentially processed during prolonged viewing. Theuse of the binocular rivalry paradigm could render an essential contribution tofurther investigations of emotional influences on visual perception.

References

Adolph, D., Alpers, G. W., and Pauli, P. (2006). Physiological Reactions toemotional Stimuli: A Comparison between Scenes and Faces. In H. Hecht, S.Berti, G. Meinhardt and M. Gamer (Eds.), Beiträge zur 48. Tagungexperimentell arbeitender Psychologen (pp. 235). Lengerich: Pabst.

Alpers, G. W. (2008). Eye-catching: Right hemisphere attentional bias foremotional pictures. Laterality: Asymmetries of Body, Brain and Cognition,13, 158-178.

Alpers, G. W., and Gerdes, A. (2006). Im Auge des Betrachters: BeiSpinnenphobikern dominieren Spinnen die Wahrnehmung. In G. W. Alpers,H. Krebs, A. Mühlberger, P. Weyers and P. Pauli (Eds.), WissenschaftlicheBeiträge zum 24. Symposium der Fachgruppe Klinische Psychologie undPsychotherapie (pp. 87). Lengerich: Pabst.

Alpers, G. W. and Gerdes, A.B.M. (2007). Here's looking at you: Emotional facespredominate in binocular rivalry. Emotion, 7, 495-506.

Alpers, G. W., Gerdes, A. B. M., Lagarie, B., Tabbert, K., Vaitl, D., and Stark, R.(submitted). Attention modulates amygdala activity: When spider phobicpatients do not attend to spiders.


Alpers, G. W., Herrmann, M. J., Pauli, P., and Fallgatter, A. J. (2005). Emotionalarousal and activation of the visual cortex: A Near Infrared SpectroscopyAnalysis [abstract]. Journal of Psychophysiology, 19, 106.

Alpers, G. W., Mühlberger, A., and Pauli, P. (2005). Angst - Neuropsychologie[Anxiety -Neuropsychology]. In H. F M. Hautzinger and G. Roth (Eds.),Neurobiologie psychischer Störungen [Neurobiology of mental disorders](pp.523-544). Heidelberg: Springer.

Alpers, G. W., and Pauli, P. (2006). Emotional pictures predominate in binocularrivalry. Cognition and Emotion, 20, 596-607.

Alpers, G. W., Ruhleder, M., Walz, N., Mühlberger, A., and Pauli, P. (2005).Binocular rivalry between emotional and neutral stimuli: a validation usingfear conditioning and EEG. International Journal of Psychophysiology, 57,25-32.

Amaral, D. G., Behniea, H., and Kelly, J. L. (2003). Topographic organization ofprojections from the amygdala to the visual cortex in the macaque monkey.Neuroscience, 118, 1099-1120.

Amaral, D. G., Price, J. L., Pitkanen, A., and Carmichael, S. (1992). Anatomicalorganization of the primate amygdaloid complex. In J. P. Aggleton (Ed.), TheAmygdala (pp. 1-66). New York: Wiley-Liss.

Anders, S., Birbaumer, N., Sadowski, B., Erb, M., Mader, I., Grodd, W., et al.(2004). Parietal somatosensory association cortex mediates affectiveblindsight. Nature Neuroscience, 7, 339-340.

Andrews, T. J., and Lotto, R. B. (2004). Fusion and rivalry are dependent on theperceptual meaning of visual stimuli. Current Biology, 14, 418-423.

Bagby, J. W. (1957). A cross-cultural study of perceptual predominance inbinocular rivalry. Journal of Abnormal and Social Psychology, 54, 331-334.

Bentin, S., and Gollanda, Y. (2002). Meaningful processing of meaninglessstimuli: The influence of perceptual experience on early visual processing offaces. Cognition, 86, B1-B14.

Berridge, K. C., and Winkielman, P. (2003). What is an unconscious emotion?(The case for unconscious "liking"). Cognition and Emotion, 17, 181-211.

Blake, R. (1988). Dichoptic reading: The role of meaning in binocular rivalry.Perception and Psychophysics, 44, 133-141.

Blake, R. (1989). A neural theory of binocular rivalry. Psychological Review, 96,145-167.

Blake, R., and Logothetis, N. K. (2002). Visual competition. Nature ReviewsNeuroscience, 3, 13-21.


Bradley, B. P., Mogg, K., Millar, N., Bonham-Carter, C., Fergussoon, E., Jenkins,J., et al. (1997). Attentional biases for emotional faces. Cognition andEmotion, 11, 25-42.

Brown, R. J., and Norcia, A. M. (1997). A method for investigating binocularrivalry in real-time with the steady-state VEP. Vision Research, 37, 2401-2408.

Catani, M., Jones, D. K., Donato, R., and Ffytche, D. H. (2003). Occipito-temporal connections in the human brain. Brain, 126, 2093-2107.

Chen, Y. P., Ehlers, A., Clark, D. M., and Mansell, W. (2002). Patients withgeneralized social phobia direct their attention away from faces. BehaviourResearch and Therapy, 40, 677-687.

Coren, S., and Russell, J. A. (1992). The relative dominance of different facialexpressions of emotion under conditions of perceptual ambiguity. Cognitionand Emotion, 6, 339-356.

Cuthbert, B. N., Schupp, H. T., Bradley, M. M., Birbaumer, N., and Lang, P. J.(2000). Brain potentials in affective picture processing: covariation withautonomic arousal and affective report. Biological Psychology, 52, 95-111.

Davis, M., and Whalen, P. J. (2001). The amygdala: vigilance and emotion.Molecular Psychiatry, 6, 13-34.

de Weert, C. M., Snoeren, P. R., and Koning, A. (2005). Interactions betweenbinocular rivalry and Gestalt formation. Vision Research, 45, 2571-2579.

Dimberg, U. (1982). Facial reactions to facial expressions. Psychophysiology, 19,643-647.

Dimberg, U., Thunberg, M., and Elmehed, K. (2000). Unconscious facialreactions to emotional facial expressions. Psychological Science, 11, 86-89.

Eastwood, J. D., Smilek, D., and Merikle, P. M. (2003). Negative facialexpression captures attention and disrupts performance. Perception andPsychophysics, 65, 352-358.

Ekman, P., Sorenson, E. R., and Friesen, W. V. (1969). Pan-cultural elements infacial displays of emotion. Science, 164, 86-88.

Engel, E. (1956). The role of content in binocular resolution. American Journal ofPsychology, 69, 87-91.

Farah, M. J., Wilson, K. D., Drain, M., and Tanaka, J. N. (1998). What is"special" about face perception? Psychological Review, 105, 482-498.

Fox, R., and Check, R. (1968). Detection of motion during binocular rivalrysuppression. journal of Experimental Psychology, 78, 388 - 395.

Freeman, A. W., and Jolly, N. (1994). Visual loss during interocular suppressionin normal and strabismic subjects. Vision Research, 34, 2043-2050.


Garavan, H., Pendergrass, J. C., Ross, T. J., Stein, E. A., and Risinger, R. C.(2001). Amygdala response to both positively and negatively valencedstimuli. Neuroreport, 12, 2279-2783.

Gilson, M., Brown, E. C., and Daves, W. F. (1982). Sexual orientation asmeasured by perceptual dominance in binocular rivalry. Personality andSocial Psychology Bulletin, 8, 494-500.

Globisch, J., Hamm, A. O., Esteves, F., and Öhman, A. (1999). Fear appears fast:Temporal course of startle reflex potentiation in animal fearful subjects.Psychophysiology, 36, 66-75.

Hamann, S., and Mao, H. (2002). Positive and negative emotional verbal stimulielicit activity in the left amygdala. Neuroreport, 13, 15-19.

Hamann, S. B., Ely, T. D., Hoffman, J. M., and Kilts, C. D. (2002). Ecstasy andagony: activation of the human amygdala in positive and negative emotion.Psychological Science, 13, 135-141.

Hamm, A. O., Weike, A. I., Schupp, H. T., Treig, T., Dressel, A., and Kessler, C.(2003). Affective blindsight: intact fear conditioning to a visual cue in acortically blind patient. Brain, 126, 267-275.

Hansen, C. H., and Hansen, R. D. (1988). Finding the face in the crowd: an angersuperiority effect. Journal of Personality and Social Psychology, 54, 917-924.

Haynes, J. D., Deichmann, R., and Rees, G. (2005). Eye-specific effects ofbinocular rivalry in the human lateral geniculate nucleus. Nature, 438, 496-499.

Helmholtz, H. v. (1924). Helmholtz's treatise on physiological optics. In J. P. C.Southall (Ed.). Rochester, N.Y.: The Optical Society of America. (Translatedfrom the 3rd German edition, 1909)

Hering, W. (Ed.). (1964). Outlines of a theory of the light sense (originallypublished in 1886). Cambridge, Mass.: Harvard University Press.

Herrmann, M. J., Huter, T. J., Plichta, M. M., Ehlis, A.-C., Alpers, G. W.,Mühlberger, A., et al. (2008). Enhancement of neural activity of the primaryvisual cortex for emotional stimuli measured with event-related functionalnear infrared spectroscopy (NIRS). Human Brain Mapping, 29, 28-35.

Humphrey, G. K., and Blake, R. (2001). Introduction [Special issue on binocularrivalry]. Brain and Mind, 2, 1-4.

Jeffreys, D. A. (1996). Evoked potential studies of face and object processing.Visual Cognition, 3, 1-38.

Johnson, M. H. (2005). Subcortical face processing. Nature ReviewsNeuroscience, 6, 766-774.

Kaplan, I. T., and Metlay, W. (1964). Light intensity and binocular rivalry.journal of Experimental Psychology, 67.


Kohn, H. (1960). Some personality variables associated with binocular rivalry.The Psychological Record, 10, 9-13.

Kovacs, I., Papathomas, T. V., Yang, M., and Feher, A. (1996). When the brainchanges its mind: Interocular grouping during binocular rivalry. Proceedingsof the National Academy of Sciences of the United States of America, 93,15508-15511.

Lang, P. J., Bradley, M. M., and Cuthbert, B. N. (2005). International affectivepicture system (IAPS) :Affective ratings of pictures and instruction manual.Technical Report A-6. University of Florida, Gainesville, FL.

Lang, P. J., Bradley, M. M., Fitzsimmons, J. R., Cuthbert, B. N., Scott, J. D.,Moulder, B., et al. (1998). Emotional arousal and activation of the visualcortex: an fMRI analysis. Psychophysiology, 35, 199-210.

Larson, C. L., Schaefer, H. S., Siegle, G. J., Jackson, C. A. B., Anderle, M. J., andDavidson, R. J. (2006). Fear is fast in phobic individuals: Amygdalaactivation in response to fear-relevant stimuli. Biological Psychiatry, 60, 410-417.

LeDoux, J. (1996). The emotional brain: The mysterious underpinnings ofemotional life. New York: Simon and Schuster.

Logothetis, N. K., Leopold, D. A., and Sheinberg, D. L. (1996). What is rivallingduring binocular rivalry? Nature, 380, 621-624.

Lundqvist, D., Esteves, F., and Öhman, A. (2004). The face of wrath: The role offeatures and configurations in conveying social threat. Cognition andEmotion, 18, 161-182.

Lundqvist, D., Flykt, A., and Öhman, A. (1998). The Karolinska DirectedEmotional Faces (KDEF). Stockholm: Karolinska Institute.

Lundqvist, D., and Öhman, A. (2005). Emotion regulates attention: The relationbetween facial configurations, facial emotion, and visual attention. VisualCognition, 12, 51-84.

Meng, M., and Tong, F. (2004). Can attention selectively bias bistable perception?Differences between binocular rivalry and ambiguous figures. Journal ofVision, 4, 539-551.

Mogg, K., and Bradley, B. P. (2002). Selective orienting of attention to maskedthreat faces in social anxiety. Behaviour Research and Therapy, 40, 1403–1414.

Morris, J. S., Friston, K. J., Buchel, C., Frith, C. D., Young, A. W., Calder, A. J.,et al. (1998). A neuromodulatory role for the human amygdala in processingemotional facial expressions. Brain, 121, 47-57.


Nguyen, V. A., Freeman, A. W., and Alais, D. (2003). Increasing depth ofbinocular rivalry suppression along two visual pathways. Vision Research, 43,2003-2008.

Nguyen, V. A., Freeman, A. W., and Wenderoth, P. (2001). The depth andselectivity of suppression in binocular rivalry. Perception and Psychophysics,63, 348-360.

O'Shea, R. P., and Crassini, B. (1981). Interocular transfer of the motion after-effect is not reduced by binocular rivalry. Vision Research, 21, 801-804.

O'Shea, R. P., Sims, A. J. H., and Govan, D. G. (1997). The effect of spatialfrequency and field size on the spread of exclusive visibility in binocularrivalry. Vision Research, 37, 175-183.

Ogawa, T., and Suzuki, N. (2000). Emotion space as a predictor of binocularrivalry. Perceptual and Motor Skills, 90, 291-298.

Ogawa, T., Takehara, T., Monchi, R., Fukui, Y., and Suzuki, N. (1999). Emotionspace under conditions of perceptual ambiguity. Perceptual and Motor Skills,88, 1379-1383.

Öhman, A. (2005). The role of the amygdala in human fear: Automatic detectionof threat. Psychoneuroendocrinology, 30, 953-958.

Öhman, A., Flykt, A., and Esteves, F. (2001). Emotion drives attention: detectingthe snake in the grass. Journal of Experimental Psychology: General, 130,466-478.

Öhman, A., Lundqvist, D., and Esteves, F. (2001). The face in the crowdrevisited: A threat advantage with schematic stimuli. Journal of Personalityand Social Psychology, 80, 381-396.

Öhman, A., and Mineka, S. (2001). Fears, phobias, and preparedness: Toward anevolved of fear and fear learning. Psychological Review, 108, 483-522.

Öhman, A., and Soares, J. J. F. (1994). "Unconscious anxiety": Phobic responsesto masked stimuli. Journal of Abnormal Psychology, 103, 231-240.

Pasley, B. N., Mayes, L. C., and Schiltz, R. T. (2004). Subcortical discriminationof unperceived objects during binocular rivalry. Neuron, 42, 163-172.

Pessoa, L., Kastner, S., and Ungerleider, L. G. (2002). Attentional control of theprocessing of neutral and emotional stimuli. Cognitive Brain Research, 15,31-45.

Pessoa, L., Kastner, S., and Ungerleider, L. G. (2003). Neuroimaging studies ofattention: from modulation of sensory processing to top-down control.Journal of Neuroscience, 23, 3990-3998.

Polonsky, A., Blake, R., Braun, J., and Heeger, D. J. (2000). Neuronal activity inhuman primary visual cortex correlates with perception during binocularrivalry. Nature Neuroscience, 3, 1153-1159.


Posner, M. I., and Raichle, M. E. (1995). Precis of Images of Mind. Behavioraland Brain Sciences, 18, 327-383.

Schupp, H. T., Junghöfer, M., Weike, A. I., and Hamm, A. O. (2003). Emotionalfacilitation of sensory processing in the visual cortex. Psychological Science,14, 7-13.

Sheinberg, D. L., and Logothetis, N. K. (1997). The role of temporal cortical areasin perceptual organization. Proceedings of the National Academy of Sciencesof the United States of America, 94, 3408–3413.

Shelley, E. L. V., and Toch, H. H. (1962). The perception of violence as anindicator of adjustment in institutionalized offenders. Journal of CriminalLaw, Criminology and Police Science, 53, 463-469.

Stolarova, M., Keil, A., and Moratti, S. (2006). Modulation of the C1 visualevent-related component by conditioned stimuli: evidence for sensoryplasticity in early affective perception. Cerebral Cortex, 16, 876-887.

Tong, F. (2001). Competing theories of binocular rivalry: A possible resolution.Brain and Mind, 2, 55-83.

Tong, F., and Engel, S. A. (2001). Interocular rivalry revealed in the humancortical blind-spot representation. Nature, 411, 195-199.

Wade, N. J., and Ono, H. (1985). The stereoscopic views of Wheatstone andBrewster. Psychological Bulletin, 47, 125-133.

Walker, P. (1978). Binocular rivalry: Central or peripheral selective processes?Psychological Bulletin, 85, 376-389.

Whalen, P. J., Kagan, J., Cook, R. G., Davis, F. C., Kim, H., Polis, S., et al.(2004). Human amygdala responsivity to masked fearful eye whites. Science,306, 2061.

Whalen, P. J., Rauch, S. L., Etcoff, N. L., McInerney, S. C., Lee, M. B., andJenike, M. A. (1998). Masked presentations of emotional facial expressionsmodulate amygdala activity without explicit knowledge. Journal ofNeuroscience, 18, 411-418.

Wheatstone, C. (1838). On some remarkable, and hitherto unobserved,phenomena of binocular vision. Philosophical Transactions of the RoyalSociety of London, 128, 371-394.

Williams, M. A., Morris, A. P., McGlone, F., Abbott, D. F., and Mattingley, J. B.(2004). Amygdala response to fearful and happy facial expressions underconditions of binocular suppression. The Journal of Neuroscience, 24, 2898-2904.

Wilson, H. R. (2003). Computational evidence for a rivalry hierarchy in vision.Proceedings of the National Academy of Sciences of the United States ofAmerica, 100, 14499-14503.


Windmann, S., Wehrmann, M., Calabrese, P., and Güntürkün, O. (2006). Role ofthe prefrontal cortex in attentional control over bistable vision. Journal ofCognitive Neuroscience, 18, 456-471.

Yu, K., and Blake, R. (1992). Do recognizable figures enjoy an advantage inbinocular rivalry? Journal of Experimental Psychology: Human Perceptionand Performance, 18, 1158-1173.

In: Binocular VisionEditors: J. McCoun et al, pp. 189-208

ISBN 978-1-60876-547-8c© 2010 Nova Science Publishers, Inc.

Chapter 9

STEREO-BASED CANDIDATE GENERATION

FOR PEDESTRIAN PROTECTION SYSTEMS

David Geronimo1∗, Angel D. Sappa1 and Antonio M. Lopez1,2

1Computer Vision Center and2Computer Science Department

Universitat Autonoma de Barcelona, 08193,Bellaterra, Barcelona, Spain

Abstract

This chapter describes a stereo-based algorithm that provides candidateimage windows to a latter 2D classification stage in an on-board pedes-trian detection system. The proposed algorithm, which consists of threestages, is based on the use of both stereo imaging and scene prior knowl-edge (i.e., pedestrians are on the ground) to reduce the candidate search-ing space. First, a successful road surface fitting algorithm provides es-timates on the relative ground-camera pose. This stage directs the searchtoward the road area thus avoiding irrelevant regions like the sky. Then,three different schemes are used to scan the estimated road surface withpedestrian-sized windows: (a) uniformly distributed through the road sur-face (3D); (b) uniformly distributed through the image (2D); (c) not uni-formly distributed but according to a quadratic function (combined 2D-3D). Finally, the set of candidate windows is reduced by analyzing their3D content. Experimental results of the proposed algorithm, together withstatistics of searching space reduction are provided.

∗E-mail address: [email protected]

190 David Geronimo, Angel D. Sappa and Antonio M. Lopez

1. Introduction

According to the World Health Organization, every year almost 1.2 mil-lion people are killed and 50 million are injured in traffic accidents worldwide[11]. These dramatic statistics highlight the importance of the research in traf-fic safety, which involves not only motor companies but also governments anduniversities.

Since the early days of the automobile, in the beginning of 20th century, andalong with its popularization, different mechanisms were successfully incorpo-rated to the vehicle with the aim of improving its safety. Some examples areturn signals, seat-belts and airbags. These mechanisms, which rely on physicaldevices, were focused on improving safety specifically when accidents wherehappening. In the 1980s a sophisticated new line of research began to pursuesafety in a preventive way: the so-calledadvanced driver assistance systems(ADAS). These systems provide information to the driver and perform activeactions (e.g., automatic braking) by the use of different sensors and intelligentcomputation. Some ADAS examples areadaptive cruise control(ACC), whichautomatically maintains constant distance to a front-vehicle in the same lane,andlane departure warning(LDW), which warns when the car is driven out thelane unadvertently.

One of the more complex ADAS arepedestrian protection systems(PPSs),which aim at improving the safety of these vulnerable road users. Attendingtothe number of people involved in vehicle-to-pedestrian accidents, e.g.,150 000injured and7 000 killed people each year in the European Union [6], it is clearthat any improvement in these systems can potentially save many human lifes.PPSs detect the presence of people in a specific area of interest around the hostvehicle in order to warn the driver, perform braking actions and deployexternalairbags in the case of an unavoidable collision. The most used sensor to de-tect pedestrians are cameras, contrary to other ADAS such as ACC, in whichactive sensors like radar or lidar are employed. Hence, Computer Vision (CV)techniques play a key role in this research area, which is not strange given thatvision is the most used human sense when driving. People detection has beenan important topic of research since the beginning of CV, and it has been mainlyfocused on applications like surveillance, image retrieval and human-machineinterfaces. However, the problem faced by PPSs differs from these applicationsand is far from being solved. The main challenges of PPSs are summarized inthe following points:

Stereo-Based Candidate Generation... 191

• Pedestrians have a high variability in pose (human body can be viewedas a highly deformable target), clothes (which change with the weather,culture, and people), distance (typically from 5 to at least 25m), sizes (notonly adults and children are different, but also there are many differenthuman constitutions), viewpoints (e.g., front, back or side viewed).

• The variability of the scenarios is also considerable, i.e., the detectiontakes place in outdoor dynamic urban roads with cluttered backgroundand illumination changes.

• The requirements in terms of misdetections and computational cost arehard: these systems must perform real-time actions at very low miss rates.

The first research works in PPSs were presented in the late 1990s. Papageor-giou et al. [10] proposed to extract candidate windows by exhaustivelyscanningthe input image and classify them with support vector machines based on HaarWavelet features. This two-step candidate generation and classification schemehas been used in a countless number of detection systems: from faces [14], ve-hicles or generic object detection to human surveillance and image retrieval [3].The simplest candidate generation approach is the exhaustive scan, alsocalledsliding window: it consists in scanning the input image with pedestrian-sizedwindows (i.e., with a typical aspect ratio around 1/2) at all the possible scalesand positions. Although this candidate generation method is generic and easyto implement, it can be improved by making use of some prior knowledge fromthe application. Accordingly, during the last decade researchers havetried to ex-ploit the specific aspects of PPSs to avoid this generation technique. Some cuesused for generating candidates are vertical symmetry [1], infrared hotspots [4]and 3D points [7]. However, the proposed techniques that exploit them poseseveral problems that make the systems not reliable in real-world scenarios. Forexample, in the case of 2D analysis, the number of false negatives (i.e., dis-carded pedestrians) can not be guaranteed to be low enough: symmetry relieson vertical edges, but in many cases the illumination conditions or backgroundclutter make them disappear. Hot spot analysis in infrared images holds a sim-ilar problem because of the environmental conditions [2]. On the other hand,although stereo stands as a more reliable cue, the aforementioned techniquesalso hold problems. In the case of [7], the algorithm assumes a constant roadslope, so the problems appear when the road orientation is not constant whichis common in urban scenarios.


This chapter presents a candidate generation algorithm that reduces thenumber of windows to be classified while minimizes the number of wronglydiscarded targets. This is achieved by combining a prior-knowledge criterion,pedestrians-on-the-ground, and using 3D data to filter the candidates. This pro-cedure can be seen as a conservative but reliable approach, which inour opinionis the most convenient option for this early step of the system.

The remainder of the manuscript is organized as follows. First, we intro-duce the proposed candidate generation algorithm with a brief description of itscomponents and their objective. Then, the three stages in which the algorithmisdivided are presented: Sect. 3. describes the road surface estimation algorithm,Sect. 4. presents the road scanning and Sect. 5. addresses the candidate filtering.Finally, Sect. 6. provides experimental results of the algorithm output. In Sect.7., conclusions and future work is presented.

2. Algorithm Overview

A recent survey on PPSs by Geronimo et al. [8] proposes a general archi-tecture that consists of six modules, in which most of the existing systems canbe fit. The modules (enumerated in the order of the pipeline process) are: 1)preprocessing, 2) foreground segmentation, 3) object classification,4) verifica-tion and refinement, 5) tracking and 6) application. As can be seen, modules2) and 3) correspond to the steps presented in the introduction. The algorithmpresented in this chapter consists in a candidate generation algorithm to be usedin the foreground segmentation module, which gets an input image and gen-erates a list of candidates where a pedestrian is likely to appear, to be senttothe next module, the classifier. There are two main objectives to be carried outin this module. The first is to reduce the number of candidates, which directlyaffects the performance of the system both in terms of speed (the fewer thecan-didates sent to the classifier the less the computation time is) and detection rates(negatives can be pre-filtered by this module). The second is not to discard anypedestrian, otherwise the later modules will not be able to correct the wrongfiltering.

The proposed algorithm is divided into three stages, as illustrated in Fig. 1.

1. Road surface estimationcomputes the relative position and orientationbetween the camera and the scene (Sect. 3.).


2. Road scanningplaces 3D windows over the estimated road surface usinga given scanning method (Sect. 4.).

3. Candidate filtering filters out windows that do not contain enough stereoevidence of containing vertical objects (Sect. 5.).

Next sections describe each stage in detail.

Vertical Objects

Road Surface Estimation Road Scanning Candidate Filtering

Discarded windows Selected windowsEstimated road position Pre-selected windows

Figure 1. Stages of the proposed algorithm.

3. Road Surface Estimation

The first stage is focused on adjusting the candidate searching space to theregion where the probability of finding a pedestrian is higher. In the context ofPPSs, the searching space is the road, hence irrelevant regions like the sky canbe directly ommited from the processing. The main targets of road surface esti-mation are two-fold: first, to fit a surface (a plane in the current implementation)to the road; second, to compute the relative position and orientation (pose) ofthe camera1 with respect to such a plane.

A world coordinate system(XW , YW , ZW ) is defined for every acquiredstereo image, in such a way that: theXWZW plane is contained in the currentroad fitted plane, just under the camera coordinate system(XC , YC , ZC); theYW axis contains the origin of the camera coordinate system; theXWYW planecontains theXC axis and theZWYW plane contains theZC axis. Due to that,the six extrinsic parameters (three for the position and three orientation angles)that refer the camera coordinate system to the world coordinate system reduce

1Also referred to as camera extrinsic parameters.


to just three, denoted in the following as(Π,Φ,Θ) (i.e., camera height, roll andpitch). Figure 2 illustrates the world and camera coordinate systems.

ZC

XC

YC

YW

XW

ZW

Camera

height

Pitch

Roll

Figure 2. Camera coordinate system(XC , YC , ZC) and world coordinate sys-tem(XW , YW , ZW ).

From the(Π,Φ,Θ) parameters, in most situations the value ofΦ (roll) isvery close to zero. This condition is fulfilled as a result of a specific cameramounting procedure that fixesΦ at rest, and because in normal urban drivingsituations this value scarcely varies [9].

The proposed approach consists of two substages detailed below (more in-formation in [13]): i) 3D data point projection and cell selection andii) roadplane fitting and ROIs setting.

3.1. 3D Data Point Projection and Cell Selection

Let D(r, c) be a depth map provided by the stereo pair withR rows andC columns, in which each array element(r, c) is a scalar that represents ascene point of coordinates(xC , yC , zC), referred to the camera coordinatesystem (Fig. 2). The aim at this first stage is to find a compact subset ofpoints, ζ, containing most of the road points. To speed up the whole al-gorithm, most of the processing at this stage is performed over a 2D space.


Initially, 3D data points are mapped onto cells in the(YCZC) plane, result-ing in a 2D discrete representationψ(o, q); whereo = ⌊DY (r, c) · ς⌋ andq = ⌊DZ(r, c) · ς⌋, ς representing a scale factor that controls the size of thebins according to the current depth map (Fig. 3). The scaling factor is aimedatreducing the projection dimensions with respect to the whole 3D data in orderto both speed up the plane fitting algorithm and be robust to noise. It is definedas: ς = ((R+ C)/2)/(∆X + ∆Y + ∆Z)/3); (∆X,∆Y,∆Z) is the workingrange in 3D space. Every cell ofψ(o, q) keeps a reference to the original 3Ddata points projected onto that position, as well as a counter with the number ofmapped points.

From that 2D representation, one cell per column (i.e., in the Y-axis) isselected, relying on the assumption that the road surface is the predominantge-ometry in the given scene. Hence, it picks the cell with the largest number ofpoints in each column of the 2D projection. Finally, every selected cell is repre-sented by the 2D barycenter(0, (

∑ni=0 yCi

)/n, (∑n

i=0 zCi)/n) of its n mapped

points. The set of these barycenters defines a compact representationof the se-lected subset of points,ζ. Using both one single point per selected cell and a2D representation, a considerable reduction in the CPU time is reached duringthe road plane fitting stage.

ZC

YC

XC

Estimated road plane

Inliers band at ±10cm of

plane hypothesis

Right camera

YW Z

W projection

4m

Camera

39m

50m

5m

Figure 3. YZ Projection and road plane estimation.


3.2. Road Plane Fitting

The outcome of the previous substage is a compact subset of points,ζ,where most of them belong to the road. As stated in the previous subsection,Φ (roll) is assumed to be zero, hence the projection is expected to contain adominant 2D line corresponding to the road together with noise coming fromthe objects in the scene.

The plane fitting stage consits of two steps. The first one is a 2D straightline parametrisation, which selects the dominant line corresponding to the road.It uses a RANSAC based [5] fitting applied over 2D barycenters intendedforremoving outlier cells. The second step computes plane parameters by meansof a least squares fitting over all 3D data points contained into inlier cells.

Initially, every selected cell is associated with a value that takes into accountthe amount of points mapped onto that position. This value will be consideredas a probability density function. The normalized probability density functionis defined as follows:pdfi = ni/N ; whereni represents the number of pointsmapped onto the celli andN represents the total amount of points contained inthe selected cells.

Next, a cumulative distribution function,Fj , is defined as:Fj =∑j

i=0 pdfi;If the values ofF are randomly sampled atn points, the application of theinverse functionF−1 to those points leads to a set ofn points that are adaptivelydistributed according topdfi.

3.2.1. Dominant 2D Straight Line Parametrisation

At the first step a RANSAC based approach is applied to find the largestset of cells that fit a straight line, within a user defined band. In order to speedup the process, a predefined threshold value for inliers/outliers detectionhasbeen defined (a band of±10 cm was enough for taking into account both datapoint accuracy and road planarity); an automatic threshold could be computedfor inliers/outliers detection, following robust estimation of standard deviationof residual errors [12]. However, it would increase CPU time since robust es-timation of standard deviation involves computationally expensive algorithms(e.g., sorting functions).


Repeat L times

(a) Draw a random subsample of 2 different barycenter points (P1, P2) ac-cording to the probability density functionpdfi using the above process;

(b) For this subsample, indexed byl (l = 1, ..., L), compute the straightline parameters(α, β)l;

(c) For this solution, compute the number of inliers among the entire set ofbarycenter points contained inζ, as mentioned above using a±10 cmmargin.

3.2.2. Road Plane Parametrisation

(a) From the previous 2D stright line parametrisation choose the solutionthat has the highest number of inlier;

(b) Compute(a, b, c) plane parameters by using the whole set of 3D pointscontained in the cells considered as inliers, instead of the correspondingbarycenters. To this end, the least squares fitting approach [15], whichminimizes the square residual error(1 − axC − byC − czC)2 is used;

(c) In case the number of inliers is smaller than 40% of the total amount ofpoints contained inζ (e.g., severe occlusion of the road by other vehi-cles), those plane parameters are discarded and the ones correspondingto the previous frame are used as the correct ones.

4. Road Scanning

Once the road is estimated, candidates are placed on the 3D surface andthen projected to the image plane to perform the 2D classification. The mostintuitive scanning scheme is to distribute windows all over the estimated planein a uniform way, i.e., in anx×nz grid, with nx sampling points in the road’sX axis andnz in theZ axis. Each sampling point on the road is used to definea set of scanning windows, to cover the different sizes of pedestrian,as will bedescribed later.


Let us defineZCmin = 5m as the minimum ground point seen from thecamera2, ZCmax = 50m as the furthest point, andτ = 100 the number ofavailable sampling positions along theZC axis of the road plane(a, b, c). Giventhat the points are evenly placed over the 3D plane, the corresponding imagerows can be computed by using the plane and projection equations. Hence,thesampled rows in the image are:

y = y0 +f

bz− f

c

b, (1)

wherez = ZCmin + iδZ ∀i ∈ {0, .., nz − 1}; δZ = (ZCmax − ZCmin)/nz isthe 3D sampling stride;(a, b, c) are the plane parameters;f is the camera focal;andy0 is they coordinate of the center point of the camera in the image. Thesame procedure is applied to the X xis, e.g., fromXCmin

toXCmaxwith thenx

sampling points. We refer to this scheme asUniform World Scanning.As can be appreciated in Fig. 4(a), this scheme has two main drawbacks:

it oversamples far positions (i.e., Z close toZCmax) and undersamples nearpositions (i.e., the sampling is too sparse when Z is close to the camera). Inorder to ammend these problems, it is clear that the sampling cannot rely onlyon the world but must be focused on the image. In fact, the sampling is aimedat extracting candidates in the 2D image. According to this, we compute theminimum and maximum image rows corresponding to theZ range:

yZCmax= y0 +

f

bZCmax− f

c

b, (2)

yZCmin= y0 +

f

bZCmin− f

c

b, (3)

and evenly place the sampling points between these two image rows using:

y = yZCmin + iδim ∀i ∈ {0..nz − 1} , (4)

whereδim = (yZCmin− yZCmax

)/nz. In this case, the correspondingz in theplane (later needed to compute the window size) is

z =f

c+ b(y − y0). (5)

2With a camera of6mm focal, oriented to the road avoiding to capture the hood, the first roadpoint seen is around 4 to 5 meters from the camera.


In the case ofX axis, the same procedure as in the first scheme can be used.This scheme is calledUniform Image Scanning. In this case, it is seen in Fig.4(b) that although the density of sampling points for the closerZC is appropiate,the farZC are undersampled, i.e., the space between sampling points is too big(see histogram of the same figure).

Figure 5 displays the sampling functions with respect to theZC scanningpositions and the imageY axis. TheUniform to Image, in dotted-dashed-blue,draws a linear function since the windows are evenly distributed over the avail-able rows. On the contrary, theUniform to Road, in dashed-red, takes the formof an hyperbola as a result of the perspective projection. The aforementionedover- and under-sampling in the top and bottom regions of this curve can bealso seen in this figure. Attending to the problems of these two approaches, wefinally propose the use of a non-uniform scheme that provides a more sensiblesampling, i.e., neither over- nor under-sampling the image or the world. Theidea is to sample the image with a curve in between the two previous schemes,and adjust the row-sampling according to our needs, i.e., mostly linear in thebottom region of the image (closeZ) and logarithmic-like for further regions(far Z), but avoiding over-sampling. In our case, we use a quadratic functionof the formy = ax2 + bx + c, constrained to pass through the intersectionpoints between the linear and hyperbolic curves and by a user defined point(iuser, yuser) between the two original functions. The curve parameters can befound by solving the following system of equations:

imax2 imax 1

imin2 imin 1

iuser2 iuser 1

abc

=

yZCmax

yZCmin

yuser

, (6)

whereimin = 0 andimax = nz − 1. For example, in the non-uniform curvein Fig. 5 (solid-black line),yuser = imin + (imax − imin)×κ and iuser =imin + (imax − imin)×λ, whereκ = 0.6 andλ = 0.25. For theXC axis wefollow the same procedure as with the other schemes. The resulting scanning,callednon-uniform scanning, can be seen in Fig. 4(c).

Once we have the set of 3D windows on the road, they are used to computethe corresponding 2D windows to be classified. We assume a pedestrian tobe h = 1.70m high, with an standard deviationσ = 0.2m. In the case ofbody width, the variability is much bigger than height, so a width margin isused to adjust most of human proportions and also leave some space for theextremities. Hence, the width is defined as a ratio of the height, specifically 1/2.


Ima

ge

Ro

ws

280 300 320 340 360 380 400 420 440 4600

1

2

3

4

5

Tim

es S

am

ple

d

Sampled Image Rows

x

z

World

Over-sampling

Under-sampling

(a) Uniform Road ScanningIm

ag

eR

ow

s

280 300 320 340 360 380 400 420 440 4600

1

2

3

4

5

Tim

es S

am

ple

d

Sampled Image Rows

x

z

World

Under-sampling

(b) Uniform Image Scanning

280 300 320 340 360 380 400 420 440 4600

1

2

3

4

5

Tim

es S

am

ple

d

Sampled Image Rows

Ima

ge

Ro

ws

x

z

World

(c) Non-Uniform Scanning

Figure 4. The three different scanning schemes. Right column shows thescan-ning rows using the different schemes and also a representation of the scan overthe plane. In order to enhance the figure visualization just50% of the lines areshown. The histograms of sampled image rows are shown on the left column;under- and over-sampling problems can be seen.


Figure 5. Scanning functions. A non-uniform road scanning with parametersκ = 0.6 andλ = 0.25 is between the uniform to road and to image curves,hence achieving a more sensible scan.

For example, the mean pedestrian window sizes1.70 × 0.85m, independentlyof the extra-margin taken by the classifier3.

5. Candidate Filtering

The final stage of the algorithm is aimed at discarding candidate windowsby making use of the stereo data (Fig. 6). The method starts by aligning thecamera coordinate system with the world coordinate system (see Fig. 2) withthe aim of compensating pitch angleΘ, computed in Sect. 3.. Assuming thatroll is set to zero, as described in the aforementioned section, the coordinates ofa given pointp(x,y,z), referred to the new coordinate system, are computed asfollows:

3Dalal et al. [3] demonstrate that adding some margin to the window (33% in their case)results in a performance improvement in their classifier.


pxR= px

pyR= cos(Θ)py − sin(Θ)pz

pzR= sin(Θ)py + cos(Θ)pz .

(7)

xx

xx

xx

xx

xx

x

z

x

x

y

3Dplane

x

xx

x

xx

xx

xx

xx

x

xx

x Discarded ROI

Preserved ROIy

ROIs over cells with few

accumulated points are discarded

Figure 6. Schematic illustration of the candidate filtering stage.

Then, rotated points located over the road4 are projected onto a uniformgrid GP in the fitted plane (Sect. 3.), where each cell has a size ofσ × σ.A given pointp(xR, yR, zR) votes into the cell(i, j), wherei = ⌊xR/σ⌋ andj = ⌊zR/σ⌋. The resulting mapGP is shown in Fig. 7(b). As can be seen,cells far away from the sensor tend to have few projected points. This is causedby two factors. First, the number of projected points decreases directly withthedistance, as a result of perspective projection. Second, the uncertainty of stereoreconstruction also increases with distance, thus the points of an ideal verticaland planar object would spread wider intoGP as the distance of these pointsincreases. In order to amend this problem, the number of points projected ontoeach cell inGP are reweighted and redistributed. The reweighting function is

GRW (i, j) = jσGP (i, j) , (8)

wherejσ corresponds to the real depth of the cell. The redistribution function

4Set of points placed in a band from0 to 2m over the road plane, assuming that this is themaximum height of a pedestrian.


consists in propagating the value ofGRW to its neighbours as follows:

G(i, j) =

i+η/2∑

s=i−η/2

j+η/2∑

t=j−η/2

GRW (s, t) , (9)

whereη is the stereo uncertainty at a given depth (in cells):η = uncertainty/σ.Uncertainty is computed as a function of disparity values:

uncertainty= f ·baselineµ

disparity2, (10)

where baseline is the baseline of the stereo pair in meters,f is the focal lengthin pixels andµ is the correlation accuracy of the stereo. The resulting mapG,after reweighting and redistribution processes, is illustrated in Fig. 7(c). Thefiltering consists in discarding the candidate windows that are over cells withless thanχ points, which is set experimentally. In our implementation, thisparameter is low in order to fulfill the conservative criterion mentioned in theintroduction, i.e., in this early system module false positives are preferred thanfalse negatives.

zx

(1)

(2)

(3)

(1)

(2)

(3)

(4)

(5)

(b) (c)

(1) (2)

(3)

zx

(1)

(2)

(3)

(4)(5)

(a)

Figure 7. Probability map of vertical objects on the road plane.(a) Originalframe. (b) Raw projectionGP . (c) Reweighted and redistributed vertical pro-jection map of the frame 3D points.

6. Experimental Results

The evaluation of the algorithm has been made using data taken from an on-board stereo rig (Bumblebee from Point Grey,http://www.ptgrey.com,Fig. 8). The stereo pair has a baseline of0.12m and each camera has a focal of


6mm and provides a resolution of640×480 pixel (the figures in the paper showthe right sensor image). The HFOV is43◦ and the VFOV is33◦, which allowsto detect pedestrians at a minimum of5m, and the camera reconstruction soft-ware provides 3D information until50m, which coincides with the parametersdescribed in Sect. 3..

As introduced in Sect. 1., one of the most used candidate generation meth-ods is sliding window. Although this method does not perform an explicit fore-ground segmentation, which is our motivation, it is useful as a reference toevaluate the benefits of our proposal. Let us say that we must detect pedestri-ans up to50m, which measure around12 × 24 pixels (of course the size willslightly differ depending on the focal and the size of the sensor pixels). On theother hand, the nearest pedestrian fully seen, at5m, is about140 × 280 pixels.Hence, a regular exhaustive scan algorithm must place windows of the scalesbetween these two distances at all the possible positions. If a scale variationisassumed to be 1.2 and the position stride is4 pixels, the number of windows isover100 000. However, smaller windows need a smaller stride between them,so the number can range between from200 000 to 400 000.

Figure 8. Stereo pair used in our acquisition system.

We have selected 50 frames taken from urban scenarios with the afore-mentioned stereo camera and applied the proposed algorithm. The parame-ters for the road surface estimation areL = 100 and ς = 0.68. In the caseof the scanning, we have used the non-uniform scheme withτ = 90 sam-pling points,κ = 0.5 andλ = 0.25. The scanning in theXC axis is madein XC = {−10, .., 10}m with a stride of0.075m. For each selected win-dow, 10 different sizes are tested (the smallest0.75 × 1.5m and the biggest0.95 × 1.8m). The algorithm selects about50 000 windows, which is a reduc-


Figure 9. Experimental results. The left column shows the original real urbanframes in which the proposed algorithm is applied. The middle column corre-sponds to the final windows after the filtering step. The right column shows thenumber of windows generated after the scanning (Gen) and after the filtering(Final). In order to enhance the visualization the different scales tested for eachsampling point are not shown, so just one candidate per point was drawn.


tion of about75 − 90% with respect to the sliding window, depending on thestride of this latter. Then, we apply the filtering stage with a cell size ofσ = 0.2andχ = 2000, reducing again a90% the number of candidates. This representsa reduction of97 − 99% compared to the sliding window. Figure 9 illustratesthe results in six of the frames used to test the algorithm. As can be seen, thepedestrians in the scenario are correctly selected as candidates, while other free-space areas are discarded to be classified. In addition, attending to the results,the number of false negatives is marginal, which is a key factor for the wholesytem performance.

7. Conclusions

We have presented a three-stage candidate generation algorithm to be usedin the foreground segmentation module of a PPS. The stages consist of roadsurface estimation, road scanning and candidate filtering. Experimental resultsdemonstrate that the number of candidates to be sent to the classifier can be re-duced by a97 − 99% compared to the typical sliding window approach, whileminimizing the number of false negatives to around0%. Future work will befocused on the research of algorithms to fuse the cues used to select the candi-dates, which can potentially improve the proposed pipeline process.

Acknowledgements

The authors would like to thank Mohammad Rouhani for his ideas with theroad scanning section. This work was supported by the Spanish Ministry ofEducation and Science under project TRA2007-62526/AUT and research pro-gramme Consolider Ingenio 2010: MIPRCV (CSD200700018); and CatalanGovernment under project CTP 2008 ITT 00001. David Geronimo was sup-ported by Spanish Ministry of Education and Science and European Social Fundgrant BES-2005-8864.

References

[1] M. Bertozzi, A. Broggi, R. Chapuis, F. Chausse, A. Fascioli, andA. Tibaldi. Shape–based pedestrian detection and localization. InProc. ofthe IEEE International Conference on Intelligent Transportation Systemspages 328–333, Shangai, China, 2003.


[2] C.-Y. Chan and F. Bu. Literature review of pedestrian detection technolo-gies and sensor survey. Technical report, Institute of TransportationStud-ies, Uni. of California at Berkeley, 2005.

[3] N. Dalal and B. Triggs. Histograms of oriented gradients for human de-tection. InProc. of the IEEE Conference on Computer Vision and PatternRecognition, volume 1, pages 886–893, San Diego, CA, USA, 2005.

[4] Y. Fang, K. Yamada, Y. Ninomiya, B. Horn, and I. Masaki. A shape-independent method for pedestrian detection with far-infrared images.IEEE Trans. on Vehicular Technology, 53(6):1679–1697, 2004.

[5] M. Fischler and R. Bolles. Random sample consensus: A paradigm formodel fitting with applications to image analysis and automated cartogra-phy. Graphics and Image Processing, 24(6):381–395, June 1981.

[6] United Nations Economic Commission for Europe. Statistics of road traf-fic accidents in Europe and North America, 2005.

[7] D.M. Gavrila, J. Giebel, and S. Munder. Vision–based pedestrian detec-tion: The PROTECTOR system. InProc. of the IEEE Intelligent VehiclesSymposium, pages 13–18, Parma, Italy, 2004.

[8] D. Geronimo, A. Lopez, A.D. Sappa, and T. Graf. Survey of pedestriandetection for advanced driver assistance systems. InIEEE Transactions onPattern Analysis and Machine Intelligence (in press), 2009.

[9] R. Labayrade and D. Aubert. A single framework for vehicle roll, pitch,yaw estimation and obstacles detection by stereovision. InProc. of theIEEE Intelligent Vehicles Symposium, pages 31–36, Columbus, OH, USA,June 2003.

[10] C. Papageorgiou and T. Poggio. A trainable system for object detection.International Journal on Computer Vision, 38(1):15–33, 2000.

[11] M. Peden, R. Scurfield, D. Sleet, D. Mohan, A.A. Hyder, E. Jarawan, andC. Mathers.World Report on road traffic injury prevention. World HealthOrganization, Geneva, Switzerland, 2004.

[12] P. Rousseeuw and A. Leroy.Robust Regression and Outlier Detection.John Wiley & Sons, New York, 1987.


[13] A.D. Sappa, F. Dornaika, D. Ponsa, D. Geronimo, and A. Lopez. Anefficient approach to onboard stereo vision system pose estimation.IEEETrans. on Intelligent Transportation Systems, 9(3):476–490, 2008.

[14] P. Viola and M. Jones. Rapid object detection using a boosted cascade ofsimple features. InProc. of the IEEE Conference on Computer Vision andPattern Recognition, pages 511–518, Kauai, HI, USA, 2001.

[15] C. Wang, H. Tanahashi, H. Hirayu, Y. Niwa, and K. Yamamoto. Compar-ison of local plane fitting methods for range data. InProc. of the IEEEConference on Computer Vision and Pattern Recognition, pages 663–669,Kauai Marriot, HW, USA, December 2001.

In: Binocular VisionEditors: J. McCoun et al, pp. 209-246

ISBN 978-1-60876-547-8c© 2010 Nova Science Publishers, Inc.

Chapter 10

DEVELOPMENT OF SACCADE CONTROL

Burkhart FischerUniv. of Freiburg, Optomotor Lab., Freiburg, Germany

Abstract

This chapter describes the development of eye movement control. Wewill consider, however, only those aspects of eye movementsthat are im-portant for reading: stability of fixation and control of saccades (fast eyemovements from one object of interest to another). The saccadic reflexand the control of saccades by voluntary conscious decisionand their rolein the optomotor cycle will be explained on the basis of the reaction timesand neurophysiological evidence. The diagnostic methods used in thenext part of the book will be explained in this chapter. The age curves ofthe different variables show that the development of the voluntary com-ponent of saccade control lasts until adulthood.

1. Introduction

Saccades are fast eye movements. We make them all the time, 3 to 5 in asecond. Due to these saccadic eye movements the brain receives 3 to 5 newpictures from the retina in each second. Without these ongoing sequencesofsaccades we would not see very much, because of the functional and anatomicalstructure of the retina: it contains in the middle a small area, called fovea, wherethe receptor cells and the other cells in the retinal layers are densely packed. Itis only this small part of the retina which allows to see sharp images. What wewant to see in detail and what we want to identify as an object or any other small

210 Burkhart Fischer

visual pattern, e.g. a letter, must be inspected by foveal vision. The solution forthis biological demand are sequences of rapid relatively small eye movements(saccades). The time periods between saccades are 150 to 300 ms long. Theyare often called ”fixations”.

Usually in everyday life these saccades are made automatically, i.e. we donot have to ”think” about them and we do not have to generate each of them by aconscious decision. But, by our own decision, we can also stop the sequence andfixate a certain small object for longer periods of times. We can also activelyandvoluntarily move our eyes from one place of interest to another. The situationis quite similar to our breathing: it works by itself but we can control it alsovoluntarily.

We are not aware of these saccades, they remain unconscious and – mostimportantly – we do not see the jumps of the retinal image. Somehow, the vi-sual system cooperates with the saccade control centres in a perfect way to dif-ferentiate between self-induced movements of the retinal image and externallygenerated movements.

Only under certain somewhat artificial conditions we can see our saccades.An example is shown by Fig. 1.

The figure shows a modification of the well known Hermann grid. As longas we look around across on this figure, we see black dots appearing sponta-neously at the crossing lines between the black squares. The picture looks likescintillations. We notice, that there are no such black dots. To avoid this illusoryblack blinks, we just have to decide to stop making saccades. The reader maytry be her/himself. Pick one of the white dots and maintain fixation at it. Aslong as one can prevent saccades, the black dots remain absent. Each saccadeoccurring spontaneously will create the illusion of dots again.

One can also see illusory movements, which are related to eye movements.The Fig. 2 shows an example. The movements disappear when we stop to makeeye movements.

An example of a geometric illusion allows also to become aware of onesown saccadic eye movements. The Fig. 3 shows the Z-illusion.

One can see that the prolongations of the short ends of the line (upper leftand lower right) will not meet the corners at the lower left and upper right.Ifone succeeds to prevent all saccades for some seconds (up to 10, which is a longtime in this context) one will see that the lines meet the corners as they do inreality. The reader may convince her/himself by using a ruler.

As long as we do not take into account the fact that vision needs saccades,

Development of Saccade Control 211

Figure 1. Scintillations of the Herman grid due to eye movements. As long aswe look around across this pattern, we see black dots jumping around. When-ever we try to look at one of them, there is no black spot. When fixating theeyes at one white spot for a few seconds black spots are no longer seen. Withstill longer fixation most of the white dots disappear also.

we will not understand the visual system. Any theory of vision which makescorrect predictions but does not include the fast movements of the eyes canhardly be considered a valid theory.

It is interesting to note, that most visual psychophysical experiments requirethe subject to fixate one spot of light while being examined on their visual expe-rience with another stimulus a distance away. The reason for this methodologi-cal detail is the sharply decreasing visual acuity from the centre to the peripheryof the visual field. Unfortunately, when it came to visual illusions, fixation wasnot longer required when testing the stability of the illusory impression. Theresult was, that the instability of geometrical illusions, shown in the examplesabove, remained undiscovered until recently [Fischer et al. 2003].

In particular, when we talk about reading, eye movements are one key forthe


Figure 2. The horizontal movements that one sees when looking at this figureare obviously illusory, because nothing moves in this figure. The illusion disap-pears, when we stop our eye movements by fixating the centre of the figure.

understanding of the reading process, which allows us to compose wordsfromletters or from syllables. We therefore have to consider the saccade system,before we may consider its role in reading. The significance of saccadic eyemovements in reading was emphasized also by a reader model resting on thebasis of experiments, where eye movements were measured while the subjectswere reading text, which was manipulated in several physical and linguisticways [Reichle et al. 2003].

In the following sections we will consider those parts of eye movement con-trol, that play an important role for reading. The other types of eye movements(vestibular ocular compensation, optokinetic nystagmus) will be neglected alltogether. However, we will consider the instability of fixation due to unwantedsaccades or to unwanted movements of the two eyes in different directions orwith different velocities. In principal, binocular vision is not needed at allforreading, but imperfections of binocular vision, which remain undetected, may


Figure 3. Z-Illusion shows the capital letter Z. The prolongations of the shortlines do not seem to hit the corners. The real geometric relations can be seenby using a ruler and draw the prolongations or they can be seen by fixatingtheeyes in the middle.

disturb vision and consequently reading may be difficult.

The Fig. 4 shows the most important anatomical connections that begin inthe retina and end up at the eye muscles.

There are hundreds of papers on eye movements in the literature. Saccadeshave always received special interest [Fischer, 1987]. Here we are interested infixation and in saccades. Today it is easy to measure eye movements in humanobserves. A sufficiently precise method uses infrared light reflection from theeyes. The data presented here are all collected by using this method. For thepurpose of clinical application a special instrument was developed. It is calledExpressEye and provides the infrared light source, the light sensitiveelements,the visual stimuli needed for basic tasks (with minilasers, see below), and theamplifiers. The instrument can deliver the raw eye position data trial by trialduring the experiment and detect saccades and provides a statistical analysis ofsaccades. The front view of the system is shown in Fig. 5.The method has beendescribed in detail elsewhere [Hartnegg and Fischer, 2002]. The rawdata canbe stored at the hard disc of a computer for further analysis to obtain differentvariables that characterize the performance of the tasks. These methodshavebeen described in detail [Fischer et al. 1997].


Parietal

FrontalTectal

VisualAssoc

ITC

FEF

PFC NC SN BS

MT

MST

LGN

III

III

Anatomical Pathways for Saccade Control

Figure 4. The figure shows a schematic diagram of the neural system of the con-trol of visually guided saccades and their connections. LGN = Lateral Genicu-late Nucleus; Assoc = Association Cortex; ITC = Infero-Temporal Cortex; FEF= Frontal Eye Field; PFC = Prefrontal Cortex; MT = Medio Temporal Cortex;MST = Medio-Superior-Temporral Cortex; NC = Nucleus Caudatus; SN =Sub-stantia Nigra; Tectal = Tectum = Superior Coilliculus; BS = Brain Stem.

2. Fixation and Fixation Stability

It may come as a surprise, that a section on eye movements starts by dealingwith fixation, i. e. with periods of no eye movements. It has been the problemover many years of eye movement research, that fixation was not considered atall as an important active function. The interest was in the moving and not inthe resting (fixating) eye. Only direct neurophysiological experiments [Munozand Wurtz, 1992] and thorough investigation of the reaction times of saccades[Mayfrank et al. 1986] provided the evidence, that fixation and saccade gener-ation are controlled in a mutual antagonistic way similar to the control of otherbody muscles. We will see, that we can observe movements of the eyes dur-ing periods where they were not supposed to move at all. It seems that therewas little doubt, that almost any subject can follow the instruction ”fixate” or


Figure 5. The front view of the Express Eye designed to measure eye move-ments. One sees the screws for the mechanical adjustment in all 3 dimensionsin front of each eye. Infrared light emitting diode and the two photocells arelocated behind and directed to the centre of the eye ball.

”do not move the eyes”. But this not the case: stability of fixation cannot al-ways be guaranteed by all subjects. This section deals with the results of thecorresponding analysis of eye movements.

2.1. Monocular Instability

As pointed out earlier, we have to consider two different aspects of distur-bances of fixation: the first aspect are unwanted (or intrusive) saccades. Theseare mostly small conjugate saccades that take the fovea from the fixation pointand back. This kind of disturbance is called a monocular instability, becausewhen it occurs one sees it in both eyes at exactly the same time and by the sameamount of saccade size. The disturbance remains if one closes one eye andtherefore it disturbs monocular vision and it does not disturb binocular vision.This is the reason why it is called a monocular instability. Below we will alsoexplain the binocular instability.

To measure the stability or instability of fixation due to unwanted saccades,we simply count these saccades during a short time period, when the subjectis instructed to fixate a small fixation point. Such a period repeatedly occursin both diagnostic tasks that are used for saccade analysis that are described in


section 3.2. on page 226.

The number of unwanted saccades counted during this task is used as ameasure of monocular fixation instability. For each trial this number is recordedand attributed to this trial. The mean value calculated over all trials will serveas a measure. The ideal value is zero for each individual trial and therefore theideally fixating subject will receive also zero as a mean value.

The Fig. 6 shows the mean values of the number of intrusive (unwanted)saccades per trial as a function of age. While children at the age of 7 produceone intrusive saccade every 2 or 3 trials, adults around 25 years of age produceone intrusive saccade every 10 trials. At higher ages the number of intrusivesaccades increases again. Of course, not every intrusive saccade leads to aninterruption of vision and therefore one can live with a number of them withoutproblems. But if the number of intrusive saccades is to high, visual problemsmay occur.

Figure 6. The curve shows the age development of the number of unwanted(intrusive) saccades per trial. The ideal value would be zero.

When we measure the monocular instability by detecting unwanted sac-cades, we should not forget, that there may be also another aspect of strengthor weakness of fixation, which cannot be detected by looking at the movementsof the eyes during periods of fixation, but rather be looking at reaction times ofsaccades that were required when the subject has to disengage from avisiblefixation point.


2.2. Binocular Instability

To understand binocular stability we have to remember that the two eyesmust be in register in order for the brain to ”see” only one image, even thougheach eye delivers its own image. This kind of convergence of the lines of sightof the two eyes at one object is achieved by the oculomotor system. We callit motor fusion. However, even with ideal motor fusion, the two images of thetwo eyes will be different, because they look at a single three dimensional objectfrom slightly different angles. The process of perceiving only one object in itsthree dimensions (stereo vision), is called perceptual fusion, or stereopsis.

When we talk about stereo vision (stereopsis) we mean fine stereopsis, i.e.single three-dimensional vision of objects. It is clear that we need both eyesfor this kind of stereopsis. However, we also have three-dimensional visionwith one eye only. The famous Necker cube shown in Fig. 7 is one of the bestknown examples. From the simple line drawing our brain constructs a three-dimensional object. Close one eye and the percept of the cube does not changeat all. This type of three-dimensional spatial vision does not need both eyes.The brain constructs a three-dimensional space within which we see objects.

In order to guarantee stable stereopsis, the two eyes must be brought inregister and they have to stay in register for some time. This means that theeyes are not supposed to move independently from each other during a periodof fixation of a small light spot. By recording the movements of both eyessimultaneously one has a chance to test the quality of the stability of the motoraspect of binocular vision.

The Fig. 8 illustrates the methods for determining an index of binocularstability.

Two trials from the same child are depicted. In the upper trial the left eyeshows stable fixation before and after the saccade. The right eye, however,converges after the saccade producing a period of non-zero relative velocity. Inthe lower case, both eyes produce instability after the saccades. The exampleshows, that the instability is sometimes produced by one eye only, or by botheyes simultaneously Often it is caused in some trials by one eye, and in othertrials by the other eye. Extreme dominance of one eye producing the instabilitywas rarely seen (see below).

In the example of Fig. 8 the index of binocular instability was 22%. Thismeans, that the two eyes were moving at different velocities during 22% of theanalysed time frame. To characterize a subject’s binocular stability as a whole,


Figure 7. The figure shows the famous Necker cube. Note that one seesathree-dimensional object even though the lines are not correctly connected. Thepercept of a three-dimensionl cube remains even when we close one eye. Note,that the lines do not really meet at the corners of the cube. Yet, our perceptionis stable against such disturbances and the impression of a cube is maintained.

the percent number of trials, in which this index exceeded 15% was used. Theideal observer will be assigned zero. The worst case would be assigned a valueof 100%.

The Fig. 9 shows the data of binocular instability of a single subject. Theupper left panel depicts the frequency of occurrence of the percentages of time,during which the eyes were not in register. The upper right panel depicts thedistribution of the relative velocity of the two eyes. The scatter plot in the lowerleft panel displays the correlation between these variables. Ideally all data pointsshould fall in the neighbourhood of zero. The lower right panel depictsthe timedevelopment of the variable percent time of limits by showing the single valuesas they were obtained trial by trial from trial 1 to trial 200. This panel allowstosee, whether or not fatigue has in influence on the binocular stability.

When the values of the binocular stability were compared among each other,several aspects became evident: (i) Within a single subject the values assignedto the trials can be very different. Almost perfect trials may be followed by trialswith long periods of instability. This means, that the subject was not completely


Figure 8. The figure illustrates the methods for determining an index of binoc-ular stability by analysing the relative velocity of the two eyes. Time runs hor-izontally. At the time of stimulus onset the subject was required to make asaccade. Up means right, down means left. Two trials from the same child aredepicted. In the upper case the left eye shows stable fixation before andafterthe saccade. The right eye, however, converges after the saccadeproducing aperiod of non-zero relative velocity. In the lower case, both eyes produce insta-bility after the saccades. For details see text.

unable to main the line of gaze for both eyes, but from time to time the eyesdrifted against each other. (ii) There was a large interindividual scatterof themean values even within a single age group. (iii) Even among the adult subjectslarge amounts of instability were observed. (iv) The test-retest reliability wasreduced by effects of fatigue or general awareness of the subjects.

The Fig. 10 shows the age development of the binocular instability usingdata from the prosaccade task with overlap conditions.

At the beginning of school large values but also small values were obtained.There was a clear tendency towards smaller values until the adult age. How-


Figure 9. The figure shows the data of binocular instability of a single subject.For details see text.

ever, the ideal value of zero is not reached at any age. This means thatsomehowsmall slow movements of the two eyes in different directions during short pe-riods of time are well tolerated by the visual system. In other words: there aresubjects with considerably instable binocular fusion not complaining about vi-sual problems. Maybe these subjects suppress the ”picture” of one eyeto avoiddouble vision all the time at the price of a loss of fine stereo vision, their visionis monocular. Because this does not create too much of a problem in everydaylife, subjects do not show up in the eye doctors praxis. Their binocular systemis never checked. This situation could be regarded as similar to the case of red-green colour blindness, which may remain undetected throughout life, becausethe subject has no reason to take tests of colour vision.


Figure 10. The percentage of trials in which the two eyes were moving relativeto each other (in more than 15% of the analysis time) is shown as a function ofage.

2.3. Eye Dominance in Binocular Instability

Since the two eyes send two different pictures to the brain, the picture of oneof the two eyes must be prevented from automatically reaching consciousness.This is true for most of the visual scene we see. Only those parts form onesingle picture in the brain that fall on corresponding points of the two retinae. Itis often forgotten, that this part covers only those objects, that are at about thesame distance from the eyes as the object, that we are just fixating with botheyes.

Because of the necessity to suppress the information from one eye most ofthe time, it has been speculated, that each subject selects one eye as the dominanteye (similar to the selection of one hand as the dominant hand). If, however, theimage of one eye is permanently suppressed, fine stereopsis is not possible. Wecan easily see, that the images of both eyes are present in our visual system, butusually, we do not perceive both of them. Fixating a near point and attendingtoan object further away leads to double vision of the object. The reader maytryby her/himself using the thumb of one hand as a near point and the thumb of theother hand as a far point.

The Fig. 11 shows the distribution of the differences between the right eyevalues and the left eye values of the index of binocular instability. The meanvalue is not significantly different from zero. But one sees that in a fewcases


clear dominances are obtained (8 subjects scored values far to the left, 3 far tothe right).

Figure 11. The distribution of the differences between the right eye and the lefteye values of binocular instability.

2.4. Independence of Mono- and Bino-Fixation Instability

In principle, the two types of instability may have the same reason: aweak fixation system allows all kinds of unwanted eye movement, includingunwanted saccades and unwanted drifts of one or both eyes. In this case oneshould see high correlations between the two variables describing these typesof instability. Because both variable depend on age, we analyse the data withinrestricted age group.

The Fig. 12 shows the scatter plots of the binocular versus the monocularinstability for two age groups. The correlation coefficients were only 0.22 forthe younger subjects (left side) and 0.21 for the older group (right side). Bothcorrelations failed to reach a significance level of 1%. This means that theproperties assessed by the two measures of fixation instability are independentfrom each other and different in nature. When we look at the data of dyslexicchildren, we will have many more data, that will support the independence ofthese two aspects of fixation instability. Also, we will see later, that a monoculartraining improves the binocular instability but not the monocular instability.


Figure 12. Scatter plot of the binocular versus the monocular instability ob-tained from overlap prosaccade trials. The left panels depict the data from sub-jects between 7 and 17 years of age (N= 129), the right panel shows thedatafrom older subjects 22 to 45 years of age (N=97). No correlations can be seen.

3. Development of Saccade Control

In the last section we have considered the stability of fixation as an impor-tant condition for perfect vision. Earlier, we have mentioned, that saccades arenecessary for vision. This might sound as a contradiction. The real require-ment is, that one should be able to generate sequences of saccades andfixationswithout an intrusion of unwanted saccades and without loosing the convergenceof the two eyes when they are in register for a given distance. Therefore, bothcomponents of gaze control should function normally. This section will showthat saccade control has to be subdivided into subfunctions describedby differ-ent variables. We have to find out first, what these subfunctions are and howthey can be assessed.

3.1. The Optomotor Cycle and the Components of Saccade Control

The control of saccades has been investigated for about 40 years and stillwe do not understand the system completely. Specialization of visual scientistsand oculomotorists has prevented that the two fields so closely related have beeninvestigated by corresponding combined research projects for a long time.Theoculomotor research groups were interested in the eye movement as a move-


ment. Their interest begins when the eyes begin to move and it stops when theeyes stop to move. The visual groups on the other hand, were interested intimeperiods when the eyes do not move. They required their subjects to fixate asmall spot, while being tested for their visual functions. Only when the interestwas concentrated on the time just before the movements, there was a chance tolearn more about the coordination of saccades and visual processes.

The time before a saccade is easily defined by the reaction time: one asksa subject to maintain fixation at one spot of light straight ahead and to make afast eye movement to an other spot of light, as soon as it appeared. Under theseconditions the reaction time is in the order of 200 ms. This is a value, whichone can find in a student handbook.

However, there were several problems. The first was: why is this time solong? While this was a question all from the beginning there was no answeruntil 1983/84, when the express saccade was discovered in monkeys [Fischerand Boch, 1983] and in human observers [Fischer and Ramsperger, 1984]. Theexpress saccades is the reflex movement to a suddenly presented light stimulusafter an extremely short reaction time (70-80 ms in monkeys and 100-120 ms inhuman observers). The reflex needs an intact superior colliculus [Schiller et al.1987].

The Fig. 17 shows in its lower part a distribution of saccadic reaction times.It exhibits 3 modes: one at about 100 ms, the next at about 150 ms, and thethird at about 200 ms. It was evident from these observation, that therewas notonly one reaction time with a (large and unexplained) scatter. Rather the reac-tion time spectrum indicated that there must be at least 3 different presaccadicprocesses that determine the beginning of a saccade, each taking its own timein a serial way. Depending on how many of the presaccadic processes are com-pleted already before the occurrence of the target stimulus, the reaction timecantake one out of three values, each with a certain amount of scatter [Fischer et al.1995].

It became clear that the shortest reaction time was 100 ms (not 200 ms) andthis was much easier to explain by the time the nerve impulses needed to begenerated in the retina (20 ms), to travel to the cortex (10 ms), to the centres inthe brain stem (10 ms) and finally to the eye muscles (15 ms). Another 5 mselapse before the eye begins to move. A much shorter time remains, that was at-tributed to a central computation time to find the correct size of the saccade to beprogrammed. One has to know at this point, that saccades are pre-programmedmovements: during the last 80 ms before a saccade actually starts, one cannot


change anything anymore.The next problem was: what is it that keeps the eyes from producing sac-

cades all the time? Or, the other way around: what is it that enables us to fixatean object on purpose? The answer came from observations of cells thatwereactive during time periods of no eye movements and that were inhibited, whensaccades were made [Munoz and Wurtz, 1993]. What could have beenfoundmuch earlier, became clear only after the neuroscientists began to think in verysmall steps: each process, that we experience as one unique action, must beeventually subdivided into a number of sub-processes. It became clearthat thebreak of fixation and/or allocated attention was a necessary step before asac-cade can be generated. There quite a number of single papers contributing to thesolution of the related problems. Most of them have been summarized and dis-cussed earlier [Fischer and Weber, 1993]. Most important for the understandingof the relation between saccades and cognitive processes is the finding that thereis a component in saccade control that relies on an intact frontal lobe [Guittonet al. 1985]

From all these consideration became clear, that sequences of fixations andreflexes form the basis of natural vision.

Cognitive ProcessesAttentionDecision

Fixation

Stop Go

Reflex

Figure 13. The figure shows the functional principle of the cooperation of the 3components of eye movement control.


The Fig. 13 shows a scheme, which summarizes and takes into account thedifferent findings: the stop-function by fixation alternates with the reflex,thego-function. These two together built up a stop-and-go traffic of fixations andsaccades. What remains open, was the question of how it was possible to inter-rupt this automatic cycling. The answer came from observations of the frontallobe functions: patients who lost parts of their frontal lobe at one side, were un-able to suppress the reflex in a simple tasks, called the antisaccade task [Guittonet al. 1985]. This task requires the subject to make a saccade to one side,whenthe stimulus is presented at the opposite side. The task became very popularduring recent years, but it was used already many years ago [Hallet, 1978].

3.2. Methods and Definition of Variables

The fundamental aspects of saccade control as described by the optomotorcycle have been discovered by using two fundamental tasks, which are surpris-ingly similar but give insight into different aspects of the optomotor cycle. Theyhave been used to quantitatively measure the state of the system of saccadecon-trol. We describe these methods and define the variables first. Then we will seesome of the results obtained by the these methods.

The two tasks are called the overlap prosaccade task and the gap antisaccadetask. The words pro and anti in their names refer to the instructions that the sub-ject is given. The words overlap and gap describe the timing of the presentationof the fixation point. The Fig. 14 shows the sequence of frames for gap and foroverlap conditions.

In both tasks a small light stimulus is shown, which the subjects is asked tofixate. This stimulus is called the fixation point.

In overlap trials a new stimulus is added left or right t of the fixation point.The subjects is asked to make a saccade to this new stimulus, the target stim-ulus, as soon as it appears. Both, the fixation point and the target are visiblethroughout the rest of the trial: they overlap in time. This overlap condition andthe task to look towards (’pro’) the stimulus explain the complete name of thetask: overlap prosaccade task.

The gap condition differs from the overlap condition in only one aspect:the fixation point is extinguished 200 ms before the target stimulus is presented.The time span from extinguishing the fixation point to the onset of the new targetstimulus is called gap. In addition to this physical difference the instruction forthe subject is also changed: the subject is required to make a saccade in the


direction opposite (’anti’) of the stimulus: when the stimulus appears left, thesubject shall look to the right and vice versa. Therefore the complete nameofthis task is: gap antisaccade task.

The prosaccade task with overlap condition allows to find too slow or toofast reaction times and to measure their scatter. The presence of a fixation pointshould prevent the occurrence of too many reflexes, the appearanceof a newstimulus should allow a timely generation of a saccadic eye movement.

The antisaccade task with gap condition challenges the fixation system tomaintain fixation and the ability to generate a saccade against the direction ofthe reflex.

+

Gap

Overlap Prosaccade Task Gap Antisaccade Task

+

+

Figure 14. The figure shows the sequence of frames for overlap and for gap con-ditions. The horizontal arrows indicate, in which direction the saccade shouldbe made: to the stimulus in the prosaccade task, in the direction opposite to thestimulus, in the antisaccade task.

Now we can define variables to describe the state of the saccade controlsystem. First of all, one has to keep in mind, that these variables may be dif-ferent for left versus right stimulation. Because left/right directed saccades aregenerated by the right/left hemisphere, side differences should not be much ofa surprise. However, for the general definition of the variables to be used inthe diagnosis, the side differences do not need to be considered at this point.The Fig. 15illustrates the definition of the variables described below. Time runsfrom left to right. The stimulus is indicated by the thick black line. Because itspresentation is identical in both conditions, it is drawn only once in the middle.The fixation point is shown by the thin black line. In the case of an overlaptrial, the fixation remains visible, in the case of a gap trial the fixation point isextinguished 200 ms before. In addition the figure shows schematically traces


of eye movements, which help to understand the definition of the variables. Theupper case shows a trace from an overlap trial. Usually one saccade is made andit contributes its reaction time, SRT. Below one sees two examples of traces.One shows a correct antisaccade, which contributes its reaction time, ANTI-SRT. The other trace depicts a trial with a direction error that was corrected alittle later. It contributes the reaction time of the error, Pro-SRT and the correc-tion time, CRT (in case the error was corrected). While these variables can betaken from every single trail, some other variables are determined by the analy-sis of the complete set of 200 trials: the percentage of express saccadesfrom alloverlap trials, the percentage of errors from all gap trials and the percentage ofcorrections among the errors.

Stimulus and Eye Movement Events

PRO - OVERLAP

ANTI - GAP

Fixationpoint

Fixationpoint

Stimulus

time

eyepositionSRT

% express

% errorsPro-SRT CRT % corrections

Anti-SRT

Gap

Figure 15. The schematic drawing of eye movement traces illustrates the defini-tion of the different variables describing the performance of the prosaccade taskwith overlap conditions and the antisaccade task with gap conditions.

List of variables:From the overlap prosaccade task the following mean values and the scatter

are used:

• SRT: saccadic reaction time in ms from the onset of the target to the be-ginning of the saccade

• % expr: the percentage of express saccades, i.e. reaction times between


80 and 130 ms.

From the gap antisaccade task:

• A-SRT: the reaction time of the correct antisaccades

• Pro-SRT: the reaction time of the errors

• CRT: the correction time

• %err: the percentage of errors

• %corr: the percentage of corrections among the errors

Note that the percentage of trials, in which the subject missed to reach theopposite side within the time limit in the trial (700 ms from stimulus presenta-tion) can be calculated as

pmis = perr · (100 − pcorr)/100

The latter variable combines errors rate and correction rate.

3.3. Prosaccades and Reflexes

The optomotor system has a number of reflexes for automatic reactions todifferent physical stimulations. The best known reflex is the vestibular-ocularreflex, which compensates head or body movements to stabilize the direction ofgaze on a fixated object: the eyes move smoothly in the direction opposite to thehead movement in order to keep the currently fixated object in the fovea. Sim-ilarly, is it possible to stabilize the image of a moving object by the optokineticreflex. Both reflexes have little or nothing to do with reading.

The saccadic reflex is a reaction of the eyes to a suddenly appearing lightstimulus. It was discovered only in 1983/84 by analysing the reaction times ofthe saccades in a situation, where the fixation point was extinguished shortly(200 ms gap) before a new target stimulus was presented. It was known atthat time that under these gap conditions the reaction times were considerablyshorter as compared to those obtained under overlap conditions [Saslow,1967].When the gap experiment of Saslow was repeated years later, it became evidentthat among the well know reactions around 150 ms after target onset therewas a


separate group of extremely short reactions at about 100 ms, the express saccade[Fischer and Ramsperger, 1984].

The Fig. 16 shows the distribution of reaction times from a single subject.One clearly sees two peaks. The first peak consists of express saccades, thesecond represents the fast regular saccades.

Figure 16. The figure shows the distributions of reaction times from a singlesubject. One clearly sees two peaks. The first represents the expresssaccades,the second the fast regular saccades.

The Fig. 17 shows the difference in the distributions of reaction times whengap and overlap trials were used. The separate peaks in the distributions in-dicate, that saccades can be generated at distinctly different reaction times de-pending on the preparatory processes between target onset and the beginningof the saccade. In the gap condition there is time during the gap to completeone or even two pre-saccadic processes. Therefore the chances of generation ofexpress saccades is high. In the overlap condition it is the target stimulus, whichtriggers the preparatory processes and therefore the chances of express saccadeare low.

If one leaves the fixation point visible throughout the trial (overlap condi-tion) the reaction times are considerably longer, even longer as compared withthe gap=0 condition (not shown here).

The consistent shortening of reaction time by introducing a temporal gapbetween fixation point offset and target onset was surprising, because the role offixation and of the fixation point as a visual stimulus was unknown. The effect iscalled the gap-effect and has been investigated in numerous studies of different


Figure 17. The figure shows the difference in the distributions of reactiontimeswhen gap and overlap trials were used. Note the separate peaks in the distribu-tions.

research groups all around the world since 1984. The effect of the gap on thereaction time is strongest if the gap lasts approximately 200 Milliseconds. Anoverview and a list of publications can be found in an overview article [Fischerand Weber, 1993].

Today it is clear, that the main reason for the increase in reaction time un-der overlap conditions is due to an inhibitory actions of a separate subsystemin the control of eye movements, the fixation system. It is activated by a fovealstimulus, which is being used as a fixation point and it inhibits the subsystemwhich generates saccades. If this stimulus is removed early enough, the inhi-bition is removed by the time the target occurs and a saccade can be generatedimmediately, i.e. after the shortest possible reaction time.

Note, that the effect of the gap is not a general reduction of reaction times,


but rather the first peak is larger and the third peak is smaller or almost absent.As a result the mean value of the total distribution is reduced.

At this point, we do not have to go through the discussion of whether ornot directed visual attention also inhibits or des-inhibits the saccade system de-pending on whether attention is engaged or disengaged. This issue has beendiscussed extensively and still today different the arguments are not finally set-tled We only have to keep in mind, that the gap conditions enables the reflexmovements to a suddenly presented visual stimulus.

It is also important to remember, that there are subjects, who produce ex-press saccades under overlap conditions [Fischer et al. 1993]. We will encounterthese so called express saccade makers [Biscaldi et al. 1996] again, when weconsider the eye movements of dyslexic subjects.

3.4. Antisaccades: Voluntary Saccade Control

It is an everyday experience, that we can stop our saccades and thatwe candirect our centre of gaze to a selected object or location in space on our owndecision. These saccades are called voluntary saccades for obviousreasons. Allfrom the beginning it will not be a big surprise to learn, that there are also differ-ent neural subsystems, that generate the automatic saccades and the voluntarysaccades.

The investigation of voluntary saccades was introduced many years ago[Hallett, 1978], but the oculomotor research community did not pay attentionto it very much. Hallett instructed his subjects to make saccades to the sideopposite to a suddenly presented stimulus. These saccades were called antisac-cades.

An early observation of neurologists did not receive much attention either,but turned out to be very important. It was reported that patients, who lostaconsiderable part of their frontal lobe in one side only, were unable to generateantisaccades to the side of the lesion, while the generation of antisaccades totheopposite side remained intact [Guitton et al. 1985]. Meanwhile the antisaccadetask has become an almost popular ”instrument” for diagnosis in neurology andneuropsychology. Reviews have been published and can be consultedby theinterested reader [Everling and Fischer, 1998]; [Munoz and Everling, 2004].

The effect of changing the instruction from ” look to the stimulus, when itappears” to ”look away from from the stimulus (the anti-instruction)” can beseen in Fig. 18.


Figure 18. The figure shows the distribution of saccadic reaction times underoverlap (left) and under gap conditions (right). The lower panels show the datafrom those trials, in which the subjects made errors by looking first to the stim-ulus. Note, that with overlap conditions there are virtually no such errors,whilewith gap conditions a considerable number of errors were made.

The introduction of the gap leads to quite a number errors. Interestingly,subjects often failed to judge their performance: some claimed that they mademany errors, but did not make many, others claimed that they made few errors,but made quite many. This indicates, that we have little conscious knowledge ofwhat we do with our eyes. The processes preparing the saccades and their execu-tion remain mostly unconscious. When the variables obtained from an overlapprosaccade task and from a gap antisaccade task were analysed by a factor anal-ysis [Gezeck et al. 1997] it turned out, that there were only 2 factors. The firstfactor contained the variables that describe prosaccades, irrespective of whether


they were generated in the prosaccade task or as erros in the antisaccade task.The second factor contained the variables that described the performance of theantisaccade task. But there was one exception: the error rate loaded onboth fac-tors. The explanation of this result becomes evident, when we remember thatthe correct performance of the antisaccade task requires 2 steps: suppression ofthe prosaccades and generation of antisaccades. The error may be high for 2reasons: (i) the suppression is not strong enough, (ii) the subject hasdifficultiesin looking to the side where there is no target.

The details of this observations finally resulted in the decision to use the 2tasks described above in order to characterize the functional state of thesystemof saccade control. The procedure of the corresponding analysis ofthe raw eyemovement data have been described in great detail [Fischer et al. 1997]. Thetasks are illustrated in Fig. 14, the definition of the variables are illustrated byFig. 15. Today, there is already a special instrument and analysis system,whichallows to measure the eye movements and to assess the variables, their meanvalues and their scatter. Test-retest reliability of saccade measures, especiallyalso for measures of antisaccade task performance are available [Klein and Fis-cher, 2005]

The data obtained from these two tasks are shown in Fig. 19 separately forleft and right stimulation. The data were combined from 8 subjects in the agerange of 14 to 17 years. The distributions show most of the important aspectsof the data, which are not as clear in the data of single subjects. The 3 peaks areseen in both left and right distributions obtained from the prosaccade taskwithoverlap conditions (upper panels). But they are not quite identical: more expresssaccades are made to the right stimulus than to the left stimulus. The antisac-cades (lower panels) have longer reaction times and a structure with differentmodes is missing.

Earlier studies of antisaccade performance as summarized recently [Munozand Everling, 2004] analyse the reaction times in the antisaccade task and thepercentage of errors. Most studies, however, failed to analyse the reaction timeof the errors. They also neglected the percentage of corrective saccades andthe correction time. We will see below, that these variables provide importantinformation about the reasons, why errors were made [Fischer et al. 2000];[Fischer and Weber, 1992].

We therefore also show the distributions of the reaction times of the errorsand the distributions of the correction times of the same subjects as in Fig. 19.

Now we can further explain the data shown in Fig. 19 and in Fig. 20. The


Figure 19. The figure shows the distributions of reaction times of 8 subjectsperforming the prosaccade task with overlap conditions (upper panels) and theantisaccade task with gap conditions. Panels at the left and right show the datafor left and right stimulation, respectively.

subjects as a group made quite a number of express saccades in the overlapprosaccade task (upper panels of Fig. 19) indicating that their ability to suppresssaccades is limited. We can see that the errors in the gap antisaccade task (upperpanels of Fig. 20) contained also more than 50% express saccades. Theerrorrate is 35% at the left and 42% at the right side. Of these errors 87% and92% were corrected after very short correction times of 131 ms and 129 ms,respectively (lower panels of Fig. 20). This indicates that the subjects have noproblem of looking to the opposite side. They do reach the destination, but theyget there with a detour because they could not suppress the saccade to the target.Their errors were mostly due to a weakness of the fixation system.

This reminds us, that we have already 2 independent factors of instability of


Figure 20. The figure shows the reaction times of the errors (upper panels) andthe distributions of the correction times (lower panels) of the same subjects asin Fig. 19.

fixation: the intrusive saccades, and the binocular instability of slow movementsof the two eyes in different directions or with different velocities. Now a thirdaspect is added by the occurrence of express saccades and in particular, whenthey occur as errors in the antisaccade task. Fixation may also by weak, when itdoes not allow to suppress the errors in the antisaccade task.

3.5. The Age Curves of Saccade Control

After these considerations and definitions we can look at the age develop-ment of the different variables. The data presented here contain many moresubjects than in an earlier study, which has shown already the developmentofsaccade control with age increasing from 7 to 70 years [Fischer et al. 1997].


Fig. 21 begins with the age curves of the performance of prosaccades withoverlap conditions. The reaction times start with about 240 ms at the age of 7 to8 years. During the next 10 years the reaction times become shorter by about 50or 60 ms. From the age of 40 years one sees a gradual increase of the reactiontimes. At about 60 years they reach the level of the 7 year old children.

Figure 21. The diagrams show the age curves of the performance of prosaccadeswith overlap conditions. The left side depicts the age dependence of the reactiontimes, the right side shows the age dependence of the percentage of expresssaccades in the distributions. N=425.

One might expect that the occurrence of reflex-like movements (expresssaccades) is also a function of age, because the reflexes receive more corticalcontrol with increasing age. However, this general aspect of the developmentmay be seen much earlier in life, i.e. during the first year of life. Yet, there isstrong tendency of a reduction of the number of express saccades with increas-ing age from a mean value just below 15% to a mean value of about 5%. Thereare however, extreme cases of subjects producing quite many express saccades.The large scatter in the data is due to these subjects.

It has been stated, that percentages of express saccades above a limitof30% must be regarded as an exceptional weakness of the fixation system. Thecorresponding subjects are called express saccade makers [Biscaldiet al. 1996].An extreme case of an express saccade maker is shown in Fig. 22. In this subjectthe express saccades occur only to the right side.

Later in the book we will look at the percentage of express saccades amongthe prosaccades generated under overlap conditions, because we want to be pre-


Figure 22. The figure shows the distributions of saccadic reaction times froma single subject, who performed the prosaccade task with overlap condition.Saccades to the left side are depicted by the left panel, those to the right side bythe right panel. Note the large peak of express saccade to the right as comparedwith no express saccades to the left.

pared for the diagnosis of saccade control in the following parts of the book,when large amounts of express saccades are made by single subjects of certainages.

The Fig. 23 shows the age curves for the variables that describe the perfor-mance of the antisaccade task. The reaction times of the correct antisaccadesare depicted by the upper left panel. The mean value of the youngest group atabout 340 ms is 100 ms is slower than that of their prosaccades. As in the caseof the prosaccades, a reduction of the reaction times is obtained within the next10 years. However, they are reduced by about 100 ms. When compared withthe prosaccades this reduction is two times as big.

The percentage of errors (middle left panel) reaches almost 80% for theyoungest group. This means that they are almost completely unable to do thetask in one step. The error rate decreases down to about 20%, stays atthis leveland increases after the age of about 40 years.

The bottom left panel depicts the correction rate. Out of the 80% errors,theyoungest group was able to correct the primary error in only 40% of cases. Thecorrection rate increases until the age of 20 to above 80%, stays at this level anddecreases again after the age of 50 years.

Combining the two measures of error production and correction results in


Figure 23. The figure shows the age development of the performance oftheantisaccade task with gap conditions. N=328. The period of the ”best” values isbetween 20 and 40 years of age.

the age curve of the percentage of uncorrected errors (misses) shown by thelower right panel of 23. The children of the youngest group reachedthe opposite


side in only half of the trials. During the following 10 years the rate of missesdrops down to almost zero. This indicates that the adult subjects in the agerange between 20 and 40 years produce 20% errors, but they correct almostall of them. During years after the age of 60 the subjects begin to have moredifficulties in correcting their increasing rate of errors.

Finally, we look at the reaction times of the errors shown by the upper rightpanel. The age curve reflects the curve for the reaction time of the prosaccadesgenerated in the overlap condition. However, all error reaction time were shorterby about the same amount of 50 ms over the complete range of ages covered.

3.6. Left – Right Asymmetries

The question of hemispheric specialisation is asked for almost any aspectof brain functions. In the case of saccade control it might be argued, that de-pending on the culture writing goes from left to right, right to left, or from toptobottom. Therefore we look at the possible asymmetries of the different variablesdescribing saccade control.

The differences between left and right variables did not show any system-atic age dependence, presumably because the age dependence for theright andthe left variables have the same development. Therefore we look at the totaldistribution of the difference values for all ages.

The Fig. 24 shows these distributions of differences for 6 variables.Theup-per left panels depicts the differences in the reaction time of the prosaccadeswith overlap conditions. The distribution looks rather symmetrical and in factthe deviation of the mean value is only 6 ms and not significantly different fromzero. However, this does not indicate that there are no asymmetries. It shows,that asymmetries occur about as often in favour of the right side as they occurin favour of the left side. The standard deviation of 30 ms to either side indi-cates that in 32% of the cases the reaction times differ by 30 ms or more. Thetendency is that reaction times are somewhat shorter for the right directed sac-cades as compared with the left directed saccades. This small differencemayberelated to the fact that the German language is written from left to right (all datain this book comes from native German speakers).

The upper right panel depicts the differences between the percentages ofexpress saccades made to the right and to the left. The mean value is -1.1% andnot significantly different from zero. But there is a tendency to more expresssaccades to right than to the left. The standard deviation if 12% indicating that


Figure 24. The figure shows the distributions of the left minus right differencesof 6 variables describing saccade control.

32% of the subjects produced more then 12% of their express saccades tooneside than to the other. Extreme cases can be seen within this relatively largegroup of normal subjects. An example can be seen in Fig. 22. The distributionof saccadic reaction times obtained with overlap conditions are shown for left


and right directed prosaccades. Almost all saccades to the left occur between130 ms and 170 ms. These are fast regular saccades. Most saccadesto theright occur between 85 ms and 140 ms. These are express saccades. The fig-ure demonstrates an extreme case of asymmetry of prosaccades. The reactiontimes of the correct antisaccades in the gap condition shows a similar result:one encounters quite a number of subjects with heavy asymmetries (32% withdifferences of more than 45 ms), but the mean value of 5 ms is statistically notsignificant from zero. The correction times exhibit even stronger asymmetries:in 32% of the subject the differences are larger than 55 ms. The percentage oferrors in the antisaccade task exhibit differences of more than 15% in 32%ofthe cases and the differences of the percentage of corrective saccades are largerthan 28% in 32% of the subjects. From the consideration of the asymmetriesin saccade control we can conclude that large asymmetries occur in quite manycases. Because the asymmetries in favour of the right or of the left side areabout the same in number as well as in size, the mean value of the distributiondoes not deviate significantly from zero.

3.7. Correlations and Independence

Large numbers of errors in the antisaccade task are often interpreted asaconsequence of a weak fixation system. This would imply that many intrusivesaccades should be observed in the overlap prosaccade task (poor mono fixationstability) along with many errors in the gap antisaccade task. We can look atthe possible correlation between these two measures. Fig. 25 shows the scatterplot of the data obtained from control subjects in the age range of 7 to 13 years.While the correlation coefficient indicates a positive significant correlation, theplot shows in detail, that the relation works only in one direction: High valuesof intrusive saccades occur along with high values of errors, but notvice versa:high values of errors may occur along with low or with high numbers of intrusivesaccades. In other words: even if a subject is able to suppress intrusive saccadeswhile fixating a small spot, he/she may not be able to suppress reflexive saccadesto a suddenly presented stimulus. But a subject, who is able to suppress thereflexive saccades, is also able to suppress intrusive saccades.

This means that the reason for many errors in the antisaccade task may bea weak fixation system, but other reasons also exits such that high errorsratesmay be produced even though the mono fixation stability was high.

The analysis of the relationship between errors, error correction, correction


Figure 25. Scatterplot of error rate in the gap antisaccade task and mono fixationinstability. High values of intrusive saccades occur along with high values oferrors, but not vice versa: high values of errors may occur along withlow orwith high numbers of instrusive saccades.

time and express saccades can also be used to learn more about fixation andits role in saccade control. Those, who produce many errors and many expresssaccades, correct their errors more often and after shorter correction times incomparison to subjects, who produce also many errors but relatively fewex-press saccades. They correct the errors not as often and the correction times arelonger. The details are described in the literature [Mokler and Fischer, 1999].

In conclusion from this section we can state, that saccade control has indeed3 main components: fixation (being weak or strong as indicated by expresssaccades), reflexive control, and voluntary control. These 3 components worktogether in the functional from of the optomotor cycle. The functioning of thecycle improves over the years from the age of 7 to adult age and has a strongtendency to deteriorate after the age of 40 years [Fischer et al. 1997].

References

Biscaldi, M; Fischer, B; Stuhr, V. (1996). Human express-saccade makers areimpaired at suppressing visually-evoked saccades.J Neurophysiol 76:199-214

Everling, S; Fischer, B. (1998). The antisaccade: a review of basic research


and clinical studies.Neuropsychologia 36: 885-899

Fischer, B. (1987). The preparation of visually guided saccades.Rev PhysiolBiochem Pharmacol 106: 1-35

Fischer, B; Biscaldi, M; Gezeck, S. (1997). On the development of voluntaryand reflexive components in human saccade generation.Brain-Res 754:285-297

Fischer, B; Boch, R. (1983). Saccadic eye movements after extremely shortreaction times in the monkey.Brain-Res 260: 21-26

Fischer, B; Breitmeyer, B. (1987). Mechanisms of visual attention revealed bysaccadic eye movements.Neuropsychologia 25: 73-83

Fischer, B; daPos, O; Strzel, F. (2003). Illusory illusions: The significance offixation on the perception of geometrical illusions.Perception 32: 1001-1008

Fischer, B; Hartnegg, K; Mokler, A. (2000). Dynamic visual perception ofdyslexic children.Perception 29: 523-530

Fischer, B; Ramsperger, E. (1984). Human express saccades: extremely shortreaction times of goal directed eye movements.Exp-Brain-Res 57: 191-195

Fischer, B; Gezeck, S; Huber, W. (1995). The three-loop-model: A neuralnetwork for the generation of saccadic reaction times.Biol Cybern 72:185-196

Fischer, B; Weber, H. (1992). Characteristics of ”anti” saccades in man. Exp-Brain-Res 89: 415-424

Fischer, B; Weber, H. (1993). Express Saccades and Visual Attention. Behav-ioral and Brain Sciences 16,3: 553-567

Fischer, B; Weber, H; Biscaldi, M; Aiple, F; Otto, P; Stuhr, V. (1993). Separatepopulations of visually guided saccades in humans: reaction times andamplitudes.Exp-Brain-Res 92: 528-541

Gezeck, S; Fischer, B; Timmer, J. (1997). Saccadic reaction times: a statisticalanalysis of multimodal distributions.Vision Res 37: 2119-2131


Guitton, D; Buchtel, HA; Douglas, RM. (1985). Frontal lobe lesions in mancause difficulties in suppressing reflexive glances and in generating goal-directed saccades.Exp-Brain-Res 58: 455-472

Hallet, PE. (1978). Primary and secondary saccades to goals defined by in-structions.Vision Res 18: 1279-1296

Hallett, P. (1978). Primary and secondary saccades to goals defined byinstruc-tions.Vision Res 18: 1279-1296

Hartnegg, K; Fischer, B. (2002). A turn-key transportable eye-tracking instru-ment for clinical assessment .Behavior, Research Methods, Instruments,& Computers 34: 625-629

Klein, C; Fischer, B. (2005). Instrumental and test-retest reliability of saccadicmeasures.Biological Psychology 68: 201-213

Mayfrank, L; Mobashery, M; Kimmig, H; Fischer, B. (1986). The role offixation and visual attention in the occurrence of express saccades in man.Eur Arch Psychiatry Neurol Sci 235: 269-275

Mokler, A; Fischer, B. (1999). The recognition and correction of involuntarysaccades in an antisaccade task.Exp Brain Res 125: 511-516

Munoz, DP; Wurtz, RH. (1992). Role of the rostral superior colliculus inactivevisual fixation and execution of express saccades.J-Neurophysiol 67:1000-1002

Munoz, DP; Wurtz, RH. (1993). Fixation cells in monkey superior colliculus.I. Characteristics of cell discharge.J Neurophysiol 70: 559-575

Munoz, DP; Everling, S. (2004). Look away: the anti-saccade task and thevoluntary control of eye movement.Nature Reviews/ Neuroscience 5:218-228

Reichle, ED; Rayner, K; Pollatsek, A. (2003). The E-Z-Reader modelof eye-movement control in reading: comparison to other models.Behavioraland Brain Sciences 26: 445-526

Saslow, MG. (1967). Latency for saccadic eye movement.J Opt Soc Am 57:1030-1033


Schiller, PH; Sandell, JH; Maunsell, JH. (1987). The effect of frontal eye fieldand superior colliculus lesions on saccadic latencies in the rhesus monkey.J Neurophysiol 57: 1033-1049


Short Commentary

OCULAR DOMINANCE

Jonathan S. PointerOptometric Research

4A Market Square, Higham Ferrers,Northamptonshire NN10 8BP, UK

The time has come to consider seriously what has hitherto possibly beenregarded as a somewhat narrow proposition; namely, that ocular dominance isbest understood by the behaviourally descriptive term ‘sighting preference’. Bythis is meant: The eye that is consciously or unconsciously selected for monoculartasks.

This definition aligns with the first description of the phenomenon recordedfour hundred years ago. Sighting dominance can be regarded as the ocularlaterality most analogous to decisions regarding limb choice, ie, undercircumstances when only one of a pair can be selected for use (eg, writing hand,ball-kicking foot). An individual’s sighting choice appears to be substantiallyconsistent within and between applicable tests, which latter are usually based onsimple motor tasks such as pointing, aiming or alignment. Such techniques haveproved to be reliable indicators of dominance across populations and within eithergender; the ocular laterality thus identified is apparently stable with advancingchronological age, and appears not to show familial traits.

In contrast, indications of ocular dominance from sensory tests, including thebinocular viewing of rivalrous stimuli or the recording of functional oculo-visual


asymmetries (often of visual acuity), have a tendency to show intra- and inter-testdisagreement. Theories of ocular dominance have developed and evolved throughthe twentieth century to acknowledge and accommodate these discrepancies, andto account for the not infrequent disagreement found in the individual between thelaterality result obtained using a sighting compared to a sensory test format. Manyof these explanations have been at best parsimonious and sometimes incontradiction of the prevailing knowledge of binocular vision, including that ofvisual suppression.

Sighting dominance is not reliably associated with limb preference or, forreasons of ocular neuro-anatomy, predicted by cortical laterality: a generalised(uniform) laterality preference of sensory organs and motor limbs in theindividual is unlikely.

Despite a burgeoning research output over recent decades, the identificationof a functional basis for ocular dominance continues to prove elusive: thephenomenon remains a ‘demonstrable habit’, adopted when a single eye must bechosen for a particular viewing task.

INDEX

A

abnormalities, 147, 152ACC, 190accessibility, 129accidents, 190, 207accommodation, 143, 144, 152, 157, 158, 160accuracy, viii, ix, 2, 4, 13, 22, 23, 31, 35, 37,

69, 81, 83, 99, 100, 103, 196, 203ACM, 59activation, 165, 168, 169, 172, 174, 176, 180,

182, 184, 185acute, 66adaptability, 74adaptation, 74, 146, 147adjustment, 2, 27, 58, 59, 88, 187, 215adult, 67, 73, 112, 123, 156, 219, 240, 243adulthood, xii, 209adults, xi, 73, 76, 78, 155, 191, 216Africa, 66age, xi, xii, 66, 67, 72, 74, 79, 144, 147, 148,

151, 155, 157, 200, 209, 216, 219, 220,221, 222, 223, 234, 236, 237, 238, 239,240, 242, 243, 247

aggregation, 130aggressiveness, 170aid, 64, 174Aircraft, 134, 136Airship, 132albinism, 109algorithm, xi, xii, 3, 4, 6, 16, 19, 21, 24, 26,

27, 29, 31, 32, 33, 34, 37, 38, 39, 43, 44,61, 116, 122, 127, 189, 191, 192, 193, 194,195, 201, 203, 204, 205, 206

alternative, 68, 71, 113, 148alternatives, 64alters, 168ambiguity, 3, 31, 35, 183, 186ambivalent, 166amblyopia, vii, x, xi, 139, 140, 147, 148, 149,

150, 152, 153, 157American Psychiatric Association, 175, 176amplitude, ix, 76, 108, 110, 114, 119, 120amygdala, 162, 163, 164, 165, 169, 176, 180,

181, 182, 183, 184, 185, 186, 187anger, 184animal studies, 162, 164animals, 72, 168aniridia, 108annealing, 104anomalous, 111, 122, 146, 147, 150, 151, 152antagonistic, 214anterior cingulate cortex, 180anxiety, 182, 186application, 59, 112, 116, 126, 129, 130, 191,

192, 196, 213argument, 17, 169Aristotle, 66, 69, 79arousal, 165, 171, 175, 181, 182, 183, 185artificial intelligence, x, 125, 129, 137aspect ratio, 191assessment, vii, xi, 40, 79, 111, 149, 150, 155astigmatism, 117astronomy, 64asymmetry, 242atropine, 148, 152attentional bias, 181Australia, 66automatic processes, 162, 180

Index250availability, 2avoidance, 127, 181awareness, x, 79, 125, 130, 132, 166, 219

B

basic research, 243battery, 68, 76beating, 111, 119behavior, ix, 81, 82, 96, 104, 162benefits, x, 5, 125, 204Bezier, ix, 81, 82, 83, 84, 85, 87, 88, 91, 103,

104, 105bias, viii, 2, 3, 4, 58, 76, 185binomial distribution, 88birds, 131birth, ix, 107, 108, 109, 119, 147blindness, 220bonding, 48, 50botulinum, 117brain, 131, 163, 164, 165, 166, 169, 185, 209,

217, 221, 224, 240brain functioning, 131brain functions, 240brain stem, 224branching, 18breathing, 210buildings, 127buttons, 176

C

calibration, 2, 95, 104, 109, 113caloric nystagmus, 115Canada, 113candidates, 16, 74, 191, 192, 197, 198, 206capacity, 131case study, 119cataract, 80, 108, 109, 148, 149cataract surgery, 80cataracts, 75, 109, 148, 150cell, 168, 194, 195, 196, 202, 206, 245cerebral hemisphere, 68channels, 168, 169chiasma, 71childhood, 67

children, 67, 73, 77, 111, 112, 113, 122, 147,148, 149, 151, 152, 153, 191, 216, 222,237, 239, 244

China, 123, 206classical, 75, 143classification, xi, 111, 129, 175, 189, 191,

192, 197clay, 64clinical approach, 74clinical assessment, 245clinical disorders, vii, xi, 140clinical examination, 111coding, 173, 175cognitive, 225cognitive process, 225coil, 112, 113communication, 58community, 232compensation, 212competition, 167, 168, 169, 177, 182compilation, 67complexity, 30, 35, 173components, 19, 103, 110, 119, 192, 223, 225,

243, 244computation, 3, 4, 35, 190, 192, 224computational performance, 99computing, ix, 6, 31, 32, 37, 38, 62, 108, 133conception, 109, 131conditioning, 178, 179, 182, 184confidence, 18, 102, 103, 130configuration, 9, 10, 15, 22, 94confusion, 146, 147congenital cataract, 109Congress, 133, 134, 136conscious awareness, 166conscious knowledge, 233conscious perception, 166, 167, 169consciousness, 166, 221consensus, 207constraints, viii, 2, 5, 6, 15, 63, 105consumption, x, 125, 129, 130, 131continuity, 5, 60, 105contrast sensitivity, 110, 122control, vii, x, xi, xii, 117, 123, 125, 126, 127,

128, 129, 130, 131, 132, 136, 158, 161,166, 169, 170, 174, 176, 177, 181, 186,188, 190, 209, 210, 212, 214, 223, 225,226, 227, 231, 234, 236, 237, 238, 240,241, 242, 243, 245

Index 251convergence, 21, 24, 26, 110, 112, 143, 144,

149, 157, 160, 217, 223convex, 35, 36, 37coordination, vii, x, 77, 139, 149, 224corneal opacities, 148correlation, 5, 16, 17, 113, 119, 128, 203, 218,

222, 242correlation coefficient, 119, 222, 242correlation function, 16, 17correlations, 69, 222, 223cortex, 71, 140, 152, 153, 163, 164, 165, 168,

169, 176, 180, 182, 184, 185, 187, 224cortical processing, 164countermeasures, 132covering, 67CPU, 195, 196CRC, 61critical period, 147crocodile, 131cross-cultural, 182cross-talk, 109CRT, 228, 229cruise missiles, 126crystalline, 143cues, xi, 6, 161, 162, 165, 176, 177, 191, 206culture, 174, 191, 240cumulative distribution function, 196Cybernetics, 59, 61, 133cycles, x, 108, 116, 117cycling, 226cycloplegic refraction, 147Cyprus, 121

D

damping, 110danger, 162, 165data set, viii, 2decisions, 247decoupling, 3defects, 117deficiency, 6deficit, 148, 150definition, 149, 227, 228, 234, 247deformation, 83demand, 74, 75, 126, 210density, 196, 199dependent variable, 172depressed, 170

depression, 159deprivation, 147, 148depth perception, xi, 72, 140, 141, 144, 149,

152detection, ix, xi, 40, 81, 82, 85, 103, 104, 126,

127, 128, 140, 151, 180, 186, 189, 190,191, 192, 196, 206, 207, 208

deviation, xi, 3, 103, 147, 155, 156, 157, 158,159, 240

diodes, 104diplopia, x, 139, 141, 142, 146, 147discomfort, 74discrimination, 150, 186diseases, ix, 107disorder, ix, 107, 108, 149displacement, 2, 82, 84, 87, 88, 90, 94, 96, 99,

104dissociation, xi, 155, 156, 157, 158, 160distribution, 66, 85, 88, 196, 218, 221, 222,

224, 230, 232, 233, 240, 241, 242division, 29, 30, 31, 32, 33, 34, 164doctors, 220dominance, vii, viii, xi, xii, 22, 63, 64, 67, 68,

69, 70, 72, 75, 76, 77, 78, 79, 80, 161, 167,168, 171, 172, 173, 176, 177, 179, 180,183, 184, 217, 247, 248

DSM-IV, 176duration, 111, 114, 117, 121, 148, 173, 174,

175, 177

E

Education, 206EEG, 165, 179, 181, 182elaboration, 73, 129electrodes, 112, 113electromagnetic, 113, 130electromagnetic wave, 130embryogenesis, 109emission, 130emitters, 113emotion, 163, 172, 174, 180, 182, 183, 184,

185emotional, xi, 161, 162, 163, 164, 165, 169,

171, 172, 173, 174, 175, 176, 177, 178,179, 180, 181, 182, 183, 184, 185, 186, 187

emotional information, 163, 164emotional reactions, 163, 174, 176, 178

Index252emotional stimuli, xi, 161, 162, 163, 164, 165,

174, 179, 180, 181, 184, 186emotional valence, 178emotions, 171encoding, 108environment, x, 13, 40, 125, 127, 130, 132,

166environmental conditions, 191EOG, 112, 121epipolar geometry, 3, 5, 6epipolar line, 10, 15, 16, 17esotropia, 147, 148, 150, 151, 152estimating, 3, 4, 59estimation process, 4, 40estimator, 59, 117, 119Euro, 62Europe, 207European Social Fund, 206European Union, 190evolution, 111, 126, 131, 133examinations, 68execution, 233, 245exotropia, 148experimental design, 175explicit knowledge, 187extraction, vii, 1, 29, 37, 38, 48, 50, 58, 131eye movement, vii, ix, xii, 68, 107, 108, 109,

111, 112, 113, 114, 115, 116, 117, 119,122, 123, 141, 143, 144, 145, 150, 209,210, 211, 212, 213, 214, 215, 222, 223,224, 225, 227, 228, 231, 232, 234, 244, 245

eyes, vii, viii, ix, x, 63, 64, 65, 66, 69, 70, 71,72, 73, 74, 75, 78, 108, 109, 110, 111, 112,139, 140, 141, 144, 148, 149, 156, 158,166, 167, 168, 210, 211, 212, 213, 214,215, 216, 217, 218, 219, 220, 221, 222,223, 224, 225, 229, 233, 236

F

facial expression, 171, 174, 175, 178, 179,183, 185, 187

factor analysis, 68, 233failure, 73false negative, 191, 203, 206false positive, 203familial, 66, 109, 247family, 3, 66, 79fatigue, 218, 219

fear, 162, 175, 176, 178, 182, 184, 186February, 65feedback, 76, 143, 180females, 67filters, 69, 193fixation, ix, xii, 65, 73, 107, 109, 110, 113,

120, 141, 143, 144, 148, 209, 210, 211,212, 213, 214, 215, 216, 217, 219, 222,223, 224, 225, 226, 227, 229, 230, 231,235, 236, 237, 242, 243, 244, 245

flight, x, 125, 126, 127, 128, 129, 130, 132,136, 163

flow, 3, 60, 127, 130fluid, 72, 75Fourier, 114fovea, ix, 107, 108, 109, 110, 117, 119, 123,

148, 209, 215, 229Fox, 150, 179, 183FPGA, 136frog, 131frontal lobe, 225, 226, 232functional magnetic resonance imaging, 78,

79, 165, 185fundus, 111fusion, 64, 72, 141, 142, 143, 144, 150, 156,

217, 220

G

Gaussian, 85gender, 67, 247generation, 113, 191, 192, 204, 206, 214, 227,

230, 232, 234, 244genetic factors, 66Geneva, 207Ger, 192, 206, 207, 208Germany, 141, 161, 209Gestalt, 171, 183glasses, 148goals, 245government, 190GPS, 129, 132grades, 142grass, 186gratings, 178grouping, 185groups, 21, 129, 131, 177, 179, 222, 223, 224,

231growth, 111

Index 253guidance, x, 125, 128

H

habituation, 172handedness, 64, 76, 78, 79harmonics, 114Harvard, 184heart, 164heart rate, 164height, 103, 194, 199, 202hemisphere, 181hemodynamic, 165Hessian matrix, 20high resolution, 129high-frequency, 113high-level, 168, 169high-speed, 112histogram, 199, 200holistic, 170Homeland Security, 136horizon, 127horse, 39host, 190hostile environment, x, 125, 130, 132hot spots, 191human, 61, 64, 66, 71, 72, 78, 108, 149, 150,

157, 169, 183, 184, 185, 186, 187, 190,191, 199, 207, 213, 224, 244

human brain, 183human development, 66human subjects, 157humans, 164, 244hyperbolic, 199hyperopia, 147hypoplasia, 109hypothesis, xi, 161, 171, 173, 175, 178, 195

I

identification, 73, 74, 111, 115, 116, 248idiopathic, 108, 109, 123Illinois, 76illumination, 82, 191illusion, 79, 104, 210, 212illusions, 211, 244image analysis, 207

images, vii, viii, ix, x, 1, 2, 4, 5, 6, 9, 10, 13,14, 16, 27, 29, 30, 35, 36, 40, 42, 43, 44,51, 58, 60, 61, 68, 72, 81, 82, 83, 84, 87,90, 99, 104, 126, 129, 139, 140, 141, 142,156, 191, 207, 209, 217, 221

imaging, ix, xi, 78, 81, 82, 83, 84, 96, 99, 103,165, 168, 189

imitation, 126, 131implementation, 40, 41, 131, 193, 203inattention, 74, 110incidence, 110independence, 164, 222Indiana, 155, 160indication, 64, 69indicators, 69, 111, 247induction, 144industrial, 103industry, 129, 131inertial navigation system, 127infancy, 109, 111, 122infants, 73, 111, 112, 113, 147, 149, 150infinite, 141information exchange, 132infrared, 112, 113, 184, 191, 213infrared light, 113, 213infrared spectroscopy, 184inheritance, 122, 123inhibition, 231inhibitory, 168, 231injury, 75, 207innervation, 79INS, 109, 133insight, 226inspection, 103instabilities, 113instability, 3, 110, 112, 113, 211, 212, 215,

216, 217, 218, 219, 220, 221, 222, 223,235, 236, 243

instruction, 185, 214, 226, 232integration, x, 125, 127, 130, 131, 149integrity, 104intelligence, 131intensity, viii, 2, 5, 84, 85, 99, 110, 111, 113,

165, 184interaction, 79, 113, 130, 180interactions, 151, 168interdependence, 119interference, 72interpretation, 60, 143interval, 85, 88, 111

Index254intraocular, 148intraocular lens (IOL), 148intrinsic, 8intrusions, 112invasive, 112, 113inversions, 119Investigations, 170IOL, 148Iran, 58Italy, 107, 207ITC, 214iteration, 40ITRC, 58ITT, 206

J

Japan, 66, 139Japanese, 76, 79Jet Propulsion Laboratory, 136judge, 233Jung, 133justification, 71

K

kinesthesis, 160kinesthetic, 156

L

language, 240laser, ix, 74, 81, 82, 83, 84, 85, 94, 96, 99,

103, 104, 130late-onset, 151laterality, viii, 63, 64, 66, 67, 68, 69, 70, 71,

72, 75, 79, 80, 247, 248law, 79lead, 112, 169learning, 186left hemisphere, 227lens, 22, 74, 75, 77, 78, 84, 95, 113, 143, 148,

157lenses, 69, 74lesions, 245, 246light emitting diode, 215likelihood, 175

limitation, 24limitations, 120, 173, 174linear, x, 3, 22, 87, 89, 90, 94, 96, 113, 119,

125, 127, 143, 199linear function, 94, 96, 199linear regression, 119linguistic, 212localization, 79, 127, 206location, 76, 104, 232London, 77, 78, 80, 122, 151, 187long period, 148, 218long-distance, 104longevity, 71longitudinal study, 67, 80low-level, 168

M

machines, 137, 191magnetic, 165magnetic resonance, 78, 165magnetic resonance imaging, 78, 79, 165males, 67management, 78, 150manipulation, 178mapping, 5, 48, 127Mars, 136masking, 126matrix, 4, 9, 13, 19, 20, 22, 27, 28, 89, 90measurement, viii, ix, x, xi, 2, 3, 81, 104, 108,

125, 127, 147, 155, 156, 157, 160measures, 6, 65, 66, 222, 234, 238, 242, 245medicine, 150Mediterranean, 121memory, 83, 99, 171mental disorder, 182mental processes, 167metabolic, 153metric, 4, 156Mexican, 170Mexico, 81microscopy, 64middle-aged, 74Ministry of Education, 206miosis, 144mirror, 166, 167, 172missiles, 126mobile robot, 126, 129mobile robots, 129

Index 255mobility, 75modalities, 79, 108model fitting, 207modeling, 3, 61models, 29, 39, 40, 41, 245modulation, 186modules, 192modus operandi, 74MOG, 112money, 131monkeys, 76, 152, 153, 224monocular clues, 143Moscow, 125, 137motion, vii, viii, x, 1, 2, 3, 4, 5, 6, 8, 9, 13, 14,

16, 17, 19, 21, 22, 23, 24, 25, 26, 27, 28,40, 41, 45, 46, 47, 48, 49, 50, 51, 52, 53,54, 55, 56, 57, 58, 60, 108, 113, 117, 125,127, 128, 131, 143, 152, 183, 186

motion control, 128motivation, 204motor control, 68, 79motor system, 109, 112motor task, 247movement, 9, 11, 13, 16, 19, 94, 111, 117,

118, 119, 122, 123, 224, 229, 234, 245multidisciplinary, 152muscle, 148muscles, 213, 214, 224myopia, 67, 80myopic, 148

N

NASA, 135National Academy of Sciences, 185, 187natural, 72, 131, 225navigation system, 127near infrared spectroscopy, 184neck, 41negative valence, 171neonatal, 109nerve, 224nervous system, 131network, ix, 81, 82, 83, 84, 85, 87, 88, 90, 91,

93, 94, 95, 96, 99, 100, 103, 163, 244neural network, ix, 81neural networks, ix, 81neuroanatomy, 71neuronal circuits, 164, 180

neurons, 83, 88, 140neuropsychology, 232neuroscientists, 225New York, 77, 122, 137, 152, 182, 185, 207noise, viii, x, 2, 3, 16, 40, 51, 108, 113, 195,

196non-human, 72nonlinear, 3, 12, 22, 60non-uniform, 199, 201, 204normal, 7, 72, 77, 110, 122, 141, 146, 147,

148, 149, 150, 151, 166, 167, 183, 194, 241North America, 67, 170, 207nucleus, 164, 168, 184nystagmus, vii, ix, x, 107, 108, 109, 110, 111,

112, 114, 115, 116, 117, 118, 119, 120,121, 122, 123, 212

O

observations, 225, 226, 234obstruction, 148occipital lobe, 165occipital regions, 179occluding, 104occlusion, 13, 18, 22, 82, 83, 84, 91, 99, 100,

101, 104, 148, 197occupational, 74, 123oculomotor, 108, 217, 223, 232offenders, 187Oklahoma, 157operations research, 105operator, 132ophthalmologists, 119optic nerve, 71, 109optical, ix, 3, 4, 23, 60, 64, 69, 81, 82, 93, 94,

97, 98, 103, 127, 147optical systems, 82optics, 184optomotor system, 229organ, 71organism, 72organization, 147, 182, 187orientation, 2, 5, 7, 8, 84, 93, 94, 96, 127, 178,

184, 191, 192, 193oscillation, 110, 111, 116, 117, 119, 122, 123oscillations, ix, 107, 108, 109, 110, 112, 117,

118, 120outliers, vii, viii, 1, 2, 3, 4, 9, 16, 21, 196

Index256

P

paper, 135, 204paradoxical, 72, 75parameter, 21, 83, 119, 203Parietal, 182, 214Paris, 78pathogenesis, ix, 107, 123pathology, 75pathophysiological, 109pathophysiological mechanisms, 109pathways, 162, 163, 186patients, ix, 73, 75, 76, 80, 107, 110, 112, 113,

117, 119, 121, 123, 147, 148, 151, 164,176, 177, 181, 226, 232

pattern recognition, 128pedestrian, xi, 128, 189, 190, 192, 193, 197,

199, 201, 202, 204, 206, 207pedestrians, xi, 189, 190, 191, 204, 206pediatric, 122, 150perception, vii, x, xi, 71, 72, 139, 140, 141,

142, 143, 144, 147, 149, 152, 161, 162,165, 166, 167, 169, 170, 171, 173, 174,175, 177, 178, 179, 180, 181, 183, 185,186, 187, 218, 244

performance, viii, ix, x, 2, 31, 37, 41, 64, 74,75, 81, 83, 84, 99, 113, 116, 125, 130, 152,183, 192, 201, 206, 213, 228, 233, 234,237, 238, 239

periodic, 21, 114, 116, 119, 121permit, 131, 143personal, 73, 170, 177personal relevance, 170, 177personality, 170, 185personality traits, 170perturbation, 4, 40perturbations, 3, 41PET, 165PFC, 214phobia, 176, 177Phobos, 134photocells, 215photographs, 178physical environment, 166physical properties, 178physicians, ix, 107, 111physiological, 73, 74, 75, 109, 141, 164, 174,

184picture processing, 183

pinhole, 84, 93, 97pitch, 127, 194, 201, 207planar, 202planets, 128planning, x, 108, 111, 130plasticity, 187play, 190, 212pond, 90poor, 58, 148, 151, 242population, 67, 72, 122posture, 110power, x, 117, 125, 129, 130, 131PPS, 206praxis, 220prediction, 141predictors, ix, 108preference, viii, 63, 64, 67, 68, 70, 76, 77, 79,

80, 148, 247, 248prefrontal cortex, 188preparedness, 186preprocessing, 192presbyopia, 74, 143, 144, 145preschool, 72, 76, 149preschool children, 76prevention, 149, 207preventive, 190primary visual cortex, 149, 186primate, 151, 182primates, 72prior knowledge, xi, 6, 189, 191PRISM, 155probability, 193, 196, 197probability density function, 196, 197probe, 179production, 129, 238prognosis, 123, 148program, 149projector, 83promote, 171propagation, 4property, viii, 2, 16, 27, 58, 127proposition, 247protection, 190protocols, 157proximal, 144, 151psychiatrists, 119ptosis, 148pupil, 113, 143, 144, 145, 152pupils, 141

Index 257

Q

quality of life, 75quantization, viii, 2, 3questioning, 69

R

radar, 130, 190radio, 113radius, 19, 20, 141random, 32, 120, 197range, viii, 2, 19, 68, 104, 108, 109, 113, 117,

132, 195, 198, 204, 208, 234, 240, 242ras, 2ratings, 171, 185reaction time, xii, 179, 209, 214, 216, 224,

227, 228, 229, 230, 231, 233, 234, 235,236, 237, 238, 240, 241, 244

reading, vii, xii, 158, 182, 209, 211, 212, 213,229, 245

real time, 61, 130, 131reality, 210reasoning, 70recognition, 104, 115, 126, 127, 128, 245reconcile, 73reconstruction, vii, viii, ix, 1, 2, 5, 26, 29, 33,

39, 48, 51, 58, 61, 81, 93, 99, 126, 127,128, 202, 204

recovery, 2, 3redistribution, 202, 203reduction, xii, 105, 148, 189, 195, 206, 231,

237, 238reference frame, 71reflection, 213reflexes, 164, 225, 227, 229, 237regression, 119regression analysis, 119regression line, 119regular, 204, 230, 242relationship, viii, 63, 70, 82, 87, 120, 141, 242relationships, 143relevance, 163, 169, 170, 177, 178reliability, x, 116, 125, 126, 127, 130, 132,

219, 234, 245renormalization, 60repeatability, xi, 83, 103, 104, 111, 121, 155,

156, 157, 158, 159, 160

research, 2, 58, 64, 66, 71, 76, 105, 113, 119,143, 147, 149, 162, 166, 170, 172, 174,180, 190, 191, 206, 214, 223, 231, 232,243, 248

Research and Development, 136researchers, 114, 191residual error, 196, 197resolution, 38, 82, 104, 113, 114, 129, 183,

187, 204resources, 129, 131respiration, 164retina, ix, 107, 108, 110, 141, 163, 209, 213,

224risk, 127, 147, 152risk factors, 152Robotics, 60, 133, 134, 135, 136robustness, 16, 40ROI, 202Royal Society, 137Russia, 125, 137Russian, 125, 136, 137Russian Academy of Sciences, 125

S

saccades, x, xii, 69, 108, 209, 210, 212, 213,214, 215, 216, 217, 219, 222, 223, 224,225, 226, 227, 228, 229, 230, 231, 232,233, 234, 235, 236, 237, 238, 240, 241,242, 243, 244, 245

saccadic eye movement, 209, 210, 227, 244,245

safety, 129, 190sample, 167, 199, 207sampling, 29, 150, 197, 198, 199, 204, 205SAR, 135scalar, 194scaling, 195scatter, 218, 219, 222, 224, 227, 228, 234,

237, 242scatter plot, 218, 222Schmid, 6, 61school, 219scientists, 162, 223sclera, 113scotoma, 148search, xi, 3, 19, 20, 80, 112, 113, 123, 128,

131, 162, 174, 180, 189searching, xi, xii, 189, 193

Index258segmentation, 192, 204, 206selectivity, 186self-report, 76, 175, 178, 179semantic, 171, 176semantic content, 171semicircular canals, 108sensitivity, 3, 22, 25, 26, 104, 110, 122sensors, 127, 130, 190separation, 113, 115series, 115, 121, 156, 179, 181shape, viii, ix, 2, 3, 8, 13, 16, 18, 26, 27, 28,

32, 33, 34, 36, 60, 81, 82, 83, 84, 90, 93,99, 100, 101, 102, 103, 104, 108, 119, 207

short period, ix, 107, 220signals, 108, 114, 118, 179, 190significance level, 222similarity, 8, 9, 13, 16simulation, 32, 37, 38Simultaneous Localization and Mapping, 127,

133sine, 119Singapore, 104sites, 128sleep, 110smoothing, 40snakes, 162social anxiety, 185social phobia, 183software, 129, 204solutions, 3somatosensory, 182sorting, 196space-time, 2Spain, 189spatial, 66, 71, 74, 104, 143, 148, 166, 178,

186, 217spatial frequency, 148spatial information, 66specialisation, 71, 240species, 72, 75, 131spectral analysis, 123spectrum, 117, 224speculation, 71speed, 29, 31, 131, 192, 194, 195, 196sports, 64SRT, 228stability, xii, 209, 211, 215, 217, 218, 219,

223, 242stabilization, 21, 108, 110, 112, 127stabilize, 109, 229

stages, x, xi, 125, 129, 161, 166, 167, 169,180, 189, 192, 206

standard deviation, xi, 103, 117, 159, 196,199, 240

standard error, 175, 177statistical analysis, 213statistics, xii, 88, 189, 190steady state, 117Stimuli, 162, 175, 178, 181stimulus, 69, 117, 147, 148, 162, 165, 166,

168, 169, 179, 211, 219, 224, 226, 227,229, 230, 231, 232, 233, 234, 242

stimulus information, 168stimulus pairs, 69Stochastic, 132strabismus, vii, x, xi, 110, 122, 139, 140, 146,

147, 148, 149, 150, 151, 153, 157streams, 60strength, 68, 170, 180, 216STRUCTURE, 1students, 157subjective, 70, 159, 171substrates, 180suffering, 115superiority, 40, 64, 68, 184suppression, x, 72, 74, 139, 146, 147, 148,

151, 152, 167, 168, 170, 183, 186, 187,234, 248

surgery, 74, 75, 77, 78, 110, 122, 123, 148,149, 150, 157

surgical, 111, 148, 150, 152surprise, 214, 227, 232surveillance, 190, 191survival, 131switching, 75Switzerland, 207symmetry, 191symptoms, 73syndrome, 109, 120syntactic, 115synthesis, 59systems, x, 108, 113, 117, 125, 126, 128, 129,

130, 131, 132, 137, 190, 191, 192, 194, 207

T

targets, 117, 128, 148, 157, 192, 193task performance, 234taxonomy, 68

Index 259technology, 112telencephalon, 164temporal, 40, 112, 113, 141, 168, 169, 187,

230test procedure, 29test-retest reliability, 219, 245thalamus, 163theory, 67, 76, 80, 109, 111, 126, 149, 152,

167, 168, 169, 176, 182, 184, 211therapy, x, 108, 111, 119, 148thinking, 131threat, 185, 186threatening, 162three-dimensional, viii, 2, 4, 11, 15, 143, 217,

218three-dimensional model, viiithree-dimensional reconstruction, 4three-dimensional space, viii, 2, 15, 217threshold, 114, 196thresholds, 115, 140time, viii, ix, x, 5, 16, 19, 27, 31, 32, 37, 38,

40, 58, 61, 64, 69, 82, 84, 103, 108, 109,110, 111, 112, 113, 114, 115, 116, 117,121, 127, 130, 131, 132, 141, 148, 163,166, 171, 172, 177, 178, 192, 195, 196,209, 210, 215, 217, 218, 219, 220, 221,223, 224, 225, 226, 228, 229, 230, 231,234, 243, 247

time consuming, 132, 163time frame, 217time periods, 210, 225time resolution, 113time series, 115, 121timing, 226Timmer, 244top-down, 165, 186topographic, 82topology, 6toxin, 117tracking, 3, 4, 5, 21, 22, 23, 24, 26, 39, 40, 42,

57, 58, 113, 117, 123, 128, 192traffic, 190, 207, 226training, 75, 222traits, 66, 247trajectory, 117trans, 27, 77transfer, 66, 186transformation, 77translation, 3, 12, 19, 27, 28, 60translational, 4, 60

transverse section, 83, 99travel, 224trend, 67, 77trial, 152, 172, 174, 177, 178, 179, 213, 216,

217, 218, 226, 227, 228, 229, 230triangulation, 9, 15triggers, 230two-dimensional, x, 139, 171

U

uncertainty, 22, 130, 202, 203uniform, 111, 197, 201, 202, 248United Nations, 207United States, 141, 185, 187universities, 190Unmanned Aerial Vehicles, x, 126, 132, 137

V

valence, 165, 171, 172, 178, 181validation, 182validity, 179values, ix, 13, 17, 58, 85, 87, 88, 89, 90, 95,

96, 99, 103, 108, 158, 160, 196, 203, 216,218, 219, 221, 222, 224, 228, 234, 239,240, 242, 243

variability, ix, 108, 115, 117, 119, 120, 121,191, 199

variable, 74, 110, 172, 218, 222, 229variables, xii, 185, 209, 213, 218, 222, 223,

226, 227, 228, 233, 234, 236, 238, 240, 241variance, 70variation, ix, 8, 22, 27, 40, 81, 82, 204vector, 6, 7, 19, 20, 22, 83, 87, 191vehicles, x, 125, 126, 132, 191, 197velocity, ix, 103, 107, 109, 110, 111, 113,

114, 115, 116, 217, 218, 219vertebrates, 108vestibular system, 108, 112violence, 187violent, 170visible, xi, 5, 48, 147, 161, 216, 226, 227, 230vision, vii, viii, ix, x, xi, 2, 3, 4, 5, 59, 63, 64,

66, 71, 72, 73, 74, 75, 76, 78, 81, 82, 104,107, 108, 110, 111, 122, 125, 126, 131,132, 137, 139, 140, 141, 142, 147, 148,149, 151, 152, 155, 156, 187, 188, 190,

Index260208, 210, 211, 212, 213, 215, 216, 217,220, 221, 223, 225, 248

visual acuity, ix, x, 65, 68, 69, 71, 75, 77, 78,79, 108, 110, 111, 112, 115, 116, 117, 119,140, 149, 150, 157, 211, 248

visual area, 149, 164, 165, 168visual attention, 185, 232, 244, 245visual field, 72, 140, 148, 151, 167, 211visual perception, 162, 163, 165, 166, 169,

173, 181, 244visual processing, xi, 149, 161, 167, 168, 169,

172, 182visual stimuli, 162, 179, 182, 213visual stimulus, 147, 148, 230, 232visual system, xi, 75, 77, 111, 141, 147, 151,

161, 181, 210, 211, 220, 221visualization, 61, 200, 205vocational, 74vulnerability, 132

W

Warsaw, 121

wavelet, 114, 115wavelet analysis, 114, 115weakness, 68, 216, 235, 237wealth, 74weapons, 128windows, xi, xii, 114, 116, 189, 191, 192,

193, 197, 199, 201, 203, 204, 205World Health Organization, 190writing, 69, 70, 78, 240, 247

X

X-linked, 109

Y

Y-axis, 195yield, 160young adults, xi, 155

binocular vision development, depth perception and disorders

Documents

northamptonshire nn10 8bp

von noorden gk

van der steen

scleral search coils

unmanned aerial vehicles

poorly textured objects

la gatta antonio

excite controversy today