This may be the author’s version of a work that was submitted/acceptedfor publication in the following source:

Low, Toby & Wyeth, Gordon(2005)Obstacle detection using optical flow.In Sammut, C (Ed.) Proceedings of the 2005 Australasian Conference onRobotics and Automation.Australian Robotics and Automation Association, Australia, pp. 1-10.

This file was downloaded from: https://eprints.qut.edu.au/32838/

c© Copyright 2005 [please consult the authors]

This work is covered by copyright. Unless the document is being made available under aCreative Commons Licence, you must assume that re-use is limited to personal use andthat permission from the copyright owner must be obtained for all other uses. If the docu-ment is available under a Creative Commons License (or other specified license) then referto the Licence for details of permitted re-use. It is a condition of access that users recog-nise and abide by the legal requirements associated with these rights. If you believe thatthis work infringes copyright please provide details by email to qut.copyright@qut.edu.au

Notice: Please note that this document may not be the Version of Record(i.e. published version) of the work. Author manuscript versions (as Sub-mitted for peer review or as Accepted for publication after peer review) canbe identified by an absence of publisher branding and/or typeset appear-ance. If there is any doubt, please refer to the published source.

http:// www.araa.asn.au/ acra/ acra2005/ papers/ low.pdf

https://eprints.qut.edu.au/view/person/Wyeth,_Gordon.htmlhttps://eprints.qut.edu.au/32838/http://www.araa.asn.au/acra/acra2005/papers/low.pdf

This is the author version published as: This is the accepted version of this article. To be published as : This is the author version published as: Catalogue from Homo Faber 2007

QUT Digital Repository: http://eprints.qut.edu.au/

Low, Toby and Wyeth, Gordon (2005) Obstacle detection using optical flow. In: Proceedings of Australasian Conference on Robotics and Automation 2005, 5 - 7 December 2005, Sydney, N.S.W.

Copyright 2005 [please consult the authors]

Obstacle Detection using Optical Flow

Toby Low and Gordon Wyeth

School of Information Technology and Electrical Engineering

University of Queensland

St Lucia, Australia.

tobylow@itee.uq.edu.au, wyeth@itee.uq.edu.au

Abstract

Motion has been examined in biology to be acritical component for obstacle avoidance andnavigation. In particular, optical flow is a pow-erful motion cue that has been exploited inmany biological systems for survival. In thispaper, we investigate an obstacle detection sys-tem that uses optical flow to obtain range in-formation to objects. Our experimental resultsdemonstrate that optical flow is capable of pro-viding good obstacle information but has obvi-ous failure modes. We acknowledge that ouroptical flow system has certain disadvantagesand cannot be solely used for navigation. In-stead, we believe that optical flow is a criticalvisual subsystem used when moving at reason-able speeds. When combined with other visualsubsystems, considerable synergy can result.

1 Introduction

Navigation and obstacle avoidance are just two of bi-ology’s main instincts for survival in unknown environ-ments. Mobile robots too must have these basic instinctsintegrated into their system before attempting to per-form any higher-level tasks. Currently sonar and lasermethods are used for obstacle detection but researchersare now looking towards an important sensory systemthat many biological creatures use everyday−vision.

Vision is a very powerful sensor providing numeroustypes of information that can be explored in many con-texts for obstacle detection. Colour and texture recogni-tion have been used to segment out the ground plane inimages thus identifying free space in the forward area[Cheng and Zelinsky, 1998]. Edge obstacle detectionmethods have been implemented in corridor type en-vironments due to the many straight line componentsand static lighting conditions [Ohya et al., 1997]. More-over edges in an image have been considered obstaclesusing the edge free ground plane assumption [Lorigo et

al., 1997] [Chao et al., 1999]. All these techniques usedifferent image aspects that have been shown to worksuccessfully in their test environments. It is only whenthey are conducted in more general environments thatfailure occurs due to the various assumptions made. Forexample, texture recognition assumes the ground tex-ture remains constant, colour recognition assumes thatobjects are not the same colour as the ground, and theedge detection method used in corridors requires strictmodel memory components plus a good simultaneous lo-calisation and mapping (SLAM) system. In biology, ithas been shown that these spatial visual systems cannotand do not function alone. Instead it suggests that thesesystems operate in conjunction with another importantsystem, the visual motion system.

Visual sensors are able to sample the amalgamation ofenvironmental motion and self-motion information of ascene through two or more successive snapshots. Thesesnapshots encode motion at a fundamental feature levelused to estimate optical flow. Optical flow can be definedby a set of vectors with each vector describing the motionof individual features in the image space. These simplevectors can provide a 2D representation of the robot’smotion and the environment’s 3D structure and motionunder the correct conditions. As such, optical flow hasbeen used in a many different ways for navigation andobstacle avoidance. The common ground plane segmen-tation using optical flow templates has been shown tobe successful in identifying flat traversable surfaces [Illicand Masciangelo, 1992]. Furthermore, the use of diver-gence maps from optical flow fields have shown promis-ing results with robots been seen to navigate betweenobstacles successfully using divergence plots [Camus etal., 1999] [Nelson and Aloimonos, 1989]. Divergence andtime to contact calculations for a central area have alsobeen implemented as a frontal collision detection sub-system [Coombs et al., 1995].

Evidence in biology also outlines the importance ofmotion with monkeys shown to contain two pathwaysknown as Magnocellular (M) and Parvocellular (P) path-

ways [Miles and Wallman, 1993], otherwise described asbeing the motion and colour/texture processing path-ways respectively. Insects such as bees have been shownto exploit optical flow by balancing right and left flowfields in corridor type environments [Srinivasan, 1992],and has also been proven to have great success on mobilerobots [Coombs et al., 1995]. No doubt visual motion isan important cue for obstacle avoidance and navigationaltasks in biology. Much work has already been done ontechniques for the colour/texture pathway but less so forthe motion pathway.

Whilst many interesting techniques have been formedusing optical flow, almost no research has been done onusing optical flow to construct a simple but robust rangemap of the environment. Optical flow under certain con-ditions can provide a 3D reconstruction of the surround-ing environment similar to that of stereo vision but onlyusing one camera.

In this paper, we aim to take advantage of this 3Denvironmental information encoded within optical flowto extract and produce a range map. We compare thisvisual range map with the traditional sonar and lasersensor information and outline any improvements andshortcomings. Our approach will be slightly different tothat from the known Structure from Motion (SFM) area.Instead of using the basic motion equations, we use thetime to contact calculations formulated by Lee [Lee andYoung, 1985] for each optical flow component. This ap-proach was taken as the time values are hypothesised tobe a key aspect in controlling mobile robots. Numerousexperiments conducted by Lee [Lee, 1980] show that ouractions and movements are not governed by the true timeto contact but rather by the time to contact if the speedof the object were to remain constant, otherwise knownas the tau-margin. We hypothesise that motion infor-mation is a key aspect for many functions, in particularobstacle detection, control and navigation.

1.1 Outline

In Section 2, we provide background motion informa-tion including the optical flow estimation methods andthe time to contact calculations used to find the rangemeasurements to the optical flow points. Section 3 willdescribe an angular Obstacle Map that is used to repre-sent the results in addition to Cartesian space. Section 4presents the system architecture including sensor hard-ware of the robot. In Section 5, errors that were encoun-tered in optical flow will be outlined and the methodschosen to solve them presented. Section 6 describes theenvironmental setup, robot paths and sensors used inthe experiments. Following this, Section 7 presents theresults of the experiments to which Section 8 will dis-cuss. Lastly the conclusion of this paper will be shownin Section 9.

2 Motion Framework

The motion field can be described as the projection ofthe 3D velocity field in a scene onto a 2D image planeand is analogous to optical flow many situations. If welet P = [X,Y,Z]⊤ be a 3D point in the camera referenceframe, Z the optical axis and f the focal length, then theimage of a scene point, P can be mapped to the imagepoint p through the following equation:

p = fP

Z(1)

From this we obtain the important basic equations ofmotion by differentiating both sides of Equation 1 toobtain:

vx =Tzx − Txf

Z− ωyf + ωzy +

ωxxy

f−

ωyx2

f.

vy =Tzy − Tyf

Z− ωxf + ωzx +

ωyxy

f−

ωxy2

f.

(2)

where v is the velocity vector in image space, T is thetranslational velocity vector in camera space, f is focallength, Z the depth to the motion point in camera spaceand ω is the rotational velocity vector in camera space.

From Equation 2, it can be seen that the rotationalor translation components are linearly separable andthat depth information Z for a point only exists ifthere is translational motion in 3D space, T. It is fromthese equations that the structure from motion field isbuilt. They give an understanding and mathematicalbackground of the information optical flow can contain.The main difference between an optical flow field andthe motion field is that optical flow only describes theapparent motion of features in an image, which may ormay not correspond to the motion field.

2.1 Optical Flow Estimation

In order to extract information from an optical flow field,it must first be estimated from our images taken withina scene. There are many optical flow estimation tech-niques but the majority can be classified into two cate-gories, differential or feature-based. Differential methodswork using the temporal and spatial variations of all pix-els in a image window and is based upon the brightnessconsistency equation [Trucco and Verri, 1998]:

(∇E)⊤v + Et = 0. (3)

where E = E(x, y, t) is the image brightness, v themotion field and the subscript t denoting partial dif-ferentiation with respect to time. This equation as-sumes that the apparent brightness of moving objects

remains constant and thus the change in image bright-ness ∆E(x(t), y(t), t) in a scene must equal zero. It iscommonly solved using the least squares solution over asmall patch Q, thus producing the optical flow v at thecentre of the patch Q.

Feature based methods on the other hand use distinctfeatures in image and a matching technique to find theoptical flow vectors. As a consequence, the optical flowfield is generally more sparse and uses less processingtime than differential methods. It is for this processingtime concern that a feature-based matching system wasemployed. Although recent studies have shown promis-ing performances for differential techniques in closed-loop control situations [McCarthy and Barnes, 2004].Some other advantages of using a feature-based methodincludes the ability to eliminate the common apertureproblem as well as the ability to cope with more sporadicmovements. To obtain both these advantages, corners(otherwise described as an area of large spatial changesin brightness in two different directions) are the featureschosen to be tracked and matched.

Corner Detection

Corner detection is an important process for obtaininggood optical flow images. There exist several corner de-tection methods from which the commonly known Har-ris corner detection method [Harris and Stephens, 1988]

was chosen. It produced superior time and corner re-sults compared to other common methods when appliedto local image tests. The Harris corner detector worksby using the matrix:

C =

[ ∑

E2x∑

ExEy∑

ExEy∑

E2y

]

(4)

where the sums of image brightness E are taken overthe neighbourhood Q of point p. Corners are found byextracting the two eigenvalues of the matrix that encodethe edge strengths thus producing a new matrix C:

C =

[

λ1 00 λ2

]

(5)

Thus when λ1 ≥ λ2 and λ2 is over a given thresholdvalue, it corresponds to two strong edges in the sameimage window thus specifying a corner. This cornerdetection method was implemented using the OpenCVfunction cvGoodFeaturesToTrack as it is analogous tothe Harris corner detector differing only in the thresh-old equation used ( OpenCV uses a simple constantmultiplied by the maximum eigenvalue). The OpenCVfunction provides adjustable variables such as the max-imum corner amount to detect, threshold constant andthe minimum spacing between corners found. This helpsscale the number of corners found and improves spatialuniformity of the optical flow.

Matching

Matching is performed via the normalised cross correla-tion coefficient for a window surrounding the corners ofinterest found. This was implemented with the help ofOpenCV’s function cvMatchTemplate as it can be easilymodified to perform other standard matching functionssuch as the sum of squared differences and cross corre-lation. With the chosen matching function, corner pairscan be eliminated using a simple threshold value rang-ing from 0 to 1 (1 being a perfect match). Matches arefurther limited with the assumption of relatively smoothmovements thus should only occur in a specified searcharea around the position of the first corner. Once match-ing has been completed, the optical flow vectors can thenbe used to extract range information through time tocontact equations.

2.2 Time To Contact

The theory of time to contact (TTC) was first intro-duced by Lee [Lee and Young, 1985]. Lee conductedmany studies upon humans and birds showing evidencethat TTC is a critical component used in the timing ofmotion and actions. As we fundamentally wish to de-velop the robot for obstacle avoidance and control, wechose to use the TTC information and believe it will helpthe task of obstacle avoidance at later stages. One canalso simply convert these time measurements to rangeestimates by multiplying it by the speed of the robot foreasy comparison.

Robot

Object

��������

r

v

��

•

(a)

Robot

Object

��������

vr

��

r

��??

?

?

?

?

?

?

?

?

?

?

?

?

?

?

φ

99

•

vo

OO

(b)

Figure 1: Time To Contact Figures: Left - Robot on astraight collision course with object, Right - Robot noton a collision course with object.

The definition of time to contact is quite simple. Givena point object travelling on a collision course with therobot, distance r from the robot and at a constant speedv (as shown in Figure 1(a)), the TTC is given by:

TTC = r/v. (6)

Another way to express the TTC is to use the direc-tion of the velocity vector v and the direction to a point

on the object [Lee and Young, 1985]. If we label theangle between these two vectors φ, the TTC can be al-ternatively calculated using:

TTC =cosφ × sinφ

φ̇. (7)

The benefit of the latter equation is that it only re-quires information that can be obtained purely from op-tical quantities. As such, this equation produces timevalues to each corner tracked within an image fromknown image-based and camera information. φ repre-sents the angle between a vector r and the optical axisof the camera. It is assumed that the robot only movesin the forward camera direction and does not move per-pendicular to the camera direction thus allowing simpleangular calculations. This is a reasonable assumptiongiven that the robot is of a wheelchair design and willbe mainly operating in flat environments. Once each ofthe time to passing or contact values are found, theycan be converted to range information by extracting thespeed of the robot from the wheel encoders. These rangevectors are then transformed into Cartesian coordinatesby using the assumption of a pinhole camera model andsimilar triangles:

X =x

fZ.

Y =y

fZ.

(8)

3 The Obstacle Map

Optical flow information was also used to form an Ob-stacle Map (OM) in order to provide a different repre-sentation of obstacle information. The Obstacle Map isa angular map with the range threshold function:

proxi = f(ri) =Kpr2i

. (9)

where ri is the range in metres to the point of inter-est i, Kp the threshold constant and proxi represent-ing range information converted to a 0 - 1 proximityscale (1 representing the closest obstacles). As the mainconcern is about detecting and actuating around threat-ening obstacles, this type of angular range map couldbe much more useful for control purposes. In particu-lar, the biological-plausible reactive navigation systemdeveloped by Browning [Browning, 2000] uses this OMcombined with a Goal Direction (GD) map to producea Motor Heading Map (MHM) that controls the direc-tion of the robot. The method is based on the RanaCmodel biologically found in frogs [Arbib, 1987] and hasbeen shown to perform well in multi-object environments[Ball, 2001].

This Obstacle Map could also be formed using timeto contact information as a relative proximity measureinstead of strict range information ri. The benefit is thatnavigation would be purely based on optical quantities.This was to be initially implemented but to allow a fairfuture comparisons of the optical flow obstacle detectionagainst traditional techniques, the strict range obstaclemap format was used.

Thus through Equation 9, the OM was constructed us-ing the Cartesian range information obtained from opti-cal flow. As optical flow points are quite sparse and sepa-rated in feature-based estimation, an error margin of ±2◦

was implemented. This helps create a more smootherand continuous Obstacle Map.

4 System Architecture

The platform used to conduct the experiment was aPioneer robot, which is developed and sold as a self-contained unit. It is a wheelchair designed robot withan inbuilt Pentium computer running a compact ver-sion of Windows XP. The Pioneer robot sensors includea sonar ring, encoders, a monocular active camera, anindustry standard SICK laser and an inertial measure-ment unit (EiMU). The laser is located in the front areaof the robot with eight (8) sonar sensors arranged be-neath. The camera is mounted on top of the laser andpositioned in the middle of the turning axis of the robot.An embedded motor board is used control motors andcommunicate encoder information back to the computer.Figure 2 shows a photo of the Pioneer mobile robot andTable 1 presents some brief information on the robot’ssensors.

Figure 2: The Pioneer Robot with a SICK Laser, SonarRing, Active Camera and the EiMU sensor.

Sensor Information

Type

Description

Sonar Cone projectedrange, 30◦ cone,max. range is 2m.

Consists of 8 sonar’ssurrounding the for-ward area.

Laser Range, max.range is 150m.

Scans up to 180◦ with1◦ angular resolutionand a 10mm rangeresolution.

Vision 392x288 @ 25fps,50◦ angular range,non interlaced im-age.

Provides a maximumresolution of 768x576PAL, supports out-puts in YUV, RGBand Grayscale.

EiMU Heading 0 − 360◦,heading rate.

Consists of a singleaxis gyroscope andthree magnetometers.Contains inbuilt soft-ware filtering and cal-culates heading direc-tion with respect tomagnetic north.

Table 1: Pioneer Sensor Information

5 Improving Optical Flow

There are three main problems associated with usingoptical flow images to obtain 3D environmental informa-tion and self-motion. Firstly, to fulfil the assumption ofsmooth movements, the robot has to move quite slowlyand as a result produces poor and coarse optical flowdata. This causes large errors in range calculations es-pecially when sub-pixel corner finding algorithms are notemployed.

The second problem is a well-known issue in computervision, whereby any rotation or rotational disturbanceswill add a constant optical flow vector to each point ofinterest. As examined in Equation 2, rotations encodeno range information thus must be removed in order toobtain the correct range information from translationalmovements.

Lastly, the problem of lens distortion also has an affecton the optical flow. Optical flow information is quite sen-sitive thus even lens distortion can alter the optical flowvectors enough to produce an incorrect observation ofthe motion field. It complicates the situation by chang-ing optical flow vectors depending on the position in theimage.

5.1 Tracking Corners

To help solve the problem of coarse optical flow data, weemployed tracking over a number of successive images.Tracking was conducted in all directions as it producedsatisfactory results with each match chosen to be tracked

from either 1 to 5 images. The tracking is interleaved be-tween images thus removing many outliers and producesa finer optical flow field suitable for range data extrac-tion. Tracking over five successive images slows downthe process of obtaining range information but the rea-son it is performed over five images is due to the robotmoving at a slow speed. If the robot were to move at afast pace, the number of tracking images can be reduced.This would produce the same resolution of optical flowat a faster rate (needed at higher speeds for control) butwith the sacrifice of outlier reduction. Although trackinghelps improve optical flow, the problem of coarse opticalflow data at very slow speeds still exists. Thus we choseto ignore any information obtained from vision motionbelow a certain forward speed.

5.2 Rotation Removal

The area of ego-motion recovery is vast thus there existsmany different algorithms to estimate rotational motioncomponents. The problem is many of these techniquesrequire numerous assumptions, are computationally ex-pensive and still produce results containing errors [Gi-achetti et al., 1998] [Gebert et al., 2003]. The Pioneerrobot was equipped with an EiMU1 thus we took advan-tage of the in-built gyroscope to help remove rotationsfrom the optical flow images. As the camera is alignedwith the robots turning axis, rotation removal becomesthe simple task of finding the heading difference in imagespace and subtracting it from the optical flow.

Initial tests on the EiMU presented huge errors due tothe heading being fused with a reference point providedfrom the magnotometers. For indoor environments, largeelectromagnetic disturbances from computers, electricalequipment as well as from the Pioneers power and motorunits affected the magnotometers greatly. Thus the mag-notometers were retired and the gyroscopes rate infor-mation fused with the Pioneer wheel odometry instead.The original sensor fusion method (a complementary fil-ter with an additional integral gain) was kept as it is sim-ple, real-time and can easily be tweaked to combine thebenefits of both datasets. Figure 3 shows a brief blockdiagram of the complementary filter (CF) where kp andki are the specified proportional and integral gains.

To calibrate the CF, 360◦ rotation tests were con-ducted and gains tuned for a turning speed of 15◦/s.The filter results are presented in Figure 4 from whichwe can see the combination of the gyroscope’s fine reso-lution and the odometry’s values helping to remove gy-roscopic drift.

1An inertial measurement unit provided by fellow re-searchers at the CSIRO ICT Centre.

Odometry

Gyro Rate

Heading�� ∫+

+� �--

-

∫

Figure 3: Complementary Filter Block Diagram for Fus-ing EiMU Data with Wheel Odometry.

7.5 8 8.5 9

−160

−155

−150

−145

−140

Complementary Filter Results

CF − Red, Triangle Rate − Blue, Plus

Odometry − Green, Asterisk

Time(s)

Ang

le(d

eg)

Figure 4: Complementary Filter Test Results: Comple-mentary Filter Heading - Red/Triangle Line, EiMU RateIntegrated Heading - Blue/Plus Line, Odometry Head-ing - Green/Asterisk Line.

5.3 Rotation Effectiveness

In order to test the effectiveness of the rotation esti-mations from our CF fusion, a simple experiment thatrotated the robot on the spot was conducted. If the CFrotation estimates, camera position and lens distortionmodel are perfect, we should obtain an optical flow imageof only dots representing corners. Figure 5(a) and Figure5(b) shows a sample of optical flow images produced be-fore and after the rotation removal process respectively.It can be seen that a significant part of rotation com-ponents are removed with a calculated average error of7.07% over the smooth rotation segments. This errormay produce a slight shortening or lengthening effectsin the range estimates but as we are concerned with ob-stacle detection and not precise mapping, this error levelis satisfactory.

(a) Before Rotation Removal

(b) After Rotation Removal

Figure 5: Optical Flow Images of the Rotation RemovalProcess using the EiMU and Odometry.

5.4 Lens Distortion Removal

To remove lens distortion, camera calibration was per-formed via the Matlab camera calibration toolbox anddistortion removed using the iterative normalisationfunction also provided by the toolbox. As removing dis-tortion blurs the images slightly, we chose to normalisethe points after the corners were tracked and matchedup. This not only improves corner detection and match-ing quality levels but also speeds up the overall process;undistorting only the matching corner points instead ofthe whole image.

6 Experimental Setup

Two (2) experiments were conducted in order to test ourthe optical flow system.

1. Straight Only - The robot was driven forward onlybetween two artificial obstacle targets against a verycorner free background.

2. Slow Curve - The robot took a slow rightwardcurving path toward real obstacles and turns intocorridor space.

The speed of the robot in the experiments ranged from0 - 250mm/s in an area of about 5 by 3m. To gain sig-nificantly better optical flow information, speeds below180mm/s were ignored. For comparison purposes thelaser and the sonar sensor information were recorded.All tests were conducted within an office environmentwhich was found to have a very consistent carpet colourthus producing a very corner free ground plane. It wasalso assumed that all corners found in an image are asso-ciated with an obstacle and considered a threat. This is areasonable assumption since obstacles close to the robotare the main concern and due to the cameras viewing an-gle, any corners produced from large heights (such as theceiling) should be classified as being relatively far awaythus not affecting the closer obstacle data. Althoughthis may not be the case in other environments such asoutdoor environments, optical flow data can provide 3Dinformation thus allowing the option of filtering valuesto those within the robot’s height space.

7 Experimental Results

Straight line results are presented in Figure 6 and Fig-ure 7 showing one of the better to one of the poorerframes respectively. The top-most image displays thecorrected optical flow overlaying the camera image. Themiddle left and right images plot the optical flow andlaser ranges in the same scale Cartesian space respec-tively. Darker dots in the left plot symbolise an obstacleof closer proximity to the robot. Note that the field ofview (FOV) of the camera is limited compared to thatof the laser and sonar thus only the points seen in thecameras view (top-most graph) are mapped to the Op-tical Flow Cartesian map. The approximate FOV lim-its of the camera are signified by a dotted-dashed linein each of the plots. The bottom left image shows thesonar range readings in Cartesian coordinates and thebottom right image contains the Obstacle Map createdfrom optical flow range information (discussed in Section3). The Obstacle Map uses an angular measure of de-grees on the x-axis with the optical axis of the cameralocated at 180◦. The FOV limits of the Obstacle Map areapproximately ±28◦. We chose an OM threshold valueof 1.5 metres and a value of 0.5 for unknown range infor-mation. All measurements are sampled relative to therobot space with flow information obtained and trackedover a sequence of five images.

Figures 8, 9 and 10 present the curve path experimentresults for frames 345, 180 and 120 respectively. Theseframes were chosen to give a view of the best, averageand worst case scenarios respectively. The layout is iden-tical to the straight line results except the depth axis

Optical Flow Overlayed with Camera Image

0

1

2

3

4

−2−1012X(m)

Optical Flow Cartesian Map

Z(m

)

140 160 180 200 2200

0.2

0.4

0.6

0.8

1

Angle(Deg)

Thr

esho

ld R

ange

(0−

1)

Obstacle Map from Optical Flow

0

1

2

3

4

−2−1012

Z(m

)

X(m)

Laser Map

0

1

2

3

4

−2−1012

Z(m

)

X(m)

Sonar Map

Figure 6: Experiment Results - Straight: Frame 115.

Z(m) was changed from 4 to 8 metres to accommodatethe long corridor measurements.

8 Discussion

Experiment 1 produced some promising results with themajority of images producing smooth flow thus providingquite accurate and consistent range information. FromFigure 6, the Optical Flow Cartesian Map shows solidevidence of two obstacles with a visually inspected errorof 0.2m with respect to the laser readings. This error ismore than acceptable for obstacle detection as seen bythe Obstacle Map which easily identifies two threaten-ing obstacles at either side of the robot with free spacebetween them. Comparing the vision-based method tosonar, it is seen that the optical flow information out-performs sonar with its much higher resolution map, theability to detect the left side obstacle correctly as wellproviding quite accurate range information.

Optical Flow Overlayed with Camera Image

0

1

2

3

4

−2−1012

Optical Flow Cartesian Map

X(m)

Z(m

)

140 160 180 200 2200

0.2

0.4

0.6

0.8

1

Angle(Deg)

Thr

esho

ld R

ange

(0−

1)

Obstacle Map from Optical Flow

0

1

2

3

4

−2−1012

Z(m

)

X(m)

Laser Map

0

1

2

3

4

−2−1012

Z(m

)

X(m)

Sonar Map

Figure 7: Experiment Results - Straight: Frame 55.

The poorer results in Figure 7 show optical flow in-formation skewing slightly and increasing the range ofthe right side obstacle. Even so, optical flow range in-formation still able to provide evidence of the right sideobstacle’s existence as well as the back wall.

In experiment 2, the optical flow system producedgood and informative results for the majority of framesbut also produced some incorrect obstacle informationthroughout frames 110 to 160. The better optical flowframes from the curve run (Figure 8) shows that opticalflow information is able to map out the corridor wall aswell as the intruding door and bin. The Obstacle Mapclearly indicates the free space to the right, examined tobe the corridor, and an object to the left relating to thebin and door in the overlayed image. Note that the lasermap is able to pick up the other side of the door but justmisses being picked up in the optical flow Cartesian mapand Obstacle Map due to the camera’s FOV. Compar-

Optical Flow Overlayed with Camera Image

0

2

4

6

8

−2−1012

Z(m

)

X(m)

Optical Flow Cartesian Map

140 160 180 200 2200

0.2

0.4

0.6

0.8

1

Angle(Deg)

Thr

esho

ld R

ange

(0−

1)

Obstacle Map from Optical Flow

0

2

4

6

8

−2−1012

Z(m

)

X(m)

Laser Map

0

2

4

6

8

−2−1012

Z(m

)

X(m)

Sonar Map

Figure 8: Experiment Results - Curve: Frame 345.

ing with sonar again, it can be seen that the detectionrange of optical flow is much better and although notas good as laser, is more suitable for obstacle avoidanceconsidering the robot’s maximum speed.

Figure 9 displays one of the more average optical flowresults from experiment 2, with some errors producedfrom the doors out of focus corners as seen in the over-layed image. Regardless, the rest of the informationclearly indicates the chair, some of the door and rub-bish bin seen in the overlayed image.

Figure 10 shows one of the failed frames with the op-tical flow ranges indicating an object on a direct col-lision course less than 1 metre away from the robot.Further investigation revealed that the incorrect rangereadings were due to the sensitivity of the TTC calcula-tions near the principle point of the image. As the anglebetween the pixels and principle point becomes smaller,the TTC equations become much more sensitive to any

Optical Flow Overlayed with Camera Image

0

2

4

6

8

−2−1012

Optical Flow Cartesian Map

Z(m

)

X(m)

140 160 180 200 2200

0.2

0.4

0.6

0.8

1

Angle(Deg)

Thr

esho

ld R

ange

(0−

1)

Obstacle Map from Optical Flow

0

2

4

6

8

−2−1012

Z(m

)

X(m)

Laser Map

0

2

4

6

8

−2−1012

Z(m

)

X(m)

Sonar Map

Figure 9: Experiment Results - Curve: Frame 180.

optical flow thus even small errors affect the range read-ing greatly. This error is further amplified as the visionsystem works using pixel resolution thus the coarse na-ture of pixel movement in the central regions only allowsvery discrete range readings.

This TTC sensitivity problem can be approached innumerous ways with the simplest method ignoring thecentral portion of the image. Another method would beto use sub-pixel interpolation or a differential optical flowtechnique in the central region to create finer optical flowvectors thus reducing quantisation errors. In biology, ithas been investigated that monkeys and humans onlyfocus on a central region of the image, and thus hypoth-esised that stereo is used to obtain depth in the centralregions whereas optical flow is mainly on the outer pe-ripheral regions of the eye. In accordance, we could alsotry combine this optical flow system with stereo and evencolour segmentation and/or texture recognition to form

Optical Flow Overlayed with Camera Image

0

2

4

6

8

−2−1012

Optical Flow Cartesian Map

X(m)

Z(m

)

140 160 180 200 2200

0.2

0.4

0.6

0.8

1

Angle(Deg)

Thr

esho

ld R

ange

(0−

1)

Obstacle Map from Optical Flow

0

2

4

6

8

−2−1012

Z(m

)

X(m)

Laser Map

0

2

4

6

8

−2−1012

Z(m

)

X(m)

Sonar Map

Figure 10: Experiment Results - Curve: Frame 120.

a complete visual obstacle-avoidance system.Aside from the bad frames seen in Figure 10, the op-

tical flow obstacle detection system was relatively suc-cessful with the ability to identify and estimate rangesto obstacles given good smooth optical flow information.It was inspected in the straight line experiment and inthe curve line experiment (ignoring the frames below thespeed threshold) that 100% and 81% of the optical flowframes were capable of providing and locating threaten-ing obstacle information respectively. With this percent-age of consistancy seen, this technique should be able tobenefit greatly from temporal filtering or path integra-tion techniques.

9 Conclusion

In this paper, we have introduced an optical flow ob-stacle detection system capable of estimating obstaclepositions. The CF rotation removal process and TTC

calculations employed were proven to be effective in prac-tical situations. In many frames, the optical flow systemproduced range information that was on par or superiorto sonar information and in some of the better frames,on par with laser information. There are some failuremodes in our system; this technique was not developedto be a standalone method but a subsystem of a com-plete vision obstacle detection system. In summary, ourresults have shown that optical flow information is capa-ble of providing useful obstacle information that can addto the strengths of other visual methods such as colour,texture and edges recognition. Motion is a critical com-ponent in biology for navigation, to which our obstacledetection system highlights the usefulness of optical flowand its promising abilities to fill the visual motion gapfor mobile robots.

References

[Arbib, 1987] M. A Arbib. Levels of modeling of visuallyguided behavior. Behavioural Brain Science, 10:407–465, 1987.

[Ball, 2001] David Ball. Intelligence system for the 2001roboroos team. Honours Undergraduate Thesis, 2001.

[Browning, 2000] Brett Browning. Biologically Plausi-ble Spatial Navigation for a Mobile Robot. PhD the-sis, Department of Computer Science and ElectricalEngineering, The University of Queensland., 2000.

[Camus et al., 1999] Ted Camus, David Coombs, Mar-tin Herman, and Tsai-Hong Hong. Real-time single-workstation obstacle avoidance using only wide-fieldflow divergence. Technical report, National Instituteof Standards and Technology., 1999.

[Chao et al., 1999] Minn Tyi Chao, Thomas Braunl,and Anthony Zaknich. Visually-guided obstacle avoid-ance. IEEE, 1999.

[Cheng and Zelinsky, 1998] Gordon Cheng and Alexan-der Zelinsky. Goal-orientated behaviour-based visualnavigation. International Conference on Robotics andAutomation., pages 3431–3436, May 1998.

[Coombs et al., 1995] David Coombs, Martin Herman,Tsai Hong, and Marilyn Nashman. Real-time obstacleavoidance using central flow divergence and peripheralflow. IEEE, 1995.

[Gebert et al., 2003] Glenn Gebert, Deryl Snyder, JuanLopez, and Naveed Siddiqi. Optical flow angular ratedetermination. IEEE, 2003.

[Giachetti et al., 1998] Andrea Giachetti, Marco Cam-pani, and Vincent Torre. The use of optical flow forroad navigation. IEEE Transactions on Robotics andAutomation., 14(1):34–48, February 1998.

[Harris and Stephens, 1988] Chris Harris and MikeStephens. A combined corner and edge detector.Proceedings of The Fourth Alvey Vision Conference.,pages 147–151, 1988.

[Illic and Masciangelo, 1992] M. Illic and S. Mascian-gelo. Ground plane obstacle detection from opticalflow anomalies: a robust and efficient implementation.IEEExplore, 1992.

[Lee and Young, 1985] David N. Lee and David S.Young. Brain mechanisms and spatial vision. Dor-drecht and Boston and Nijhoff., 1985.

[Lee, 1980] D. N. Lee. The optic flow field: The foun-dation of vision. Philosophical Transcations of theRoyal Society of London. Series B, Biological Sci-

ences., 290(1038):169 – 178, 1980.

[Lorigo et al., 1997] Liana M. Lorigo, Rodney A.Brooks, and Eric L. Grimson. Visually-guided obsta-cle avoidance in unstructured environments. IROS,1997.

[McCarthy and Barnes, 2004] Chris McCarthy and NickBarnes. Performance of optical flow techniques forindoor navigation with a mobile robot. Proceedings ofthe 2004 IEEE, April 2004.

[Miles and Wallman, 1993] F.A. Miles and K. Wallman.Visual Motion and Its Role in the Stabilization of

Gaze. Elsevier, 1993.

[Nelson and Aloimonos, 1989] Randal C. Nelson andJhon Aloimonos. Obstacle avoidance using flow fielddivergence. IEEE Transactions on Pattern Analysisand Machine Intelligence., 11(10), October 1989.

[Ohya et al., 1997] Akihisa Ohya, Akio Kosaka, andAvia Kak. Vision-based navigation of mobile robotswith obstacle avoidance by single camera vision andultrasonic sensing. IROS, 1997.

[Srinivasan, 1992] Mandyam V. Srinivasan. How beesexploit optic flow: Behavioural experiments andneural models. Philosophical Transactions: R. Soc.Lond. B, 1992.

[Trucco and Verri, 1998] Emanuele Trucco and Alessan-dro Verri. Introductory Techniques for 3-D ComputerVision. Prentice Hall, 1998.