additional material harris detector

8/13/2019 Additional Material Harris Detector

1/5

COMBINEDCORNER ND EDGE DETECTORChris Harris Mike Stephens

Plessey Research Roke Manor, United KingdomO The Plessey Company plc. 988

Consistency of image edge filtering is of prime importancefor 3 interpretation of image sequences using featuretracking algorithms. To cater or image regions containingtexture and isolated features, a com bined corner and edgedetector based on the local auto-correla tion function isutilised, and it is shown to perform with good consistencyonnatural imagery .

INTRODUCTIONThe problem we are addressing in Alvey Project MMI149is that of using computer vision to understand theunconstrained 3D world, in which the viewed scenes willin general contain too wide a diversity of objects for top-down recognition techniques to work. For example, wedesire to obtain an understanding of natural scenes,containing roads, buildings, trees, bushes, etc., as typifiedby the two frames from a sequence illustrated in Figure 1.The solution to this problem that we are pursuing is touse a computer vision system based upon motion analysisof a monocular image sequence from a mobile camera. Byextraction and tracking of image features, representationsof the 3D analogues of these features can be constructed.To'enabie explicit tracking of image features to be.performed, the image features must be discrete, and notform a continuum like texture, or edge pixels (edgels). For'this reason, our earlier work1 has concentrated on theextraction and tracking of feature-points or corners, since

they are discrete, reliable and meaningful2. However, thelack of connectivity of feature-points is a major limitationin our obtaining higher level descriptions, such as surfacesand objects. We need the richer information that isavailable from edge .

T H E EDGE TR CKING PROBLEMMatching between edge images on a pixel-by-pixel basisworks for stereo, because of the known epi-polar camerageometry. However for the motion problem, where thecamera motion is unknown, the aperture problem preventsus from undertaking explicit edge1 matching. This could beovercome by solving for the motion beforehand, but weare still faced with the task of tracking each individual edgepixel and estimating its 3D location from, for example,Kalman Filtering. This approach is unattractive incomparison with assembling the edgels into edgesegments, and tracking these segments as the features.Now, the unconstrained imagery we shall be consideringwill contain both curved edges and texture of variousscales. Representing edges as a set of straight linefragments4, and using these as our discrete features will beinappropriate, since c w e d lines and texture edges can beexpected to fragment differently on each image of thesequence, and so be untrackable. Because of ill-conditioning, the use of parametrised curves (eg. circulararcs) cannot be expected to provide the solution, especiallywith real imagery.

Figure I Pair of images rom an outdoor sequence.


2/5

Having found fault with the above solutions to theproblem of 3D edge interpretation, we question thenecessity of trying to solve the problem at all Psycho-visual experiments (the ambiguity of interpretation inviewing a rotating bent coat-hanger in silhouette), showthat the problem of 3D interpretation of curved edges mayindeed be effectively insoluble. This problem seldomoccurs in reality because of the existence of smallimperfections and markings on the edge which act astrackable featu re-points.Although an accurate, explicit 3 D representation of acurving edge may be unobtainable, the connectivity itprovides may be sufficient for many purposes indeed theedge connectivity may be of more im portance than explicit3D measurements. Tracked ed ge connectivity, supplement-ed by 3 D locations of comers and junctions, can provideboth a wire-frame structural representation, and delimitedimage regions which can act as putative 3D surfaces.

This leaves us w ith the probIem o f perform ing reliable (ie,consistent) edge filtering. The state-of-the-art edge filters,such as5, are not designed to cope with junctions andcomers, an d are reluctant to provide any edge connectivity,This is illustrated in Figure 2 for the Canny edge operator,where the above- and below-thresho ld edgels are represented :respectively in black and grey. Note that in the bushes,some, but not all, of the edges are readily matchable by -eye. A fter hysteresis has been undertaken, followed by thedeletion of spurs and short edges, the application ofjunction completion algorithm results in the edges andjunctions shown in Figure 3, edges being shown in grey,and junctions in black. In the bushes, very few of theedges are now readily matched . The problem here is that ofedges with responses close to the detection threshold: asmall change in edge strength or in the pixellation causes alarge change in the edge topology. The use of edges todescribe the bush is suspect, and it is perhaps better todescr ibe it in terms of feature-points alone.

Figure 2 Unlinked Canny edges or the outdoor images

a bFigure 3 Linked Canny edgesfo r the outdoor images


3/5

AUTO CORRELATIO N DETECTORThe solution to this problem is to attempt to detect bothedges and corners in the image: junctions would then; consist of edges meeting at corners. To pursue thisapproach, we shall start from M oravec s comer d etectofi.

MORAVEC REVISITEDMoravec s comer detector functions by co nsidering a loca lwindow in the image , and determining the averag e changesof image intensity that result from shifting the w indow bya small amount in various directions. Three cases need tobe considered:A. If the windowed image patch is flat (ie. approximatelyconstant in intensity), then all shifts will result in onlya sma ll change;0 If the w indow straddles an edge, then a sh ift along theedge will result in a small change, but a shiftperpendicular to the edg e will result in a large change;C. If the windowed patch is a comer or isolated point, thenall shifts will result in a large change. A comer canthus be detected by finding when the minimum changeproduced by any of the shifts is large.We now give a mathematical specification of the above.Denoting the image inten sities by I , the ch ange E producedby a shift (x,y) is given by:

where w specifies the image window: it is unity within aspecified rectangular region , and zero elsewhere. The shifts,(x,y), that are considered comprise ((1,0), (1,1), (0,1),(-1.1)). T hus Moravec s corner detector is sim ply this:look for local maxima in min(E) above some thresholdvalue.

c dFigure 4 Corner detection on test image

The performance of Moravec s corner detector on a testimage is shown in Figure 4a; for comparison are shownthe results of the Beaudet7 and Kitchen ~ o s e n f e l d ~operators (Figures 4b and 4c respectively). The M oravecoperator suffers from a number of problems; these arelisted below, together with appropriate correctivemeasures:1 The response is anisotropic because only adiscrete set of shifts at every 45 degrees isconsidered all possible small shifts can be covered byperforming an analytic expansion about the shift origin:

where the first gradients are approximated by

Hence, for small shifts, E can e written

whereA x 2 8 wB y 2 @ wC = ( X Y ) @ w

2 The response is noisy because the window isbinary and rectangular use a smooth circularwindow, for example a Gaussian:

3 The operator responds too readily to edgesbecause only the minimum of E is taken intoaccount reformulate the comer measure to make use ofthe variation of with the direction of shift.The change, E, for the small shift (x,y) can be conciselywritten as

where the 2x2 symm etric matrix M is

Note that E is closely related to the local autocorrelationfunction, with M describing its shape at the origin(explicitly, the quadratic terms in the Taylor expansion).Let a be the eigenvalues of M. a and will beproportional to the principal curvatures of the local auto-


4/5

correlation function, and form a rotationally invariantdescription of M. As before, there are three cases to beconsidered:A. If both curvatures are small, so that the local auto-correlation function is flat, then the windowed imageregion is of approximately constant intensity (ie.arbitrary shifts of the image patch cause little change inEl;B. If one curvature is high and the other low, so that thelocal auto-conelation function is ridge shaped, thenonly shifts along the ridge (ie. along the edge) causelittle change in E: this indicates an edge;C. If both curvatures are high, so that the local auto-correlation function is sharply peaked, then shifts inany direction will increase E: this indicates a comer.

iso-response contours

Figure 5 Auto-correlationprincipal curvature space-heavy lines give cornerledgelflat classijkation,fine lines are equi-response contours.

Conside r the graph of (a ) space. An ideal edge will havelarge and /3 zero (this will b e a surf ace o f translation),

but in reality J will merely be small in comparison to adue to noise, pixellation and intensity quantisation.com er will be indicated by both and being large, and afla t image region by both nd being small. Since anincrease of image contrast by a factor of p will increase aand p proportionately by p2, then if (a ) is deemed tobelong in an edge region, then so should (ap2,pp2), forpositive values of p. Sim ilar considerations apply tocomers. T hus (a ) space needs to be divided as shown bythe heavy lines in Figure 5.

ORNERIEDGE RESPONSE FUN TIONNot only do w e need comer and edg e classification regions,but also a measure of comer and edge quality or response.The size of the response will be used to select isolatedcom er pixels and to thin the edge pixels.Let us first consider the measure of comer response, R,which we require to be a function of and P alone, ongrounds of rotational invariance. It is attractive to useTr(M) and D et(M) in the formulation, as this avoids theexplicit eigenvalue decomposition of M, thus

Consider the following inspired formulation for the comerresponse~ = ~ e t - k ~ r ~ S

Contours of constant R are shown by the fine lines inFigure 5. R is positive in the corner region, negative inthe edge regions, and small in the flat region. Note thatincreasing the co ntrast (ie. mo ving radially away from the

Figwe 6 . Edgelcorner class icafion for the outdoor images(grey corner regiom, white thinned edges).


5/5

a bFigure 7 Completed edges or the outdoor imageswhite= corners, black = edges).

origin) in all cases increases the magnitude of theresponse. The flat region is specified by Tr falling belowsome selected threshold.comer region pixel (ie, one w ith a positive response) isselected as a nominated co mer pixel if its response is an 8-way local maximum: comers so detected in the test imageare shown in Figure 4d. Similarly, edge region pixels aredeemed to be edgels if their responses are both negative andlocal minima in either the x or y directions, according towhether the magnitude of the first gradient in the x or ydirection respectively is the larger. This results in thin

edges. The raw edgelcorner classification is shown inFigure 6, with black indicating corner regions, and grey,the thinned edges.By applying low an d high thresholds, edge hysteresis canbe carried out, and this can enhance the continuity ofedges. These classifications thus result in a 5-level imagecomprising: background, two com er classes and two edgeclasses. Further processing (similar to junctiond completion) will delete edge spurs and short isolated edges,nd b ri dg e s ho rt bre ak s in ed ge s. Th is re su lt s i ncontinuous thin edges that generally terminate in thecomer regions. The e dge term inators are then linked to thecomer pixels residing within the com er regions, to form aconnected edge-vertex graph, as shown in Figure 7. Notethat many of the comers in the bush are unconnected toedges, as they reside in essentially textural regions.

i Although not readily apparent from the Figure, many ofthe corners and edges are directly matchable. Further workremains to be undertaken concerning the junction;Completion algorithm, which is currently quiterudimentary, and in the area of ad aptive thresholding.

CKNOWLEDGMENTSThe authors gratefully acknowledge the use of imagerySupplied by Mr J Sherlock of RSRE, and of the results

(the comparison of c om er operators, Figure 4) obtainedunder MOD contract. The grey-level images used in thispaper are subject to the following copyright: Copyright 6Controller HMSO Lo ndon 1988.

R E F E R E N C E S1. Hams, C G M Pike, D Positional Integrationfrom Image Sequences, Proceedings third Alvey VisionConference (AVC87), pp. 233-236, 1987; reproduced inImage and Vision Computing, vol 6, no 2, pp. 87-90,May 1988.2 Charnley, D R J Blissett, Surface Reconstructionfrom Outdoor Image Sequen ces, Proceedings fourthAlvey Vision Club (AVC88). 1988.3. Stephens, M J C G Harris, 3 Wire-FrameIntegration from Image Sequences, Proceedings fourthAlvey Vision Club (AVC88), 1988.4. Ayache, N F Lustman, Fast and Reliable PassiveTrinocular Stereovision, Proceedings first ICCV, 1987.5 . Canny, J F, Finding Edges and Lines in Images, MITtechnical report AI-T R-720,1983 .6 Moravec, H, Obstacle Avoidance and Navigation in theReal World by a Seeing Robot Rover, Tech ReportCMU-RI-TR-3, Carnegie-Mellon University, RoboticsInstitute, September 1980.7. Beaudet, P R, Rotationally Invariant Image Operators,International Joint Conference on Pattern Recognition,pp. 579-583 (1987).8. Kitchen, L, and A Rosenfeld, Grey-level CornerDetection, Pattern Recognition Letters, 1, pp. 95-102(1982).

additional material harris detector

Documents