COMPACT - A Surface Representation Scheme

Pavel Grossmann*Long Range Research Laboratory

GEC Hirst Research CentreEast LaneWembley

Middlesex HA9 7PP

This paper describes the continuation of our work on ex-tracting simple surfaces from a 3D line-segment repre-sentation of stereo image data. The surface parametersform a basis of a scene representation which is suitablefor navigation by a mobile robot and also for recognitionand manipulation of simple objects by a robot arm.

Following our earlier work on planes we have nowdeveloped methods and algorithms for recovering spher-ical, cylindrical and conical surfaces. The essenceof our approach is testing of small groups of SDline segments for compatibility with a particular sur-face type. Maximal sets of segments supportingdifferent surfaces are then identified. In this wemake a novel use of the Dual Space Representation.

1 Introduction

Choice of the best representation of image data obviouslydepends on the purpose for which the representation isbeing designed - e.g. an object recognition and manip-ulation task may require a more complex and accuraterepresentation of the visible surfaces than a simple in-door navigation task. The choice also depends on thenature of the image data itself - e.g. dense depth mapsgenerated by a laser range finder and the sparse depthdata produced by a stereo edge-based system may giverise to very different representations.

In principle we can use all of the usual representationprimitives based on points, lines, surfaces or volume el-ements [1,2]. Many present systems, however, use pre-dominantly only one of the geometrical primitives andtry to explore its full potential. In some applicationspoints [3] or volumes offer specific advantages, but in thedomains of indoor scenes or manufactured objects, simplelines and surfaces can provide a basis for a more compactrepresentation.

These are the domains of interest in our collaborativeESPRIT project P940 - "Depth and motion analysis".The aim of the project is to develop a vision system ca-pable of guiding a mobile robot in indoor navigation tasksand also a robot arm in object recognition and manipu-lation tasks.

'This research was supported in part by the ESPRIT projectP940.

Many indoor scenes can be adequately described interms of planes, and also a large category of manufactedobjects that are being considered for our recognitionand manipulation tasks can be adequately described interms of four surface types : planes, cylinders, cones andspheres. We have therefore proposed (and are currentlyimplementing at the Long Range Research Laboratory,HRC, Wembley) a compact scene representation scheme(COMPACT) based on those four surface primitives.

The input to the higher level representation is a set of3D line segments produced by the initial stages of ourthree-camera stereo vision system [4].

It has been shown that, given a set of 3D line seg-ments, one can construct closed continuous contours (orparts thereof) that directly imply surfaces (usually pla-nar faces of simple polyhedral objects [5], although thisapproach has also been used to locate cylinders in im-ages [6]). There are two problems associated with thisapproach : it often requires complete surface contoursand also it makes no use of the segments "inside" thesurface boundaries that often make up most of the data(e.g. segments corresponding to windows, pictures andother features of walls in interior scenes).

Our approach is based on direct extraction of sur-face candidates from groups of segments and subsequentmerging of the compatible candidates into finite surfaces.

2 The method

2.1 General description

We shall explain the basic method on the example of aplanar surface (which has already been described in detailin [7,8]). Here we consider pairs of segments : each pairis tested for coplanarity and each coplanar pair is usedeither to update an existing surface candidate or to forma new candidate. When all the pairs are exhausted, somesurface candidates are removed if the number of segmentssupporting them is small, and some merged (if compati-ble) to yield a small number of significant surfaces. Thesurface parameters (in the planar case the surface normaland the perpendicular distance to the origin) are beingcontinuously updated. We shall refer to this as the "sur-face growing" method.

The Dual Space Representation (DSR - see AppendixA and [9]) offers an alternative way of finding planes.Here a point, a line and a plane in space are represented

97

AVC 1988 doi:10.5244/C.2.15

in Dual Space by a plane, a line and a point respectively.Thus a set of coplanar lines in space is mapped onto aset of lines intersecting in a single point in Dual Space.When Dual Space is divided into small cells (similar tothe Hough accumulator) such intersections can be foundas maxima in the number of lines intersecting each cell.

The main advantage of this (DSR) method (apart fromits elegance) is its speed - instead of being proportionalto the number of segment pairs, the computing time hereis linear in the number of segments (with some additional- fixed - time for the maxima search). On the other handthere is a limit to the number of maxima we can identifyin a noisy Dual Space population. Also the treatment oferrors is far less straightforward due to the granularity ofDual Space and the nature of the mapping transforma-tions.

2.2 Treatment of errors

The description of uncertainties associated with all thequantities in our calculations is based on the assumption[10] that a covariance matrix always provides a sufficientparametrization of the relevant probability distributionand that the covariant matrices can be propagated usingthe standard formula [ll] :

where y is a set of m functions which all depend onthe set of n random variables x (\/k = !/&(*))• The erroron any quantity is then given by the square root of therelevant variance.

Starting with the experimentally determined covari-ance matrices of the segment end points we first computethe errors for the segment parameters (unit vector andlength), then for the intermediate quantities (e.g. vectorproducts) and finally for the resultant surface parame-ters.

In the surface growing approach the surface param-eters are continuously updated by the addition of newestimates and the updated values are determined as theweighted means of all the previous measurements. Ourmethod of computing the (in general correlated) param-eters and the relevant variances is sketched out in Ap-pendix B.

In addition, to take into account the departure fromideal planes of many real quasi-planar surfaces, that weperceive (and hence wish to identify) as planar, we haveintroduced the concept of "perceptual resolution". Asan example, consider a set of segments representing awindow. For the purposes of recognition we may wish toregard all the segments as belonging to the same planarsurface although they may be contained in a rectangularvolume of space some 10-20 cms thick. When a segment istested for consistency with a particular plane, we considernot only the errors on the segment and plane parameters,but also the perceptual resolution.

Hence it is not always easy to test the errors except insome global sense. As an example consider an office scenerepresented by the 3D segments in Figure 1. There aretwo main vertical planes there corresponding to the left

c)

Figure 1: INRIA office scenea) one of the triplet of imagesb) front view of the 3D segmentsc) top view of the 3D segments

wall (with windows) and the cabinets on the right (withposters). While the distribution of segments associatedwith each plane is quite "noisy", we would expect theplane normals nx and n? to be well determined and inour case perpendicular (see Figure lc). In our analysiswe get :

cos 8 = nx • n2 = 0.020 ± 0.027

i.e. the walls are perpendicular within the (small) error.

2.3 Spheres

A spherical surface can be identified by examiningtriplets of line segments [7,12]. Strictly speaking ourmethod is based on triplets of surface tangents. Herewe make an assumption that a line segment, which is anapproximation to a section of a surface curve, is also agood approximation to a surface tangent u,- at a surfacepoint close to the segment midpoint P,-. Experimentswith real data (see our results shown below) suggest thatthis assumption is reasonable. A triplet of surface tan-gents defines a triplet of planes (fui = pi, ru^ = P2 an (iru3 = P3) normal to the tangents, that intersect in a sin-gle point R (Figure 2) whose position can be computed[13]:

R =Pi ("2 X "3) + P2("3 X ttl) + P3("l X

« i ( u 2 X U3)

unless ui(u2 x "3) = 0. If the distance from R to thethree surface points Pi is the same (= D), the segmentsare consistent with a spherical surface which is describedby its centre R and its radius D.

98

Figure 2: Test of the spherical surface hypothesis

2.4 Cylinders and Cones

The cylinders and cones are treated in the same way. Thebasic idea is to find the axis of the cylinder or cone asan intersection of several planes that are normal to thesurface and that contain the axis. In order to find suchplanes we make an assumption that our line segments areparts (or approximations thereof) of the surface lines ofcurvature that are planar and whose associated curvatureis constant. Such lines have been found to be importantdescriptors of surfaces [14]. On cylindrical and conicalsurfaces these are either straight lines (parallels) or circles(meridians).

For each pair of parallels we can construct their planeof mirror symmetry, which passes through the axis of thesurface. Hence the axis can be found as the intersectionof several such planes (Figure 3a).

For each segment that approximates a part of a merid-ian we can find a plane normal to the segment whichalso passes through the axis. And similarly the axis canbe found as the intersection of several such planes (Fig-ure 3 b).

The task of searching for such plane intersections ismade simpler when we use the Dual Space Representa-tion (DSR). Here a set of planes intersecting in a singleline in 3-space is transformed into a set of points lyingon a line in Dual Space. This (dual) line can be found byusing the Hough transform or other standard methods.

The requirement that a significant fraction of segmentsis of the "parallel" or "meridian" type may appear some-what restricting. There are many objects with more orless random distribution of surface markings or, indeed,with no markings al all. For such cases we proposed[12] a way of generating a set of parallels from the ex-tremal (self-occluding) boundaries of the object (we arestill talking about cylinders and cones!).

In the mode of operation of the optical system in ques-tion, when the cameras move around the object (e.g. acylinder), accumulating and combining information in or-der to build a more complete description of the visiblesurface, different projections of the "real" lines in spacewill coincide when projected back into space. The ex-tremal boundaries, however, will move along the surfaceas the cameras move and so they can be identified as

Figure 3: Determining the axis of a cylinder or a conea) from a set of parallelsb) from a set of meridians

such. Their projections in 3-space will not coincide; theywill form a set of different rulings that can be used in ourmethod. This way of generating useful sets of segmentsis currently being tested.

3 Implementation and results

The methods described above have been implemented asa suite of programs written in Franz Lisp and running onour VAX 11-750 with the Unix operating system. It isanticipated that some parts will be implemented in 'C.

The algorithms have been tested with simulated data,both ideal and with superimposed errors, and also withreal sets of 3D line segments from the pool of imagedata provided for the collaboration by our ESPRIT col-leagues. The planar algorithms were tested using officescenes (Figure 1) from INRIA (Rocquencourt) and totest the non-planar methods we used triplets of imagesof a sphere and a cylinder obtained at ELSAG (Genoa)and prosessed at INRIA where the 3D segments were ex-tracted (Figures 6 and 7).

Time taken to process a typical office scene (~150 seg-ments, Figure l) is of the order of several minutes. TheDSR mapping of segments into the Dual Space is faster,while the spherical algorithm based on segment tripletscan take several tens of minutes. At the moment thealgorithms are not optimized for speed and we expectconsiderable improvement in the running times in thefuture.

The planar case (Figure 1) has already been discussedelsewhere [7,8] and so here we shall concentrate on thequadric surfaces.

99

« #

b)

Figure 4: Simulated (ideal) dataa) two coaxial cylinders - 3D segmentsb) a projection of Dual Space

Figure 5: Simulated (noisy) data - a conea) 3D segments and the reconstructed axisb) x-y projection of the Dual Space

Figure 4a shows an example of the synthetic (ideal)data. Here we have two coaxial cylinders and all the seg-ments are "parallels" on the surfaces. Because the cylin-ders are coaxial, both separate sets of segments map ontothe same straight line in Dual Space (Figure 4b). Thereis, however, also a contribution from the "mixed" pairs ofsegments (each segment from a different cylinder) whichconstitutes a background. The straight line parameterswere determined by the Principal Axis method used inan iterative mode to reject outliers at every step and togradually "home in" onto the line. This line was thenmapped back to the ordinary space to give the correctposition of the surface axis (shown in Figure 4a). Oncethe axis is known, the two radii can easily be computed.

Figure 5 shows the effect of realistic errors being im-posed on the ideal segments (5a). There is a scatter ofpoints about the "best line fit" in Dual Space (Figure 5b)that translates into a finite spread of the angle betweenthe segments and the computed axis about the ideal (i.e.zero errors) value.

Finally in Figures 6 and 7 we show the results obtainedwith real data. For the test of the spherical algorithm weused an image of a ball painted with a simple pattern,resting on a table. For the corresponding set of 3D seg-ments shown in Figure 6a the program found only onesignificant spherical surface candidate and the segmentsassigned to it are shown in Figure 6b. The radius of thesphere has been computed to be 46 mm compared withthe measured value of 45 mm.

Similarly to test the cylindrical algorithm we used animage of a cylinder decorated with a simple pattern.Each of the corresponding segments in Figure 7a was as-

sumed to be a part of a meridian although many segmentshave very different orientation. Nevertheless, a straightline fit to the corresponding Dual Space distribution (Fig-ure 7b) gave a reasonable estimate of the cylinder axis asshown superimposed on the data in Figure 7a.

4 Future workHaving demonstrated our ability to determine the surfaceparameters of each of the four surface types from theline segment representation of image data, we now haveto integrate our algorithms into a single system capableof dealing with any mixture of the basic surface typespresent in complex scenes.

Considering the use of our scheme for navigation andobject recognition we have to develop ways of matchingthe representation of a scene to the representation of thesame scene viewed from a different viewpoint or to anappropriate representation of familiar objects likely tobe found in the scene. We are also investigating meth-ods for interpretation of image data without referring togeometrical models. This will be done by analyzing thespatial relations between the surfaces and using domainspecific knowledge to provide additional constraints.

5 Acknowledgements

Thanks are due to my colleagues in the ESPRIT P940collaboration (INRIA and ELSAG) for the use of theirimage data (in Figures 1,6 and 7) and to Mike Brady formany inspiring discussions.

100

V

\

'• /

\

A Dual Space Representation

A.I Dual point to a plane

A plane with an unit surface normal N = (Nx, Ny, Nz)passing through a point V is mapped to a dual point

f

^ M\

Figure 6: A real spherea) all 3D segmentsb) segments found to belong to the sphere

> i—k—^w /1

\

. . . * in

a)

Figure 7: A real cylindera) 3D segments and the computed axisb) a projection of the Dual Space

A.2 Dual plane to a point

A point V = (x, y, z) is mapped to a dual plane with anunit surface normal N = (§•, ̂ , jj) and passing througha dual point V = N{- f ) where B = y/(l + x2 + y2).

A.3 Dual line to a line

A line with a unit vector L = (Lx, Lv, Lz) and passingthrough a point V ( r = V + XL) is mapped to a dualline joining the following two points Si and 52 :

NV—,—,-—)

NV

which themselves are dual to the two planes passingthrough the point V and having the following unit surfacenormals :

N1 =LxV

Lx[LxV)

Lx [LxV)\

As points, lines and planes are interrelated, so are theirduals.

B Errors for correlated vectorquantities

Let us consider estimation of a vector quantity Jl givenn measurements (xy, x^, •••, xn) and also the (different)covariance matrices V,. We shall use the MaximumLikelihood principle to derive the appropriate estimator.Starting from the likelihood function [ll] :

we require that

101

d\nL= 0

In the case of 3-dimensional vector quantity (k =1,2,3) the above condition leads to the following 3 in-homogeneous equations (for better readability we shalldrop the summation index t and denote the elements ofthe inverse covariance matrices V~l by MM) '•

xx + IMV 5 3 Mxy + fiz 5 3 Mxz =

+ + ] £= 53 *

x 53 Mv= 53 xMyx + 53 yMvy + X) zM

= ]T xMxx + 5 3 yM*yThe solution can be given as follows :

D D

where :

and the determinants Dx, Dy and Dz are formed byreplacing the "x,y or z" column in D by the right handside of the equations above :

yMxyY.x

E A/y

and similarly for Dy and Dz.Using a similar approach we can also derive the estima-

tor for the covariance matrix associated with the vectorp. The result is simple - to obtain the inverse of the co-variance matrix we sum up the inverse covariant matricesof the individual measurements :

[4] Nicolas Ayache and Francis Lustman. Fast and re-liable passive trinocular stereovision. In Proceedingsof the First International Conference on ComputerVision, 1987.

[5] Martin Herman and Takeo Kanade. Incremental re-construction of 3D scenes from multiple, compleximages. Artificial Intelligence, 30, 1986.

[6] Kashipati Rao and R. Nevatia. Generalized conedescriptions from sparse 3-d data. In ProceedingsCVPR'86, 1986.

[7] Pavel Grossmann. COMPACT - a 3D shape repre-sentation scheme for polyhedral scenes. In Proceed-ings of the Third Alvey Vision Conference, 1987.

[8] Pavel Grossmann. Building Planar Surfaces fromRaw Data. Technical Report R4.1.2, ESPRITProject P940, 1987.

[9] M.C.Err. Computer Interpretation of EngineeringDrawings. PhD thesis, University of Essex, 1985.

[10] Hugh F. Durrant-Whyte. Uncertain geometry inrobotics. IEEE Journal of Robotics and Automa-tion, 4(1), 1988.

[11] O.Skjeggestad A.G.Frodesen and H.T0fte. Probabil-ity and Statistics in Particle Physics. Universitets-forlaget Oslo, 1979.

[12] Pavel Grossmann. Planes and Quadrics fromSD Segments. Technical Report R4.1.6, ESPRITProject P940, 1988.

[13] I.D.Faux and M.J.Pratt. Computational Geometryfor Design and Manufacture. Ellis Horwood Ltd.,1981.

[14] Michael Brady et al. Describing surfaces. ComputerVision, Graphics and Image Processing, 32, 1984.

References

[l] J. B. Bowen and J. E. W. Mayhew. ConsistencyMaintenance in the REVgraph Environment. Tech-nical Report 20, AI Vision Research Unit, SheffieldUniversity, 1986.

[2] J. L. Crawley. Representation and maintenance ofa composite surface model. In IEEE Int. Conf. onRobotics and Automation, 1986.

[3] C. G. Harris. Determination of ego-motion frommatched points. In Proceedings of the Third AlveyVision Conference, 1987.

102