View Planningfor SiteModeling

PeterK. Allen MichaelK. Reed IoannisStamosDepartmentof ComputerScience,ColumbiaUniversity�

allen,m-reed,istamos� @cs.columbia.edu�

Abstract

3-D modelsof complex environments,known assitemodels,areusedin many differentapplicationsrangingfrom city planning,urbandesign,fire andpoliceplanning,military applications,virtual real-ity modelingandothers.Sitemodelsaretypicallycreatedby handin a painstakingand error proneprocess.Thispaperfocusesontwo importantprob-lemsin site modeling. Thefirst is how to createageometricandtopologicallycorrect3-D solid fromnoisydata.Thesecondproblemis how to planthenext view to alleviate occlusions,reducedatasetsizes,and provide full coverageof the scene. ToacquireaccurateCAD modelsof the scenewe areusingan incrementalvolumetricmethodbasedonsetintersectionthatcanrecover multipleobjectsina sceneandmerge modelsfrom differentviews ofthe scene. Thesemodelscan serve as input to aplannerthatcanreducethenumberof viewsneededto fully acquirea scene.Theplannercanincorpo-ratedifferentconstraintsincludingvisibility, field-of-view and sensorplacementconstraintsto findcorrectview pointsthatwill reducethemodel’sun-certainty. Resultsarepresentedfor acquiringageo-metricmodelof asimulatedcity sceneandplanningviewpointsfor targetsin aclutteredurbanscene.

1 Introduction

Realistic3-D computermodelsarefastbecomingastapleof oureverydaylife. Thesemodelsarefoundon TV, in the movies, video games,architecturalanddesignprogramsandahostof otherareas.One�This work was supportedin part by an ONR/DARPA

MURI award ONR N00014-95-1-0601,DARPA AASERTawards DAAHO4-93-G-0245and DAAH04-95-1-0492,andNSFgrantsCDA-96-25374andIRI-93-11877.

of the more challengingapplicationsis in build-inggeometricallyaccurateandphotometricallycor-rect3-D modelsof complex outdoorurbanenviron-ments. Theseenvironmentsare typified by largestructures(i.e. buildings) that encompassa widerangeof geometricshapesandavery largescopeofphotometricproperties.3-D modelsof suchenvi-ronments,known assitemodels,areusedin manydifferent applicationsrangingfrom city planning,urbandesign,fire andpoliceplanning,military ap-plications,virtual realitymodelingandothers.Thismodelingis doneprimarily by hand,andowing tothecomplexity of theseenvironments,is extremelypainstaking.Researcherswantingtousethesemod-elshaveto eitherbuild theirown limited, inaccuratemodels,or rely onexpensive commercialdatabasesthat are themselves inaccurateand lacking in fullfeaturefunctionality thathigh resolutionmodelingdemands.For example,many of theurbanmodelscurrentlyavailablearea mix of graphicsandCADprimitivesthatvisually maylook correct,but uponfurtherinspectionarefoundto begeometricallyandtopologicallylacking. Buildingsmay have unsup-portedstructures,holes,danglingedgesandfaces,andothercommonproblemsassociatedwith graph-ics vs. topologicallycorrectCAD modeling. Fur-ther, photometricpropertiesof thebuildingsareei-ther missing entirely or are overlaid from a fewaerialviewsthatfail to seemany surfacesandhencecannotaddtheappropriatetextureandvisualprop-ertiesof theenvironment.Ourgoalis to haveamo-bile systemthatwill autonomouslymove aroundasite andcreateanaccurateandcompletemodelofthatenvironmentwith limited humaninteraction.

Thereareanumberof fundamentalscientificissuesinvolved in automatedsite modeling. The first is

ho�

w to createageometricandtopologicallycorrect3-D solid from noisy data. A key problemhereismergingmultipleviewsof thesamescenefrom dif-ferentviewpoints to createa consistentmodel. Inaddition, the modelsshouldbe in a format that isCAD compatiblefor further upstreamprocessingandinterfacingto higherlevel applications.A sec-ond fundamentalproblemis how to plan the nextview to alleviate occlusionsandprovide full cov-erageof the scene.Given the large datasetsizes,reducingthenumberof views while providing fullcoverageof the sceneis a major goal. If a mo-bile agentis usedto acquirethe views, thenplan-ningandnavigationalgorithmsareneededto prop-erly position the mobile agent. Third, the mod-els needto integratephotometricpropertiesof thescenewith theunderlyinggeometryof themodeltoproducea realisticeffect. This requiresdevelopingmethodsthat canfuseandintegraterangeandim-agedata.Fourth,methodsthatreducethecomplex-ity of themodelswhile retainingfidelity areneeded.This paperfocuseson solving the first two prob-lems,modelacquisitionandview planning.

Previous work in the modelacquisitionphasefo-cuseson constructionof models of 3-D objectsfrom rangedata,typically smallobjectsfor reverseengineeringor virtual reality applications.Exam-ples of theseefforts include the groupsat Stan-ford [17, 4], CMU [18], UPENN [7], and Utah[16]. However, thesemethodshave not beenusedon larger objectswith multiple parts. Researchspecificallyaddressingthe modelingof large out-door environmentsincludesthe FACADE systemdevelopedat Berkeley [5]. This is an exampleofasystemthatmergesgeometric3-D modelingwithphotometricpropertiesof the sceneto createreal-istic modelsof outdoor, urbanenvironments. Thesystemhowever, requireshumaninteractionto cre-ate the underlying3-D geometricalmodel and tomake the initial associationsbetween2D imageryandthemodel. Teller et al. [15, 3] aredevelopinga systemto modeloutdoorurbanscenesusing2-Dimageryandlargesphericalmosaics.A numberofothergroupsarealsocreatingImage-Basedpanora-masof outdoorscenesincluding[11, 6].

Our approachto automaticsitemodelingis funda-mentallydifferentfrom othersystems.First,weare

explicitly usingrangedatato createtheunderlyinggeometricmodelof thescene.Wehaveadevelopeda robustandaccuratemethodto acquireandmergerangescansinto topologically correct3-D solids.This systemhasbeentestedon indoormodelsandwe areextendingit to outdoorsceneswith multi-ple objects. Secondly, we areusingour own sen-sor planningsystemto limit the numberof viewsneededto createa completemodel. This plannerallows a partially reconstructedmodelto drive thesensingprocess,whereasmostotherapproachesas-sumecoverageof thesceneis adequateor usehu-maninteractionto decidewhich viewing positionswill beneeded/used.Detailsonourapproachareinthefollowing sections.

The testbedwe areusingfor this researchconsistsof a mobilevehiclewe areequippingwith sensorsandalgorithmsto accomplishthis task. A pictureof the vehicle is shown in figure 1. The equip-mentconsistsof anRWI ATRV mobilerobotbase,arangescanner(80 meterrangespotscannerwith 2-DOF scanningmirrorsfor acquiringa wholerangeimage), centimeteraccuracy onboardGPS,colorcamerasfor obtainingphotometryof thescene,andmobile wirelesscommunicationsfor transmissionof dataandhighlevel controlfunctions.Briefly, wewill describehow a sitemodelwill beconstructed.The mobile robot basewill acquirea partial, in-complete3-D modelfrom asmallnumberof view-points. This partial solid modelwill thenbe usedto plan thenext viewpoint, taking into accountthesensingconstraintsof field of view and visibilityfor thesensors.Therobotwill benavigatedto thisnew viewpoint and merge the next view with thepartial model to updateit. At eachsensingposi-tion, both rangeandphotometricimagerywill beacquiredand integratedinto the model. By accu-rately calculatingthe position of the mobile basevia the onboardGPSsystem,we canintegratetheviews from multiple scansandimagesto build anaccurateandcompletemodel. Both 3-D and2-Ddata,indexed by the locationof the scan,will beusedto capturethefull complexity of thescene.

2 Model Acquisition

We have developeda methodwhich takesa smallnumberof rangeimagesand builds a very accu-

Figure 1: Mobile robot baseand sensors(laserrangefindernotshown).

rate 3-D CAD model of an object [8, 10, 9, 2].The methodis an incrementalonethat interleavesa sensingoperationthat acquiresand merges in-formation into the model with a planning phaseto determinethe next sensorposition or “view”.Themodelacquisitionsystemprovidesfacilitiesforrangeimageacquisition,solid modelconstruction,and modelmerging: both meshsurfaceand solidrepresentationsare usedto build a model of therangedatafrom eachview, which is thenmergedwith the model built from previous sensingoper-ations. The planningsystemutilizes the resultingincompletemodel to plan the next sensingoper-ation by finding a sensorviewpoint that will im-provethefidelity of themodelandreducetheuncer-tainty causedby object occlusion(including self-occlusion).

We now describehow our systemworks. For eachrangescan,ameshsurfaceis formedand“swept” tocreateasolidvolumemodelof boththeimagedob-jectsurfacesandtheoccludedvolume.Thisis doneby applyinganextrusionoperatorto eachtriangularmeshelement,sweepingit alongthe vectorof therangefinder’s sensingaxis,until it comesin contactwith a far boundingplane. The result is a 5-sidedtriangularprism. A regularizedsetunionoperationis appliedto the set of prisms,which producesapolyhedralsolidconsistingof threesetsof surfaces:a mesh-like surfacefrom the acquiredrangedata,a numberof lateral facesequal to the numberof

verticeson theboundaryof themeshderived fromthesweepingoperation,andaboundingsurfacethatcapsone end. Eachof thesesurfacesare taggedas“imaged”or “unimaged”for thesensorplanningphasethatfollows.

Each successive sensingoperationwill result innew informationthatmustbemergedwith thecur-rentmodelbeingbuilt, calledthecompositemodel.Themerging processitself startsby initializing thecompositemodelto betheentireboundedspaceofour modelingsystem.Theinformationdeterminedby a newly acquiredmodel from a single view-point is incorporatedinto the compositemodelbyperforminga regularizedsetintersectionoperationbetweenthe two. The intersectionoperationmustbeableto correctlypropagatethesurface-typetagsfrom surfacesin the modelsthroughto the com-positemodel. Retainingthesetagsafter mergingoperationsallowsviewpointplanningfor unimagedsurfacesto proceed.

3 View Planning

The sensorplanningphaseplans the next sensororientationso that eachadditionalsensingopera-tion recovers object surfacethat hasnot yet beenmodeled.Usingthis planningcomponentmakesitpossibleto reducethenumberof sensingoperationsto recover a model: systemswithout planningtendto usehumaninteractionor overly large datasetswith significantoverlapbetweenthem. This con-ceptof reducingthe numberof scansis importantfor reducingthetime andcomplexity of themodelbuilding process.

In clutteredandcomplex environmentssuchasur-ban scenes,it can be very difficult to determinewherea sensorshouldbe placedto view multipleobjectsand regions of interest. It is importanttonotethatthissensorplacementproblemhastwo in-tertwinedcomponents.The first is a purely geo-metricplannerthatcanreasonaboutocclusionandvisibility in the scene. The secondcomponentisanunderstandingof theopticalconstraintsimposedby the particularsensor(i.e. camerasand rangescanners)that will affect the view from a particu-lar chosenviewpoint. Theseincludedepth-of-field,resolutionof the image,and field-of-view, whicharecontrolledby aperturesettings,lenssizefocal

Figure 2: a) Simulated city environment onturntable. b) Visibility volume after 4 scans. c)Discretizedsensorpositionsusedto determinenextview.

Figure 3: Recovered 3-D models- all 3 objectswererecoveredat once,using12 scanswith plan-ning after the initial 4 scans.Visibility andocclu-sion volumeshave beenusedto plan the correctnext views for the sceneto reducethe uncertaintyin themodel.Noterecoveredarchesandsupports

length�

for camerasandkinematicconstraintsin thecaseof a spot rangingsensor. To properlyplan acorrectview, all of thesecomponentsmustbecon-sidered.

Thecoreof oursystemis asensorplanningmodulewhichperformsthecomputationof thelocusof ad-missibleviewpoints in the 3-D spacewith respectto a 3-D modelof objectsanda setof target fea-turesto beviewed. This locusis calledtheVisibil-ity Volume. At eachpoint of the visibility volumea camerahasanunoccludedview of all target fea-tures,albeit with a possibly infinite imageplane.Thefinite imageplaneandfocal lengthconstraintswill limit thefield of view, andthis imposesa sec-ond constraintwhich leadsto the computationoffield of view coneswhich limit the minimum dis-tancebetweenthe sensorand the target for eachcameraorientation. The integration of visibilityand optical constraintsleadsto a volume of can-didateviewpoints.Thisvolumecanthenbeusedasthe goal region of the mobile robot navigation al-gorithm which will move the robot to a viewpointwithin thisvolume.

The computationof the visibility volumeinvolvesthe computationof the boundaryof the freespace(the part of the 3-D spacewhich is not occupiedby objects)andthe boundarybetweenthe visibil-ity volumeandtheoccludingvolume, which is thecomplementof the visibility with respectto thefree space.In order to do that we decomposetheboundaryof thesceneobjectsinto convex polygonsandcomputethepartial occludingvolumebetweeneachconvex boundarypolygonandeachof thetar-gets which are assumedto be convex polygons.Multiple targetscanbeplannedfor, andthesystemcanhandleconcave targetsby decomposingtheminto convex regions. We discardthosepolygonswhichprovide redundantinformation,thusincreas-ing theefficiency of our method.Theboundaryoftheintersectionof all partialvisibility volumes(seenext section)is guaranteedto be the boundarybe-tweenthevisibility andtheoccludingvolume.Theboundaryof the freespaceis simply theboundaryof thesceneobjects.

We now describehow the plannercomputesvisi-bility takinginto accountocclusion.Themethodisbasedonourpreviouswork in automatedvisualin-

spection[13, 1]. Our modelbuilding methodcom-putesa solid modelat eachstep. Thefacesof thismodelconsistof correctlyimagedfacesandfacesthat arethe resultof the extrusion/sweepingoper-ation. We can label thesefacesas “imaged” or“unimaged” and propagate/updatetheselabelsasnew scansareintegratedinto thecompositemodel.Thefaceslabeled“unimaged”arethenthefocusofthe sensorplanningsystemwhich will try to posi-tion thesensorto allow these“unimaged”facestobescanned.

Given an unimagedtarget face � on the partialmodel, the plannerconstructsa visibility volume��� �������

. This volume specifiesthe set of all sen-sor positionsthat have an unoccludedview of thetarget.Thiscancomputedin four steps:

1. Compute���������������� � �

, thevisibility volumefor� assumingtherewereno occlusions- a halfspaceononesideof � .

2. Compute� , the setof occludingmodelsur-facesby including model surface � if �! ��������������"� � �$#%'&

3. Computetheset ( of volumescontainingtheset of sensorpositionsoccludedfrom � byeachelementof � .

4. Compute�)��*��� �+�

=�)�����������,��� � �.-0/214356187 (

Thevolumedescribedby���������������� � �

is ahalf-spacewhosedefiningplaneis coincidentwith thetarget’sface,with thehalf-space’s interior beingin thedi-rectionof thetarget’ssurfacenormal.Eachelementof O is generatedby the decomposition-basedoc-clusionalgorithmpresentedin [14], anddescribesthesetof sensorpositionsthata singlemodelsur-faceoccludesfrom thetarget. It is importantto notethat this algorithmfor determiningvisibility doesnotuseasensormodel,andin factpartof its attrac-tivenessis that it is sensor-independent. However,for reasonsof computationalefficiency it makessenseto reducethe numberof surfacesin � , andthereforethe numberof surfacesusedto calculate( . Thiscanbedoneby embodyingsensor-specificconstraintsinto theplanner.

3.19

Example: City Scene

We now show a planningexampleof a complexsceneusingmultiple targets. Figure2a is a simu-latedcity scenemadeup of threemodelbuildingsplacedon a laserscannerturntable. This sceneiscomposedof multipleobjectsandhashighself oc-clusion.Themodelingprocesswasinitiatedby theacquisitionof four rangeimages,with 90 turntablerotationsbetweenthem, to producea preliminarymodelthatcontainedmany unimagedsurfaces.Ap-proximately 25% of the entire acquirablemodelsurface is at this point composedof “occluded”surface (“acquirablemodel surface” in this con-text meansthose“occluded” surfacesthat arenotin a horizontalorientation,suchasthe roofs). Af-terdecimatingtheoccludedsurfaces,the30 largestby areawerechosenanda planwasgeneratedforthem.Figure2b shows

�)��*�����+�for eachof these30

surfaces,with adecimatedcopy of thecity sceneatthecenterto allow thereaderto observe therelativeorientations.Thesevisibility volumesarethenin-tersectedwith

�:�� � ��;<� � �, which is thevolumerep-

resentingthesensorplacementconstraints,to yieldthe setsof occlusion-freesensorpositionsfor thetargets,asshown in figure2c. In this imagingsetupof a turntable-laser,

�:�� � ��;<� � �is a cylindrical vol-

ume. A discretesolutionis desiredfor the propernumberof degreesto rotate the turntablefor thenext view. To accomplishthis, the sensorspacehasbeendiscretizedevery =?> , with the total targetareaacquiredat eachpositionfoundby testingthecontinuous-spaceplansfor intersectionwith a ver-tical line at the appropriatepositionon the cylin-der representingthe sensorplacementconstraint.This is a planninghistogramwherethe height ofeachbarrepresentstheareaof targetsurfacesvisi-ble from that sensorlocation,with higherbarsde-noting desirablesensorlocations,lower oneslessso. Theangleof turntablerotationis foundby se-lecting the peakin the planninghistogram. Afterthenext rangeimageis taken, its modelis mergedwith the existing compositemodel, and the plan-ningprocessis repeated.After a total of 12 imageshavebeenautomaticallyacquired,modeled,andin-tegrated,thefinal modelis shown in figure3. Themodelshave beentexturemappedwith aMondrianpaintingCheckerboard with Light Colors to high-

light thegeometricrecovery.

3.2 Analysis: City Scene

Figure4 shows somequantitative resultsfrom themodelbuilding phase.Theentriesin thetableare:

@ Vol - Thetotal volumeof themodel.

@ SurfaceArea - The total surfaceareaof themodel.

@ Occ. Area - The total areaof all occluded“unimaged” surfacesthat have a significantcomponentof their surface normals in theworld x-y plane. This preventsthe inclusionof ”roof” features,which cannot beacquiredandshouldnotbeplannedfor, in thissum.

@ PlanArea- Thetotalsurfaceareaof thetargetsfor whichplanshave beengenerated.

@ PercentPlanned- Thesurfaceareaof planned-for targets,as a percentageof the total ”oc-cluded”surfacearea.

Eachof thesemetricswas calculatedalgorithmi-cally on the computermodel. As shown in figure4, thefirst 4 viewswereacquiredwithoutany plan-ning (View 0 is just theentirevolumebeforescan-ning). In the datadescribingthe remainingviewstherearesomefeaturesthatseemintuitive. Theto-tal modelvolumedecreasesover time, asindeeditmustfor a systemthat usessetintersectionfor in-tegrationandhasnot duplicatedany sensorview-points. Of particularimport is thedatain thefinalcolumn. Becausethe plansarecomputedusingafixednumberof surfacesat eachiteration,it is in-terestingto seewhat percentageof the total avail-abletargetareais beingplannedfor. Clearly, if ev-ery target surfacewereconsidered,this would be100%eachtime. Eventhoughonly 30of thelargesttargetsby areaareplannedfor, the percentof theplannedareanever dropsbelow 10% of the totalarea,and in mostcasesis over 20%. This showsthat the considerablecomputationalcostsaved byselectinga subsetof the targetsto plan for is a vi-ablestrategy. The actualvolumeof the city scenehasbeencalculatedfrom measurementsmadebyhandas362A�BDC .

View Vol. Surface Occ. Plan PercentNo. Area Area Area Planned0 4712 1571 15711 1840 1317 9422 1052 1151 5903 506 733 2004 432 658 1405 416 656 121 61 50%6 404 659 104 28 27%7 391 657 90 12 13%8 386 647 84 8 10%9 382 644 75 15 20%10 380 651 62 7 11%11 374 622 53 16 30%12 370 604 36 9 25%

Figure4: Analysisof theplanner’sability to reduceuncertaintyandcreateaccuratemodels.

4 Integrating the Field of View Constraintfor Cameras

To fully planviews, we needto take into consider-ation the constraintson the rangescannerto scanunimagedsurfacesas we did in the previous sec-tion. We also needto understandthe constraintson cameraswhich will be usedto acquirephoto-metricpropertiesof thescene.We now discusstheconstraintsrelatedto 2-D imagingsensors.A view-point which lies in thevisibility volumehasanun-occludedview of all targetfeaturesin thesensethatall lines of sight do not intersectany object(otherthan the target) in the environment. This is a ge-ometricconstraintthat hasto be satisfied. Visualsensorshowever imposeoptical constraintshavingto dowith thephysicsof thelens(Gaussianlenslawfor thin lens),thefinite aperture,thefinite extentofthe imageplaneandthefinite spatialresolutionofthe resultingimageformedon the imageplane,aswell aslensdistortionsandaberrations.

An importantconstraintis the field of view con-straint for a camerawhich is relatedto the finitesizeof the active sensorareaon the imageplane.Givena partialmodelwith sometarget surfacesE�Fwecanplanaviewpoint to acquireacorrectcameraimageof thesesurfaces.Thetargets E F areimagedif their projectionlies entirelyon theactive sensorareaon theimageplane.This active sensorareais

a 2-D planarregion of finite extent. Thusthe pro-jectionof thetargetfeaturesin their entiretyon theimageplanedependsnotonly on theviewpoint GIH ,but alsoon theorientationof thecamera,theeffec-tivefocal lengthandthesizeandshapeof theactivesensorarea.

For a specificfield of view angle J anda specificviewing direction K we can computethe locusofviewpointswhichsatisfythefieldof view constraintfor thesetof targets � . If we approximatethe setof targetswith asphereLMH of radiusNOH andof cen-ter P?Q containingthem, then this locus is a circu-lar coneRSH ��T?U K 3 J 3 P Q 3 NOHMV , calledthefield of viewcone(seefigure 5). The coneaxis is parallelto Kandits angleis J . Viewpointscantranslateinsidethis volume(theorientationis fixed)while the tar-getsareimagedontheactivesensorarea.Thelocusof the apicesof theseconesfor all viewing direc-tions is a spherewhosecenteris P?Q and its radiusis NOHXWZY\[�] U JW^=^V (figure5). For every viewpoint ly-ing outsideof this spherethereexists at leastonecameraorientationwhich satisfiesthefield of viewconstraint,sincethis region is definedas:

_*` RSH��T?U K 3 J 3 P?Q 3 NOHMV

For viewpointsinsidethespheretheredoesnot ex-ist any orientationwhich could satisfythe field ofview constraint(the camerais too closeto the tar-gets).Theapproximationof thetargetsby a spheresimplifies the field of view computation. It pro-vides,however, a conservative solutionto thefieldof view problemsincewerequirethewholesphereto beimagedon theactive sensorarea.

Thefield of view anglefor acircularsensorhavinga radiusof a ; F � , is J % =Zb"c^]Zdfe U a ; F � W^=^gfV , whereg is the effective focal lengthof the camera.Forrectangularsensorsthesensorareais approximatedby the enclosingcircle. The field of view angle Jdoesnotdependon theviewpointor theorientationof the camera. In our system,both the visibilityvolumeand field of view coneare representedassolid CAD models.This allows us to usebooleansetintersectionon theseregionsof spaceto find anadmissibleviewing volumethatencodesbothcon-straints.Intuitively, this solid is theresultof thein-tersectionof thevisibility volumefor a targetwith

R f

αv

rk

Rf/sin(α/2)

rs

Figure 5: Field of view cone(shadedregion) forviewing direction K andfield of view angle J . Thetargetsareenclosedin thesphereof radiusNOH

a field of view cone.For multiple targets,we sim-ply intersectall thevolumesandfind theadmissibleregion. Examplesof theseregionsaregiven in thenext section.

4.1 Example: Merging Visibility and Field-of-View Constraints

To testthesealgorithms,we have built an interac-tive graphicalsimulatorwheresensorplanningex-perimentscan be performed[12]. Figure 6 is anoverview of this systemwhich allows us to gener-ate, load andmanipulatedifferent typesof scenesandinteractively selectthetargetfeaturesthatmustbe visible by a camera.The resultsof the sensorplanningexperimentsaredisplayedas3-D volumesof viewpoints that encodethe constraints.Virtualcamerasplacedin thosevolumesprovide a meansof synthesizingviews in real-timeand evaluatingviewpoints.Thesystemcanalsobeusedto provideanimated“fly throughs”of thescene.

We now describean exampleof how this systemworksfor aplanningcameraviewpointsin anurbanenvironment.Figure 7 is a sitemodelof Rosslyn,Virginia. Our inputwasanOpenInventormodelofthecity (given to usby GDE SystemsInc.), that isa setof polygonswithout topologicalinformation.While visually compelling,the model is not topo-logically correct. As statedearlier, this is not un-usual- thesemodelstypically have danglingfaces,unsupportedstructuresand empty voids that cancauseproblemsin upstreamapplicationsthat ex-pecta correctCAD model. Oncewe have created

a correctCAD model,we canthenusethe sensorplannerto improve themodelvia navigation to re-gionsin thescenethatwill allow visibility andcor-rect field of view for imagingsensors.To do this,wehave transformedthismodelinto aCAD modelusinga setof interactive tools we have developed[12]. TheCAD modelconsistsof 488buildingsandwe testedour sensorplanningalgorithmson a por-tion of thismodelwhoseboundaryconsistedof 566planarfaces(seefigure 8).

In thefirst experiment(figure9a)onetarget (blackface)is placedinsidetheurbanareaof interest.Thevisibility volumeis computedanddisplayed(trans-parentpolyhedralvolume). For a viewing direc-tion of Kih % Ukj � 3 =^= � 3lj � V (Euler angleswith re-spectto theglobalCartesiancoordinatesystem)andfield of view angleof J e %nm^m � , the field of viewlocus is the transparentconeon the left. The setof candidateviewpoints a e U Kih 3 J e V (intersectionofvisibility with field of view volume) is the partialconeon the left. For a differentviewing directionKio % Ukj � 3lp^q � 3lj � V thesetof candidateviewpointsa e U Kio 3 J e V is thepartialconeon theright.

In the secondexperiment(figure9b) a secondtar-getis addedsothattwo targets(blackplanarfaces)needto bevisible. Thevisibility volume,thefieldof view conefor thedirection Kih andthecandidatevolumesa�r U K h 3 J e V (left) and a�r U Kis 3 J e V (right) aredisplayed.The viewing orientationK s is equaltoUkj � 3lt^q � 3lj � V . The visibility volume and the can-didatevolume a r U Kih 3 J e V aresubsetsof the corre-spondingonesin thefirst experiment.

If we place a virtual camerainside the volumea e U Kio 3 J e V (pointUku^j^j^v�p^j^3lw^x^v�q^y^3lu = w^v�w?x V ), set the

field view angle to J e and the orientationto Kio ,then the synthesizedview is displayedon figure10a. The target is clearly visible. Placinga vir-tual cameraoutsideof thevisibility volume(pointUkw^j^p^v�p = 3 m q^v�t^j^3lu^x?x^v�y?j V ) resultsin the synthesizedview of figure10b. Clearlythetargetis occludedbyoneobjectof thescene.Theorientationof thecam-erais

Ukj � 3ly m � 3lj � V (for every viewpoint outsidethevisibility volume theredoesnot exist any cameraorientationthatwouldresultin anunoccludedviewof the target). If we placea virtual cameraon theboundaryof the the candidatevolume a e U Kio 3 J e V(point

Uku^t^w^v�w^p^3lw = v�u^x^3lu m y^v m t V ), thenin the result-

Figure6: Interactive sensorplanningsystem

Figure7: VRML Graphicsmodelof Rosslyn

ing synthesizedview (figure 10c) we seethat theimageof the target is tangentto the imageof oneobjectof thescene.AgainthecameraorientationisKio andthefield of view angleJ e .In figure 10d we seea synthesizedview whenthecamerais placedon the conical boundaryof thecandidatevolume a r U K s 3 J e V . The camera’s po-sition is

Ukq^w^p^v m = 3lu^j^v = m 3lu m t^vzu^w V . The transparentsphereis thesphereLMH usedto enclosethetargets.We seethat LMH is tangentto thebottomedgeof theimage,becausetheviewpoint lies on theboundaryof thefield of view cone.Finally thefigure10ehasbeengeneratedby a cameraplacedon thepolyhe-dral boundaryof the candidatevolume a r U K s 3 J e V(position

U = w m v�t^y^3 m p^v = y^3lu^w?j^v m w V ).5 Conclusions

This paperdescribesa methodfor acquiringcom-plex 3-D modelsfrom outdoorurbanscenesthatin-

Figure 8: Solid CAD model computedfrom thegraphicsmodel.

cludesanintegral planningcomponent.Themodelacquisitionprocessis basedupon an incrementalvolumetric methodthat can merge multiple rangedatascansinto a coherent,topologicallycorrect3-D model. The plannercan incorporatevisibility,field-of-view and sensorplacementconstraintsindeterminingwhereto take the next view to reducemodeluncertainty. Resultshave beenpresentedforrangedataacquisitionof amodelof asimulatedur-banscenewith highocclusionandfor planningcor-rectviewpointsfor acamerain amodelof Rosslyn,Virginia. usingan interactive planningsystem. Itcan computevisibility and field of view volumesandtheir intersectionwhichyieldsa locusof view-points which are guaranteedto be occlusion–freeand placestargets within the field of view. Ob-jectmodelsandtargetscanbeinteractively manipu-latedandcamerapositionsandparametersselectedto generatesynthesizedimagesof the targetsthatencodetheviewing constraints.

Given a partial site model of a scene,the systemcanbeusedto planview positionsfor a varietyof

tasks.{

We are currently extendingthis systemtoincluderesolutionconstraintsandto createmobilerobot navigation algorithmsbasedupon the plan-ner’s output. Futurework alsoincludesfusing therangedatamodelswith the2-D cameraimagerytocreateevenmorerealisticsitemodels.

References

[1] S.Abrams.SensorPlanningin anactiverobotwork-cell. PhD thesis,Departmentof Com-puterScience,ColumbiaUniversity, January1997.

[2] P. K. Allen and R. Yang. Registering,inte-grating,andbuilding CAD modelsfrom rangedata. In IEEE Int. Conf. on RoboticsandAu-tomation, May 18-201998.

[3] S. Coorg, M. Master, andS. Teller. Acquisi-tion of largepose-mosaicdataset.In Confer-enceon ComputerVision andPatternRecog-nition, pages872–878,June23-251998.

[4] B. Curless and M. Levoy. A volumetricmethod for building complex models fromrangeimages.In Proceedingsof SIGGRAPH,1996.

[5] P. Debevec,C. J. Taylor, andJ. Malik. Mod-eling and renderingarchitecturefrom pho-tographs: A hybrid geometry and image-basedapproach. In Proceedingsof SIG-GRAPH, August1996.

[6] L. McMillen andG. Bishop. Plenopticmod-eling: An image-basedrenderingsystem. InSIGGRAPH’95, pages39–46,August1995.

[7] R. Pito andR. Bajcsy. A solutionto thenextbestview problemfor automatedCAD modelacquisitionof free-form objectsusing rangecameras.In ProceedingsSPIESymposiumonIntelligent Systemsand AdvancedManufac-turing, Phila. PA, 1995.

[8] M. ReedandP. K. Allen. 3-D modelingfromrangeimagery:An incrementalmethodwith aplanningcomponent.Image andVisionCom-puting, (to appear)1998.

[9] M. Reed,P. K. Allen, andI. Stamos. Auto-matedmodel acquisitionfrom rangeimageswith view planning. In ComputerVision andPattern Recognition Conference, pages72–77,June16-201997.

[10] M. Reed,P. K. Allen, and I. Stamos. Solidmodel constructionusing meshesand vol-umes.In Proc.DARPA Image UnderstandingWorkshop, pages921–926,May 12-141997.

[11] H. ShumandR. Szeliski. Panoramicimagemosaics. TechnicalReport TR-97-23, Mi-crosoftResearch,1997.

[12] I. Stamosand P. K. Allen. Interactive sen-sorplanning.In ComputerVisionandPatternRecognition Conference(CVPR), pages489–494,June21-231998.

[13] K. Tarabanis,P. K. Allen, andR. Tsai. TheMVP sensorplanningsystemfor robotic vi-sion tasks. IEEE Transactionson RoboticsandAutomation, 11(1):72–85,February1995.

[14] K. Tarabanis, R. Y. Tsai, and A. Kaul.Computingocclusion-freeviewpoints. IEEETransactionsPattern Analysisand MachineIntelligence, 18(3):279–292,March1996.

[15] S. Teller. Automaticacquisitionof hierarchi-cal, textured 3d geometricmodelsof urbanenvironments:Projectplan,view planningforsite modeling. In Proc. DARPA Image Un-derstandingWorkshop, pages767–770,NewOrleans,1997.

[16] W. Thompson,H. de St. Germain,T. Hen-derson, and J. Owen. Constructinghigh-precisiongeometricmodelsfrom sensedpo-sition data. In Proceedings1996 ARPA Im-age Understanding Workshop, pages987–994,1996.

[17] G. Turk and M. Levoy. Zipperedpolygonmeshesfrom rangeimages. In Proceedingsof SIGGRAPH, pages311–318,1994.

[18] M. Wheeler. AutomaticModelingand local-ization for object recognition. PhD thesis,Schoolof ComputerScience,Carnegie Mel-lon University, October1996.

Target

v|

v|1

2

Field of View Cone

Visibility Volume

Field of View Cone

Visibility Volume

v1

3v

Targets

Figure9: Two experiments.a) (top figure) Onetarget andb) (bottomfigure) two targetsareplacedintheurbanarea.Thetargetsareplanarfaces.TheVisibility Volumes(transparentpolyhedralvolumes),theField of View Conesfor thedirection Kih (transparentcones)andtheCandidateVolumes(intersectionofthevisibility volumeswith thefield of view cones)for theviewing direction Kih (left partialcones)andforthe directionsKio (right partial cone,top figure)and K s (right partial cone,bottomfigure) aredisplayed.TheFieldof View Conesfor thedirectionsKio (top)and K s (bottom)arenotshown.

Figure10: Synthesizedviews. Singletarget (black face): thecamerais placeda) (left image)insidethecandidatevolume,b) out of thevisibility volumeandc) on theboundaryof thecandidatevolume. Twotargets:thecamerais placedon d) theconicalboundaryande) thepolyhedralboundaryof thecandidatevolume.