camera-based calibration techniques for seamless multi-projector

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. X, NO. X, MONTH YEAR 1

Camera-Based Calibration Techniquesfor Seamless Multi-Projector Displays

Michael Brown∗, Aditi Majumder†, Ruigang Yang‡

Abstract— Multi-projector, large-scale displays are usedin scientific visualization, virtual reality and other visu-ally intensive applications. In recent years, a number ofcamera-based computer vision techniques have been pro-posed to register thegeometryand color of tiled projection-based display. These automated techniques use cameras to“calibrate” display geometry and photometry, computingper-projector corrective warps and intensity correctionsthat are necessary to produce seamless imagery acrossprojector mosaics. These techniques replace the traditionallabor-intensive manual alignment and maintenance steps,making such displays cost-effective, flexible, and accessible.

In this paper, we present a survey of different camera-based geometric and photometric registration techniquesreported in the literature to date. We discuss several tech-niques that have been proposed and demonstrated, eachaddressing particular display configurations and modesof operation. We overview each of these approaches anddiscuss their advantages and disadvantages. We examinetechniques that address registration on both planar (videowalls) and arbitrary display surfaces and photometriccorrection for different kinds of display surfaces. Weconclude with a discussion of the remaining challengesand research opportunities for multi-projector displays.

Index Terms— Survey, Large-Format Displays, Large-Scale Displays, Geometric Alignment, Photometric Align-ment, Graphics Systems, Graphics.

I. I NTRODUCTION

Expensive monolithic rendering engines and special-ized light projectors have traditionally made projector-based displays an expensive “luxury” for large-scalevisualization. However, with advances in PC graphicshardware and light projector technology, it is now pos-sible to build such displays with significantly cheapercomponents. Systems, such as Li et al.’s Scalable DisplayWall [23], Matusik and Pfister’s 3D TV [31], and dis-plays constructed using Humphreys et al.’sWireGL [19]and Chromium [18] PC-cluster rendering architecture,

∗M. S. Brown is with the Hong Kong Univ. of Science andTechnology.

†A. Majumder is with the University of California, Irvine.‡R. Yang is with the University of Kentucky, Lexington, KY.

Fig. 1. Camera-based geometric registration is used to calculateimage-based corrections that can generate a seamless image fromseveral (unaligned) overlapping projectors.

have demonstrated the feasibility of cost-effective, large-format displays assembling from many commodity pro-jectors and PCs.

Images from a multi-projector display must be seam-less, i.e., they must appear as if they were being projectedfrom a single display device. This involves correctingfor geometric misalignment and color variation withinand across the different projectors to create a finalimage that is both geometrically and photometricallyseamless. This correction process is commonly referredto as “calibration”. Calibration involves two aspects:geometric registrationand color correction. Geometricregistration deals with geometric continuity of the entiredisplay, e.g., a straight line across a display made frommultiple projectors should remain straight. Photometriccorrection deals with the color continuity of the display,e.g., the brightness of the projected imagery should notvary visibly within the display.

Calibration can be achieved through mechanical andelectronic alignment, a common approach adopted bymany research and commercial systems [12], [23], [19],[14], [36]. Such alignment procedures often require aspecialized display infrastructure and a great deal ofpersonnel resources, to both set up and maintain. Thissignificantly increases the cost and effort needed todeploy such large-scale, high-resolution displays. Often,half of a display’s total cost is related to the displayinfrastructure, including the mounting hardware and dis-


Fig. 2. Left: This image illustrates the geometric misalignment problem at the boundary of two overlapping projectors. The geometry isnoticeably unaligned. Right: This image shows the final seamless imagery of the same projector alignment. This registration is the goal ofgeometric registration methods.

play screens. In addition, most reasonably sophisticatedmounting hardware does not have the capability or theprecision to correct non-linear distortions such as projec-tor radial distortion and intensity non-linearities. Further,manual methods tend to be unscalable. Calibrating evena four-projector system can be severely time consuming.

Recently, techniques have been developed that use oneor more cameras to observe a given display setup in arelaxed alignment, where projectors are onlycasuallyaligned. Using feedback obtained from a camera ob-serving the display setup, the necessary adjustments toregister the imagery, both in terms of geometry and color,can be automatically computed and applied throughsoftware [45], [43], [40], [10], [54], [9], [29], [21], [41],[28], [32]. The key idea in these approaches is to usecameras to provideclosed-loop control. The geometricmisalignments and color imbalances are detected by acamera (or cameras) that monitors the contributions ofmultiple light projectors using computer vision tech-niques. The geometric- and color-correction functionsnecessary to enable the generation of a single seam-less image across the entire multi-projector display aredetermined. Finally, the image from each projector isappropriately pre-distorted by the software to achievethe correction (see Figure 1). Thus, projectors can becasually placed and the resulting inaccuracies in geome-tries and color can be corrected automatically by thecamera-based calibration techniques in minutes, greatlysimplifying the deployment of projector-based displays.In comparison with traditional systems relying on precisesetups, camera-based calibration techniques provide thefollowing advantages in particular:

• More flexility. Large-format displays with camera-based calibration can be deployed in a wide varietyof environments, for example, in the corner of aroom, or across a column. These irregularities cancause distortions that traditional systems may finddifficult to work with.

• Easy to setup and maintain. Camera-based cal-ibration techniques can completely automate thesetup of large-format displays. This is particularlyattractive for temporary setups in trade-shows orfield environments. Labor-intensive color balancingand geometric alignment procedures can be avoidedand automated techniques can be used to calibratethe display in just minutes.

• Reduced costs. Since precise mounting of projectorsis not necessary, projectors can be casually placedusing commodity support structures (or even as sim-ple as laying the projectors on a shelf). In addition,it is not necessary to hire trained professionals tomaintain a precise alignment to keep the displayfunctional. Further, since the color variations canalso be compensated, expensive projectors with highquality optics (that assure color uniformity) can bereplaced by inexpensive commodity ones.

While camera-based calibration techniques requirecameras and support hardware to digitize video signals,these costs are amortized by savings from long-termmaintenance costs. Overheads like warping and blendingat rendering time to correct for various distortions arereduced or eliminated by the recent advances in graphicshardware.

In this paper, we present a survey of different camera-based calibration techniques. Our goal is to providepotential developers of large-format displays a usefulsummary of available techniques and a clear under-standing of their benefits and limitations. We start withgeometric registration techniques in Section II. We orga-nize different approaches by the types of configurationsaddressed and the modes of operation accommodated.We discuss techniques for planar or arbitrary displaysurfaces with stationary or moving viewers. We comparethese approaches and point out their positive and negativepoints for given environments and tasks. In Section III,we focus on color correction techniques. Until recently,


expensive color measurement instruments were used tocalibrate the color of such displays and the spatial vari-ation in color was largely ignored. Recent camera-basedtechniques address photometric variation across suchdisplays for surfaces with different reflectance properties.We discuss all these methods along with their advan-tages and disadvantages. In Section IV, we list severalrepresentative systems that use camera-based calibrationand discuss their features. In Section V, we introducesome interesting technological hardware advancementsthat reduce or even remove the rendering overheadthat is typical in performing geometric and photometriccorrections. Finally, we conclude in Section VI with adiscussion of the remaining challenges that still need tobe addressed for projector-based displays.

II. GEOMETRIC REGISTRATION

When building a multiple-projector display, two typesof geometric distortion must be addressed –intra-projectionandinter-projectiondistortion.Intra-projectordistortions are distortions within a single projectorcaused by off-axis projection, radial distortion, andin some cases, display on non-planar surfaces.Inter-projector distortions are found between adjacent projec-tors where edge boundaries do not match. Geometricregistration techniques are used to detect and correct bothtypes of distortion (see example in Figure 2).

Camera-based display registration techniques can bedivided into two categories based on the type of displaysurfaces addressed, eitherplanar or non-planar. Wefirst discuss techniques that assume a planar displaysurface. These are used to construct large-scale videowalls. Later, we extend the discussion to arbitrary displaysurfaces, for example, multiple planar walls or semi-spherical screens. These scenarios are particularly suitedfor immersive displays.

A. Planar Display Surfaces

When the display surface is planar, each projectorPk’s image can be related to a reference frame,R, onthe display surface, via a 2D planar homography (wesuggest [17] for readers unfamiliar with homographies).This projector-to-reference frame homography is denotedasRPk wherek is the index of the projector, and the sub-scriptR denotes that the homography maps the image ofPk to the reference frameR (notation adopted from [9]).To compute the homography, it is necessary to establishfour point-correspondences between coordinate frames.Using more than four point-correspondences allows aleast-squares fit solution which is often desirable in theface of errors and small non-linearities. In practice, most

techniques project many known features per projector tocompute the homographies [10], [23], [42], [9], [54],[41].

The alignment of the projected imagery is achieved bypre-warping the image from every projector,Pk, usingthe homography,RP−1

k . This pre-warp can be performeddirectly in the rendering pipe-line [54] or by using apost-rendering warp [43].

Thus, the key is to determine the correctRPk foreach projectorPk. In essence, we need to establishpoint-correspondences between each projector and thedisplay’s reference frame,R. This can be accomplishedby using a camera (or cameras) to observe the projectedimagery, as shown in Figure 3.

1) Using a Single Camera:We first consider the casewhen only one camera is used. Figure 3 (left) showsan example of this setup. A homography between thecamera and the display reference frameR, denoted byRC, is first computed. Typically, manually selected point-correspondences between the camera image and known2D points on the display surface are used to calculateRC.After RC has been computed, projected imagery fromeachPk is observed by the camera and a projector-to-camera homography for each projectork, is calculated,and denoted asRCk. The projector-to-reference framehomography,RPk, is then derived fromRC andRCk as:

RPk = RC× RCk, (1)

where the operator× represents a matrix multiplication.Raskar et al. [42] presented a system using a single

camera that could compute this mapping for a2 × 2projector array in roughly a few seconds. In this work,a camera was first registered to the display’s refer-ence frame manually. Each projectorPk projected acheckerboard pattern that was observed by the camera.Corners on the checkerboard pattern where determinedin the camera’s image plane, establishing the point cor-respondences between the camera an the projector. Fromthis information, projector-to-camera homographiesRCk

were computed. Next, the necessaryRPk could be com-puted using Eq. 1. This allowed the projected imageryto be correctly aligned to the display reference frame.Raskar et al. reported that this approach could align theprojected imagery with sub-pixel accuracy [42].

More recently, techniques that do not require manuallyselecting points on the display surface have been intro-duced [34], [38]. They show that planar auto-calibration,proposed by Triggs in [51], can be used to determine theintrinsics of an array of projectors projecting on a singleplane.

While these above approaches are effective, the use ofa single camera limits the scalability of these techniques


Single Camera Multiple Cameras

Fig. 3. (Left) 3 × 3 linear homographies are computed that relate the projectors to the display reference frame,R. A camera is used toobserve projected imagery from each projector. (Right) For large field-of-view projector arrays, multiple cameras are used. Each cameraobserves a region of the display. Projector-to-camera homographies concatenated with camera-to-reference frame homographies are used tocompute the necessary projector-to-reference frame mapping.

to displays composed of a larger number of projectors.Such large displays made of40 − 50 projectors areused in many national institutes like Sandia, LawrenceLivermore National Laboratories and the National Centerfor Supercomputing Applications (NCSA) at Universityof Illinois at Champagne Urbana (UIUC).

To address this scalability issue, Y. Chen et al. [10]proposed a method that used a single camera mountedon a pan-tilt unit (PTU). The PTU allowed the camerato move such that it could see a very large field-of-view.Controlled by a PC, the camera could be automaticallymoved to observe points and lines projected by theindividual projectors (the experiment in [10] registeredeight projectors). The camera could relate points andlines from each projector to the display’s global referenceframe,R. A rough alignment of projectors was assumed.This meant that projected points and lines between pro-jectors should be aligned, but were not because of slightmis-registrations. Using the collected data from the cam-era a simulated annealing algorithm was used to computeeachRPk, such that errors between the correspondingprojector points and angles between the correspondinglines where minimized. Y. Chen et al. reported that thisapproach could achieve near pixel accuracy in projectoralignment [10]. While this approach proved to work,it suffered from being slow. Overall time to collectdata from the PTU-mounted camera and to perform thesimulated annealing was reported to be around one hour.Implementation improvements can undoubtedly reducethe overall time.

2) Using Multiple Cameras:More recently, H. Chenet al. [9] proposed a more scalable approach that usesmultiple cameras. Several cameras observing the display

are related to one another by camera-to-camera homogra-phies. A root cameraRC is established as the referenceframe. Adjacent cameras,i and j, are related to oneanother by theiHj homographies. Point correspondencesare established between adjacent cameras by observingprojected points from which theiHjs are computed.Next, each camera is registered to the root camera andthus to the reference frameR. This can be done bycomputing a homographyRHj , which is constructed byconcatenating adjacent camera-to-camera homographiesuntil the root camera is reached as follows (see Fig-ure 3 (right)):

RHj = RC× RHi × · · · ×i Hj . (2)

The homographyRHj maps points in camera,j, to thereference frameR. To determine the path of this camera-to-reference frame concatenation, a minimum-spanning“homography tree” is built that minimizes registrationerrors in the camera-to-camera reference frame [9].

Each projector is now observed by one of the camerasin the system. A single camera in the setup can typicallyobserve only 2-4 projectors from the entire display wall.The projectors can be related to their correspondingcameras via a homography, denoted askCj , wherej isthe camera index andk is the projector index. Usingthe homography tree computed between the cameras,the projector-to-reference homographyRPk for a givenprojectork can be computed as:

RPk = RHj × jCk, (3)

where RHj has been constructed using Eq. 2. Exper-iments in [9] showed that this approach can be veryaccurate in registering projectors. In examples using up


to 32 projectors, sub-pixel local alignment accuracieswere reported. In simulation, this technique was shownto be scalable to scores of projectors and cameras. Inaddition, this approach took only a few minutes to reacha solution with large numbers of projectors.

B. Arbitrary Display Surfaces

The previous homography-based approaches were ap-plicable only if the display surface is planar. Here,we discuss approaches that address non-planar displaysurfaces. These include surround environments such asvideo domes and immersive environments. In addition,these techniques are geared for very flexible deploymentin existing environments, like an office, where largeempty planar display surfaces may be difficult to find.

The approaches we present address two modes ofoperation. One that assumes a stationary viewer and onethat allows a moving user (i.e., a head-tracked viewer).These techniques can of course be applied to planardisplay surfaces as well.

1) Stationary Viewer: Raskar [44] and Surati [45]proposed a registration algorithm that uses a two-passrendering technique to create seamless imagery on arbi-trary display surfaces. In this approach, a single camerais placed at the location from where the viewer is sup-posed to observe the displayed imagery. A set of equallyspaced features are projected from each projectorPk

and registered in the camera image plane. The projectedfeaturesPk(x, y) are typically used to form a tessellatedgrid in the projector space as well as the camera im-age space (see Figure 4). This establishes a non-linearmapping from the projector’s featuresPk(x, y) to theirpositions in the camera’s image planeC(u, v), denotedasC(u, v) 7→ Pk(x, y).

To correct the displayed imagery, a two-pass renderingalgorithm is used. In the first pass, the desired image tobe seen by the viewer is rendered. This desired imagefrom the first pass is then warped to the projected imagebased on theC(u, v) 7→ Pk(x, y) non-linear mapping.This non-linear warp can be realized by a piecewisetexturing between tessellated meshes in the projector andthe camera image space. Thw warping constitutes thesecond rendering pass. For clarity, Figure 4 shows thisprocedure using only one projector. This technique will,however, produce a seamless image even when multipleoverlapping projectors are observed by the camera. Notethat any camera distortion (such as radial distortion) willbe encoded in theC(u, v) 7→ Pk(x, y) mapping. Forcameras with severe radial distortion, e.g. a camera usinga fish-eye lens, this distortion will be noticeable in theresulting image created by the projector mosaic. Care

should be taken to first calibrate the camera to removesuch distortion. Routines to perform this calibration aretypically available in computer vision software packages,such as Intel’s OpenCV [20].

The warp specified from theC(u, v) 7→ Pk(x, y)mapping generates a geometrically correct view fromwhere the camera is positioned. For this reason, thecamera is positioned close to where the viewer will bepositioned while viewing the display. However, as theviewer moves away from this position, the imagery willbegin to appear distorted. Thus, this technique is suitablefor a stationary viewer only.

Yang et al. [54] incorporated this two-pass renderingalgorithm into the University of North Carolina at ChapelHill’s PixelFlexdisplay system. In the PixelFlex system,mirrors are mounted on pan-tilt units (PTU) positionedin front of the projectors. Software allows a user todynamically modify the projectors’ spatial alignmentby moving the mirrors via the PTUs. New projectorconfigurations are registered using the technique de-scribed above. Brown et al. [5] incorporated the sametechnique into the WireGL and Chromium [19] render-ing architecture. This allows users to deploy PC-basedtiled display systems that support unmodified OpenGLapplications. Both Brown et al. [5] and Yang et al. [54]reported sub-pixel projector registration when using thistwo-pass approach. In addition, these approaches canregister the displays in a matter of minutes. Further, thenon-linear warp corrects for both non-linear projectorlens distortion and display surface distortion. Thus, thisapproach allows for very flexible display configurations(non-rectangular projector arrangements on non-planardisplay surfaces). However, the need for two renderingpasses can affect performance. Brown et al. [5] reporteda drop in performance from 60 fps to 30 fps, whenthe second-pass warp was used on a2 × 2 projectorarray using four PCs with nVidia GeForce3 cards. Thisoverhead may be alleviated in the future by having thesecond-pass conducted directly on the projectors (seeSection V). In addition, this technique uses a singlecamera, limiting its scalability.

2) A Moving (Head-Tracked) Viewer:For a movingviewer in an arbitrary display environment, the necessarywarping function between each projector and the de-sired image must be dynamically computed as the viewchanges. Raskar et al. [43] presented an elegant two-passrendering algorithm to address this situation. Figure 5(a)illustrates this two-pass rendering approach. The desiredimage from the viewer’s position is rendered in the firstpass. This image is then projected from the viewer’spoint of view onto a 3D model of the display surfaceusing projective textures. This textured 3D model is then


Fig. 4. (Left) Projectors display features that are observed by a camera placed near the desired viewing location. (Right) The desired imageis (1) rendered and then (2) warped to the projected imagery based on its mapping to the camera.

3D Display Surface

Projector

Moving

Viewer

(Pass-1) Render

desired image.

Project onto a 3D

model of the display

surface.

(Pass-2) Render texture

3D model using

projectors view frusta

Stereo-Camera

Pair S1

3D Global Registration

Stereo-Camera

Pair S2P1 P2

D1

D2

Moving

Viewer

(a) (b)

Fig. 5. a) A two-pass rendering algorithm for a moving viewer and an arbitrary display surface. The first pass renders the desired imageto be observed by the user. This is used as a projective texture and projected from the viewer’s point of view onto the display surface. Thetextured display surface is then rendered from the projector’s point of view constituting the second pass render. When projected, secondpass rendered image will look correct to the viewer. (b) Stereo-camera pairs are used to determine the 3D display surfacesD1 andD2 andprojector locationsP1 andP2. These are then registered into a common coordinate system along with the head tracker.

rendered from the view point of the projector as thesecond rendering pass. When projected by the projector,this second pass image will appear geometrically correctto viewer.

In this algorithm, three components must be known:(1) a 3D model of the display surface, (2) the pro-jectors’ locations (in the form of a view frustum withrespect to the display surface) and (3) the viewer’slocation (with respect to the display surface). Thesethree components need to be registered in a commoncoordinate frame for the algorithm to work. Raskar etal. [40] presented a system that uses several cameras to

determine automatically the 3D geometry of the displaysurface and the location of the projectors within thedisplay environment. The data is then integrated witha head tracker to provide the third necessary componentof viewer location. Figure 5(b) shows an overview of theapproach.

In this system, multiple cameras are first used to formseveral stereo-camera pairsSi to observe the projectedimagery. Typically, one stereo pair is established for eachprojector. Each stereo pair is calibrated using a largecalibration pattern. Note that a particular camera may bea member of more than one stereo pair. Using the stereo


pair Si, the display surfaceDi seen by the projectorPi can be determined using a structured-light technique.Each recovered 3D display surfaceDi is represented as a3D mesh. FromDi, the projector’sPi’ view frustum (i.e.,3D location) with respect to the display surface can becomputed. This completes the computation of the initialunknowns of the 3-D display surface and the projectorlocation for every projector in the display. However, eachDi andPi pair is still registered to different coordinateframes. The next step is to unify them within a commoncoordinate frame.

A stereo pairSi can see its corresponding projectorPi

and any overlapping portion from an adjacent projectorPj . Using 3-D point correspondences acquired in theoverlapping region between two display surfacesDi andDj , a rigid transformation consisting of a rotationiRj ,and a translationiTj can be computed to bringDi andDj

into alignment. Once the display surfaces are in align-ment view frustums,Pi and Pj can also be computedin the same common coordinate frame. Finally, the headtracker is registered to the global coordinate frame. Thiscan be done by registering tracker positions to 3D pointson the displays surface. A rotation and translation tobring the tracker’s coordinates into alignment with thedisplay surface can be computed. With all three nec-essary components registered to a common coordinateframe, the two-pass rendering algorithm for a movinguser can be used to generate seamless imagery.

This approach allows flexible projector alignment anddisplay surfaces and, hence, recovers the underlyingdisplay surface and projector locations automatically. Inaddition, this technique is scalable, allowing immersivedisplays to be deployed in a wide-range of environments.However, due to the large number of parameters thatneed to be estimated (e.g. camera parameters,3D surface,projector parameters) the accuracy of this system isroughly 1-2 pixels.

More recently, Raskar et al. [41] presented a moreefficient method to solve for a smaller set of 3D surfaces,specifically quadratic surfaces. Examples of quadricsurfaces are domes, cylindrical screens, ellipsoids, andparaboloids. Such specialized surfaces are used in sys-tems built for training and simulation purposes. Forquadratic surfaces, the warping to register the images onquadric surfaces can be expressed by a parameterizedtransfer equation. In comparison with the full 3D recon-struction approach in [40], this parameterized approachhas substantially fewer unknown parameters to estimate.A registration accuracy of one pixel was reported for thismethod [41].

Fig. 6. Digital photographs of tiled displays showing the colorvariation problem. (a): Example of severe photometric variationacross a display made of abutting projectors. Though difficult tobelieve, it is true that every pixel of this display is projecting theidentical input of the maximum intensity for green. (b): A tileddisplay made of a3× 5 array of fifteen projectors (10′ × 8′ in size)with perfect geometric registration, but with color variation.

III. PHOTOMETRIC CORRECTION

In this section, we address the color variation problem.Current commodity projectors, our target products forbuilding large-area displays inexpensively, do not havesophisticated lens systems to assure color uniformityacross the projector’s field-of-view. Thus, the color vari-ation in multi-projector displays made of commodityprojectors, can be significant. Figure 6(a) shows thesevere color variation of the40 projector display used byNCSA of UIUC. Even after perfect geometric alignment,the color variation problem can be the sole factor incausing a ‘break’ in creating the illusion of a single largedisplay, as shown in Figure 6(b). Thus, color variationproblems need to be addressed to achieve truly seamlessdisplays.

Color is a three-dimensional quantity defined by one-dimensional luminance (defining brightness) and two-dimensional chrominance (defining hue and saturation).The entire range of luminance and chrominance thatcan be reproduced by a display is represented by a 3Dvolume called thecolor gamut of the display. Sincecolor is defined by luminance and chrominance, thecolor variation problem involves spatial variation in bothluminance and chrominance. It has been shown thatmost current tiled displays composed of projectors of thesame manufacturer modelshow large spatial variationin luminance while the chrominance is almost constantspatially [30], [25]. Also, humans are at least an order ofmagnitude more sensitive to luminance variation than tochrominance variation. For example, humans have higherspatial and temporal frequency acuity for luminancethan for chrominance; in addition, humans can resolvea higher luminance resolution than chrominance resolu-tion. Detailed discussion of such perceptual capabilitiescan be found in the psychophysics literature [11], [15],[52]. It has been shown that perceptually, the subprob-lem of photometric variation(luminance variation) is


Fig. 7. From left: (1) Correction done by luminance matching for a display made of two abutting projectors; (2), (3), and (4) respectively:fifteen-projector tiled display, before blending, after software blending, and after optical blending using physical mask, respectively.

the most significant contributor to the color variationproblem.

The color variation in multi-projector displays hasbeen classified in three different categories [30].

1) Intra-Projector Variation: Luminance varies sig-nificantly across the field-of-view of a single pro-jector. Luminance fall-off of about40 − 80%from the center to the fringe is common in mostcommodity projectors. This is caused by severalreasons like the distance attenuation of light andthe angle at which the light from the projector fallson the screen. This also results in an asymmetricfall-off, especially with off-axis projection. Thenon-Lambertian nature of the screen further pro-nounces the problem. There are many front projec-tion screens available that are close to Lambertianin nature. However, this is a rare property amongthe rear projection screens making intra-projectorvariation more pronounced for rear-projection sys-tems. However, the chrominance remains almostspatially constant within a single projector.

2) Inter-Projector Variation: Luminance can varysignificantly across different projectors. This iscaused by differences in the properties of theprojector lamps and their ages, by differences inthe position and orientation of the projector withrespect to the screen, and also by differencesin the projector settings like brightness, contrastand zoom. However, chrominance variation acrossprojectors is relatively much smaller and is almostnegligible for same model projectors.

3) Overlap Variation: The luminance in the regionwhere multiple projectors overlap is multiplied bythe number of overlapping projectors, creating avery high brightness region. If the chrominanceproperties of the overlapping projectors are notclose to each other, this can also lead to visiblechrominance variations in the overlapped region.However, these variations are at least an order ofmagnitude smaller than the luminance variation.

A related problem is that of theblack offset. Anyideal display device should project no light for thered, green, and blue (RGB) input(0, 0, 0). This is true

for most cathode ray tube (CRT) projectors since theelectron beam inside can be switched off completelyat zero. However, most commodity projectors use lightblocking technology like liquid crystal display (LCD) ordigital light projector (DLP), through which some lightis always projected. This is called theblack offset. Thisreduces the contrast of projectors and current technologyis driven towards reducing this black offset as much aspossible.

In abutting projector displays, traditionally, the colorcompensation was done by manipulating the projectorcontrols (like brightness, contrast and zoom) manuallyusing feedback from a human user on the quality of thecolor uniformity achieved. Unfortunately, this is a verylabor-intensive process and ideal color uniformity is notalways achievable given the limited control allowed tothe user. Therefore, automatic color calibration methodswere devised to create scalable displays.

A. Gamut Matching

This approach was the first to automate the processof using manual feedback and manipulation of controls.A point light measuring instrument (like a spectrora-diometer) [47], [48] is used to measure thecolor gamutof each projector at one spatial location. The spatialvariation of color within a single projector is assumedto be negligible and the colors between the differentprojectors are matched by a two-step process. First, acommon color gamutis identified that is the intersectionof the gamuts of different projectors. This represents therange of colors that all the projectors in the display arecapable of producing. Second, by assuming projectors tobe linear devices, 3D linear transformations are used toconvert the color gamut of each display to the commoncolor gamut.

This method is applicable to devices that use threeprimary colors (most devices use red, green and blue ascolor primaries). Since three primary colors form a basisfor describing all colors, a color in the three-primary-system can be represented by auniquecombination ofthe three primaries. However, some DLP projectors usea clear filter to project the grays instead of project-ing the superposition of light from the red, green and


blue filters. This makes these DLP projectors behavelike four primary devices (like printers that use cyan,magenta, yellow and black as primaries). Adding thefourth primary brings in linear dependencies in suchsystems. As a result, a color cannot be represented usingunique combinations of the four primaries. The gamut-matching method depends on the linear independenceof the primaries and becomes inapplicable in such four-primary systems. [53] presents a solution by which agamut-matching method can be extended to be appliedto the DLP projectors.

The theoretical disadvantage of the gamut-matchingmethod lies in the fact that there is no practical methodto find the common color gamut. [4] presents an optimalmethod to find the intersection ofn color gamuts inO(n6) time. This is clearly not a scalable solution,especially for large-scale displays of over 10 projectors.[27] tries to address this problem by matching onlyluminance across different projectors. Since most displaywalls are made of the same model projectors, whichdiffer negligibly in chrominance, achieving luminancematching across different projectors can suffice. The re-sult of this method is shown in Figure 7. However, sincespatial color variation is ignored, these methods cannotproduce entirely seamless displays. Further, expensiveinstrumentation makes these methods cost prohibitive.A relatively inexpensive radiometer costs at least fourtimes more than a projector. Expensive radiometers cancost as much as a dozen projectors.

B. Using a Common Lamp

Using a common lamp is a wonderful engineering feat[35]. In this method, the lamps of the multiple projectorsare taken off and replaced by a common lamp of muchhigher power. Light is distributed from this commonlamp to all the different projectors using optical fibres.However, this method is cost intensive because it requiresskilled labor. Further, power and thermal issues (heatgenerated by the high-power lamp) make this approachunscalable. So far, a maximum of nine projectors can beilluminated by a common lamp using this method. Also,this approach addresses only the color variation causedby the differences in the lamp properties. All the otherkinds of variations still exist.

C. Blending

Blending or feathering techniques, adopted fromimage-mosaicing techniques, address overlapped regionsand try to smooth color transitions across these regions.The smooth transitions can be achieved by using alinear or cosine ramp which attenuate pixel intensities

Fig. 9. Aperture blending by mounting metal masks on the opticalpath of the projector that attenuates the light physically.

in the overlapped region. For example, considering apixel x in the overlap region of projectorsP1 and P2,as illustrated in Figure 8(Left). Let the contributionsof these projectors atx be given byP1(x) and P2(x)respectively. When using linear ramping the intensity atx is computed by a linear combination of the intensitiesP1(x) andP2(x), i.e,

α1(x)P1(x) + α2(x)P2(x)

where α1 + α2 = 1. These weights,α1 and α2, arechosen based on the distance ofx from the boundariesof the overlapped region. For example, when using alinear ramp, these functions can be chosen as,

α1(x) =d1

d1 + d2; α2(x) =

d2d1 + d2

.

.This two-projector example can be extended to an ar-

bitrary number of projectors [40]. To do so, the hull,Hi,in the camera’s image plane of observed projectorPi’spixels is computed. The alpha-weight,Am(x), associatedwith projector,Pm’s pixel, x, is evaluated as follows:

Am(x) =αm(m,x)∑

i αi(m, x), (4)

whereαi(m,x) = wi(m,x)∗di(m,x) andi is the indexof the projectors observed by the camera (includingprojectorm).

In the above equation,wi(m,x) = 1 if the camera’sobserved pixel of projectorPm’s pixel, x, is insidethe convex hull,Hi; otherwise,wi(m,x) = 0. Theterm di(m,x) is the distance of the camera’s observedpixel of projectorPm’s pixel, x, to the nearest edge ofHi. Figure 8 (right) shows the alpha-masks created forfour overlapping projectors. The alpha-masks are appliedafter the image has been warped. This can be performedefficiently as a single alpha-channel textured quad thesize of the framebuffer.


Fig. 8. Blending Techniques: (Left) The intensity at any pixelx in the overlapped region of two projectors,P1 andP2, is the combinationof the intensity ofP1 andP2 at x. (Right) The resulting alpha-masks computed for four projectors.

Blending can be achieved in three ways. First, it canbe done in software [43] where the distances from theprojector boundaries and the number of projectors con-tributing to every pixel in the overlap region can be accu-rately calculated using geometric calibration information.Thus, the ramps can be precisely controlled by software.However, this cannot attenuate the black offset, which isespecially important with scientific or astronomy data,which often have black backgrounds. Alternate opticalmethods thus try to achieve this blending by physicalattenuation of lights so that it can also affect the blackoffset. In one method, physical masks mounted at theprojector boundaries on the optical path attenuate thelight in the overlapped region [24], as shown in Figure9. In another method, optical masks are inserted in frontof the projection lens to achieve the attenuation [8]. Theresults of blending are shown in Figure 7. Though blend-ing methods are automated and scalable, they ignore theinter- and intra-projector spatial color variation. Also,the variation in the overlapped region is not accuratelyestimated. Thus, blending works well if the overlappingprojectors have similar luminance ranges, which is oftenassured by an initial manual brightness adjustment usingthe projector controls. However, for displays where theluminance has a large spatial variation (like for most rearprojection systems), blending results in only softeningof the seams in the overlapping region, rather thanremoving them.

D. Camera-based Photometric Uniformity

All the methods mentioned so far address only theinter-projector or overlapped variation. None addressesthe intra-projector variation that can be significant. Also,only the gamut matching method makes an effort toestimate the color response of the projectors. However,since the spatial variation in color is significant, a high-resolution estimation of the color response is the onlymeans towards an accurate solution. Thus, the use of

Fig. 10. To compute the display luminance surface of each projector,we need only four pictures per channel. Top: Pictures taken for adisplay made of a2× 2 array of4 projectors. Bottom: The picturestaken for a display made of a3× 5 array of15 projectors (both forgreen channel).

a camera is inevitable. However, since a camera has alimited color gamut (as opposed to a spectroradiometer),estimating the color gamut of the display at a high resolu-tion is difficult. However, different exposure settings of acamera can be used to measure the luminance accuratelyand faithfully. Exploiting this fact, [29], [30] use acamera to correct for the photometric variation (variationin luminance) across a multi-projector display. Sincemost current displays use the same model projectorsthat have similar chrominance properties, this methodachieves reasonable seamlessness.

This camera-based method aims at achievingidenticalphotometric response at every display pixel. This iscalled photometric uniformity. We describe the methodfor a single channel. All three channels are treatedsimilarly and independently. The method comprises twosteps. The first step is a one-timecalibration step thatuses the camera to estimate the luminance response ofthe multi-projector display. At the end of this step, a per-projector per-pixel map called theluminance attenuationmap (LAM) is generated. In theimage correctionstep,the LAM is used to correct any image to be displayed.

1) Calibration: Let the displayD be made ofN pro-jectors, each denoted byPj . Let the camera used for cal-ibration beC. First, geometric calibration is performedto find the geometric warpsTPj→C relating the projectorcoordinates(xj , yj) with the camera coordinates(xc, yc),


Fig. 11. Left: The luminance surface for one projector. Middle and Right: The display luminance surface for a2 × 2 array of a fourprojector and a3× 5 array of fifteen projectors, respectively. (all for the green channel)

and TPj→D relating the projector coordinates with theglobal display coordinate(xd, yd). Any of the geometriccalibration methods described in Section II can be usedfor this purpose. After that, photometric calibration hasthree steps.

a) Capturing the Display Luminance Response:Note that the camera should be in the same location asthe geometric calibration method throughout this processof photometric calibration. Using the digital camera, twofunctions are acquired to perform photometric calibra-tion.

The variation of the projected intensity from a channelof a projector with the variation in the input is defined bythe intensity transfer function (ITF). This is commonlycalled the gamma function. In projectors, this functioncannot be expressed by a power function and hencewe prefer to call it the intensity transfer function. [25]shows this function to be spatially invariant i.e variesonly with input and does not change from one pixel toanother within the projector. Hence, the ITF for eachprojector is first estimated using a point light measuringinstrument like a photometer at one location for eachprojector. However, since such instruments can be costprohibitive, [37] presents a method in which the highdynamic range (HDR) imaging method developed byDebevec and Malik [13] is applied to measure the ITFof all the projectors at a time using a inexpensive videocamera.

Next, the display luminance surface is captured. Im-ages with maximum luminance are projected from eachprojector and captured using the camera. More than onenon-overlapping projector can be captured in the sameimage. The images taken for this purpose for a fourand fifteen projector display are shown are in Figure10. From these images the luminance surface for eachprojector LPj

is generated by using the warpTPj→C .StandardRGB to Y CrCb conversion is used for thispurpose. The luminance surface from these projectorsare then added up spatially using the warpTPj→D to

create the display luminance surfaceLD. The luminancesurfaces generated for a projector and the whole displayare shown in Figure 11.

Fig. 12. Left: The display luminance attenuation map for a3 × 5array of fifteen projectors. Right: The LAM for a single projector cutout from the display LAM on the left. (both examples are for thegreen channel)

b) Finding the Common Achievable Response:Next, the luminance response that can be achieved atevery pixel of the display is identified. Since the dimmerpixels cannot match the brighter pixel, the commonachievable response is given by

Lmin = min∀(xd,yd)

LD.

c) Generating the Attenuation Maps:The lumi-nance attenuation map (LAM),AD, for the whole displayis first generated by

AD(xd, yd) =Lmin

LD(xd, yd).

¿From this display LAM, a luminance attenuation mapfor each projector is generated using the inverse of thewarp TPj→D. The display and projector LAMs thusgenerated are shown in Figure 12. This concludes thecalibration.

2) Image Correction:Once the per-projector LAMsare generated, the per-projector image correction is donein two steps. These correction steps are applied to anyimage that is projected from the display. First, the perpixel multiplication of the image with the LAM isperformed. This multiplication assumes linear ITF. Inpractice, however, the ITF is non-linear. To compensatefor that, an inverse of the ITF is applied to the image


Fig. 13. The top row shows the image before correction and the bottom row shows the image after luminance matching. Left and middle:Digital photograph of a2 × 2 array of projectors. Right: Digital photograph of a5 × 3 array of projectors. In this case, the image aftercorrection was taken at a higher exposure.

after the LAM is applied. The results of this method areshown in Figure 13.

The corrections required to achieved photometric uni-formity or to compensate for the surface reflectance areencoded as per-pixel linear operations and a 1D colorlook-up-table (LUT). These form a very efficient wayof representing the non-linear correction because theseoperations can be applied in real time using commoditygraphics hardware. Recent advances in programmablegraphics hardware make it possible to implement com-plex per-pixel operations that can run natively on thegraphics processor without taking a toll on the mainCPU [33], [2]. Details of how these can be used to createinteractive displays are available at [28].

However, since this method aims at photometric uni-formity, the photometric response of every pixel ismatched to the ‘worst’ pixel on the display, ignoring allthe ‘good’ pixels that are very much in the majority. Thisresults in a compression in dynamic range making themethod unscalable. Ongoing research [26] is trying toaddress this issue by achieving a perceptual uniformityrather than a strict photometric uniformity.

E. Camera-Based Compensation for Non-White Sur-faces

The methods described so far can be used to compen-sate for color variation in a multi-projector display whenprojecting on a white screen. Recent work addresses theissue of using projectors to project on displays that arenot necessarily white but have colors and textures, like

brick walls or a poster boards [32], for scenarios whereit may not be possible to find white display surfaces. Inthis approach, the camera and projectors are assumed tobe linear devices and the color transformation betweenthem is expressed by a3×3 matrix, V . The RGB color,C, measured by a camera for a projector input,P , isrelated by the matrix multiplicationC = V P .

The camera is first used to measure the responseof several images projected from the projector. Eachimage is made of identical input at every projector pixel.With the projector pixel inputs and the correspondingmeasured outputs from the camera established,V canbe estimatedfor each pixelby solving a set of over-determined linear equations. OnceV is estimated,V −1

is applied to the input image to generate the desiredresponse that would look seamless on an imperfect sur-face. The estimatedV is further refined by a continuousfeedback and an estimation loop between the projectorand the camera. The non-linearities of the projector andthe camera are also considered to validate the assumptionof linear devices. Greater details of this method areavailable in [32]. This method has not yet been scaledto displays made of multiple projectors.

IV. D ISCUSSION

All of the approaches discussed in the previous sec-tions have been used and tested in deploying variousprojector-based display systems. Table I provides a listof representative systems and supporting publicationreferences. Table I lists these systems in chronological


order based on publication date and itemizes key aspectseach system, including the types of display surface,the number of cameras and projectors in the system,geometric and photometric registration approach used,the number of rendering passes required, and targetedviewer mode (stationary vs. moving). Since many ap-proaches are available for different applications anddisplay configurations, we use this section to discuss thepositive and negative aspects of the various approachesfor geometric and photometric registration.

On the geometric front, restricting the display surfaceto be planar has many benefits. First, there are morescalable techniques to register very large arrays withsub-pixel accuracy, such as the homography tree ap-proach [9]. In addition, the alignment procedure using a2D linear homography can be performed in the graphicspipeline, allowing for efficient rendering [39], [54]. Pla-nar homography-based approaches, however, can correctfor only linear geometric distortions. For instance, non-linear radial distortion introduced by a projector’s opticalsystem cannot be corrected by this method. Yang et al.[54] showed that the zoom setting of some projectorsaffected the radial distortion enough to introduce pixelerrors in homography-based approaches. As a result, theprojectors usable zoom range had to be fixed to thepositions that minimize radial distortion.

The parameterized transfer equation introduced byRaskar et al. [41] extends planar surface algorithmsto quadric surfaces. While some screens (i.e., dome andcylindrical screens) can be modelled as quadric surfaces,this requires precise manufacturing. For applications thatuse cheaper constructed surfaces that do not requirehead-tracking, it may still be better to use the direct map-ping technique (see Section II-B.1) that can compensatefor the imperfections in the display surface geometry.

For arbitrary display surfaces, the direct mappingfrom the camera space to the projector space is a veryefficient way to generate seamless images from onefixed view location. The resulting two-pass renderingalgorithm compensates for display surface distortion aswell as projector lens distortion. For small arrays (4-5projectors), this approach is very flexible and can allowquick deployment of projector-based displays in a widerange of environments. However, because this techniquerequires the camera to see the entire display and it is notscalable to large projector arrays.

The technique presented by Raskar et al. [40] for amoving user and arbitrary display surfaces involves afull 3D modeling of the display environment includingthe projector positions and display surface geometry.While this approach is the most general solution to largescale display deployment, it is non-trivial to implement

a robust and practical system. Due to its complexity, thebest registration error reported so far is about 1-2 pixels.

For correcting the color variation problem, solutionslike blending (Section III-C) do not estimate the spatialvariation and hence cannot achieve entirely seamlessdisplay, especially for large displays. However, for smallsystems of2− 4 projectors, blending can achieve effec-tive results if it is preceded by color balancing acrossdifferent projectors. This color balancing can be manualor can be automated using gamut matching techniques(Section III-A). The camera based technique (SectionIII-D) can achieve reasonable photometric seamlessnessacross the display, which is sufficient for displays madeof same brand projectors. The advantage of this methodlies in its complete automation and scalability. However,the limitation of both gamut matching or photometricuniformity lies in degrading the color quality of thedisplay in terms of dynamic range and color resolution.Thus, achieving perceptual uniformity (rather than strictphotometric uniformity) while maintaining high displayquality is the current area of research. Finally, currentcamera-based correction do not address chrominancevariation, arbitrary display geometry or a moving user,which are still active areas of research.

V. HARDWARE SUPPORT FORIMAGE CORRECTION

To correct geometric and photometric distortions in aprojector-based display requires changes to be made tothe desired image, which causes overhead during ren-dering time. This shortcoming has been ameliorated byrecent advances in computer hardware. Modern graphicshardware provides a tremendous amount of image pro-cessing power. Thus, many of the correction operationscan be off-loaded to the graphics board. The overheadto warp and blend a screen resolution image becomesnegligible. In addition, certain correction operations canbe integrated into the graphics rendering pipeline, such asthe one-pass rendering algorithm for off-axis projectionon planar surfaces [39]. These approaches completelyeliminate the image correction overhead when render-ing 3D contents. With increasing programmability ingraphics hardware, we expect that new techniques thatleverage the power of programmable graphics hardwarewill emerge to reduce the rendering overhead in a widerange of configurations.

The need for more flexibility in projector-based dis-plays is also being addressed by projector manufacturers.Recent projectors are equipped with more options to ad-just the projected images. For example, projectors fromEPSON provide automatic keystone correction using abuilt-in tilt sensor [46]. 3D-Perception CompactViewwas one of the first companies to offer a projector that


TABLE I

CHARACTERISTICS OF REPRESENTATIVE LARGE-FORMAT DISPLAYS USING CAMERA-BASED CALIBRATION

System Display number of number of Resolution Geometric Photometric Rendering

surfaces projectors cameras (mega pixels) registration correction passes

Surati [45] arbitrary♥ 4 one 1.9 fixed warping color attenuation twoRaskar et al. [40] arbitrary♦ 5 multiple 3.8 full 3D model software blending threeY. Chen et al. [10] planar 8 one on PTU 5.7 simulated annealing optical blending one

PixelFlex [54] arbitrary♥ 8 one 6.3 fixed warping software blending twoH. Chen et al. [9] planar 24 multiple 18 homography tree optical blending one

Metaverse [21] multiple walls♦ 14 one 11 homography software blending oneiLamp [41] quadric surfaces 4 one/projector 3.1 full 3D model software blending two

♦ head-tracked moving viewer.♥ static viewer (image is correct for a fixed location).

performed real-time corrective warping to the incomingvideo stream [1]. This feature is used to help compensatefor projection on smooth, curved surfaces, such as thosein video domes. Recently, other projector manufacturershave provided similar options. The Barco Galaxy-WARPprojector is also capable of performing real-time cor-rective warping to the incoming video stream [3]. Bothproducts allow for non-linear image mapping. Thus,a wide range of configurations can be accommodatedwithout incurring any rendering overhead.

Currently, these products allow control of the non-linear warping via user interfaces. However, it is amatter of time before an interface between the projectorand camera-based registration techniques will allow thiswarping to be specified automatically. With the projec-tors performing the warping in real-time, performanceoverhead will not be an issue. This should be a greatbenefit to the current two-pass rendering algorithms.Merging this technology with camera-based registrationwill truly allow a new generation of flexible and highlyconfigurable projector-based display environments.

VI. CONCLUSION AND FUTURE WORK

Camera-based calibration techniques have enabled amuch wider range of configurations for projector-baseddisplays. The capability of automatic geometric align-ment and photometric correction of multiple projectedimages eases the setup and reduces the cost of large-format displays. Coupled with advances in distributedrendering software and graphics hardware, the possibilityof creating inexpensive and versatile large format dis-plays using off-the-shelf components becomes a reality.It is our hope that the information provided in this surveywill provide projector-based display users a useful guideto the currently available techniques and their associatedadvantages and disadvantages.

Looking forward, there are a number of research topicsthat can further advance the state of the art.

a) Geometric Registration Quality:Registrationquality is often reported as pixel registration accuracyin local overlapped regions and not in the context ofthe global display coordinate frame. Moreover, using thepixel as a unit of measure is ill-defined when imagery isprojecting on arbitrary display surfaces or contributingprojector pixels are not uniform in size. Better metricsand analytical approaches are needed to fully evaluateoverall registration accuracy.

b) Color Correction: The shortcoming of the au-tomated color correction method presented here is thesevere degradation in image quality. Methods should bedevised that optimize the available resources in termsof brightness and contrast of the display and achieveperceptual uniformity, which may not require strict pho-tometric uniformity. Also, only the problem of spatialphotometric variation has been addressed while assumingthat most current displays have negligible chrominancevariation. However, when using different model pro-jectors, the chrominance variation cannot be ignored.Finally, arbitrary 3D display surfaces with arbitraryreflectance properties for moving users is still to beaddressed.

c) Image Resampling:Most of the geometric cor-rection techniques involved a resampling of an originalrendered image. How this resampling affects the overallresolution of the display and ways to avoid fidelity lossneed to be addressed.

d) Continuous Calibration: Almost all camera-based techniques treat the calibration procedure as a pre-processing routine. The correction function derived fromthe calibration information remains fixed until the nextcalibration. During the normal operation of a displaysystem, however, there are many factors that affect thevalidity of the calibration, such as vibrations, electronic


drift, aging of projector light bulbs, or even transientevents such as temporary occlusion of projector light. Todeal with these problems, techniques could be developedto continuously monitor the projected imagery and cor-rect any undesired distortions online. Promising works,such as continuous monitoring of display surfaces [55]and shadow removal [22], [49], [7] have demonstratedthe potential of this research area.

e) Display and User Interaction:The real-timefeedback of cameras in the display environment makeit possible to develop interaction techniques betweenthe user and the display. For example, laser pointerinteraction inside a camera-registered displays can beeasily realized [50], [6]. Significantly more ambitiousgoals have been set forth in UNC’s Office of the Fu-ture project [43] and Gross et al’s [16]blue-c system.These systems aim to provide immersive 3D telecom-munication environments where cameras capture real-time 3D information of users inside the display. Thetightly coupled relationship between the camera anddisplay environment offers great potential for novel userinteraction metaphors within such environments.

REFERENCES

[1] 3D Perception AS, Norway. CompactView X10, 2001.http://www.3d-perception.com/.

[2] ATI Technologies Inc. ATI Radeon 9800, 2003.http://www.ati.com/products/radeon9800.

[3] Barco, Kortrijk Belgium. Barco Galaxy Warp.http://www.barco.com/.

[4] M. Bern and D. Eppstein. Optimized color gamuts for tiled dis-plays. ACM Computing Research Repository, cs.CG/0212007,19th ACM Symposium on Computational Geometry, San Diego,2003.

[5] M. S. Brown and W. B. Seales. A Practical and Flexible TiledDisplay System. InProc of IEEE Pacific Graphics, pages 194–203, 2002.

[6] M. S. Brown and W. Wong. Laser Pointer Interaction ForCamera-Registered Multi-Projector Displays. InProceedings ofthe Proceedings of International on Image Processing (ICIP),Barcelona, Sept 2003.

[7] T.J. Cham, J. Rehg, R. Sukthankar, and G. Sukthankar. Shadowelimination and occluder light suppression for multi-projectordisplays. Proceedings of Computer Vision and Pattern Recog-nition, 2003.

[8] C. J. Chen and Mike Johnson. Fundamentals of Scalable HighResolution Seamlessly Tiled Projection System.Proceedings ofSPIE Projection Displays VII, 4294:67–74, 2001.

[9] H. Chen, R. Sukthankar, and G. Wallace. Scalable Alignmentof Large-Format Multi-Projector Displays Using Camera Ho-mography Trees. InProceeding of IEEE Visualization 2002,pages 339–346, 2002.

[10] Y. Chen, D. Clark, A. Finkelstein, T. Housel, and K. Li. Auto-matic Alignment Of High-Resolution Multi-Projector DisplaysUsing An Un-Calibrated Camera. InProceeding of IEEEVisualization 2000, pages 125–130, 2000.

[11] R.A. Chorley and J. Laylock. Human Factor Consideration forthe Interface between Electro-Optical Display and the HumanVisual System. InDisplays, volume 4, 1981.

[12] C. Cruz-Neira, D. Sandin, and T. DeFanti. Surround-ScreenProjection-Based Virtual Reality: The Design and Implementa-tion of the CAVE. InProceedings of SIGGRAPH 1993, pages135–142, 1993.

[13] P. E. Debevec and J. Malik. Recovering High DynamicRange Radiance Maps from Photographs.Proceedings of ACMSiggraph, pages 369–378, 1997.

[14] Fakespace Systems Inc. PowerWall, 2000.http://www.fakespace.com.

[15] E. Bruce Goldstein. Sensation and Perception. WadsworthPublishing Company, 2001.

[16] Markus Gross, Stephan Wuermlin, Martin Naef, Edouard Lamb-oray, Christian Spagno, Andreas Kunz, Esther Koller-Meier,Thomas Svoboda, Luc Van Gool, Silke Lang, Kai Strehlke,Andrew Vande Moere, and Oliver Staadt. blue-c: A SpatiallyImmersive Display and 3D Video Portal for Telepresence. InProceedings of SIGGRAPH 2003, pages 819–827, San Diego,July 2003.

[17] R. I. Hartley and A. Zisserman. Multiple View Geometryin Computer Vision. Cambridge University Press, ISBN:0521623049, 2000.

[18] G. Humphreys, I. Buck, M. Eldrige, and P. Hanrahan.Chromium: A Stream Processing Framework for InteractiveRendering on Clusters. InProceedings of SIGGRAPH, July2002.

[19] G. Humphreys, M Eldridge, Ian B., G Stoll, M Everett, andP Hanrahan. WireGL: A Scalable Graphics System for Clusters.In Proceedings of SIGGRAPH 2001, August 2001.

[20] Intel. Open Source Computer Vision Library (OpenCV).http://www.intel.com/research/mrl/research/opencv/.

[21] C. Jaynes, B. Seales, K. Calvert, Z. Fei, and J. Griffioen.The Metaverse - A Collection of Inexpensive, Self-configuring,Immersive Environments. InProceeding of 7th InternationalWorkshop on Immersive Projection Technology, 2003.

[22] C. Jaynes, S. Webb, M. Steele, M. S. Brown, and B. Seales.Dynamic Shadow Removal from Front Projection Displays. InProceeding of IEEE Visualization 2001, pages 174–181, SanDiego, CA, 2001.

[23] K. Li, H. Chen, Y. Chen, D.W. Clark, P. Cook, S. Damianakis,G. Essl, A. Finkelstein, T. Funkhouser, A. Klein, Z. Liu,E. Praun, R. Samanta, B. Shedd, J.P. Singh, G. Tzanetakis,and J. Zheng. Early Experiences and Challenges in Buildingand Using A Scalable Display Wall System.IEEE ComputerGraphics and Applications, 20(4):671–680, 2000.

[24] K. Li and Y. Chen. Optical Blending for Multi-ProjectorDisplay Wall System. InProceedings of the 12 th Lasers andElectro-Optics Society 1999 Annual Meeting, 1999.

[25] A. Majumder. Properties of Color Variation Across Multi-Projector Displays.Proceedings of SID Eurodisplay, 2002.

[26] A. Majumder. A Practical Framework to Achieve PerceptuallySeamless Multi-Projector Displays, PhD Thesis. Technicalreport, University of North Carolina at Chapel Hill, 2003.

[27] A. Majumder, Z. He, H. Towles, and G. Welch. Achieving ColorUniformity Across Multi-Projector Displays.Proceedings ofIEEE Visualization, 2000.

[28] A. Majumder, D. Jones, M. McCrory, M. E. Papka, andR. Stevens. Using a Camera to Capture and Correct SpatialPhotometric Variation in Multi-Projector Displays.IEEE Inter-national Workshop on Projector-Camera Systems, 2003.

[29] A. Majumder and R. Stevens. LAM: Luminance AttenuationMap for Photometric Uniformity in Projection Based Displays.Proceedings of ACM Virtual Reality and Software Technology,2002.

[30] A. Majumder and R. Stevens. Color Nonuniformity inProjection-Based Displays: Analysis and Solutions.IEEE


Transactions on Visualization and Computer Graphics, 10(2),2003.

[31] W. Matusik and H. Pfister. 3D TV: A Scalable System for Real-Time Acquisition, Transmission, and Autostereoscopic Displayof Dynamic Scenes. InProceedings of SIGGRAPH, pages 814–824, 2004.

[32] S. K. Nayar, H. Peri, M. D. Grossberg, and P. N. Belhumeur. AProjection System with Radiometric Compensation for ScreenImperfections. IEEE International Workshop on Projector-Camera Systems, 2003.

[33] NVIDIA Corporation. GeForce FX, 2003.http://www.nvidia.com/page/fx desktop.html.

[34] T. Okatani and K. Deguchi. Autocalibration of a Projector-Screen-Camera System: Theory and Algorithm for Screen-toCamera Homography Estimation. InProceedings of Interna-tion Conference on Computer Vision (ICCV), volume 2, pages125–131, 2002.

[35] B. Pailthorpe, N. Bordes, W.P. Bleha, S. Reinsch, and J. More-land. High-Resolution Display with Uniform Illumination.Proceedings Asia Display IDW, pages 1295–1298, 2001.

[36] Panoram Technologies Inc. PanoWalls, 1999.http://www.panoramtech.com/.

[37] A. Raij, G. Gill, A. Majumder, H. Towles, and H. Fuchs. Pix-elFlex2: A Comprehensive, Automatic, Casually-Aligned Multi-Projector Display.IEEE International Workshop on Projector-Camera Systems, 2003.

[38] A Raij and M. Pollefeys. Auto-Calibration of Multi-ProjectorDisplayWalls. InProceedings of International Conference onPattern Recognition (ICPR), 2004.

[39] R. Raskar. Immersive Planar Display using Roughly AlignedProjectors. InIEEE VR 2000, pages 109–116, 2000.

[40] R. Raskar, M. S. Brown, R. Yang, W. Chen, G. Welch,H. Towles, B. Seales, and H. Fuchs. Multi-projector displaysusing camera-based registration. InProceeding of IEEE Visu-alization 1999, pages 161–168, 1999.

[41] R. Raskar, J. van Baar, P. Beardsley, T. Willwacher, S. Rao, andC. Forlines. iLamps: Geometrically Aware and Self-configuringProjectors.ACM Transactions on Graphics (SIGGRAPH 2003),22(3):809–818, 2003.

[42] R. Raskar, J. vanBaar, and J. Chai. A Low Cost Projector Mo-saic with Fast Registration. InFifth International Conferenceon Computer Vision (ACCV.02), 2002.

[43] R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin, andH. Fuchs. The Office of the Future: A Unified Approachto Image-Based Modeling and Spatially Immersive Displays.Computer Graphics, 32(Annual Conference Series):179–188,1998.

[44] R. Raskar, G. Welch, and H. Fuchs. Seamless ProjectionOverlaps using Image Warping and Intensity Blending. InProc. of 4th International Conference on Virtual Systems andMultimedia, 1998.

[45] R.Surati. Scalable Self-Calibrating Display Technology forSeamless Large-Scale Displays. PhD thesis, Departmentof Computer Science, Massachusetts Institute of Technology,1998.

[46] SEIKO EPSON Corp, Japan. Epson PowerLite 730p.http://www.epson.com/.

[47] M. C. Stone. Color Balancing Experimental Projection Dis-plays. 9th IS&T/SID Color Imaging Conference, 2001a.

[48] M. C. Stone. Color and Brightness Appearance Issues in TiledDisplays. IEEE Computer Graphics and Applications, 2001b.

[49] R. Sukthankar, T.J. Cham, and G. Sukthankar. Dynamicshadow elimination for multi-projector displays.Proceedingsof Computer Vision and Pattern Recognition, 2001.

[50] R. Sukthankar, R. Stockton, and M. Mullin. Smarter Presen-tations: Exploiting Homography in Camera-Projector Systems.

In Proceedings of the Proceedings of International Conferenceon Computer Vision (ICCV), Vancouver, July 2001.

[51] B. Triggs. Autocalibration from Planar Scenes. InProceedingsof Fifth European Conference on Computer Vision(ECCV), page89C105, 1998.

[52] R. L. De Valois and K. K. De Valois.Spatial Vision. OxfordUniversity Press, 1990.

[53] G. Wallace, H. Chen, and K. Li. Color Gamut Matchingfor Tiled Display Walls. Immersive Projection TechnologyWorkshop, 2003.

[54] R. Yang, D. Gotz, J. Hensley, H. Towles, and M. Brown.PixelFlex: A Reconfigurable Multi-Projector Display System.In Proceeding of IEEE Visualization, pages 167–174, 2001.

[55] R. Yang and G. Welch. Automatic Projector Display SurfaceEstimation Using Every-Day Imagery. In9th International Con-ference in Central Europe on Computer Graphics, Visualizationand Computer Vision, 2001.

Michael Brown received his B.Eng. and PhDdegrees, both in Computer Science, from theUniversity of Kentucky in 1995 and 2001,respectively. he was a visiting PhD studentat the University of North Carolina at ChapelHill from 1998-2000. In 2001, he joined theComputer Science faculty at the Hong KongUniversity of Science and Technology. Hisresearch interests include, image processing

and computer graphics.

Aditi Majumder is an Assistant Professor atthe Department of Computer Science in Uni-versity of California, Irvine. She received herBE in Computer Science and Engineering fromJadavpur University, Calcutta, India in 1996and PhD from Department of Computer Sci-ence, University of North Carolina at ChapelHill in 2003.

Her research focuses large area displays,computer graphics and vision, image processing and human computerinteraction. Her significant contributions include development of pho-tometric vision techniques that use limitation of human perception toachieve seamless high-quality multi-projector displays and geometricregistration of images from multi-camera sensors to create real-timepanoramic video for immersive teleconferencing.

Ruigang Yang is an Assistant Professor in theComputer Science Department at the Univer-sity of Kentucky. He received his Ph.D. degreein Computer Science from University of NorthCarolina at Chapel Hill in 2003. Prior tocoming to UNC-Chapel Hill, he earned a M.S.degree in Computer Science from ColumbiaUniversity in 1998.

Dr. Yang’s research interests include com-puter graphics, computer vision, and multimedia. He is a member ofthe IEEE Computer Society and ACM.

camera-based calibration techniques for seamless multi-projector

Documents