Calibration of a Multi-Projector System for Displayon a Cylindrical Surface

Brandon B. MayCenter for Imaging Science

Rochester Institute of TechnologyRochester, NY 14623

Email: bbm7344@rit.edu

Nathan D. CahillSchool of Mathematical SciencesRochester Institute of Technology

Rochester, NY 14623Email: nathan.cahill@rit.edu

Mitchell R. RosenCenter for Student Innovation

Rochester Institute of TechnologyRochester, NY 14623

Email: mitchell.rosen@rit.edu

Abstract—In this paper we present a method for geometrically,photometrically & colorimetrically calibrating a multi-projectorsystem for display on a cylindrical surface. Using a single camerawe reconstruct the 3D surface of the display and determinethe projector-screen relationship to accurately register projectedimages. Given these relationships, we apply chrominance gamutmorphing in the overlap regions to smoothly transition fromone projected image to the next. After white point balancingand perceptual brightness constraining, the final registered andblended images are shown by each respective projector to createa seamless high resolution image.

I. INTRODUCTION

In immersive display environments it is often desirable todisplay imagery of very high resolution to provide a morerealistic experience for the observers. This may be donethrough combining many lower resolution projectors to createa very high resolution presentation. However, in order forthis solution to work well, the projectors need to behaveas though they are one seamless projector. Thus, they mustundergo a careful calibration procedure to ensure that this isthe case. Commonly, cylindrical type display surfaces are usedto provide a wide field of view that tends to wrap aroundthe observer. As such, the geometric registration proceduremust be able to account for these types of non-linear sur-faces. Additionally, in areas where the projectors overlap, acolorimetric and photometric calibration (blending) must beemployed to smoothly transition from one projected image tothe next. In this paper, we present a method to completely cali-brate multiple overlapping projectors on a cylindrical surface.Typical calibration procedures force all projectors to adhereto one common characteristic. Here we utilize an approachthat allows each projector to attempt to maximize its use ofintrinsic dynamic range while reducing noticeable projector-to-projector differences by taking advantage of the human visualsystem’s forgiveness for slow variation.

The calibration procedure is performed in two main parts:geometric registration and colorimetric & photometric calibra-tion. We use the method of [1] as a basis for geometricallyregistering multiple projectors on a cylindrical surface throughcamera calibration, projector calibration and geometric mod-eling. Then our blending step is based on the method of [2]which consists of white point balancing, chrominance gamutmorphing, and perceptual brightness constraining. Using these

Fig. 1. Flow chart describing the proposed calibration algorithm.

we are able to create a seamless high-resolution display frommultiple overlapping projectors.

Related work is presented in section II. The calibrationprocedure is detailed in section III. Next, our implementationand results are presented in section IV. Finally, conclusionsare presented in section V.

II. RELATED WORK

In general, when registering multiple projectors on a non-planar surface, multiple cameras are needed in order to recon-struct the 3D surface of the display. In [3] they use a stereocamera rig to reconstruct non-planar quadric surfaces usingsuch techniques as conformal mapping and quadric transfer tominimize distortion after the geometric registration. Anothermethod, [4], first achieves camera and projector calibrationthrough 3D fiducials and then reconstructs the surface ofthe display by using many structured-light patterns. Thereare other registration methods that make use of only onecamera for non-planar surfaces, however these are typicallycamera-view dependent. For example, [5] registers multipleprojectors with respect to the viewpoint of the camera andavoids reconstructing the display geometry entirely.

Intra-projector variation, inter-projector variation, and over-lap variation are the three main categories of spatial colorchanges in multiple projector displays. Most existing blendingmethods don’t address all of these issues and thus have

978-1-4244-9300-5/10/$26.00 ©2010 IEEE6

suboptimal solutions. One of these methods, [6], proposesa gamut matching method for tiled display walls, howevergamut matching significantly degrades the color quality ofthe display by restricting the common achievable gamut [5].Another method, [7], matches the luminance transfer functionsto achieve a luminance balancing without considering theprojector chrominance variations.

In [1], they present a technique to calibrate multiple casuallyaligned projectors on a cylindrical surface using a singlecamera, where the cylinder is a vertically extruded surfaceand the aspect ratio of the rectangle formed by the cornersof the screen is known. They achieve accurate geometriccalibration of multiple projectors on a cylindrical displaywithout performing an extensive stereo reconstruction. Ourmethod, based on [1], is likewise able to recreate the 3Dsurface of the display using a single camera without needingto restrict the final viewpoint to that of the cameras position.The constrained gamut morphing algorithm created by [2]removes variations due to differences in chromaticity gamutsacross the projectors, the vignetting effect of each projector,and the overlap across adjacent projectors. They demonstratecolor seamlessness across multiple projectors for both planarand curved displays. Basing our approaches on [2], the col-orimeteric & photometric calibration steps implemented in thispaper consider, in similar fashion, both spatial variations inluminance within the projectors and differences in chromatic-ity between the projectors. By combining techniques from [1]and [2], we have created a complete system for seamlesslypresenting content across multiple overlapping projectors on acylindrical surface.

III. METHODS

Our system does calibration through two main steps: geo-metric registration and colorimetric & photometric calibration.Fig. 1 gives an overview of the entire procedure. The geometricregistration step results in a viewpoint-specific rendered imagewhich acts as input to the final rendering step. Colorimetric& photometric calibration is processed as one multiplicativemask (alpha mask) applied in projector luminance spaceduring the final rendering step. Once the image is transformedback to projector digital counts via the inverse transfer func-tions, it is ready to be displayed.

A. Geometric Calibration

The first step necessary to calibrate this system is to get anaccurate understanding of the geometric relationships betweeneach overlapping projector and the screen. This knowledgemakes it possible to accurately align content displayed in theoverlap regions and provides a starting point for colorimetric& photometric calibration. The algorithm implemented hereis able to determine the geometric relationships from a singleimage of the entire screen and each projector displaying aparticular pattern.

1) Corner-based Optimization: In order to determine therelationships between the projectors and screen there mustbe a way to reconstruct the approximate surface geometry

Fig. 2. Typical arrangement of projectors, camera, and cylindrical surface.

of the screen so that a particular 3-D location is known forevery projected pixel. This can be done by imaging the screenwith a camera, assuming a geometric scene model as well asknowing the intrinsic and extrinsic camera parameters (Fig. 2& 3). However, the camera’s focal length, f , and extrinsicparameters (rotation, R, and translation, T ) are unknownand must be determined from the camera image. The cornerbased optimization procedure takes as input the aspect ratio,a, of the plane created by the screen corners and initialguesses of the intrinsic and extrinsic camera parameters. Ituses these predicted versus projected corners as input to aLevenberg-Marquardt nonlinear minimization that minimizesthe reprojection error between projected screen corners, cproj ,and predicted screen corners, cpred,

minθ

(∑(cpred(θ)− cproj)

2)

(1)

θ = [f,R, T ] (2)

and outputs a rough estimate of the camera’s focal length andorientation.

2) Curve-based Optimization: To refine the camera param-eters even further, the projections of the top and bottom curvesof the screen are used as optimization constraints. Points thatlie along the top curve and bottom curve in the camera imageare chosen and fit with a quadratic curve. Using the currentcamera parameters the top curve is back-projected, each raybeing intersected with the Y = 1 plane to determine its 3-Dlocation which by our convention represents the top of thescreen. Since the screen is assumed to be cylindrical the topand bottom curves should have the same shape in 3-D. Thereprojected top curve is translated to the Y = −1 plane wherethe bottom curve lives and then projected back onto the image.This predicted curve, xp, is compared with the original bottomcurve, xb, in a least squares sense via Levenberg-Marquardtoptimization

minθ

(∑(xp(θ)− xb)

2)

(3)

that refines the camera parameters to make these curves matchas closely as possible. Once complete, the camera parametershave been accurately estimated and can be used to providereliable estimates of the 3-D projected pixel coordinates.

3) Projector Calibration: In this step, each projector dis-plays a calibration pattern that allows us to determine thetopmost and bottommost lines achievable by the projector, aswell as its four corners. Using this information it is possibleto estimate the intrinsic and extrinsic parameters of each

7

Fig. 3. Cylindrical surface and world coordinate system.

projector. The extrinsic parameters are defined by a coordinatesystem represented by three orthogonal axes XP , YP , and ZPand a center OP (Fig. 4). XP is defined as the line created bythe intersection of the top and bottom planes of the projectorview frustum as determined by the top and bottom lines ofthe projected pattern. Once XP is known, the center OPcan be estimated by assuming the projector view frustum issymmetric in the horizontal direction and thus lies on the lineXP with the constraint that the two vertical planes formedby the view frustum make the same angle with the line XP .Due to the equal angle constraint, OP will be given by aweighted average of the corners projected onto XP . By similartriangles the weights are inversely proportional to the distancesbetween the original and projected corners. To find ZP simplydetermine the plane that intersects the top and bottom planesin equal lengths; ZP is the normal to that plane. Finally,YP = ZP ×XP .

B. Colorimetric & Photometric Calibration

At this point in the calibration, the imagery being displayedby each projector can be geometrically registered by using therecovered geometry of the screen, projector-screen relation-ships, and a viewing position to ray-trace the correct imagesfor each projector. But, there are still obvious differences inbrightness and color between projectors and in the overlapregions that need to be addressed. This is done by applying analpha mask to the image. The alpha mask can be constructedby composition of the three intermediate masks that performwhite point balancing, chrominance gamut morphing, andperceptual brightness constraining.

1) White Point Balancing: This step finds a per channelscale factor, αl, for each projector to match a desired whitepoint with chromaticity coordinates (xD, yD). In order toperform this step it is necessary to have measured the colorprimaries, (xl, yl), for each projector with a colorimeter. Wesolve for the scale factors using,∑

l αlBl

(xlyl

)∑l αlBl

=

(xDyD

)(4)

and by fixing αR = 1, where Bl = Xl + Yl + Zl.The valuesof αl are then normalized by the scale factor with the largestvalue.

2) Chrominance Gamut Morphing: To ensure a smoothtransition in color differences across projectors, this stepimplements a pixel-by-pixel chrominance gamut morphingin the overlap regions. We measured the luminance of eachprojector across all pixels using a camera with the methodfound in [8]. To preserve the white balance achieved in the

Fig. 4. Local coordinate system for projector calibration.

previous step we look for one common scale factor for all colorchannels. In each overlap region we first define the desiredluminance profile as a smoothly varying “s-curve” betweenthe luminance at the first edge to the luminance at the secondedge,

LD = τ ∗ LE1+ (1− τ) ∗ LE2

(5)

where LD is the desired luminance, LE1 is the luminanceof the first edge, LE2 is the luminance of the second edge,and τ is a value from 0-1 representing the contributionsof each defined by the “s-curve”. Then, to distribute theluminance contributions across both projectors to create thedesired luminance profile, we also require that each individualprojector’s luminance in the overlap region falls off in asmoothly varying “s-curve” to avoid any hard edges. Oncethe luminance contribution from each projector has beendetermined, per projector alpha maps are generated by takingthe ratio of the desired contribution to maximum availableluminance from that projector at that pixel.

3) Perceptual Brightness Constraining: To refine thebrightness variations leftover after the gamut morphing, aperception based gradient constraint is applied to the entireluminance map [2][9]. There are three constraints enforced,the first of which is called the Capability Constraint. Itensures that the modified luminance, W ′

l , never exceeds themaximum luminance achievable by the display. The PerceptualUniformity Constraint maintains a smooth variation in theluminance by requiring

5W ′l <

1

λ∗W ′

l , (6)

where λ is the perceptually based smoothing parameter [2].Finally, the Display Quality Objective Function maximizesthe dynamic range by ensuring that the integration of theluminance function over the display is maximized. Enforcingthese constraints allows greater image fidelity in the systemby taking advantage of the human visual system’s forgivenessfor slow variation.

C. Rendering

Before an image can be shown, it must first be renderedto the screen using the geometric model constructed duringthe initial calibration. This viewpoint specific image is thentransformed into the projector luminance space by applyingindividually measured projector transfer functions. Once the

8

(a)

(b)

(c)

Fig. 5. (a) Uncorrected image (b) Exemplar rendered imagery aftergeometric registration only and (c) geometric registration with photometric& colorimetric calibration.

white balancing, gamut morphing, and brightness constraininghave been applied, the image must be transformed back viathe inverse projector transfer functions.

IV. IMPLEMENTATION & RESULTS

Our implementation made use of four Panasonic PT-DW6300 projectors arranged linearly projecting onto a cylin-drical screen of aspect ratio a = 4.29. The primaries ofeach projector were measured using a Konica Minolta CS-100A spot colorimeter while displaying maximum intensityred, green, and blue values. The photographs taken for bothgeometric calibration, luminance and transfer function mea-surements were done with a Canon PowerShot SX210ISarranged on a tripod to the right of the screen. To measurethe luminance full intensity red, green, blue, and white weredisplayed from each projector and photographed while notwo overlapping projectors were simultaneously active. Theprojector transfer functions were found by projecting andcapturing ramps uniformly sampling the red, green, and bluechannels from digital counts 0-255. Coding for each step ofthe calibration was done in the MATLAB environment.

Results of calibration are shown in Fig. 5. Fig. 5(b) showsthe system after only geometric calibration. The overlappingregions are still obvious and delineated by sharp changes inluminance. After photometric & colorimetric calibration, Fig.5(c), the sharp changes have been removed and the imagerytransitions smoothly between one projector and the next.

V. CONCLUSION

In this paper we have shown how to geometrically calibrateas well as photometrically & colorimetrically calibrate amultiple projector system on a cylindrical surface. The geo-metric calibration allows us to reconstruct the projector-screenrelationship and a 3D representation of the screen surface byusing only one camera. This allows us to accurately registerthe imagery displayed by each projector so that overlapping

pixels correspond to and display the same content. Knowingthis geometric relationship, we are then able to address thecolor and luminance variations across projectors and in theoverlap regions. We are able to smooth the chrominance andluminance variations between projectors in the overlap regionsusing white point balancing, chrominance gamut morphing,and perceptual brightness constraining. This results in theability to display seamless high-resolution imagery on curved,semi-immersive surfaces through the use of multiple low-resolution projectors providing quality comparable to that ofa single ultra-high resolution projector.

One limitation of this work is that we assume the chromi-nance of a projector is spatially consistent, unchanging withluminance level and is thus independent of the imagery beingdisplayed. In fact, the vignetting effect is also assumed tobe a consistent function across all luminance levels. A moreaccurate calibration model would analyze spatial and lumi-nance dependence of chrominance and vignetting. However,this would require a “brute-force” approach to characterizingevery combination of spatial location and luminance level. Ifthis information were available, then at runtime an optimizedblending mask could be constructed for the specific imagerybeing projected.

ACKNOWLEDGEMENT

The authors would like to acknowledge the National ScienceFoundation for supporting this research through NSF project0909588. See http://geovis.cis.rit.edu. We would also like tothank the RIT Center for Student Innovation for providing thedisplay facilities.

REFERENCES

[1] B. Sajadi and A. Majumder, “Auto-calibration of cylindrical multi-projector systems,” in Virtual Reality Conference (VR), 2010 IEEE, March2010, pp. 155 –162.

[2] B. Sajadi, M. Lazarov, M. Gopi, and A. Majumder, “Color seamlessnessin multi-projector displays using constrained gamut morphing,” IEEETransactions on Visualization and Computer Graphics, vol. 15, no. 6,pp. 1317–1326, 2009.

[3] R. Raskar, J. van Baar, T. Willwacher, and S. Rao, “Quadric transfer forimmersive curved screen displays,” in Eurographics, vol. 23, no. 3, 2004.

[4] R. Raskar, M. S. Brown, R. Yang, W.-C. Chen, G. Welch, H. Towles,B. Seales, and H. Fuchs, “Multi-projector displays using camera-basedregistration,” in VISUALIZATION ’99: Proceedings of the 10th IEEEVisualization 1999 Conference (VIS ’99). Washington, DC, USA: IEEEComputer Society, 1999.

[5] M. Brown, A. Majumder, and R. Yang, “Camera-based calibrationtechniques for seamless multiprojector displays,” IEEE Transactions onVisualization and Computer Graphics, vol. 11, no. 2, pp. 193–206, 2005.

[6] G. Wallace, H. Chen, and K. Li, “Color gamut matching for tileddisplay walls,” in EGVE ’03: Proceedings of the workshop on Virtualenvironments 2003. New York, NY, USA: ACM, 2003, pp. 293–302.

[7] A. Majumder, Z. He, H. Towles, and G. Welch, “Achieving coloruniformity across multi-projector displays,” in VIS ’00: Proceedings ofthe conference on Visualization ’00. Los Alamitos, CA, USA: IEEEComputer Society Press, 2000, pp. 117–124.

[8] A. Majumder and R. Stevens, “Lam: luminance attenuation map forphotometric uniformity in projection based displays,” in VRST ’02:Proceedings of the ACM symposium on Virtual reality software andtechnology. New York, NY, USA: ACM, 2002, pp. 147–154.

[9] ——, “Perceptual photometric seamlessness in projection-based tileddisplays,” ACM Trans. Graph., vol. 24, no. 1, pp. 118–139, 2005.

9