projector with radiometric screen compensation shree k. nayar, harish peri michael grossberg, peter...
Post on 22-Dec-2015
214 views
TRANSCRIPT
Projector with Radiometric Screen
Compensation
Shree K. Nayar, Harish Peri
Michael Grossberg, Peter Belhumeur
Support:National Science Foundation
Computer ScienceColumbia University
Procams WorkshopICCV 2003, Nice, France
Projecting on Any Surface ?
Textured Screen
Camera
Computer
Projector
Projector-Camera System
Geometric Calibration
Projector Input Camera Output
Maximum Error: 0.6 pixels, RMS Error: 0.18 pixels
Display Device
Projector
ScreenI D
P
E
xixd
xs
CameraCapture Device
BM
CL
xbxm
Projector-Camera Pipeline: Radiometry
What Display Image (I) will produce a Desired Captured Image (M) ?
Radiometric Model (for each pixel)
Display Device
Projector
ScreenI DP
E
CameraCapture Device
BMC L
))(( KKKK IdpP
display reponseprojector reponse
))(( 11LLLL MmbC
capture reponsecamera reponse
dqswV LKKL )()()(projector channel
screen camera channel
B
G
R
BBBGBR
GBGGGR
RBRGRR
B
G
R
P
P
P
VVV
VVV
VVV
C
C
C
camera irradiance projector brightness
color mixing matrix
Finding the Model Parameters (at each pixel)
Display Device
Projector
ScreenI DP
E
CameraCapture Device
BMC L
B
G
R
BBBGBR
GBGGGR
RBRGRR
B
G
R
P
P
P
VVV
VVV
VVV
C
C
C
camera irradiance projector brightness
color mixing matrix
)(
)(
)(
1
1
1
BBB
GGG
RRR
PgI
PgI
PgI
display image color projector color
composite response
B
G
R
BGBR
GBGR
RBRG
B
G
R
P
P
P
VV
VV
VV
C
C
C
1
1
1
camera irradiance projector brightness
modified color mixing matrix
(known)
Finding the Color Mixing Matrix
B
G
R
BGBR
GBGR
RBRG
B
G
R
P
P
P
VV
VV
VV
C
C
C
1
1
1
camera irradiance projector brightness
modified color mixing matrix
(known)
arbitrary image only change in red
1I 2I
RR CP
R
GGR C
CV
R
BBR C
CV
only change in green only change in blue
3I 4I
GG CP
G
RRG C
CV
G
BBG C
CV
BB CP
B
RRB C
CV
B
GGB C
CV
00.10134.0012.0
0363.000.10361.0
0010.05534.000.1
00.10253.00172.0
0652.000.10344.0
0010.00434.000.1
00.10218.00094.0
0500.000.10471.0
0086.05973.000.1
00.10588.00119.0
0465.000.10357.0
0001.06764.000.1
00.10213.00100.0
0476.000.10500.0
0095.05561.000.1
00.10195.00338.0
0632.000.10593.0
0012.05804.000.1
00.10278.00217.0
0425.000.10543.0
0010.06972.000.1
00.10321.00367.0
1016.000.10461.0
0169.04545.000.1
Color Mixing: V Matrices
Radiometric Calibration: Color
Projector Input Camera Output
)(
)(
)(
1
1
1
BBB
GGG
RRR
PgI
PgI
PgI
desired projector output
compensated display image
composite response
Texture of Screen
Color Experiment
Uncompensation outputs (flat gray inputs: 100, 150, 200 gray levels)
Compensation images (flat gray inputs: 100, 150, 200 gray levels)
Compensated outputs (flat gray inputs: 100, 150, 200 gray levels)
Compensation Accuracy
50,50,50
100,100,100
150,150,150
200,200,200
25,26,26
66,70,71
75,98,95
50,99,90
2,3,7
6,8,14
19,15,21
39,29,25
24.1,24.2,24.5
58.9,57.2,64.9
49.0,44.5,69.1
27.7,47.5,42.7
0.6, 0.8, 1.7
1.6, 1.9, 5.4
4.8, 3.1, 5.8
12.7,4.0,10.7
ProjectedBrightness
Max. Error RMS Error
Without Comp.
With Comp. (R, G, B)
(R, G, B) (R, G, B)
Without Comp.
With Comp.
Original (desired) image (I) Uncompensated output (M)
Face Image on Checkerboard Screen
Compensated output (M)~
Compensation image ( I )~
Face Video on Brick Texture
Normal Projector Projector With Compensation
Scene Video on Brick Texture
Normal Projector Projector With Compensation
Face Video on Butterfly Poster
Normal Projector Projector With Compensation
Summary
• Off-Line Method for Radiometric Calibration
• On-Line Algorithm for Screen Compensation
• Refinement using Continuous Feedback
• Radiometric Model for Projector-Camera System