color and brightness constancy jim rehg cs 4495/7495 computer vision lecture 25 & 26 wed oct 18,...
Post on 18-Jan-2016
222 Views
Preview:
TRANSCRIPT
Color and Brightness ConstancyColor and Brightness Constancy
Jim RehgJim Rehg
CS 4495/7495 Computer VisionCS 4495/7495 Computer Vision
Lecture 25 & 26Lecture 25 & 26
Wed Oct 18, 2002Wed Oct 18, 2002
2 J. M. Rehg © 2002
OutlineOutline
Human color inferenceHuman color inference Land’s RetinexLand’s Retinex Dichromatic reflectance modelDichromatic reflectance model Finite dimensional linear modelsFinite dimensional linear models Color constancy algorithmColor constancy algorithm
3 J. M. Rehg © 2002
Human Color ConstancyHuman Color Constancy
Distinguish betweenDistinguish between Color constancy, which refers to hue and saturationColor constancy, which refers to hue and saturation Lightness constancy, which refers to gray-level.Lightness constancy, which refers to gray-level.
Humans can perceiveHumans can perceive Color a surface would have under white light Color a surface would have under white light
(surface color)(surface color) Color of the reflected light (limited ability to separate Color of the reflected light (limited ability to separate
surface color from measured color)surface color from measured color) Color of illuminant (even more limited)Color of illuminant (even more limited)
4 J. M. Rehg © 2002
Spatial Arrangement and Color Spatial Arrangement and Color PerceptionPerception
5 J. M. Rehg © 2002
Spatial Arrangement and Color Spatial Arrangement and Color PerceptionPerception
6 J. M. Rehg © 2002
Spatial Arrangement and Color Spatial Arrangement and Color PerceptionPerception
7 J. M. Rehg © 2002
Land’s Mondrian ExperimentsLand’s Mondrian Experiments
The (by-now) familiar phenomena: Squares of The (by-now) familiar phenomena: Squares of color with the same color radiance yield very color with the same color radiance yield very different color perceptionsdifferent color perceptions
Photometer: 1.0, 0.3, 0.3 Photometer: 1.0, 0.3, 0.3
Audience: “Red” Audience: “Blue”White light Colored light
8 J. M. Rehg © 2002
Basic Model for Lightness Basic Model for Lightness ConstancyConstancy
Modeling assumptions for cameraModeling assumptions for camera Planar frontal scenePlanar frontal scene Lambertian reflectanceLambertian reflectance Linear camera response Linear camera response
Camera model:Camera model: Modeling assumptions for sceneModeling assumptions for scene
Albedo is piecewise constantAlbedo is piecewise constant– Exception: ripening fruitException: ripening fruit
Illumination is slowly-varyingIllumination is slowly-varying– Exception: shadow boundariesException: shadow boundaries
)()()( xpxIkxC c
9 J. M. Rehg © 2002
Algorithm ComponentsAlgorithm Components
The goal is to determine what the surfaces in the The goal is to determine what the surfaces in the image would look like under white light.image would look like under white light.
A process that compares the brightness of patchs A process that compares the brightness of patchs across their common boundaries and computes across their common boundaries and computes relative brightness.relative brightness.
A process that establishes an absolute reference A process that establishes an absolute reference for lightness (e.g. brightest point is “white”)for lightness (e.g. brightest point is “white”)
10 J. M. Rehg © 2002
1-D Lightness “Retinex”1-D Lightness “Retinex”
Threshold gradient image to find surface (patch) boundaries
11 J. M. Rehg © 2002
1-D Lightness “Retinex”1-D Lightness “Retinex”
Integration to recover surface lightness (unknown constant)
12 J. M. Rehg © 2002
Extension to 2-DExtension to 2-D
Spatial issuesSpatial issues Integration becomes much harderIntegration becomes much harder
– Integrate along many sample paths (random walk)Integrate along many sample paths (random walk)– Loopy propagationLoopy propagation
Recover of absolute lightness/color referenceRecover of absolute lightness/color reference Brightest patch is whiteBrightest patch is white Average reflectance across scene is knownAverage reflectance across scene is known Gamut is knownGamut is known Specularities can be detectedSpecularities can be detected Known reference (color chart, skin color, etc.)Known reference (color chart, skin color, etc.)
13 J. M. Rehg © 2002
Color RetinexColor Retinex
Images courtesy John McCann
14 J. M. Rehg © 2002
Finding SpecularitiesFinding Specularities Dielectric materialsDielectric materials
Specularly reflected light has the color of the sourceSpecularly reflected light has the color of the source Reflected light has two components, we see their sumReflected light has two components, we see their sum
Diffuse (body reflection)Diffuse (body reflection) Specular (highlight)Specular (highlight)
Specularities produce a “Skewed-T” in the color Specularities produce a “Skewed-T” in the color histogram of the object.histogram of the object.
15 J. M. Rehg © 2002
R
G
B
Illuminant color
Diffuse component
T
S
Skewed-T in HistogramSkewed-T in Histogram
A Physical Approach to Color Image UnderstandingA Physical Approach to Color Image Understanding – Klinker, – Klinker, Shafer, and Kanade. IJCV 1990Shafer, and Kanade. IJCV 1990
16 J. M. Rehg © 2002R
G
B
R
G
B
Diffuseregion
Boundary ofspecularity
Skewed-T in HistogramSkewed-T in Histogram
17 J. M. Rehg © 2002
Recent Application to StereoRecent Application to Stereo
Motion of camera causes highlight location to change. Thiscue can be combined with histogram analysis.
Synthetic scene:
18 J. M. Rehg © 2002
Recent Application to StereoRecent Application to Stereo“Real” scene:
19 J. M. Rehg © 2002
Finite Dimensional Linear ModelsFinite Dimensional Linear Models
E ii i1
m
rj j j1
n
pk k ii i1
m
rj j
j1
n
d
irj k i j di1, j1
m,n
irjgijki1, j1
m,n
20 J. M. Rehg © 2002
Obtaining the illuminant from Obtaining the illuminant from specularitiesspecularities
Assume that a specularity Assume that a specularity has been identified, and has been identified, and material is dielectric.material is dielectric.
Then in the specularity, we Then in the specularity, we havehave
Assuming Assuming we know the sensitivities we know the sensitivities
and the illuminant basis and the illuminant basis functionsfunctions
there are no more there are no more illuminant basis functions illuminant basis functions than receptorsthan receptors
This linear system yields This linear system yields the illuminant coefficients.the illuminant coefficients.
pk k E d
i k i di1
m
21 J. M. Rehg © 2002
Obtaining the illuminant from Obtaining the illuminant from average color assumptionsaverage color assumptions
Assume the spatial Assume the spatial average reflectance is average reflectance is knownknown
We can measure the We can measure the spatial average of the spatial average of the receptor response to getreceptor response to get
AssumingAssuming g_ijk are knowng_ijk are known average reflectance is average reflectance is
knownknown there are not more there are not more
receptor types than receptor types than illuminant basis functionsilluminant basis functions
We can recover the We can recover the illuminant coefficients from illuminant coefficients from this linear systemthis linear system
r j j j1
n
pk i r jgijki1, j1
m,n
22 J. M. Rehg © 2002
Normalizing the GamutNormalizing the Gamut
The gamut (collection of all pixel values in image) The gamut (collection of all pixel values in image) contains information about the light sourcecontains information about the light source
It is usually impossible to obtain extreme color It is usually impossible to obtain extreme color readings (255,0,0) under white lightreadings (255,0,0) under white light
The convex hull of the gamut constrains illuminantThe convex hull of the gamut constrains illuminant Gamut mapping algorithm (Forsyth ’90)Gamut mapping algorithm (Forsyth ’90)
Obtain convex hull W of pixels under white lightObtain convex hull W of pixels under white light Obtain convex hull G of input imageObtain convex hull G of input image The mapping M(G) must have property The mapping M(G) must have property WGM )(
top related