object recognition under varying illumination. lighting changes objects appearance
Post on 21-Dec-2015
226 views
TRANSCRIPT
Object recognition under varying illumination
Lighting changes objects appearance.
SpecularLambertian
How do we recognize these objects?
Few Definitions: Reflection
• Reflection - The scattering of light from an object.
• Two extreme cases: diffuse reflection and specular reflection.
• Real objects reflect light as a mixture of these two extremes.
Few Definitions: Lambertian Reflection
• Surface reflects equally in all directions.– Examples: chalk, clay,
cloth, matte paint
• Brightness doesn’t depend on viewpoint.
• Amount of light striking surface proportional to cos θ.
LN DD KI LN DD KI
intensity
albedo surface normal
(light intensity)* (light direction)
L,0 Nmax DD KI L,0 Nmax DD KI
Few Definitions: Specular Reflection
• Specular surfaces reflect light more strongly in some directions than in others.
• Appearance of a surface depends on the direction L of the light source, direction of the surface normal N, and direction V of viewing.
The vectors L, N and R all lie in one plane
Few Definitions: Specular Reflection
• Perfect mirror: The angle of incidence equals the angle of reflection.
rough specular
RN
L
mirror
RN
L θθ
• Rough specular : Most specular surfaces reflect energy in a tight distribution (or lobe) centered on the optical reflection direction– Examples: metals,glass
NNLL
ll
RR
VVrr
nss KI cosL n
ss KI cosL
Few Definitions: Phong Model
• Determine the angle α between the direction V of viewing and the direction R of reflection by an ideal mirror.
• Assume the intensity of reflected light is proportional to cos(α)
• The exponent n (“shine”) is determined empirically.
• Large values of n make the surface behave more like an ideal mirror.
• Phong’s exponent controls how fast the highlight “falls-off”
Lambertian
Main Approaches
2D methods based on quasi-invariance to lighting
-50 0 50 100 150 200
50
100
150
200
-50 0 50 100 150 200
50
100
150
200
250
Model- based: 3D to 2D
3D
imagerendering
Low dimensional representation of an object’s image set under different lightings
comparecompare
Main Approaches
Specular
2D Methods: will be distracted by highlights and lack of real edges.
3D Methods: Specular objects cannot be well approximated by low-dimensional linear sub-spaces.
Apply Lambertian methods and treat specularities as noise
?
Use specularities for recognition
Matching Specularities
hypothesized pose
approximate
3D model
Mapping
imageimage
Gaussian sphereGaussian sphere
Finding Specularity
query
map onto the sphere
consistent
specularity disk
map back
recovered highlights
threshold
specular candidates
Wrong Match
query
inconsistent
map onto the sphere
specularity disk
map back
recovered highlights
threshold
specular candidates
Combined Method for Recognition of General Objects
• Integrate knowledge about highlights with the Lambertian component.
• No prior knowledge of lighting.
Recover light direction from Lambertian component.
• No prior knowledge of how specular and how Lambertian the object is.
Comparison
renderrender
Lambertian Lambertian componentcomponent
highlighthighlight
Lambertian Lambertian componentcomponent
highlighthighlight
Same objectSame object
Uncontrolled Lighting
• First step: allow multiple unknown light sources.– Extend the highlight recovery to work with
known multiple light sources. – Detect multiple light source directions from the
Lambertian component.– Use both Lambertian and specular parts to
more robust detection of light sources.
PROJECT 5
Extend the specular recognition algorithm* to multiple light sources. Collect a test set of several rotationally symmetric glass objects:- Take images of these objects filled with opaque liquid for 3D model construction.- Take 3 images of each object with 2 and 3 light sources and different backgrounds.
Test the algorithm on these objects.*M. Osadchy, D.W. Jacobs and R. Ramamoorthi, Using specularities for recognition, IEEE International Conference on Computer Vision (ICCV), 2003
Multiple Light Source Detection
Given an image of known shape, recover the light sources.
)0,smax(N m
mdKI
Sphere Illumination
0 50 100 150 200 250 300
50
100
150
200
250
300
0sn p
s,0nmax ppI
0sn p
0sn p
Critical Boundary
Multiple Light Sources
0 50 100 150 200 250 300
50
100
150
200
250
300
1s
2s
3skp
Lmmpp
k
I vnsn
kL Set of lights that illuminate pixels in kR
kv Virtual light associated with region kR
kR
Finding Critical Boundaries
0 50 100 150 200 250 300
50
100
150
200
250
300
wI
2
2-vN
1ww I
wf smalllarge
• Threshold f• Windows with large f
correspond to points on critical boundaries.
• Apply Hough Transform to fit points to critical boundaries.
Real light source
If two regions and are adjacent on theimage, with and the corresponding virtual lights then
1R2R
1v 2v
ms 21 vv
0 50 100 150 200 250 300
50
100
150
200
250
300
322v ss 3211v sss
121 v-v s
1s
s1
s3
s2
PROJECTS 6
• Implement V2R algorithm* on sphere with 3 light sources (no opposite lights).
• Extend V2R algorithm to textured spherical objects.
• Large bonus: extend this algorithm to run on arbitrary convex objects.
*Christos-Savvas Bouganis, Mike Brookes. "Multiple Light Source Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 4, pp. 509-514, April, 2004.