ICCV 2003Colour Workshop
1
Recovery of Chromaticity Image Free from Shadows via
Illumination Invariance
Mark S. Drew1, Graham D. Finlayson2,
& Steven D. Hordley2
2School of Information Systems, University of East Anglia, UK
1School of Computing Science, Simon Fraser University, Canada
ICCV 2003Colour Workshop
2
Overview
Introduction
Shadow Free Greyscale images
- Illuminant Invariance at a pixel -- 1D image
Shadow Free Chromaticity Images
- Better-behaved 2D-colour image invariant to lighting
Application- For shadow-edge-map aimed at re-integrating to obtain full colour, shadow-free image
ICCV 2003Colour Workshop
3
The Aim: Shadow Removal
We would like to go from a colour image with shadows to the same colour image, but without the shadows.
ICCV 2003Colour Workshop
4
Why Shadow Removal?For Computer Vision, Image Enhancement, Scene Re-lighting, etc.
- e.g., improved object tracking, segmentation etc.
Two successive video frames
Motion map, original colour space
Motion map, invariant colour space
snake
ICCV 2003Colour Workshop
5
What is a shadow?
Region Lit by Sunlight and
Sky-light
Region Lit by Sky-light only
A shadow is a local change in illumination intensity and (often) illumination colour.
ICCV 2003Colour Workshop
6
Removing Shadows
So, if we can factor out the illumination locally (at a pixel) it should follow that we remove the shadows.
Can we factor out illumination locally? That is, can we derive an illumination-invariant colour representation at a
single image pixel?
Yes, provided that our camera and illumination satisfy certain restrictions ….
ICCV 2003Colour Workshop
7
Conditions for Illumination InvarianceAssumptions (but works anyway…!):
(1) If sensors can be represented as delta functions (they respond only at a single wavelength)
(2) and illumination is restricted to the Planckian locus
(3) then we can find a 1D coordinate, a function of image chromaticities, which is invariant to
illuminant colour and intensity
(4) this gives us a greyscale representation of our original image, but without the shadows
(so takes us a third of the way to the goal of this talk!)
(5) But the greyscale value in fact lives in a 2D log- chromaticity colour space, (so takes us a 2/3 of the way) [and exponentiating goes back to a
rank-3 colour].
ICCV 2003Colour Workshop
8
Chromaticity: )/(},,{ BGRBGR
2D chromaticity is much more information than 1D greyscale:
Can we obtain a shadowless chromaticity image?
greycolour chromaticity
ICCV 2003Colour Workshop
9
dQSEB
dQSEG
dQSER
)()()(
)()()(
)()()(
3
2
1
)(S
)(E )()( SE
Image Formation
Camera responses depend on 3 factors: light (E), surface (S),
and sensor (Q) is Lambertian shading
ICCV 2003Colour Workshop
10
Q2()
Sen s
i tiv
i ty
Q1() Q3()
=
Delta functions “select” single wavelengths:
R R1 qQ
Using Delta-Function Sensitivities
RRRRR SEqdESq
GGqQ 2
BBqQ 3
RRR SEqR
GGG SEqG
BBB SEqB
ICCV 2003Colour Workshop
11
Characterizing Typical Illuminants
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
r/(r+g+b)
g/(
r+g
+b
)
Illuminant Chromaticities Most typical illuminants lie on, or close to, the Planckian locus (the red line in the figure)
So, let’s represent illuminants by their
equivalent Planckian black-body
illuminants ...
ICCV 2003Colour Workshop
12
1
51 1)(
2
T
c
ecIE Here I controls the overall intensity of light, T is the
temperature, and c1, c2 are constants
Planckian Black-body Radiators
For typical illuminants, c2>>T.
So, Wien’sapproximation:
T
c
ecIE 2
51)(
ICCV 2003Colour Workshop
14
For delta-function sensors and Planckian illumination we have:
Back to the image formation equation
T
c
kkkkkecIqSR 2
51)(
Surface Light
ICCV 2003Colour Workshop
15
Band-ratio chromaticity
G
R
B
Plane G=1
Perspective projection onto G=1
,2..1,/ kRR pkk
Let us define a set of 2D band-ratio chromaticities:
p is one of the channels, (Green, say)
ICCV 2003Colour Workshop
16
Let’s take log’s:
Band-ratios remove shading and intensity
Teess pkpkkk /)()/log()log('
with ,)(51 kkkk qScs kk ce /2
Gives a straight line:
)(
)())/log(()/log(
1
21
'12
'2
p
ppp ee
eessss
Shading and intensity are gone.
ICCV 2003Colour Workshop
17
Calibration: find invariant direction
Log-ratio chromaticities for 6 surfaces under 14 different Planckian
illuminants, HP912 camera
Macbeth ColorChecker:
24 patches
ICCV 2003Colour Workshop
18
Deriving the Illuminant Invariant
This axis is invariant to shading + illuminant
intensity/colour
ICCV 2003Colour Workshop
19
Algorithm:
,2..1,/ kRR pkk
)log('kk
Plot, and subtract mean for each colour patch:
SVD (2nd eigenvector) gives invariant direction.
ICCV 2003Colour Workshop
20
Algorithm, cont’d:
eI k''
Form greyscale I’ in log-space:
)'exp(II exponentiate:
ICCV 2003Colour Workshop
21
Obtaining invariant Chromaticity image (1):
We observe: line in 2D chromaticity space is still 2D, if we use projector,
rather than rotation:
,||||
)(
e
eeP
T
e
''~kek P 2-vector
eI '~'
ICCV 2003Colour Workshop
22
Obtaining invariant Chromaticity image (2):
However, we have removed all lighting! put back offset in e-direction equal
to regression on top 1% brightness pixels:
ICCV 2003Colour Workshop
23
Obtaining invariant Chromaticity image (3):
offset in e-direction:
)'~exp(~,'~'~'~ extralight
We are most familiar with L1-chromaticity)(/},,{ BGRBGR
)1(/},1,{ 2121
ICCV 2003Colour Workshop
24
Obtaining invariant Chromaticity image (4):
In terms of L1-chromaticity:
orig. recovered
ICCV 2003Colour Workshop
25
Obtaining invariant Chromaticity image (5):
Projection line becomes a rank ~3
curve in L1 chromaticity space
ICCV 2003Colour Workshop
26
Obtaining invariant Chromaticity image (6):
We can do better on fitting recovered chromaticity to original — regress on
brightest quartile:
ICCV 2003Colour Workshop
29
Main Advantage: chromaticity invariant (in [0,1]) is better-behaved than greyscale invariant –– betterfor shadow-free re-integration (ECCV02)