Colors and sensors
Slides from Bill Freeman, Fredo Durand, Rob Fergus, and David Forsyth, Alyosha Efros
Agenda
• Project 1 delayed till Thursday October 9th
• Color• Sensors• Matlab intro
Project 1: Demosaicing
• Warning; it might take some time– Getting familiar with Matlab– Writeup solutions in html and submit to EEE– I put up a project template (with sample code &
writeup)
Image Formation
Digital Camera
The Eye
Film
Digital camera
• A digital camera replaces film with a sensor array– Each cell in the array is light-sensitive diode that converts photons to electrons– Two common types
• Charge Coupled Device (CCD) • CMOS
– http://electronics.howstuffworks.com/digital-camera.htm
Slide by Steve Seitz
Sensor Array
CMOS sensor
Sampling and quantizing brightness
The real world of colored light: why is color useful?
• Find things to eat
• Spot dangerousthings
What’s the physics behind color?
The Physics of Light
Any patch of light can be completely describedphysically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm.
400 500 600 700
Wavelength (nm.)
# Photons(per ms.)
© Stephen E. Palmer, 2002
The Physics of Light
.
# P
ho
ton
s
D. Normal Daylight
Wavelength (nm.)
B. Gallium Phosphide Crystal
400 500 600 700
# P
ho
ton
s
Wavelength (nm.)
A. Ruby Laser
400 500 600 700
400 500 600 700
# P
ho
ton
s
C. Tungsten Lightbulb
400 500 600 700
# P
ho
ton
s
Some examples of the spectra of light sources
© Stephen E. Palmer, 2002
The Physics of Light
Some examples of the reflectance spectra of surfaces
Wavelength (nm)
% P
hoto
ns R
efle
cted
Red
400 700
Yellow
400 700
Blue
400 700
Purple
400 700
© Stephen E. Palmer, 2002
Electromagnetic Spectrum
http://www.yorku.ca/eye/photopik.htm
Human Luminance Sensitivity Function
Computer Vision - A Modern Approach
Set: Color Slides by D.A. Forsyth
Black body radiators• Construct a hot body with near-zero albedo (black body)
– Easiest way to do this is to build a hollow metal object with a tiny hole in it, and look at the hole.
• The spectral power distribution of light leaving this object is a simple function of temperature
• This leads to the notion of “color temperature” --- the temperature of a black body that would look the same
€
E λ( )∝1λ5 ⎛ ⎝
⎞ ⎠
1exp hc kλT( ) −1
⎛
⎝ ⎜
⎞
⎠ ⎟
Why do we see light of these wavelengths?
© Stephen E. Palmer, 2002
.
0 1000 2000 3000
En
erg
y
Wavelength (nm)
400 700
700 C
2000 C
5000 C
10000 C
VisibleRegion
…because that’s where theSun radiates EM energy
Visible Light
Plank’s law for Blackbody radiationSurface of the sun: ~5800K
The Psychophysical Correspondence
There is no simple functional description for the perceivedcolor of all lights under all viewing conditions, but …...
A helpful constraint: Consider only physical spectra with normal distributions
area
Wavelength (nm.)
# Photons
400 700500 600
mean
variance
© Stephen E. Palmer, 2002
The Psychophysical Correspondence
Mean Hue
yellowgreenblue
# P
hoto
ns
Wavelength
© Stephen E. Palmer, 2002
The Psychophysical Correspondence
Variance Saturation
Wavelength
high
medium
low
hi.
med.
low# P
hoto
ns
© Stephen E. Palmer, 2002
The Psychophysical Correspondence
Area Brightness#
Pho
tons
Wavelength
B. Area Lightness
bright
dark
© Stephen E. Palmer, 2002
Spectral Image Formation
I(λ)
R(λ)
Si(λ)
I(λ) R(λ)
From Foundation of Vision by Brian Wandell, Sinauer Associates, 1995
Spectral Image Formation
I(λ) – Illumination Spectrum
R(λ) - Surface reflectance/transmission
Si(λ) - Spectral sensitivity of photoreceptor i
Pixel value / Perceived color depends on all 3 terms! Problem of color constancy
Pi (λ) = I(λ)R(λ)Si (λ)
Color appearance depends on nearby colors
Top pink should look stronger
Color names for cartoon spectra
400 500 600 700 nm
400 500 600 700 nm
400 500 600 700 nm
red
gree
nbl
ue
400 500 600 700 nm
cyan
mag
enta
yell
ow
400 500 600 700 nm
400 500 600 700 nm
Slide credit:W. Freeman
Additive color mixing
400 500 600 700 nm
400 500 600 700 nm
red
gree
n
Red and green make…
400 500 600 700 nm
yell
ow
Yellow!
When colors combine by adding the color spectra. Example color displays that follow this mixing rule: CRT phosphors, multiple projectors aimed at a screen, Polachrome slide film.
Slide credit:W. Freeman
Additive color mixing of illuminants
Subtractive color mixing
When colors combine by multiplying the color spectra. Examples that follow this mixing rule: most photographic films, paint, cascaded optical filters, crayons.
400 500 600 700 nm
cyan
yell
ow
400 500 600 700 nm
Cyan and yellow (in crayons,called “blue” and yellow) make…
400 500 600 700 nmGreen!gr
een
Slide credit:W. Freeman
Subtractive color mixing of materials
• Light reflecting off colored object
• E.g. printing inks
Wikipedia
Color matching experiment
Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995
Color matching experiment 1
Color matching experiment 1
p1 p2 p3
Color matching experiment 1
p1 p2 p3
Color matching experiment 1
p1 p2 p3
The primary color amounts needed for a match
Color matching experiment 2
Color matching experiment 2
p1 p2 p3
Color matching experiment 2
p1 p2 p3
Color matching experiment 2
p1 p2 p3 p1 p2 p3
We say a “negative” amount of p2 was needed to make the match, because we added it to the test color’s side.
The primary color amounts needed for a match:
p1 p2 p3
Measure color by color-matching paradigm
• Pick a set of 3 primary color lights.• Find the amounts of each primary, e1, e2, e3, needed
to match some spectral signal, t.• Those amounts, e1, e2, e3, describe the color of t. If
you have some other spectral signal, s, and s matches t perceptually, then e1, e2, e3 will also match s, by Grassman’s laws.
• Why this is useful—it lets us:– Predict the color of a new spectral signal– Translate to representations using other primary lights.
Goal: compute the color match for any color signal for any set of primary colors
• Examples of why you’d want to do that:– Want to paint a carton of Kodak film with the Kodak
yellow color.
– Want to match skin color of a person in a photograph printed on an ink jet printer to their true skin color.
– Want the colors in the world, on a monitor, and in a print format to all look the same.
Color matching functions for a particular set of monochromatic primaries
p1 = 645.2 nmp2 = 525.3 nmp3 = 444.4 nm
Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995
Questions?
Some other color spaces…
NTSC color components: Y, I, Q
B
G
R
Q
I
Y
312.0523.0211.0
322.0274.0596.0
114.0587.0299.0
NTSC - RGB
HSV hexcone
Forsyth & Ponce
Hue Saturation Value• Value: from black to white
• Hue: dominant color (red, orange, etc)
• Saturation: from gray to vivid color
• HSV double cone
value
saturation
saturation
hue
CCD color sampling
The eye’s approach to color imaging
Cross-section of eye
Ganglion cell layer
Bipolar cell layer
Receptor layer
Pigmentedepithelium
Ganglion axons
Cross section of retina
© Stephen E. Palmer, 2002
Cones cone-shaped less sensitive operate in high light color vision
Two types of light-sensitive receptors
cone
rod
Rods rod-shaped highly sensitive operate at night gray-scale vision
Human eye photoreceptor spectral sensitivities
Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995
What colors would these look like?
Color Sensing in Camera (RGB)• 3-chip vs. 1-chip: quality vs. cost
• Why more green?
http://www.cooldihttp://www.cooldictionary.com/words/Bayer-filter.wikipediationary.com/words/Bayer-filter.wikipedia
Why 3 colors?
Slide by Steve Seitz
CCD color filter pattern
detector
The cause of color moire
detector
Fine black and white detail in imagemis-interpreted as color information.
Typical color moire patterns
Blow-up of electronic cameraimage. Notice spuriouscolors in the regionsof fine detail in the plants.
Color sampling artifacts
Human receptors vs CCD sensors
Distribution of incoming luminance into CCD sensors
Gamma correction
• Iout = Iin^(gamma), where gamma < 1