color images color image processing - home | department of ...elec4245/sp16/lec6-2x2.pdf · imaging...

Color Image Processing

Dr. Edmund Lam

Department of Electrical and Electronic EngineeringThe University of Hong Kong

ELEC4245: Digital Image Proessing(Second Semester, 2015–16)

http://www.eee.hku.hk/˜elec4245

E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 1 / 46

Color images

Color = RGB (red, green, blue)?

Yes, and no.


Physics of color and human vision

1 Physics of color and human vision

2 Colorimetry: the science of color

3 Representing colors in different color spaces

4 Capturing colors with photodetectors



Physics of color

Light: electromagnetic waves with a range of frequencies (or,equivalently, wavelengths)

c = 3.0 × 108 m s−1; 400 nm ≤ λ ≤ 700 nm for visible light.There is color because our eyes interpret those frequencies



Physics of color

We interpret colors based on wavelengthsλ ≈ 400 nm =⇒ blueλ ≈ 550 nm =⇒ greenλ ≈ 700 nm =⇒ redλ < 400 nm: ultra-violet (UV)λ > 700 nm: infra-red (IR)UV and IR are gaining importance in imaging, e.g., in medicalimaging and remote sensing respectively

Electromagnetic waves with a range of frequencies at various strength hit oureyes (or photodetectors). How is color perceived?



Illumination

We can see things because there are illuminations, or light sources.Virtually all illuminations consist of light with a range of wavelengths.

We have to go to extraordinary length to create lasers that are of a“single” wavelength (more precisely, a very narrow bandwidthlight source).Each illumination is described with a curve showing the strengthof the electromagnetic radiations at different values of λ.Its normalization is called the spectral power distribution ofillumination as a function of wavelength.Mathematically, we denote it as I(λ)



Illumination

Examples of spectral power distribution:



Object

When the electromagnetic radiations from the illumination reach anobject, they are partially absorbed, and partially reflected ortransmitted.

the proportion of the reflection or transmission varies withwavelengthsit is irrespective of illuminationspectral reflectance or spectral transmittanceMathematically: R(λ) or T(λ)



Object

Examples of object reflectance:



Object

A “Macbeth Color Checker”



Color stimulus

What reaches our eye: reflectance of object upon illumination

S(λ) = I(λ)R(λ). (1)

Color stimulus influences the color perceptionImportant corollary: Color is not an inherent feature of an objectWe can make a “red” T-shirt looks “blue” with careful change ofillumination



Human Visual System

We often don’t need the entire color stimulus curveWe summarize in 3 numbersPossibility of ambiguity due to information lossEnough for color reproduction due to human visual system (HVS)



Photoreceptors

Our eyes have “sensors” called photoreceptors. There are two types:cones and rods.

Cones Rods

6 to 7 million 75 to 150 million

Located together at fovea Distributed all over the retina

Highly sensitive to color Not for color vision

Help resolve fine details Provide broad field of view

Photopic (bright-light) vision Scotopic (dim-light) vision



Photoreceptors

Cones are responsible for color vision.

3 types of cones:L-cones (long-wavelength peak sensitivity)M-cones (middle-wavelength peak sensitivity)S-cones (short-wavelength peak sensitivity)

They have different spatial sampling distributionsL- and M-cones have similar wavelength sensitivityS-cone is significantly differentS-cone is relatively rare in the retina, and widely spaced



Photoreceptors



Spectral sensitivities

We use x(λ), y(λ), and z(λ) to denote the response of the L-, M-, andS-cones respectively.

Color is formed by integration!

X =

∫ 700

400x(λ)I(λ)R(λ) dλ

Y =

∫ 700

400y(λ)I(λ)R(λ) dλ (2)

Z =

∫ 700

400z(λ)I(λ)R(λ) dλ



Spectral sensitivities

If discretized, we can compute a summation:

XYZ

=

x(λ1) . . . x(λn)y(λ1) . . . y(λn)z(λ1) . . . z(λn)

I(λ1)R(λ1)...

I(λn)R(λn)

(3)

The triplet (X,Y,Z) is called trichromatic responses. These three arethe “magic numbers” sent to our brain for color interpretation.

It is possible that two objects with different stimuli produce the same color.We call them metamers.



Color formation

Our eyes make these interpretations, e.g.,Strong L-cone response =⇒ redStrong L-cones and M-cones responses =⇒ yellowMore generally, we represent the addition of colors withred/green/blue (RGB) as they correspond to the peaks of theresponses of L-, M-, and S-cones.

RGB are often referred to as primary colors.



Describing colors

Other than quantifying with the RGB “color components”, we oftenhave different ways to describe colors:

luminance: the “energy” of the light, and chrominance: the “color”hue: the “dominant wavelength” of the light, and saturation: the“purity” of the color (i.e., how much it is mixed with white)


Colorimetry: the science of color







Mixing colors

We can mix the primary colors to form other colors.

Many displays make use of RGB “light sources” to simulate differentcolors, e.g., your television / monitor (experiment: put a magnifyingglass in front of your LCD screen). This is called additive color system.



Mixing colors

Each color has its opposite, called color complement:

0◦30◦

60◦

90◦

120◦

150◦180◦

210◦

240◦

270◦

300◦

330◦

yellow

greenblue

magenta

red

cyanE. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 22 / 46


Mixing colors

Red’s complement is Cyan (C)Green’s complement is Magenta (M)Blue’s complement is Yellow (Y)

CMY are often referred to as secondary colors.

We often use them for subtractive color system, e.g., mixing inks, whichblock the reflection of that color. So colors subtract: if all colors arepresent, you perceive black.



Mixing colors

We can assign different weights to the “primary colors” to generatedifferent colors. Let’s say we have X amount of “red”, Y amount of“green”, Z amount of “blue”: (X,Y,Z) are called tri-stimulus values.

CIE in 1931 standardizes the following: (CIE stands for “CommissionInternationale de l’Eclairage” — the International Commission onIllumination)

red = 700 nmgreen = 546.1 nmblue = 435.8 nm



Mixing colors

We can normalize the X, Y, Z values by dividing the sum of the weights

x =X

X + Y + Z

y =Y

X + Y + Z(4)

z =Z

X + Y + Z

Note that x + y + z = 1. So, (x, y) determines the color!How can we reduce 3 parameters to 2? We lose the normalization,i.e., the brightness (luminance) of the color.



Mixing colors

Use (x, y) as the coordinates, and each point determines a color: theCIE chromaticity diagram. The boundaries are the “pure colors” atvarious wavelengths, known as the spectral locus.

y

x

z = 1 − (x + y)

380 nm

480 nm

490 nm

500 nm

510 nm

520 nm

540 nm

570 nm

600 nm

780 nmwhite



Mixing colors

The chromaticity diagram is useful to visualize mixing colors: supposewe have three light sources, “r”, “g”, and “b” respectively. The trianglerepresents all possible colors from mixing them: the color gamut

y

x

r

g

b



Mixing colors

Insight from the chromaticity diagram and color gamut:Bigger gamut means more color can be reproduced from the lightsourcesNo triangle can cover the entire chromaticity diagram: i.e., for anyfixed set of three light sources, there will always be colors thatcannot be reproduced. These colors are out of gamut.Having more light sources can generally increase the gamut, butstill cannot cover the entire space.

Color reproduction is a hard problem! You take a picture of an object,and you want the color to show up “correctly” on your computermonitor, your mobile phone, your photographic print, etc.


Representing colors in different color spaces







RGB color space

To represent a color image, each pixel should have three numbers. Thesimplest way is to let them represent red, green, and blue; this is calledthe RGB color space. We can visualize with a cube:

R

G

B

red

green

blue

yellow

cyan

magenta

K

W

K: black

W: whitegray

Each color is a linear combination from no (R,G,B) to full (R,G,B)E. Lam (The University of Hong Kong) ELEC4245 Jan–Apr, 2016 30 / 46


CMY color space

Remember that CMY are complements of RGB. Assume all values gofrom 0 to 1.

CMY

=

111

−RGB

. (5)

CMY are usually used in printers.In theory, C = M = Y = 1 gives black. For inkjet printers, however,mixing all inks usually give a rather muddy color.Black is used very frequently in printing =⇒most inkjet printersactually use at least 4 colors, i.e. CMYK. (K for black)



RGB to CMYK

Application: To print your digital photograph, a process known ascolor space conversion has to be done.

Often, this is RGB to CMYK.A non-linear mapping (one-to-many mapping theoretically).Has to be performed on every pixel =⇒ a time-consuming process!Often implemented in a look-up table (LUT).Often proprietary and specifically tuned for the printer engine.

A further complication: inks are high-density binary (ink or no ink),while digital images are usually lower-density 8-bit-per-color-channel.Printing involves a process known as halftoning.



NTSC and PAL

To give you a flavor of the use of different color spaces, consider thoseused in our TVs. YIQ is used for NTSC. YUV is used for PAL.

YIQ

=

0.299 0.587 0.1140.596 −0.275 −0.3210.212 −0.523 0.311

RGB

(6)

YUV

=

0.299 0.587 0.114−0.147 −0.289 0.4360.615 −0.516 −0.100

RGB

(7)

Note that

I = −U sin 33◦ + V cos 33◦

Q = U cos 33◦ + V sin 33◦



NTSC and PAL

How to interpret YIQ and YUV color spaces?Y is the (positively) weighted sum of RGB. It describes theluminance of the signal. Technically it’s also called the luma.Weights are different for RGB because of our eyes: e.g. we aremost responsive to green.In the old days, we have monochrome TV =⇒ it only decodes theY signal!

How about I, Q, U, and V? {I,Q} or {U,V} describe the hue (“purecolor”) and saturation (“degree of pure color diluted by white”)collectively. Remember we need three components to describe a color.

They are designed such that we can limit their spatial bandwidthwithout noticeable image degradation.YUV was proposed first; YIQ was an improvement because it wasfound that it could reduce the bandwidth to a greater degree.



HSI color space

In applications that deal with human more directly, shouldn’t we use amore direct description of color? Yes, we have HSI!

H = hueS = saturationI = intensity, a.k.a. “energy”

As with before, there should be a one-to-one mapping between RGBand HSI.



HSI color space

white

black

IS HI = 0.5



RGB to HSI

H =

{θ if B ≤ G360 − θ otherwise

S = 1 − 3(R + G + B)

[min(R,G,B)] (8)

I =13

(R + G + B)

where

θ = cos−1

12 [(R − G) + (R − B)]

√(R − G)2 + (R − B)(G − B)

and R ∈ [0, 1], G ∈ [0, 1], and B ∈ [0, 1].



HSI to RGBFor 0 ≤ H < 120◦, use H = H.

B = I(1 − S)

R = I[1 +

S cos Hcos(60◦ − H)

](9)

G = 1 − (R + B).

For 120◦ ≤ H < 240◦, compute H = H − 120◦.

R = I(1 − S)

G = I[1 +


](10)

B = 1 − (R + G).

For 240◦ ≤ H < 360◦, compute H = H − 240◦.

G = I(1 − S)

B = I[1 +


](11)

R = 1 − (G + B).



Color processing

How to do engineering applications that involve color?

First decision: what color space / modelSeldom a good idea to operate on RGBUsually much better on a luminance-chrominance model

Second decision: operate on 3-planes together or separately?Separately: would results from the 3 “layers” in the color modelmis-align? e.g. edge detection finds edges at different locations?Together: how? May not be easy.



Color processing

Example: median filter

3 separate planes:Color may shift drastically! Let’s say we work on RGB. If R and Gdon’t change much but B changes a lot, then the color suddenlybecomes much more yellow!

Together:Each point is a vector of 3 intensity valuesWe don’t know how to order vectors!


Capturing colors with photodetectors







Sensor technology

Two dominant technology: CCD and CMOS

Charged-couple device (CCD):Most common; dominant technology; lowest costInvented at Bell Labs in 1970. Nobel prize in 2009.Optimized photodetectors→ good SNR; low nonuniformityDifficult to integrate other functions on the sensor; high powerconsumption; limited frame rate due to serial readout

Complementary metal-oxide semiconductor (CMOS):Integrated fabrication; cheapLow power consumption (∼ 1/10 of CCD)Very high frame rate can be achievedIntegrate processing with sensingNon-uniformity due to multiple levels of amplification (pixel,column, chip); fixed pattern noise and dark current



Capturing in color

Both CCD and CMOS respond to the “amount of light” (number ofphotons), within the range of visible spectrum. In other words, they donot differentiate color.

To capture color images, each pixel needs to output at least threevalues (e.g. Red, Green, Blue)We can then do color space conversion, and other color processingA few possible architectures



Dichroic mirror

R

GB

R

G

B

Red sensor

Green sensor

Blue sensor

Three separate sensorsHigh spatial resolution; little optical loss of the signal; costly



Sequential imaging

R

G

B

R

G

B

Red sensor

Green sensor

Blue sensor

Time-varying color filterThree exposures; possibility of mis-alignment



Color filter array

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

G

R

B

G

R

G

B

Red sensor

Green sensor

Blue sensor

Capture 1/3 information; interpolate with demosaicingLow cost; most common in practice


color images color image processing - home | department of ...elec4245/sp16/lec6-2x2.pdf · imaging...

Documents