digital image processingqil.uh.edu/dip/media/cosc6380/lecture_-_2.pdfperspective projection •a...

Digital Image ProcessingCOSC 6380/4393

Lecture – 2

Jan 16th, 2020

Slides from Dr. Shishir K Shah, Rajesh Rao and Frank (Qingzhong) Liu

1

Digital Image ProcessingCOSC 6380/4393

• Instructor

– Pranav Mantini

– Email: pmantini@uh.edu

– Office: PGH 550E

– Office Hours: TTh 2-3PM

• TA

2

Sara Bouhalisarabouhali4@gmail.comM 10:00-12:00Office: (PGH 223)

Kadhija Khalidikkhaldi@uh.edu

F 12:00-2:00Office: (PGH 550E)

qil.uh.edu/dip

Review: Pre-Introduction

• Example: Measure depth of the water in meters at a certain pier

• Yet another representation

3

Review: Pre-Introduction

• Example: Measure depth of the water in meters at a certain pier

• Yet another representation

• Image as a mode/format to convey information usually for human consumption

4

WHAT ARE DIGITAL IMAGES?• Images are as variable as the types of radiation that exist and

the ways in which radiation interacts with matter:

5

GENERAL IMAGE TYPES• We can distinguish between three types of imaging, which create different

types of image information:• Reflection Imaging

– Image information is surface information; how an object reflects/absorbsincident radiation• - Optical (visual, photographic, laser-based)• - Radar• - Sonar, ultrasound (non-EM)• - Electron microscopy

• Emission Imaging– Image information is internal information; how an object creates radiation

• - Thermal, infrared (FLIR) (geophysical, medical, military)• - Astronomy (stars, nebulae, etc.)• - Nuclear (particle emission, e.g., MRI)

• Absorption Imaging– Image information is internal information; how an object modifies/absorbs

radiation passing through it• - X-Rays in many applications• - Optical microscopy in laboratory applications• - Tomography (CAT, PET) in medicine• - “Vibro-Seis” in geophysical prospecting

6

Image formation

• Let’s design a method to capture reflection

– Idea 1: put a piece of film in front of an object

– Do we get a reasonable image?

7

Light Source

Image formation

• Let’s design a method to capture reflection

– Idea 1: put a piece of film in front of an object

– Do we get a reasonable image?

8

Pinhole camera

• Add a barrier to block off most of the rays

– This reduces blurring

– The opening is known as the aperture

– How does this transform the image?

9

Camera Obscura

• The first camera

– Known to Aristotle

– Analyzed by Ibnal-Haytham(Alhazen, 965-1039 AD) in Iraq

10

Shrinking the aperture

11

Shrinking the aperture

• Why not make the aperture as small as possible?

12

Shrinking the aperture

13

Shrinking the aperture

• Less light gets through

• Diffraction effects...

14

Pinhole camera

• Add a barrier to block off most of the rays

– This reduces blurring

– The opening is known as the aperture

– How does this transform the image?

15

Lenses

16

Lenses

• A lens focuses parallel rays onto a single focal point

– focal point at a distance f beyond the plane of the lens

• f is a function of the shape and index of refraction of the lens

– Aperture of diameter D restricts the range of rays

• aperture may be on either side of the lens

– Lenses are typically spherical (easier to produce)

17

Adding a lens

• A lens focuses light onto the film

– There is a specific distance at which objects are “in focus”• other points project to a “circle of confusion” in the image

– Changing the shape of the lens changes this distance

18

OPTICS OF THE EYE

19

OPTICS OF THE EYE

• The important optical structures in the eye are :– the cornea (clear front surface of the eye),

– the iris (a sphincter muscle that determines the size of the pupil) and

– the lens (a flexible lens that can change shape to adjust for different object distances)

• The human optical system is approximately radially symmetric about a line running through the center of the cornea, pupil and lens – optic axis

20

OPTICS OF THE EYE

• The optic axis is shifted approximately 5 degrees towards the temple

• Cornea is the main refractive surface of the eye as the index of refraction between the air and cornea is much bigger than between any other adjoining media within the eye

• Iris determines the size of the pupil. (Pupil is pigmented and determines the color of the eyes)

• Pupil serves as an aperture in the eye’s optical system

21

OPTICS OF THE EYE

• Lens allows for variation in optical power by changing the curvature. This allows for focus of the retinal image while varying distance to object

• Greatest curvature for close objects; the least for distant objects

• Aqueous humor and vitreous humor provide nutrients to non-vascular structures within the eye and maintain the shape of the eye

• Transmittance of light varies with wavelengths and with age – in young eye, the cornea absorbs most of radiation below 300nm and the lens filters out wavelength below 380nm

22

Adding a lens

• A lens focuses light onto the film

– There is a specific distance at which objects are “in focus”• other points project to a “circle of confusion“ in the image

– Changing the shape of the lens changes this distance

23

OPTICS OF THE EYE

• Retina receives wavelengths between 380-950nm

• As the lens yellows, transmittance of all wavelengths decreases and optical density of the eye increases

• In the visible range, 380-770nm, eye transmits more red light (longer wavelength) than blue light. 70-85% of white light reaches the retina

24

PHOTORECEPTORS• Photoreceptors forms one of several layers of neurons in

the retina, a thin layer of tissue that lines the inside of the eyeball

• Photoreceptors come in two basic shapes; the cylindrical-shaped receptors are known as rods, and the conical-shaped receptors are known as cones

• Rods and cones play very different functional roles in vision; specifically rods are responsible for encoding images under low light conditions and cones under high light conditions

25

PHOTORECEPTORS

• Rods are 1-2 microns in diameter; the cones are 2-3 microns in diameter in the fovea, but increase in diameter away from the fovea (No rods in the fovea)

• Cones are densely packed in the fovea and quickly decrease in density as a function of eccentricity

• Rods increase in density out to approximately 20 degree eccentricity, beyond which their density begins to decline

26

http://www.indiana.edu/~p1013447/dictionary/rodcone.htm

COLOR VISION

• Human visual system perceives the range of light wave frequencies as a smoothly varying rainbow of colors

• This range of light frequencies is the visual spectrum

• The eye’s peripheral vision system only supports low resolution imaging but offers an excellent ability to detect movement through a wide range of illumination levels

27

COLOR VISION

• Peripheral vision provides very little color information

• The eye’s high resolution color vision system has a much narrower angle of coverage

• This system can flexibly adapt to widely varying illumination colors and levels

• It evolved primarily as a daylight system and ceases to work well at very low illumination levels

• Rods and cones have different spectral sensitivities and different absolute sensitivities to light, so visual response is not the same over the retina

28

COLOR VISION

• Our eyes have three sets of sensors with peak sensitivities at light frequencies that we call red (580 nm), green (540 nm) and blue (450 nm)

• Our perception of which color we are seeing is determined by which combination of sensors are excited and by how much

• The spectral sensitivity of the typical human visual system is:

29

Scotopic Vision

• The human visual system has much greater sensitivity in low ambient illumination. The spectral sensitivity of the rods is:

30

FACTORS AFFECTING VISIBILITY

• Contrast – relationship between the luminance of an object and the luminance of the background. These luminances can be affected by location of light sources and room reflectance (glare problems)

• Size – larger the object, the easier it is to see. However, it is the size of the image on the retina, not the size of the object per se that is important. Therefore we bring smaller objects closer to the eye to see details

• Time – there is a time lag in the photochemical processes of the retina, therefore the time available for viewing is important. When objects are briefly viewed we need bright light, when lots of time is available even small details can be seen

31

HUMAN VISUAL SYSTEMS

• Much of digital image processing is motivated by the capabilities of the human visual system

• Fully 2/3 of all sensory neurons in the human brain come from the two eyes

• Extremely large fraction of the cerebral cortex is devoted to basic visual processing

32

STEPS IN PROCESS OF VISION

• Can divide the process of vision in five distinct steps

– Formation of external visual stimuli (physics of light)

– Imaging process (optics of the eye)

– Visual sensors (photoreceptors)

– Low-level image processing (retinal mechanisms)

– Higher-level processes (central brain mechanisms)

• Lowest levels of visual processing such as image formation, sampling, and spatial filtering are relatively well understood

• Higher levels are less well understood

• In some sense, it is the higher levels of processing that most concerns artificial vision research

33

VISUAL TASKS AND PERFORMANCE

• Visual performance refers to the ability of a vision system to perform specific tasks – identify a defective part, predict trajectory of a baseball

• Visual tasks that humans perform include:– Simple detection and discrimination

– Object and/or material identification

– Navigation through environment

– Prediction of motion trajectories

– Estimation of physical dimensions

– Object manipulation

34

MULLER-LYER ILLUSION

• Vision is a learned sense – relies on feedback for development and learning or tuning-up of the visual processing mechanisms

• Sometimes, providing feedback is inappropriate

35

Find the black dot

36

What is this?

37

Which lines are straight?

38

OPTICAL IMAGING GEOMETRY

• We will quantify how the geometry of a 3-D scene projects to the geometry of the image intensities:

object

lens

image

sensingplate,

emulsion, etc

light source(point source)

emitted rays

reflectedrays

focallength

39

PERSPECTIVE PROJECTION

• A reduction of dimensionality is projection - in this case perspective projection

• A precise geometric relationship between space (3-D) coordinates and image (2-D) coordinates exists under perspective projection

• We will require some coordinate systemsReal-World Coordinates

– (X, Y, Z) denote points in 3-D space– The origin (X, Y, Z) = (0, 0, 0) is taken to be the lens

centerImage Coordinates

– (x, y) denote points in the 2-D image– The x - y plane is chosen parallel to the X - Y plane– The optical axis passes through both origins

40

PIN-HOLE PROJECTION GEOMETRYZ

Y

X

f = focal length

image plane

Idealized "Pinhole"

Camera Model

lens center

(X, Y, Z) = (0, 0, 0)

41

UPRIGHT PROJECTION GEOMETRY

X

Y

Z

lens center

f = focal lengthimage plane

Upright Projection Model

x

y

(X, Y, Z) = (0, 0, 0)

42

PROJECTION• This diagram shows all of the coordinate axes and

labels

X

Y

Z

f = focal lengthimage plane x

y

(0, 0, 0)

(X, Y, Z) = (A, B, C)

A

B

C

(x, y) = (a, b)

43

PROJECTION (contd.)

• This equivalent simplified diagram shows only the relevant data relating (X, Y, Z) = (A, B, C) to its projection (x, y) = (a, b):

a

b

A

B

C

f44

SIMILAR TRIANGLES

• Triangles are similar if their corresponding angles are equal:

b

g

a ab

g

45

SIMILAR TRIANGLES

• Similar Triangles Theorem - Similar triangles have their side lengths in the same proportions.

b

g

a ab

g

D

E

F

d

e

f

D

E=

d

eE

F=

e

fF

D=

f

d46

SOLVING PERSPECTIVE PROJECTION• Using similar triangles we can solve for the relationship

between 3-D coordinates in space and 2-D image coordinates• Redraw the imaging geometry once more, this time making

apparent two pairs of similar triangles:

b

f

B

C

a

f

A

C

a

b

A

B

C

f

47

SOLVING PERSPECTIVE PROJECTION

• By the Similar Triangles Theorem, we conclude that

and

OR

(a, b) = · (A, B) = (fA/C, fB/C)

a

f=

A

Cb

f=

B

C

f

C

48

PERSPECTIVE PROJECTION EQUATION

• Thus the following relationship holds between 3-D space coordinates (X, Y, Z) and 2-D image coordinates (x, y) :

(x, y) = · (X, Y)

where f = focal length.

• The ratio f/Z is the magnification factor, which varies with the range Z from the lens center to the object plane.

f

Z

49

EXAMPLE

• There is a man standing 10 meters (m) in front of you

• He is 2 m tall

• The focal length of your eye is about 17 mm

• Question: What is the height H of his image on your retina?

H2 m

10 m 17 mm

50

ANSWER

2 m

10 m=

H

17 mm

• By similar triangles,

H = 3.4 mm

51