csc 8610 & 5930 multimedia technology lecture 2 digital image representation
TRANSCRIPT
CSC 8610 & 5930Multimedia Technology
Lecture 2
Digital Image Representation
2Chapter 1
Today in class (1/30)
6:15 Recap, Reminders 6:25 Lecture – Digital Image Representation 7:30 Break 7:40 Workshop 1 discussion & presentations 8:20 Workshop 2 – Digital Image Exploration 9:00 Wrap
3
ANALOG VS. DIGITAL
4
Analog InformationExamples: time, weight temperature line length width and length of a sheet of paper sound loudness light brightness color saturation and hue Continuous information An infinite number of divisions exist between any two
measurements
5
What is the length of the pencil?
6
What is the length of the pencil?
7
What is the temperature?
8
Analog Thermometer vs. Digital Thermometer
digital thermometer
analog thermometer
9
Analog vs. Digital Analog information
– continuous– made up of infinite number of data points
Digital data– discrete
10
Discrete DataExamples: number of persons
There is no in-between one person and two persons.
choices in multiple-choice questionsThere is no in-between choice A and choice B.
11
Analog vs. DigitalTherometers and Scales What are the limitations of these analog and digital
devices?
What are the advantages of these analog and digital devices?
12
Analog vs. Digital Sight and sound we peceive in our natural world are
analog information--continuous and infinite number of points between any two points.
Computers handle discrete digital data. In addition, the amount of data has to be finite.
Sight and sound must be converted into finite discrete digital data in order for the computer to handle.
13
MONITORING A PUPPY'S WEIGHT IN HIS FIRST YEAR
14
Suppose you use an analog scale to weigh the puppy
15
Now, what is the weight you would note down for this puppy?
16
See the problem in picking a number to represent an analog measurement?
17
Number of Decimal Places In recording the weight, you must decide the number of
decimal places to use.
This determines the precision or exactness of the measurement.
How many will give an exact measurement? How many is enough? How many is too many?
18
Using More Decimal Places• Pros :
– increase the precision in general(But how many is meaningful?)
– Will allow finer distinction between values(will explain in the next slide)
• Cons:– Require more paper and paperwork.– Take longer to read through and interpret the numbers.
19
Distinction Between ValuesWith one decimal place:
– You can have 10 different values between say 2 and 3:2.1, 2.2, ...3.0
• You can distinct between 2.5 and 2.8.• But 2.5 and 2.8 would have been rounded to the same
value of 3 the values do not allow decimal places.
20
Distinction Between ValuesSuppose the allowable weight read outs are
these 10 levels:
0, 5, 10, 15, 20, 25, 30, 35, 40, 45
Then,
2 pounds: rounded to 0 pound
3 pounds: rounded to 5 pounds
The difference between 2 and 3 pounds is 1 pound.
But now, it become 5 pounds if we use these levels.
21
How many measurements to make?
A. once a yearB. once a monthC. every two weeksD. every weekE. every dayF. every hourG. every minuteH. every second
Now, how often would you weigh the puppy to produce a "good" monitoring of his weight over his first year?
22
What are your considerations in deciding how often to weigh the puppy?
23
Considerations• What happens if you weigh the puppy not often enough?
• What happens if you weigh the puppy too often?
• Is there one right answer?
• Will you use the same weighing schedule to monitor the weight of an adult dog?
24
DIGITIZATION:SAMPLING AND QUANTIZATION
Back to the Computer
25
Digitization To convert analog information into digital data that
computers can handle
2-step process:1. sampling
2. quantization
26
Sampling Analogous to weighing and recording the puppy's weight
During the sampling step, you need to set a sampling rate.
Sampling rate: how often you take a data
27
Suppose this is the true timeline of the puppy's first-year growth
28
Suppose you weigh the puppy once a month
29
You get these data points
30
You then interpolate the points
31
You would miss the changes that occur during the first month
32
But the rest matches with the true growth pretty well
33
What about weighing the puppy once a week?
34
You get these data points
35
The data is catching the changes occurring in the first month better
36
But is it exactly?
37
Now for the rest of the year, the data points seem too many
38
Sampling Rate
Weighing Puppy Scenario
Digitization
high(i.e. taking data often)
Pros: can catch more weight changes
Cons: produce more paperworkand thus take longer to read through all the data
Pros: can capture details (e.g. some changes of color within a small region in a picture or amplitude changes in sound within a short period of time)
Cons: produce larger file and thus take longer to process
low(i.e. taking data infrequently)
Pros: less paperwork and thus take shorter time to read through all the data
Cons: may miss weight changes
Pros: produce smaller file and thus take shorter time to process
Cons: may miss details (e.g. color changes in a picture or changes in sound)
39
Quantization• Analogous to rounding the weight to fix number of digits
in the weighing puppy scenario
• During the quantization step, you need to set bit depth.
• Bit depth refers to the number of allowable levels you map (or round) the values to.
40
Example: 10 levels of weightFor 10 discrete levels, you may have the 10 allowable
values as 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, and 2.9 0, 5, 10, 15, 20, 25, 30, 35, 40, 45 2, 4, 6, 8, 10, 12, 14, 16, 18, 20 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 ... and so forth
41
Suppose you choose 2.0, 2.1, ..., 2.9For 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9 Any weight data below 2.0 will be recorded as 2.0.
Any weight data higher than 2.9 will be capped at 2.9.
It works well if the puppy's weight falls in this range. But it does not seem to be the case.
42
Suppose you choose 0, 5,..., 45For 0, 5, 10, 15, 20, 25, 30, 35, 40, 45 A weight of 2 pounds would be rounded to 0 and a weight of
3 pounds to 5.
Cons: For example, the difference between 2 and 3 pounds is altered after they are mapped to the allowable value on this 10-level scale. The difference becomes 5 pounds not 1 pound.
Pros:Wider range.
Again, it works well if the puppy's weight falls in this range.
43
Well, what if we choose this:
2.0, 2.1, 2.2, ..., 44.8, 44.9, 45.0
44
Suppose you choose 2.0, 2.1,...,44.9, 45.0
You have increased the number of levels from 10 to 431.
Pros:– Increase precision compared to using
0, 5, 10, 15, 20, 25, 30, 35, 40, 45– Increase range compared to using
2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6, 2.7, 2.8, 2.9
45
Increase Number of Allowable Levels There does not seem to be any cons in the weighing
puppy scenario.
However, for digitization, increasing the number of allowable levels (i.e. increasing bit depth) will increase the file size.
46
Sampling and QuantizationDigitizing media involves sampling and quantization
regardless of the type of media:– images– video– audio
47
Overview of how sampling rate and bit depth affect digital media file quality
Sampling rate is related to:
Bit depth is related to:
digital images image resolution, or number of pixels
number of allowable colors in an image
digital video number of pixels in the video, frame rate
number of allowable colors
digital audio sampling rate of the audio (it limits how high the pitch of the audio can be captured)
number of allowable levels of amplitude
Details will be covered in chapters for each media type.
48
BITMAPS & DIGITIZATION
49
Pegboard Analogy
A 10 holes 10 holes pegboard
50
Pegboard Analogy
Suppose you want to copy this music note graphic on the pegboard.
51
Pegboard Analogy
Place one peg.
52
Pegboard Analogy
2 pegs.
53
Pegboard Analogy
3 pegs.
54
Pegboard Analogy
Suppose we only put peg in a hole if more than half of its area is covered by the musical note graphic.
55
Pegboard Analogy
Now remove the musical note overlay.
56
Pegboard Analogy
Details are lost because the gridis too coarse for this musical note.
57
How would you improve the details of the musical notes on the pegboard?
58
Using a pegboard with more holes
59
Using a pegboard with more holes
Now it looks closer to the original musical note.
60
Pixels Each peg hole on the pegboard is a sample point. The sample points are discrete.
In digital imaging, each of these discrete sample points is called a picture element, or pixel for short.
61
Pixel Dimensions Refer to an image’s width and height in pixels In the pegboard analogy, the dimension of this pegboard
would be 10 holes 10 holes.
62
Sampling Step
63
Let's look at the sampling step of digitizing a natural scene as if we are taking a digital photo of a natural scene.
64
A natural scene
Look up and let your eyes fall on the scene in front of you. Draw an
imaginary rectangle around what you see. This is your “viewfinder.”
Imagine that you are going to capture this view on a pegboard.
65
Sample into a grid of 25 20 discrete samplesSuppose you are going to sample the scene you see in the "viewfinder" into a pegboard with 25 20 holes.
66
One color for each peg hole.
Each peg hole takes only one peg. Suppose each peg has one solid color. Suppose the color of each of these discrete samples is determined by averaging the corresponding area.
67
This sampled image looks blocky. Details are lost because the grid is too coarse for this image.
68
For 25 20 sample points, it means you get a digitized image of 25 20 pixels.
69
Let's try a different grid size.
70
Sample into a grid of 100 80 discrete samples
Suppose you are going to sample the scene you see in the "viewfinder" into a pegboard with 100 80 holes.
71
Again, one color for each peg hole.
72
For 100 80 sample points, it means you get a digitized image of 100 80 pixels.
73
Sampling Rate Refers to how frequent you take a sample For an image, sampling frequency refers to how close
neighboring samples are in a 2-D image plane.
74
Sampling Rate For example, when we change the grid from 25 20 to
100 80, we say that we increase the sampling rate.– You are sampling more frequently within the same spatial distance.
75
Resolution In digital imaging, increasing the sampling rate is
equivalent to increasing the image resolution.
76
Consequences of Higher Resolution
With higher resolution, You have more sample points (pixels) to represent the
same scene, i.e., the pixel dimensions of the captured image are increased.
The file size of the digitized image is larger. You gain more detail from the original scene.
77
Resolution of Digital Photos Note that 25 20 and 100 80 pixels are by no means
realistic pixel dimensions in digital photography.
They are only for illustration purposes here. Most digital cameras can capture images in the range of thousand pixels in each dimension—for example, 3000 pixels 2000 pixels.
78
A Pixel is not a Square Block A pixel is a sample point. It does not really have a physical dimension associated
with it.
79
A Pixel is not a Square Block When you zoom in on a digital image in an image
editing program, you often see the pixels represented as little square blocks.
This is simply an on-screen representation of a sample point of an digitized image.
80
COLORS
81
Problems A natural image is colored in continuous tones, and thus
it theoretically has an infinite number of colors.
The discrete and finite language of the computer restricts the reproduction of an infinite number of colors and shades.
82
Quantization Step To encode an infinite number of colors and shades with
a finite list.
Quantizing the sampled image involves mapping the color of each pixel to a discrete and precise value.
83
Quantization Step First, you need to consider how many possible colors
you want to use in the image.
To illustrate this process, let’s return to the example of the 100 80 sampled image.
84
The sampled 100 80 image
85
Say, we want to map the color of each sample points into one of these four colors:
Mapping colors
86
Quantized with 4 Colors
87
Quantized with 8 Colors
88
Consequences of Quantization Reduce the number of allowed colors in the image.
When we reduce the colors, different colors from the original may bemapped to the same color on the palette. This causes the loss of the image fidelity and details.
The details that rely on the subtle color differences are lost during quantization.
89
The area outlined in red is made up of many different green colors.
The same area in the 4-color image now has only one color.
90
Bit Depth The number of colors used for quantization is related to
the color depth or bit depth of the digital image.
A bit depth of n allows 2n different colors. Examples:– A 2-bit digital image allows 22 (i.e., 4) colors in the image.– An 8-bit digital image allows 28 (i.e., 256) colors in the image.
The most common bit depth is 24. A 24-bit image allows 224 (i.e., 16,777,216) colors.
91
Increasing colors
It depends, and in most cases, can be yes.
The number of colors or the bit depth is not the only determining factor for image fidelity in quantizing an image.
The choice of colors for the quantization also plays an important role in the reproduction of an image.
Will increasing the number of colors in the palette improve the image fidelity?
92
Quantized with 8 Different Colors
93
Effect of Bit Depth on File Size Higher bit depth means more bits to represent a color.
Thus, an image with a higher bit depth has a larger file size than the same image with a lower bit depth.
94
VECTOR GRAPHICS
95
Vector Graphics Examples: graphics created in
– Adobe Flash– Adobe Illustrator
96
Vector Graphics Characteristics Generated mathematically,
i.e. instructionsnot pixel-based
Resolution independent
97
Vector Graphics Instruction Example A simple postscript example:
%!
newpath
200 200 moveto
300 200 lineto
stroke
showpage
98
Bitmap Images vs. Vector Graphics
An analogy:Driving Directions:
A visual map vs. a written instruction
Bitmap ~~> map
Vector Graphics ~~> a written instruction
99
Bitmap Images vs. Vector GraphicsAn analogy:
Driving Directions:A visual Triptik map vs. a written instruction
Which one takes up more storage space?Triptik map ~~> bitmap
Which one takes you more time to translate the direction into a mental image?written instruction ~~> vector graphics (take more computation to display on computer because it is mathematically generated)
100
Bitmap Images vs. Vector Graphics
An analogy:
Driving Directions:
A visual Triptik map vs. a written instruction
If you are going to draw out a map based on a written instruction, how big can you draw?
as big as you physically can ~~> the instruction is resolution independent
101
Bitmap Images vs. Vector Graphics
1111111111111111111111011111101111110111111011111101111111111111
%!newpath2 1 moveto6 5 linetostrokeshowpage
vector graphic bitmap image
102
Vector Graphics
%!newpath2 1 moveto6 5 linetostrokeshowpage
vector graphic
The unit is arbituary, i.e. when you print out an image, you may set one unit as an inch or a foot.
This means vector graphic is resolution independent.
103
Printing Bitmap Images
bitmap image
print bigger print smaller
have the same amount of data, i.e. same level of details
104
Printing Vector Graphics
vector graphics
print bigger print smaller
105
Bitmap Images vs. Vector Graphics Example
(a) Vector graphics:A line defined by two end points.
(b) Vector graphics: The same line is stroked with a certain width.
(c) & (d) The line is converted to a bitmap.
106
Curve Drawing in Vector Graphics Programs
defined by a set of points;we call them anchor points
the direction handles or tangent handles of a point controls the tangent at that point on the curve
107
Rasterizing Vector Graphics Raster means convert vector graphics into pixel-based
images.
Most vector graphics programs let you rasterize vector graphics.
Need to specify a resolution for rasterizing, that is, how coarse or how fine the sampling.
108
ALIASINGBlurriness, Blockiness, Moire Patterns, Jagged Edges
109
Aliasing
The rasterized image will appear jagged.
This jagged effect is a form of aliasing caused by undersampling (which means insufficient sampling rate.) Recall the musical note on a pegboard example.
Original vector graphics
Rastered vector graphics without anti-aliasing
110
Anti-aliasing Techniques
To soften the jaggedness by coloring the pixels with intermediary shades in the areas where the sharp color changes occur.
Original vector graphics
Rastered vector graphics without anti-aliasing
Rastered vector graphics with anti-aliasing
111
FREQUENCY IN DIGITAL IMAGES
112
Function over spatial domain Any image can be represented as a function Frequency – the rate at which color values change over
the space the image occupies
113
DISCRETE COSINE TRANSFORM
114
Discrete Cosine Transform Transforming the image data (function) from the spatial
domain to the frequency domain We won’t explore the mathematics Awareness of functions and transform is goal
115
COLOR MODELS
116
Color Models Used to describe colors numerically, usually in terms of
varying amounts of primary colors.
Common color models:– RGB– CMYK– HSB– CIE and their variants.
117
RGB Color Model Primary colors:
– red– green– blue
Additive Color System
118
Additive Color System
119
Additive Color System of RGB Full intensities of red + green + blue = white
Full intensities of red + green = yellow
Full intensities of green + blue = cyan
Full intensities of red + blue = magenta
Zero intensities of red, green, and blue = black
Same intensities of red, green, and blue = some kind of gray
120
Color Monitors
From a standard CRT monitor screen
121
Color Monitors
From a SONY Trinitron monitor screen
122
Color Monitors
From a LCD screen
123
RGB Color Model
depicted graphically as a cube defined by three axes in 3-D space
The maximum value on eachaxis is normalized to 1.
124
RGB Color Model
x-axis: red values y-axis: green values z-axis: blue values
125
RGB Color Model
The coordinates within the color cube represent the relative intensities of red, green, and blue colors.
126
RGB Color Model
origin (0,0,0): black
127
RGB Color Model
(1,0,0): full intensity of red
128
RGB Color Model
(0,1,0): full intensity of green
129
RGB Color Model
(0,0,1): full intensity of blue
130
RGB Color Model
(1,1,1): white
131
RGB Color Model
(1,1,0): full red + full green = full yellow
132
RGB Color Model
(1,0,1): full red + full blue = full magenta
133
RGB Color Model
(0,1,1): full green+ full blue = full cyan
134
Color Picker
So where is the color whose RGB values are (150, 200, 100) in the RGB cube?
135
Correlating RGB Color Cube with Color Picker
This is a 2-D slice containing all the colors with red = 150.
136
Correlating RGB Color Cube with Color Picker
This is a 2-D slice containing all the colors with green = 200.
137
Correlating RGB Color Cube with Color Picker
This is a 2-D slice containing all the colors with blue = 100.
138
Correlating RGB Color Cube with Color Picker
The color (150, 200, 100) is located in the 3-D space of the RGB color cube.It is at the intersection of the three 2-D slices.
139
CMYK Color Model Primary colors:
– cyan– magenta– yellow– black
Subtractive Color System
140
Subtractive Color System of CMY
141
Subtractive Color System of CMY Full intensities of cyan + magenta + yellow = black
(theoretically, but in practice with inks, it is not full black)
Full intensities of cyan + magenta = blue
Full intensities of cyan + yellow = green
Full intensities of magenta + yellow = red
Zero intensities of cyan, magenta, and yellow = white
142
HSB Color Model Hue:
– basic color– 0o to 360o : the location on a color wheel– in the order of colors in a rainbow
Saturation:– purity of the color– how far away from the neutral gray of the same brightness
Brightness
143
HSB Color Model
144
HSB Color Model
145
HSB Color Model
A slice of the color wheel from the HSB model
146
HSB Color Model
Matches well with the way humans intuitively think about colors
For example, how would you describe this color?
• Would you think of it in terms of how much red, green, and blue?
• Would you first think of it as a yellowish color and then figure out the lightness and how washed out the color is?
147
Problems with RGB and CMYK Color Space
Do not encompass all the colors human can see
148
CIE XYZ Color Model Primaries:
– X– Y– Z
Primaries are not physical colors
Its color space encompasses all the colors human can see.
Normally not used in digital image editing because monitors and printers cannot reproduce all the colors in the CIE XYZ color space anyway.
149
Color Gamuts
Refers to the range of colors of a specific system or device can produce or capture
150
Color Gamuts
a) Colors that human can see
b) RGB color gamut of typical CRT monitors
c) CMYK color gamut of typical inkjet printers
151
Color Gamuts
Monitors and inkjet printers cannot reproduce all the colors that human can see
Some of the colors that monitors can reproduce cannot be reproduced by inkjet printers. Most of these colors lie at the corners of the color gamut of the monitor, which means these are highly saturated colors.
152
Identifying Out-of-Gamut Colors in Images
Out-of-gamut colors will not be printed correctly.
In digital image editing programs such as Adobe Photoshop, you can tell whether a color is out of gamut based on your CMYK setting.
Click on the icon to use the closest color in gamut for printing.
153
Identifying Out-of-Gamut Colors in Images
Out-of-gamut colors will not be printed correctly.
In digital image editing programs such as Adobe Photoshop, you can tell whether a color is out of gamut based on your CMYK setting.
154
Identifying Out-of-Gamut Colors in Images
155
Identifying Out-of-Gamut Colors in Images
Click on the icon to replace the out-of-gamut color with the closest color in gamut for printing.
156
Identifying Out-of-Gamut Colors in Images
The out-of-gamut color is replaced with the closest color in gamut for printing.
157
Difficulties in Reproducing Colors in Digital Images
Digital devices cannot produce all of the colors visible to human
Difficulties exist in reproducing color across devices– different devices have different color gamuts– additive color system for screen display vs. subtractive color system for
printing
158
Indexed Color
159
Indexed Color