image processing and computer vision: 91. image and video coding compressing data to a smaller...
TRANSCRIPT
Image Processing and Computer Vision: 9 1
Image and Video Coding
Compressing data to a smaller volume without losing (too much) information
Image Processing and Computer Vision: 9 3
Why Code Data? To reduce storage volume To reduce transmission time
One colour image 760 by 580 pixels 3 channels, each 8 bits 1.3 Mbyte
Video data Same resolution 25 frames per second 33 Mbyte/second
Image Processing and Computer Vision: 9 4
Redundancy Spatial
Correlation between adjacent pixels Chromatic
Correlation between colour channels Temporal
Correlation between adjacent frames Perceptual
Unnoticed losses
Image Processing and Computer Vision: 9 5
Example
Image Processing and Computer Vision: 9 6
Redundancy - Consequences
Data exceeds information More data than content justifies
Can lose data without losing information
Image Processing and Computer Vision: 9 7
Compression Ratio
image compressed of volume
image eduncompress of volumeC
Image Processing and Computer Vision: 9 8
Lossy vs Lossless Compression Can lose data without losing
information Lossily compressed images can
look similar to the original Lossy compression has greater C
Image Processing and Computer Vision: 9 9
Quality of Decoded Images Measure differences between
Original Coded/decoded images
Options Maximum difference Average difference Subjective scales
Image Processing and Computer Vision: 9 10
Example
Image Processing and Computer Vision: 9 11
Subjective Quality Measurement Scales - Example
0 Unusable1 Annoying degradation2 Adequate images3 Barely perceptible degradation4 No observable degradation
Image Processing and Computer Vision: 9 12
Difference coding Adjacent pixels are similar
Difference is small
Uncompressed:
Compressed:
,,,,,43210 sssss
,,,,,3 42 3210 10 sssssssss
Image Processing and Computer Vision: 9 13
Coding Assign 4 bits to a difference Can code –7, … , +8
Overflow? Use –7 and +8 to show larger differences Code –6, … , +7 directly Use overflow codes to indicate shift of
codes
Image Processing and Computer Vision: 9 14
Image Processing and Computer Vision: 9 15
Predictive Coding If signals are well sampled
Adjacent samples are correlated They have similar values Differences are small Can guess next sample from value of
current
Image Processing and Computer Vision: 9 16
Constants are correlation coefficient and mean grey value
Difference between real and predicted values are smaller
Code as for difference coding
II ttˆ 1
Image Processing and Computer Vision: 9 17
Run Length Coding Replace runs of equal brightness values
by (length of run, value)
1 2 2 3 3 4 4 4 5 6 5(1 1) (2 2) (2 3) (3 4) (1 5) (1 6) (1 5)
More use when few brightness values e.g. fax
Image Processing and Computer Vision: 9 18
Huffman Coding Uses variable length codes Most frequently occurring grey
value has shortest code Least frequently occurring values
have longest codes
Image Processing and Computer Vision: 9 19
ExampleSymbol Probability 21 43
ABCDEF
0.40.30.10.10.060.04
0.40.30.10.10.1
0.40.30.20.1
0.40.30.3
0.60.4
Image Processing and Computer Vision: 9 20
SymbolProbability 21 43
ABCDEF
0.4 10.3 000.1 0110.1 01000.06 010100.04 01011
0.4 10.3 000.1 0110.1 01000.1 0101
0.4 10.3 000.2 0100.1 011
0.4 10.3 000.3 01
0.6 00.4 1
Image Processing and Computer Vision: 9 21
GIF Applicable to images with 256
colours Replace sequences of bytes with
shorter codes Most common sequences use
shortest codes
Image Processing and Computer Vision: 9 22
Wavelet coders Wavelet transform organises
image content efficiently Can easily select data to be
discarded
Image Processing and Computer Vision: 9 23
Wavelet coders
Image Processing and Computer Vision: 9 24
JPEG standard A subcommittee of ISO Optimised for a range of image
subject matter Compression rates can be defined Quality inversely proportional to C
Image Processing and Computer Vision: 9 25
Block Transform Codingoriginal image
decompose
transform
quantise
entropy code
Sequence of 8 by 8 blocks - different planes treated separately (RGB, YUV etc.)
Transformed blocks reduce redundancy and concentrate signal energy into a few coefficients discrete cosine transformation (DCT)
Blocks with discarded information - goal is to smooth picture and discard information that will not be missed, e.g. high frequencies
Image Processing and Computer Vision: 9 26
Block Transform Encoding
DCT
Zig-zag
run length code entropy
code
quantise
010111000111…..
Image Processing and Computer Vision: 9 27
Block Encoding
139 144 149 153144 151 153 156150 155 160 163159 161 162 160
1260 -1 -12 -5 -23 -17 -6 -3 -11 -9 -2 2 -7 -2 0 1
DCT
Zig-zag
run length code
Huffman code
quantise
10011011100011….
79 0 -1 0-2 -1 0 0-1 -1 0 0 0 0 0 0
79 0 -2 -1 -1 -1 0 0 -1 0 0 0 0 0 0 0
0 791 -20 -10 -10 -12 -10 0
Original image
Image Processing and Computer Vision: 9 28
Block Transform Decoding
DCT
Zig-zag
run length code entropy
code
quantise
010111000111…..
Image Processing and Computer Vision: 9 29
Result of Coding and Decoding
139 144 149 153144 151 153 156150 155 160 163159 161 162 160
Original block
144 146 149 152148 150 152 154155 156 157 158160 161 161 162
Reconstructed block
-5 -2 0 1-4 1 1 2-5 -1 3 5-1 0 1 -2
errors
Image Processing and Computer Vision: 9 30
Discrete Cosine Transform
F[u,v] =4C(u)C(v)
n2
n-1
j=0
n-1
k=0
f(j,k) cos(2j+1) u
2ncos
(2k+1) v 2n
with
C(w) =
12
For w = 0
1 otherwise
Image Processing and Computer Vision: 9 31
Discussion Where are lossy steps?
Quatisation and subsampling before coding
How is quantisation matrix chosen? Its predefined by the standard after
much experimentation
Image Processing and Computer Vision: 9 32
Video coding Many specific standards
AVS, MOV, QT, … One universal standard
MPEG
Image Processing and Computer Vision: 9 33
MPEG Standards Standard specifies audio, video
and system layers MPEG-1: low data rates, poor
quality: VHS quality at 1.5Mbits-1
MPEG-2: high quality hence high data rates: studio quality, 15Mbits-1
MPEG-4: low data rates, small images, 64 kbits-1
Image Processing and Computer Vision: 9 34
MPEG-1 Audio and video designed to work
at CD ROM speeds: 1.5Mbits-1
Video 1.150Mbits-1
Audio 0.256Mbits-1
System 0.094Mbits-1
Image Processing and Computer Vision: 9 35
MPEG-2 Released in 1994 Aimed at digital TV, ATM. Additions for
Interlaced video Scalable video coding Graceful degradation with noise
Implementation of full standard impractical Varying profiles/levels of conformity
Image Processing and Computer Vision: 9 36
MPEG-4 Coding specifically for multimedia
objects
Image Processing and Computer Vision: 9 37
Coding Algorithms Frame sequence Motion compensation Frame coding
Image Processing and Computer Vision: 9 38
Frame Sequence I frames (Intraframes)
Coded independently of any other frame P frames (Predicted frames)
Derived from previous I frame by motion prediction
B frames (Bidirectionally interpolated) Interpolate motion compensated blocks
between I and P frames
Image Processing and Computer Vision: 9 39
Motion Compensation Image is divided into macroblocks
(16 x 16 pixels) Matching macroblocks are found
by minimising differences Code differences and macroblock
displacement
Image Processing and Computer Vision: 9 40
Frame Coding Use JPEG algorithms
Image Processing and Computer Vision: 9 41
Summary Why code data? Redundancy Assessment of compression Lossy vs. lossless compression Algorithms
JPEG, MPEG
Image Processing and Computer Vision: 9 42
But what … is it good for?Engineer at the Advanced Computing
Systems Division of IBM, commenting on the microchip in 1968