quantization and error last updated on june 15, 2010 doug young suh [email protected] 8/30/2015
TRANSCRIPT
amount of information = degree of surprise
Entropy and average code length
Information source and codingMemoryless source : no correlation
04/19/23 Media Lab. Kyung Hee University 2
Entropy and compression
))(()(][log)( 2 xIaverageXHbitspxI
Red blue yellow yellow red black red
00011010001100
bitslXH 2)(
byteslbitsXH 42)(
∙∙∙∙∙
∙∙∙
∙∙∙
Dice vs. coin
Effects of quantizationData compression Information loss, but not all
04/19/23 Media Lab. Kyung Hee University 3
Fine-to-coarse Quantization
1/6
1 2 3 4 5 6
{1,2,3} head
{4,5,6} tail
1/2
H T
3 5 2 1 5 4 ∙∙∙ H T H H T T ∙∙∙ quantization
5849.2)( XH 1)( XH
analog-to-digit-al quantization In order to cook in binary computers digital TV, digital comm., digital control…
fine-to-coarse digital quantization
04/19/23 Media Lab. Kyung Hee University 4
Quantization
ADC
Infinite numbers finite numbers
Digital Selectable accuracy : scale for human vs.
gold [dynamic range, required accuracy, pdf]
open questions1) Weights of soldiers are ranged from 50 kg to 100
kg, while that of new born baby is less than 5kg.2) Voice signal of mobile phones is quantized in 8bits,
while CD quality audio is quantized in 16bits. Why is 8bits enough for voice?
04/19/23 Media Lab. Kyung Hee University 5
Quantization
Quantization/de-quantization
04/19/23 Media Lab. Kyung Hee University 6
,25,45,65,55 4321 kgxkgxkgxkgx
Representing values and error (-5kg ~ 5kg) x1=50.341kg, x2=67.271kg, x3=45.503kg,
x4=27.91kg, …
000 010 001 111
Dynamic range of R, B bits Step size Δ = R/2B
Quantization noise power = E[e2]
Noise in dB (log102=3.01)
Effect of 1 additional is 6.02dB
04/19/23 Media Lab. Kyung Hee University 7
B
RdeeeE
2
222/
2/
22
21212
1][
2log2012log10log20212
log10 1010102
2
10 BRR
B
1/Δ
-Δ/2 Δ/2 e
probability
Effect of quantization in image
04/19/23 Media Lab. Kyung Hee University 8
PSNR Inf PSNR 25dB
DCT Q
IDCT
Q-1
IDCT
Media signal
pdf and quantization error
The narrower pdf, the less number of bits at the same error
The narrower pdf, the less error at the same number of bits
pdf (probability density function)
Media signal
Non-uniform pdf
Variable step size Less error
Fixed step size More error
bitsXH 811.1)( bitsXH 2)(
otherwise
xforx
xforx
xf
0
101
011
)(
Media signal
Error for fixed step size
1
5.0
25.0
0
2
0
5.0
25.0
1
2
4948.0)1()75.0()1()25.0(
)1()25.0()1()75.0(
dxxxdxxx
dxxxdxxx
otherwise
xforx
xforx
xf
0
101
011
)(
Representing values at all intervals are
-0.75, -0.25, 0.25, 0.75, respectively, then mean square errors become,
Media signal
Error for variable step size
otherwise
xforx
xforx
xf
0
101
011
)(
What representing value minimizes mean square error in each interval? For example, in the interval 00, the
following equation is differentiated by p to find minimum.
2
1
0
22
1
0
2 )1()()()()( dxxpxdxxfpxp
memory-less and memory I(x) = log2 (1/px) = “degree of surprise”
qu-, re-, th-, -tion, less uncertain Of course, there are exceptions... Qatar,
Qantas Conditional probability p(u|q) >> p(u) Then, I(u|q) << I(u) accordingly, I(n|tio) << I(n)
04/19/23 Media Lab. Kyung Hee University 13
Correlation in text
Media signal
Differential Pulse-Coded Modulation (DPCM)
Quantize not x[n] but d[n]. Principle : Pdf of d[n] is narrower than that of x[n].
Less error at the same number of bits. Less amount of data, at the same error.
Prediction
][nx ][nd
][nd][ˆ nx
Quantize][nd
Histograms in images
simple image complex image
Effects of DPCM
04/19/23 Media Lab. Kyung Hee University 15
x[n]
Prob
.
x[n]
d[n] d[n] 0 0
Pred
][nx ][nd Q
H(D1)<H(D2)
Prob
.
Prob
.
Prob
.
Media signal
Differential Pulse-Coded Modulation (DPCM)
Prediction
][nx ][nd
][nd][ˆ nx
Quantize][nd
predicted the: ][ˆ where][ˆ][][ nxnxnxnd ] ][ [ ][ ndquantand
][][ˆ][ ndnxnx One - Tap Prediction ]1[][][ naxnxndN – Tap Prediction
][]3[]2[]1[][][ 321 Nnxanxanxanxanxnd N
Determine “a” which minimizes
where R(1) is the auto-correlation
for zero mean signal
DPCM
04/19/23 Media Lab. Kyung Hee University 17
time
a ≈ 0
a > 0 ]1[][][ naxnxnda << 0
22 ])1[][(])[( naxnxnd
0]]1[][[2]]1[[2]))[(( 2
2
nxnxEnxaEda
ndEd
][/)1( 2XERa 2/)1( Ra
Media signal
Adaptive DPCM Prediction filter coefficients are estimated periodically and sent as side information.
CDMA IS-95, CELP, EVRC (update interval 50 or 100 ms) LPC (linear predictive coding) Drawbacks
1. Correlation should be given and stationary.
2. Error propagation : needs refreshment Open questions
1. Why is quantized difference used for prediction? 2. Will quantization noise be accumulated? 3. How often do we have to refresh? 4. How about non-stationary case?
Trade-off between bit-rate and quality[dynamic range, accuracy, pdf]
Narrower pdf is preferred, w.r.t. H(X) Prediction for narrower pdf
Widely used in audio-video codecs Adaptation for better prediction Error propagation
Summary
04/19/23 Media Lab. Kyung Hee University 19