Download - Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University
![Page 1: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/1.jpg)
Adaptive DenoisingAdaptive Denoisingfor Video Compressionfor Video Compression
Eren SoyakEECS 463
Winter 2006
Northwestern University
![Page 2: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/2.jpg)
Video Compression and You
Demand for video where no video has gone before
Source You
Source
Source
![Page 3: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/3.jpg)
Video Compression and You
Source You
Source
Source
Encode Medium
Decode
Demand for video where no video has gone before
![Page 4: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/4.jpg)
Video Compression and You
Demand for video where no video has gone before
Source You
Source
Source
Encode Channel
Decode
Pos t proces sing
Preproces sing
![Page 5: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/5.jpg)
Video and Compression Video compression works by identifying and
exploiting redundancy in source video The more information there is in the source,
the more difficult it is to compress into a smaller form
Foreman
Foreman.264
![Page 6: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/6.jpg)
Noise and Compression Noise is usually present in source video due to
various reasons (capture, film grain, quantization, transmission errors etc)
Wide spectrum noise is very difficult to compress
The ever-popular AWGN-type noise
Deprecated old analog-type noise
![Page 7: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/7.jpg)
Dealing with Noise Pre/post filtering methods very useful Simple denoising method: averaging filter
3 pels 5 pels 7 pels
![Page 8: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/8.jpg)
Good, Bad and Ugly Denoising Denoising must distinguish between original signal
and noise, filter out only the noise. Prediction of the noise and/or the original video is usually required for this.
Smoothing, edge loss and blurring are all undesirable
Despeckle
“Smart” blur
10 pel average
![Page 9: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/9.jpg)
Case Study: AWGN Additive White Gaussian Noise (AWGN) can be
introduced by capture devices, especially due to poor lighting and sometimes weather.
AWGN breaks most compression algorithms. Consider signal independent AWGN.
Foreman + AWGN
![Page 10: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/10.jpg)
Advanced Denoising (Wiener) The Wiener filter is commonly used by the
ambitious for generic denoising. Requires little information about noise. Few “catastrophic” corner cases.
Wiener(Foreman + AWGN)
![Page 11: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/11.jpg)
Global Denoising Issues The visibility (and usually compression hindrance)
of noise is a function of the source even if the severity of the noise itself is not – noise is more visible on smooth regions as opposed to texture.
It would be highly desirable to filter noise such that the final video retains local shape/texture characteristics as well.
Adaptive methods begin to suggest themselves.
![Page 12: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/12.jpg)
LMMSE Filtering Linear Minimum Mean Squared Error filter (IIR)
1 2i i212121 = ˆ -i)-i, n)g(n, ih(i), n(ns
Noisy image
Impulse response
LMMSE estimate of ideal image s(n1, n2)
(1)
![Page 13: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/13.jpg)
The Unrealizable Wiener Filter The principle of orthogonality states that the
estimation error s(n1, n2)- (n1, n2) should be orthogonal to every sample of the observed image.
s
0
ˆ =
, ˆ
212121
212121
), kg(k), n(ns), ns(nE
), kg(k), n(ns), ns(n(2)
![Page 14: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/14.jpg)
The Impossible Wiener IR Substituting (1) into (2) and simplifying we can
express the the impulse response of the filter as a 2D convolution
Is impossible to realize since infinite time is required before an output sample is computed.
), n(nR), n(nR), nh(n sggg 212121 =**
“Discrete Wiener-Hopf equation”
autocorrelation of observations
cross correlation between ideal and observed image
![Page 15: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/15.jpg)
Adaptive LMMSE Kuan et al. proposed in ’85:
), n(n)-μ, ng(nσ), n(nσ
) , n(nσ) + , n(n) = μ, n(ns g
vs
sg 21212
212
212
2121ˆ
), n(ns 21ˆ
), n(nμg 21
) , n(nσs 212
), ng(n 21
2vσ
= observation
= filtered output = local variance
= estimated noise variance
= local mean
![Page 16: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/16.jpg)
Adaptive LMMSE Performance Given its adaptive nature to local image properties
the filter is better at preserving edges/texture while removing noise.
It is very process-intensive and sensitive to misestimation of noise variance.
Adaptive LMMSE(Foreman + AWGN)
![Page 17: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/17.jpg)
Comparing Filter Outputs
![Page 18: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/18.jpg)
Comparing Filter Outputs
Adaptive LMMSEWiener
![Page 19: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/19.jpg)
Comparing Compressed Video Compressed at 512 kbps at H.264 Main Profile
Wiener
Adaptive LMMSE
![Page 20: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/20.jpg)
Weighed Adaptive LMMSE Directionally weighed variance matrix
May better account for edges due to 2D direction component
Choice of weight matrix could be optimized
1 2 1
2 3 2
1 2 1
![Page 21: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/21.jpg)
Weighed Adaptive LMMSE Prone to blurring if matrix weights poorly chosen..
Poorly Weighed Adaptive LMMSE(Foreman + AWGN)
![Page 22: Adaptive Denoising for Video Compression Eren Soyak EECS 463 Winter 2006 Northwestern University](https://reader033.vdocuments.net/reader033/viewer/2022050714/56649d005503460f949d2990/html5/thumbnails/22.jpg)
Bibliography A. Murat Tekalp, Digital Video Processing, ‘95 J. S. Lim, Two Dimensional Signal and Image
Processing, ‘90 D.T. Kuan, A.A. Sawchuk, T.C. Strand, P. Chavel,
Adaptive noise smoothing filter for images with signal-dependent noise, ‘85