block loss recovery techniques for image communications
DESCRIPTION
Block Loss Recovery Techniques for Image Communications. Jiho Park, D-C Park, Robert J. Marks, M. El-Sharkawi The Computational Intelligence Applications (CIA) Lab. Department of Electrical Engineering University of Washington May 29, 2002. Projections based Block Recovery – Motivation. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/1.jpg)
Block Loss Recovery Techniques for Image Communications
Jiho Park, D-C Park, Robert J. Marks, M. El-Sharkawi
The Computational Intelligence Applications (CIA) Lab.
Department of Electrical Engineering
University of Washington
May 29, 2002
![Page 2: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/2.jpg)
2
Projections based Block Recovery – Motivation
Conventional Algorithms use information of all surrounding area. Using only highly correlated area
![Page 3: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/3.jpg)
3
Alternating Projections is projecting between two or more convex sets iteratively.
Alternating Projections
Converging to a common point
![Page 4: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/4.jpg)
4
Projections based Block Recovery – Algorithm
2 Steps Pre Process : 1) Edge orientation detection
2) Surrounding vector extraction
3) Recovery vector extraction
Projections : 1) Projection operator P1
2) Projection operator P2
3) Projection operator P3
![Page 5: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/5.jpg)
5
Edge orientation in the surrounding area(S) of a missing block(M). In order to extend the geometric structure to the missing block.
Simple line masks at every i, j coordinate in surrounding area(S) of the missing block(M) for edge detection.
Pre Process 1 –Edge Orientation Detection
121
121
121
vL
111
222
111
hL
Horizontal Line Mask Vertical Line Mask
![Page 6: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/6.jpg)
6
Pre Process 1 – Edge Orientation Detection
Responses of the line masks at window W :
Total magnitude of responses :
Th > Tv ; Horizontal line dominating area
Th < Tv ; Vertical line dominating area
987
654
321
www
www
www
W987654321 w-w-w-w2w2w2w-w--w hR
987654321 w-w2w-w-w2w-w-w2-w vR
,||T S
hh R S
vv R ||T
![Page 7: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/7.jpg)
7
Pre Process 2 – Surrounding Vectors
Surrounding Vectors, sk, are extracted from surrounding area of a missing block by N x N window.
Each vector has its own spatial and spectral characteristic. The number of surrounding vectors, sk, is 8N.
}W),(),,(:{ jijixxks
![Page 8: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/8.jpg)
8
Pre Process 3 – Recovery Vector Recovery vectors are extracted to restore missing pixels. Two positions of recovery vectors are possible according to the
edge orientation.
Recovery vectors consist of known pixels(white color) and missing pixels(gray color).
The number of recovery vectors, rk, is 2.
}W),(),,(:{ jijixxkr
Vertical line dominating area Horizontal line dominating area
![Page 9: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/9.jpg)
9
Projections based Block Recovery –Projection operator P1
Recovery vectors, ri, for i = 1, 2
Surrounding vectors, sj , for j = 1 ~ 8N
Surrounding vectors, S, form a convex hull in N2-dimensional space
Recovery vectors, R, are orthogonally projected onto the line defined by the closest surrounding vector, si, j : Projection Operator P1.
![Page 10: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/10.jpg)
10
Projections based Block Recovery –Projection operator P1
Projection operator P1 :
Convex hull (formed by surrounding vectors, containing information of local image structure)
![Page 11: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/11.jpg)
11
Projections based Block Recovery –Projection operator P1
Surrounding vectors, sj , for j = 1 ~ 8N Recovery vectors, ri, for i = 1, 2
The closest vertex, sdi , from a recovery vector, ri.
or equivalently in DCT domain,
P1 :
Njiford jij
i 81,21||||minarg sr
Njiford jij
i 81,21||||minarg SR
21,||||
,)(
2
ii
i
d di
idiiP S
R
RSRS
![Page 12: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/12.jpg)
12
Convex set C2 acts as an “identical middle”.
Projection operator P2 :
Projections based Block Recovery –Projection operator P2
otherwise
nforFFC
o
n
ff
ff
:
L: maxmin2
otherwise
nFforF
nFforF
P
n
n
n
n
f
f
f
f L
L
max,max
min,min
2
![Page 13: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/13.jpg)
13
Convex set C3 acts as a convex constraint between missing pixels and adjacent known pixels, (fN-1 fN).
where,
and is a N x N recovery vector in
column vector form.
Projections based Block Recovery – Projection operator P3
fN-1 fN
}||:{3 EC n gg
)}(....,),{( ,,10,0,1 NNNNNN ffffg
}....,,,{ 21 Nffff
Projection operator P3 :
otherwise
nEforE
nEforE
P
mn
nmn
nmn
mn
,
,1
,1
,3 L,
L,
f
gf
gf
f
![Page 14: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/14.jpg)
14
Projections based Block Recovery –Iterative Algorithm
Missing pixels in recovery vectors are restored by iterative algorithm of alternating projections :
N x N windows moving :
ii fPPPf 3211
Vertical line dominating area Horizontal line dominating area
![Page 15: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/15.jpg)
15
Projections based Block Recovery - Summary
Edge Orientation Detection
Surrounding Vector Extraction
Recovery Vector Extraction
Projection Operator P1
Projection Operator P2
Projection Operator P3
Iteration=I?
All pixels?
![Page 16: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/16.jpg)
16
Simulation Results –Lena, 8 x 8 block loss
Original Image Test Image
![Page 17: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/17.jpg)
17
Simulation Results –Lena, 8 x 8 block loss
Ancis, PSNR = 28.68 dB Hemami, PSNR = 31.86 dB
![Page 18: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/18.jpg)
18
Simulation Results –Lena, 8 x 8 block loss
Ziad, PSNR = 31.57 dB Proposed, PSNR = 34.65 dB
![Page 19: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/19.jpg)
19
Simulation Results –Lena, 8 x 8 block loss
Ancis
PSNR = 28.68 dB
Hemami
PSNR = 31.86 dB
Ziad
PSNR = 31.57 dB
Proposed
PSNR = 34.65 dB
![Page 20: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/20.jpg)
20
Simulation Results – Each StepLena 8 x 8 block loss
(a)
(b)
(c)
![Page 21: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/21.jpg)
21
Simulation Results –Peppers, 8 x 8 block loss
Original Image Test Image
![Page 22: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/22.jpg)
22
Simulation Results – Peppers, 8 x 8 block loss
Ancis, PSNR = 27.92 dB Hemami, PSNR = 31.83 dB
![Page 23: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/23.jpg)
23
Simulation Results – Peppers, 8 x 8 block loss
Ziad, PSNR = 32.76 dB Proposed, PSNR = 34.20 dB
![Page 24: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/24.jpg)
24
Simulation Results –Lena, 8 x one row block loss
Original Image Test Image
![Page 25: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/25.jpg)
25
Simulation Results –Lena, 8 x one row block loss
Hemami, PSNR = 26.86 dB Proposed, PSNR = 30.18 dB
![Page 26: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/26.jpg)
26
Simulation Results –Masquerade, 8 x one row block loss
Original Image Test Image
![Page 27: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/27.jpg)
27
Simulation Results –Masquerade, 8 x one row block loss
Hemami, PSNR = 23.10 dB Proposed, PSNR = 25.09 dB
![Page 28: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/28.jpg)
28
Simulation Results –Lena, 16 x 16 block loss
Original Image Test Image
![Page 29: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/29.jpg)
29
Simulation Results –Lena, 16 x 16 block loss
Ziad, PSNR = 28.75 dB Proposed, PSNR = 32.70 dB
![Page 30: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/30.jpg)
30
Simulation Results –Foreman, 16 x 16 block loss
Original Image Test Image
Ziad, PSNR = 25.65 dB Proposed, PSNR = 30.34 dB
![Page 31: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/31.jpg)
31
Simulation Results –Flower Garden, 16 x 16 block loss
Original Image Test Image
Ziad, PSNR = 20.40 dB Proposed, PSNR = 22.62 dB
![Page 32: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/32.jpg)
32
Simulation Results – Test Data and Error
512 x 512 “Lena”, “Masquerade”, “Peppers”, “Boat”, “Elaine”, “Couple”
176 x 144 “Foreman” 352 x 240 “Flower Garden”
8 x 8 pixel block loss 16 x 16 pixel block loss 8 x 8 consecutive block losses
Peak Signal – Noise – Ratio
)|),(ˆ),(|
255log(10
1 1
2
2
N
i
M
j
jixjix
MNPSNR
![Page 33: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/33.jpg)
33
Simulation Results – PSNR (8 x 8)
Lena Masqrd Peppers Boat Elaine Couple
Ancis 28.68 25.47 27.92 26.33 29.84 28.24
Sun 29.99 27.25 29.97 27.36 30.95 28.45
Park 31.26 27.91 31.71 28.77 32.96 30.04
Hemami 31.86 27.65 31.83 29.36 32.07 30.31
Ziad 31.57 27.94 32.76 30.11 31.92 30.99
Proposed 34.65 29.87 34.20 30.78 34.63 31.49
![Page 34: Block Loss Recovery Techniques for Image Communications](https://reader036.vdocuments.net/reader036/viewer/2022062801/5681444f550346895db0edb0/html5/thumbnails/34.jpg)
34
Simulation Results – PSNR (Row, 16 x 16)
(16 x 16) Lena Foreman Garden
Ziad 28.75 25.65 20.40
Proposed 32.70 30.34 22.62
(8 x Row) Lena Maskrd Peppers Boat Elaine Couple
Hemami 26.86 23.10 25.41 24.54 26.87 24.30
Proposed 30.18 25.09 28.31 26.06 30.11 26.12