1
Symmetric Disparity Estimation in Distributed Coding of Stereo Images
Xin LiLane Dept. of CSEE
West Virginia UniversityMorgantown, WV 26506-6109
2
Outline Background: distributed source coding (DSC)
The gap between theory and practice: modeling nonstationary and location-related correlation structure
Distributed coding of stereo images Asymmetric vs. symmetric approach Side information (SI) generation: EM-like iterative disparity
estimation Coded information (CI) exploitation: jointly refine intensity
and disparity uncertainty
Experimental results and summary
3
What is Distributed Coding?
Traditional (centralized) Coding Paradigm
(e.g., TV broadcasting)
New (distributed) Coding Paradigm
(e.g., sensor network)
sourceX
sourceY
centralizedencoder
centralizeddecoder
sourceX
sourceY
distributedencoder X
distributedencoder Y
centralizeddecoder
Power-hungrycomponent
4
Current State of the Art Theoretic sources (e.g., i.i.d. binary or Gaussian)
Trellis codes: Pradhan and Ramchandran’1999, Wang and Orchard’2001
Turbo codes: J. Garcia-Frias’2001, Aaron and Girod’2002 LDPC codes: Stankovic, Liveris and Xiong’2002
Practical sources (e.g., image and video) Wyner-Ziv coding of video still significantly falls behind
H.263+ (refer to “Distributed Video Coding” by Girod, Aaron, Rane and Rebollo-Monedero, Proc. of IEEE, pp.71-83 2005)
5
Why is Practical Source Difficult? The gap between theory and practice: the
correlation structure of practical sources is often characterized by Nonstationary instead of stationary Location-based instead of intensity-based
The distributed constraint of exploiting source correlation Adaptation at the encoder is more difficult Optimization at the decoder instead of encoder
6
Distributed Coding of Stereo Images
Disparity estimation& compensation
X Y
Centralized encoderHow to shift disparity estimation
from encoder to decoder?
Quantization& entropy coding
X Y
Distributedencoder X
Distributedencoder Y
Centralizeddecoder
7
Asymmetric vs. Symmetric Coding
Amblyopia (Lazy Eye)
Left eye: X Right eye: Y
Left-eye and right-eye images with varying acuity are jointly decodedby HVS to estimate disparity.
Asymmetric coding protocol
Left sensor: X Right sensor: Y
Symmetric coding protocol
Interlaced versions of left and right images are jointly decoded along with disparity estimation
Interlaced sampling
8
Symmetric Coding Protocol
Xeven,Yodd
(finely-quantized)
Xodd,Yeven
(coarsely-quantized)Left image: X Right image: Y
Correlation model:
Primary
Secondary
WXDYWYDX evenevenoddodd )(,)(
Side Information (SI) from primary
D: disparity compensationW: additive noise term Coded Information
(CI) from secondary
9
Side-Information (SI) Generation
To estimate disparity maps,We need to know missing rows
To estimate missing rows (SI),We need to know disparity maps
How to solve such problem of chicken-and-egg flavor ?
10
EM-like Iterative Decoding
Update the estimationof missing rows(disparity-based compensation)
Update the estimationof disparity maps(stereo matching/
disparity estimation)
Initialization (spatial interpolation)
11
Convergence and Caveat
Iteration number
Ene
rgy
of in
tens
ity u
pdat
e
left
right
reconstructed
original
Phase misalignment (trapped in local minimum)
column variable
Inte
nsity
pro
file
Fast convergence ofour iterative algorithm
12
Coded Information (CI) Exploitation
Kill two Birds with one Stone (secondary channel)
1) Work as the synchronization marker to resolve location ambiguity2) Work as the quantization constraint to correct intensity errors
Xeven,Yodd
(finely-quantized)
Xodd,Yeven
(most significant bit)Left image: X Right image: Y
Primary
Secondary
SI
CI
13
Benefits of Secondary Channel Phase Alignment
Intensity RefinementDecoded
(from SI)
52 53 201 200 51 55
MSB
(from CI)
0 0 1 1 1 0 0
50201
CI SI
Work as the synchronization markerto resolve location uncertainty
Work as the quantization constraint to correct intensity errors
original
14
Experimental Setup
tsukubaroomsquare
low highComplexity of disparity field
middle
15
Improvement of Generated SI
0 2 4 6 8 1020
40
60
80
100
120
140
iteration number
MS
E
room
1 1.5 2 2.5 3 3.5 420
25
30
35
40
45
iteration number
MS
E
tsukuba
Blue line: left image, green line: right image
16
Benefits of Secondary Channel
Original EM-like scheme(no phase alignment orintensity refinement)
Improved EM-like scheme(with phase alignment butno intensity refinement)
Improved EM-like scheme(with both phase alignmentand intensity refinement)
17
Disparity Map Comparison
Distributed decoding Centralized encoding(The stereo matching scheme we
used is fast yet less accurate)
18
Disparity Field Comparison
Distributed decoding(uncorrected errors due to
local minimum trap)
Centralized encoding
19
0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.2420
21
22
23
24
25
26
27
28
29
Rate(bpp)
PS
NR
(dB
)
H264 I+I
OursH264 I+P
Preliminary Coding Results
Room pair(256×256)
H264 I+I
H264 I+POurs
20
Summary Distributed coding of stereo images
Why? – It calls for the modeling of nonstationary location-based correlation
How? – We propose a symmetric coding protocol in which SI is generated by EM-like iterative disparity estimation and CI is exploited to jointly refine intensity and disparity uncertainty at the decoder
What is next? – We are working on modeling more complicated correlation structure in distributed video coding