1

Symmetric Disparity Estimation in Distributed Coding of Stereo Images

Xin LiLane Dept. of CSEE

West Virginia UniversityMorgantown, WV 26506-6109

2

Outline Background: distributed source coding (DSC)

The gap between theory and practice: modeling nonstationary and location-related correlation structure

Distributed coding of stereo images Asymmetric vs. symmetric approach Side information (SI) generation: EM-like iterative disparity

estimation Coded information (CI) exploitation: jointly refine intensity

and disparity uncertainty

Experimental results and summary

3

What is Distributed Coding?

Traditional (centralized) Coding Paradigm

(e.g., TV broadcasting)

New (distributed) Coding Paradigm

(e.g., sensor network)

sourceX

sourceY

centralizedencoder

centralizeddecoder

sourceX

sourceY

distributedencoder X

distributedencoder Y

centralizeddecoder

Power-hungrycomponent

4

Current State of the Art Theoretic sources (e.g., i.i.d. binary or Gaussian)

Trellis codes: Pradhan and Ramchandran’1999, Wang and Orchard’2001

Turbo codes: J. Garcia-Frias’2001, Aaron and Girod’2002 LDPC codes: Stankovic, Liveris and Xiong’2002

Practical sources (e.g., image and video) Wyner-Ziv coding of video still significantly falls behind

H.263+ (refer to “Distributed Video Coding” by Girod, Aaron, Rane and Rebollo-Monedero, Proc. of IEEE, pp.71-83 2005)

5

Why is Practical Source Difficult? The gap between theory and practice: the

correlation structure of practical sources is often characterized by Nonstationary instead of stationary Location-based instead of intensity-based

The distributed constraint of exploiting source correlation Adaptation at the encoder is more difficult Optimization at the decoder instead of encoder

6

Distributed Coding of Stereo Images

Disparity estimation& compensation

X Y

Centralized encoderHow to shift disparity estimation

from encoder to decoder?

Quantization& entropy coding

X Y

Distributedencoder X

Distributedencoder Y

Centralizeddecoder

7

Asymmetric vs. Symmetric Coding

Amblyopia (Lazy Eye)

Left eye: X Right eye: Y

Left-eye and right-eye images with varying acuity are jointly decodedby HVS to estimate disparity.

Asymmetric coding protocol

Left sensor: X Right sensor: Y

Symmetric coding protocol

Interlaced versions of left and right images are jointly decoded along with disparity estimation

Interlaced sampling

8

Symmetric Coding Protocol

Xeven,Yodd

(finely-quantized)

Xodd,Yeven

(coarsely-quantized)Left image: X Right image: Y

Correlation model:

Primary

Secondary

WXDYWYDX evenevenoddodd )(,)(

Side Information (SI) from primary

D: disparity compensationW: additive noise term Coded Information

(CI) from secondary

9

Side-Information (SI) Generation

To estimate disparity maps,We need to know missing rows

To estimate missing rows (SI),We need to know disparity maps

How to solve such problem of chicken-and-egg flavor ?

10

EM-like Iterative Decoding

Update the estimationof missing rows(disparity-based compensation)

Update the estimationof disparity maps(stereo matching/

disparity estimation)

Initialization (spatial interpolation)

11

Convergence and Caveat

Iteration number

Ene

rgy

of in

tens

ity u

pdat

e

left

right

reconstructed

original

Phase misalignment (trapped in local minimum)

column variable

Inte

nsity

pro

file

Fast convergence ofour iterative algorithm

12

Coded Information (CI) Exploitation

Kill two Birds with one Stone (secondary channel)

1) Work as the synchronization marker to resolve location ambiguity2) Work as the quantization constraint to correct intensity errors

Xeven,Yodd

(finely-quantized)

Xodd,Yeven

(most significant bit)Left image: X Right image: Y

Primary

Secondary

SI

CI

13

Benefits of Secondary Channel Phase Alignment

Intensity RefinementDecoded

(from SI)

52 53 201 200 51 55

MSB

(from CI)

0 0 1 1 1 0 0

50201

CI SI

Work as the synchronization markerto resolve location uncertainty

Work as the quantization constraint to correct intensity errors

original

14

Experimental Setup

tsukubaroomsquare

low highComplexity of disparity field

middle

15

Improvement of Generated SI

0 2 4 6 8 1020

40

60

80

100

120

140

iteration number

MS

E

room

1 1.5 2 2.5 3 3.5 420

25

30

35

40

45

iteration number

MS

E

tsukuba

Blue line: left image, green line: right image

16

Benefits of Secondary Channel

Original EM-like scheme(no phase alignment orintensity refinement)

Improved EM-like scheme(with phase alignment butno intensity refinement)

Improved EM-like scheme(with both phase alignmentand intensity refinement)

17

Disparity Map Comparison

Distributed decoding Centralized encoding(The stereo matching scheme we

used is fast yet less accurate)

18

Disparity Field Comparison

Distributed decoding(uncorrected errors due to

local minimum trap)

Centralized encoding

19

0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2 0.22 0.2420

21

22

23

24

25

26

27

28

29

Rate(bpp)

PS

NR

(dB

)

H264 I+I

OursH264 I+P

Preliminary Coding Results

Room pair(256×256)

H264 I+I

H264 I+POurs

20

Summary Distributed coding of stereo images

Why? – It calls for the modeling of nonstationary location-based correlation

How? – We propose a symmetric coding protocol in which SI is generated by EM-like iterative disparity estimation and CI is exploited to jointly refine intensity and disparity uncertainty at the decoder

What is next? – We are working on modeling more complicated correlation structure in distributed video coding