drift management and adaptive bit rate allocation in scalable video coding
DESCRIPTION
Drift Management and Adaptive Bit Rate Allocation in Scalable Video Coding. H. Yang, R. Zhang and K. Rose Signal Compression Lab ECE Department University of California, Santa Barbara. Outline. Introduction ROPE for scalable coding R-D optimized mode selection Simulation results - PowerPoint PPT PresentationTRANSCRIPT
09/24/02 ICIP2002 1
Drift Management and Adaptive Bit Rate Allocation in Scalable Video Coding
H. Yang, R. Zhang and K. Rose Signal Compression Lab
ECE Department
University of California, Santa Barbara
09/24/02 ICIP2002 2
Outline Introduction
ROPE for scalable coding
R-D optimized mode selection
Simulation results
Conclusions
09/24/02 ICIP2002 3
Introduction
Scalable video coding
Drift problem
Multicast scenario & existent framework
Point-to-point scenario & proposed framework
Proposed coding approach
09/24/02 ICIP2002 4
Scalable video coding and drift problem
Scalable video coding – Error resilience
– Multiple QoS
Drift problem– Whether to use enhancement layer information for prediction
If used better prediction improve coding gain
If lost mismatch / error error propagation
09/24/02 ICIP2002 5
Scalable video coding and drift problem H.263 and MPEG4 favor no-drift system.
Drift management – Goal: Achieve a good trade-off.
– Key : Accurately measure and thus effectively control the
amount of incurred error propagation.
EI
I
EP
P
EP
P
09/24/02 ICIP2002 6
Multicast scenario & existent framework
Independent channels with different capacities– Some receivers have only access to the base layer, while others have access
to both.– A coarse but acceptable base layer video quality is necessary.– Bit rates of different layers are determined by channel capacities.
Existent coding framework:
EI
I
EP
P
EP
P
09/24/02 ICIP2002 7
Point-to-point scenario & proposed framework
Only one channel is considered.– Scalable coding only provides error resilience.– An acceptable base layer video quality is not necessary.– Bit rates of the layers don’t need to be specified before encoding.
Proposed coding framework:
Research Purposes:– How much we can gain by using the proposed framework.
– Investigate the importance of accurate end-to-end distortion estimation in effective management of drift.
EI
I
EP
P
EP
P
09/24/02 ICIP2002 8
Proposed coding approach
Macroblock(MB) based SNR scalable video coding
Objective: To minimize the expected end-to-end distortion given the packet loss rate and the total bit rate.
Drift management and adaptive bit rate allocation are fulfilled via R-D optimized coding mode selection for each MB.
To accurately estimate the end-to-end distortion, ROPE is adopted.
09/24/02 ICIP2002 9
Proposed coding approach
Coding mode selection is an efficient means to optimize the tradeoff between coding efficiency and error resilience.
EI
I
EP
P
EP
P
Intra: Stop error propagation & most bits.
Inter B B: No new error & less bits.
Inter E B:New error & least bits.
09/24/02 ICIP2002 10
ROPE for Scalable Coding
Recursive Optimal per-Pixel Estimate (ROPE):– Take account of all the relevant factors as quatization, packet loss
and error concealment.
– Accurate & low complexity.
Adapt ROPE to scalable coding:– All the data of one frame is transmitted in one packet.
– The channel is modeled as a Bernoulli process with packet loss only in the enhancement layer.
09/24/02 ICIP2002 11
}))(~
{()}(~
{2)(
}))(~
{()(22
2
bfEbfEff
bffEbdi
ni
ni
ni
n
in
in
in
}))(~
{()}(~
{2)(
}))(~
{()(22
2
efEefEff
effEedi
ni
ni
ni
n
in
in
in
Overall expected decoder distortion of pixel i in frame n:
inf
)(ˆ bf in )(ˆ ef i
n
)(~
bf in )(
~ef i
n
Original value
, Encoder reconstruction values
, Decoder reconstruction values
p Packet loss rate of the enhancement layer
)(ˆ bein )(ˆ eei
n, Quantized prediction residues
09/24/02 ICIP2002 12
Intra:
Calculation of and
22 ))(ˆ(}))(~
{(
)(ˆ)}(~
{
bfbfE
bfbfEi
ni
n
in
in
Inter B B:
}))(~
)(ˆ{(}))(~
{(
)}(~
{)(ˆ)}(~
{2
12
1
bfbeEbfE
bfEbebfEj
nin
in
jn
in
in
Inter E B:
}))(~
)(ˆ{(}))(~
{(
)}(~
{)(ˆ)}(~
{2
12
1
efbeEbfE
efEbebfEk
nin
in
kn
in
in
)}(~
{ bfE in }))(
~{( 2bfE i
n
09/24/02 ICIP2002 13
Intra:
Calculation of and assuming upward error concealment
Upward:
Inter E E:
)}(~
{ efE in }))(
~{( 2efE i
n
}))(~
{())(ˆ()1(}))(~
{(
)}(~
{)(ˆ)1()}(~
{222 bfEpefpefE
bfEpefpefEi
ni
ni
n
in
in
in
}))(~
{(}))(~
)(ˆ{()1(}))(~
{(
)}(~
{)})(~
{)(ˆ()1()}(~
{222 bfEpbfeeEpefE
bfEpbfEeepefEi
ni
nin
in
in
in
in
in
}))(~
{(}))(~
)(ˆ{()1(}))(~
{(
)}(~
{)})(~
{)(ˆ()1()}(~
{22
12
1
bfEpefeeEpefE
bfEpefEeepefEi
nj
nin
in
in
jn
in
in
09/24/02 ICIP2002 14
RD Optimized Coding Mode Selection
Unconstrained minimization:– J can be independently minimized for each MB.– The coding mode and quantization step size of each MB are jointly
selected.
Joint optimization:– Global optimal but with non-trivial complexity
RDJ
MBi
inMB edeD )()( .For simple retransmission, .2
))}()(()({min)}({min bReReDeJ MBMBMBMB modemode
.
09/24/02 ICIP2002 15
RD Optimized Coding Mode Selection
Sequential optimization:– Sub-optimal but with low complexity
– For the base layer:
– For the enhancement layer:
))}(()({min)}({min bRbDbJ MBMBMB modemode
)}()({min)}({min eReDeJ MBMBMB modemode
09/24/02 ICIP2002 16
Simulation Results
UBC H.263+ codec with two-layer scalability.
Mean luminance PSNR: average first over the frames and then over the packet loss patterns.
QCIF sequences: Carphone and Salesman, first 150 frames, frame rate: 30 f/s, total bit rate: 300 kb/s.
50 packet loss patterns.
Assuming simple retransmission:
2
09/24/02 ICIP2002 17
30
31
32
33
34
35
36
5 10 15 20
packet loss rate (%)
PSN
R (
dB)
B&E driftE driftno driftsequential opt.
(a) QCIF “Carphone” (b) QCIF “Salesman”
29
30
31
32
33
34
35
36
37
38
39
5 10 15 20
packet loss rate (%)
PSN
R (
dB)
B&E driftE driftno driftsequential opt.
Fig.1 PSNR Performance of different coding frameworks
Gain of “B&E drift” over “E drift” : 0.78 dB ~ 2.80 dB Gain of “E drift” over “no drift”: 0.65 dB ~ 2.59 dB
Sequential opt. captures much of the gain of the joint opt., while their complexity ratio is approximately 1:13.
09/24/02 ICIP2002 18
(a) QCIF “Carphone” (b) QCIF “Salesman”
27
29
31
33
35
37
5 10 15 20
packet loss rate (%)
PSN
R (
dB)
B&E driftE driftno driftQDE-RD
26
28
30
32
34
36
38
40
5 10 15 20
packet loss rate (%)
PSN
R (
dB)
B&E driftE driftno driftQDE-RD
Fig.2 Performance of different distortion estimation methods
“ROPE-RD” always largely outperforms “QDE-RD”.
At high packet loss rates, “QDE-RD” performs even worse than “no drift”.
09/24/02 ICIP2002 19
Conclusions
In the context of point-to-point video transmission overlossy networks:
Decoder drift due to prediction and packet loss should be controlled but not altogether disallowed.
Bit rates of different layers should be adaptively allocated for each frame.
Reaping the full benefits of drift management and adaptive bit rate allocation requires accurate estimation of end-to-end distortion.
09/24/02 ICIP2002 20
Thank you!