1
Error Concealment for Shape in MPEG-4 Object-Based
Coding
Student: Jieh-Yuean Hsu
Advisor: Prof. David W. Lin
2
Reference
[1]Meng-Yuan Liu, “Real-Time Implementation of MPEG-4 Video Encoder Using SIMD-Enhanced Intel Process” , M.S. thesis, Department of Electrical Engineering, Nation Chiao Tung University, Hsinchu, Taiwan, July 2004
[2]Cheng Huang and Paul Salama,” Error Concealment for Shape in MPEG-4 Object-Based Video Coding”, IEEE Transactions on Image Processing, Vol. 14, No. 4, April 2005.
3
Reference
[3]Guido M. Schuster, Aggelos K. Katsaggelos,” Motion Compensated Shape Error Concealment”, IEEE Transactions on Image Processing, Vol. 15, NO. 2, February 2006
4
Outline Introduction of MPEG-4 Object-
Based Coding Error Concealment
Boundary Extraction Boundary patching Boundary Filling
Experimental Result Conclusion
5
Outline Introduction of MPEG-4 Object-
Based Coding Error Concealment
Boundary Extraction Boundary patching Boundary Filling
Experimental Result Conclusion
6
VO and VOP
VO: Video Objects VOP: Video Object Planes A sequence of VOPs is referred to as
a VO I-VOP: Intra-coded P-VOP: Predictive-coded B-VOP: Bidirectionally predictive-coded
7
VO and VOP
8
VOP information
Spatial (Intra) Shape (Object-Based Only) Texture
Temporal (Inter) Motion
9
Structure of VO Encoder
10
Shape Information
Alpha plane Binary alpha plane Gray scale alpha plane
11
Binary Mask
16
16BAB
Transparent
Opaque
Boundary
12
Outline Introduction of MPEG-4 Object-
Based Coding Error Concealment
Boundary Extraction Boundary patching Boundary Filling
Experimental Result Conclusion
13
Basic Assumption The shape of the objects does not
drastically change between the current VOP and the reference VOP
It is assumed that the reference VOP is undamaged
14
Boundary Extraction
Use the 8-neighborhood If any pixel in the 8-neighborhood of a
current pixel doesn’t belong the object boundary
The boundary will be 4-connected
15
Boundary Extraction
16
4-connected Boundary
17
Boundary Patching Find the number of endpoints
A boundary pixel having only one 4-connected neighbor
The number of endpoints must be even
Partition the endpoints in the current VOP
18
Boundary Patching
Endpoints
Aa
c
CB
b
19
Boundary Patching Patching by Global Motion
Compensation Some parameter:
Scale:
Centroid:
,
,
2( 2)
( 1) 1
i ji j
i ji j
barea VOP
scalearea VOP b
( , )x y
,( )
( )
i ji jb j
xarea VOP
,( )
( )
i ji jb i
yarea VOP
20
Boundary Patching orientation angle:
Rotation angle:
2, ,( )i j i ji j
a x x b , , ,( )( )i j i j i ji j
b x x y y b 2
, ,( )i j i ji jc y y b
1tan ( )( )ba c
cur ref
21
Boundary Patching
Current VOP Reference VOP
[( ) cos( ) ( ) sin( )]cur cur cur curref ref
x x y yx x
scale
[( ) cos( ) ( ) sin( )]cur cur cur curref ref
y y x xy y
scale
22
Boundary Patching
Reference VOP Current VOP
[( ) cos( ) ( ) sin( )]cur ref ref ref ref curx x x y y scale x
[( ) cos( ) ( ) sin( )]cur ref ref ref ref cury y y x x scale y
23
Boundary Patching
Mapping
Aa
c
CB
b
A
a
c
CB
b
Current VOP Reference VOP
24
Boundary Patching Traverse the boundary of the ref.
VOP, and record the order of endpoints
ex:
Map each pixel in curve whose endpoints belong to different pair
ex , ,
A a c C B b
a c C B b A
25
Boundary Patching
Reference VOP Current VOP
a
A
c
C
B
b
Recovered pixel
26
Boundary Filling Filling in the closed boundary of the
VOP with opaque pixels After filling, we can get the recovered
VOP
Recovered VOP
27
Outline Introduction of MPEG-4 Object-
Based Coding Error Concealment
Boundary Extraction Boundary patching Boundary Filling
Experimental Result Conclusion
28
Experimental Result brea_qcif, 30 fps
29
Experimental Result
30
Experimental Result Akiyo_cif, 10fps
31
Experimental Result
32
Experimental Result brea_qcif, 10 fps
33
Outline Introduction of MPEG-4 Object-
Based Coding Error Concealment
Boundary Extraction Boundary patching Boundary Filling
Experimental Result Conclusion
34
Conclusion The method for shape concealment based on
global motion compensation It works well for video sequence with slow motion
and high frame rate It can achieve good result, even the shape is
severely damaged Extra information added resulted in 5% increase
for stream size, and it is acceptable