mpeg 系統中錯誤隱藏之 vlsi 晶片設計 nsc90-2213-e-327-010
DESCRIPTION
MPEG 系統中錯誤隱藏之 VLSI 晶片設計 NSC90-2213-E-327-010. 國立高雄第一科技大學 電腦與通訊工程系/所 Speaker : 夏世昌 博士 中華民國 94 年 10 月 28 日. IC Type RF IC Memory IC Power IC Mixing Mode IC Micro Processor & Interface chip Analog IC DSP IC. DSP Chip=Algorithm +VLSI architecture +Chip Implementation. 內容大綱. - PowerPoint PPT PresentationTRANSCRIPT
-
Speaker
941028
MPEGVLSINSC90-2213-E-327-010
-
IC Type
RF ICMemory ICPower ICMixing Mode ICMicro Processor & Interface chipAnalog ICDSP IC
DSP Chip=Algorithm +VLSI architecture +Chip Implementation
-
Inter-coding/Predictive-coding Intra-coding
I
P3
P1
P2
B0
B1
B0
B1
B0
B1
1
10
9
8
7
6
5
4
3
2
........
Forward Prediction
Bidirectional Prediction
B0
B1
I
11
12
13
-
Blockinput
DCT
Quantization
Zig-ZagandRun-Length
VLC
Buffer
InverseDCT
InverseQuantization
MotionEstimation
FrameMemory(coded)
CodingControl
Intra-coding
Inter-coding
Intra-coding
Inter-coding
+
+
+
-
To Quantization
To Mode Switch
To Motion Estimation
-
Buffer
VLCDecoder
Inverse Quantization
InverseDCT
Frame Memory
Motion Compensation
Loop Filter
Mux
0
Motion vector
Inter/intra mode
Input data
Decoded data
Inverse Zig-ZagInverse Run-length
intra
inter
-
Video sequence
sequence header
GOP
GOP
GOP
K
Sequence end code
GOP header
Picture
Picture
Picture
K
Picture
Picture
Picture header
Slice
Slice
Slice
K
Slice
Slice
Slice header
MB
MB
MB
K
MB
MB
MB header
block
block
block
K
block
block
Transform Coefficients(TCOEFF)
K
End of block(EOB)
Sequence Layer
GOP Layer
Picture Layer
Slice Layer or GOB Layer
MB Layer
block Layer
-
VLCVLCIBBPBBPBBPBBI (Forward error correction) (ARQ)
-
>Bilinear
dT
dL
dB
dR
bT(0,k)
b(i,k)
bB(N+1,k)
bL(i,0)
bR(i,N+1)
N=16
-
>zero motion vector
t
t-1
-
>above motion vector
t
t-1
MV=(x,y)
MV=(x,y)
-
>average motion vector for I-frame
tIntra-frame
t-1P-frame
MV=(x,y)
MV=(TLx,TLy)
MV=(BLx,BLy)
-
>average motion vector for P-frame
tP-frame
t-1P-frame
MV=(x,y)
MV=(TLx,TLy)
MV=(BLx,BLy)
-
>median motion vector of P-frame
tP-frame
t-1P-frame
MV=median{candidate MB}
MV=(x,y)
Corrupted MB
Candidate MB
-
>median motion vector of I-frame
Corrupted MB
tIntra-frame
t-1P-frame
MV=median{candidate MB}
MV=(x,y)
Candidate MB
-
>Boundary Matching Algorithm(BMA)
:Lam W.M., Reibman A.R., Liu B. , Recovery of lost or erroneously received motion vectors , 1993 IEEE International Conference on Acoustics, Speech, and Signal Processing, 1993. ICASSP-93., Vol. 5, p.p. 417-420, 1993.
Region A
Region B
Region C
-
>Decoder Motion-Vector Estimation (DMVE) Algorithm
:Jian Zhang , John F. Arnold and Michale R. Frater , A Cell-Loss Concealment Technique for MPEG-2 Coded Video IEEE Trans. on Circuits and Systems for Video Technology, Vol. 10, No. 4, p.p. 659-665, June 2000.
Lost macroblock
Decoded pixels fromcurrent frame
-
Median Vector for P or I frame for P frameP-frameI-frameShih-Chang Hsia , Shyi-Chyi Cheng and Shih Wen Chou, Efficient adaptive error concealment technique for video decoding system, IEEE Trans. Multimedia. IEEE Trans. Multimedia, pp.860-868, Oct. 2005.
-
>
t-1
t
V
Corrupted MB
VqLW
-
t-1
t
Vq
Corrupted block
V
MV=(x,y)
t-1
V
V
Vq
t-2
MV=(x,y)
-
P-frame at time t
P-frame at time t-1
Corrupted MB
-
LTDlLTDtlLTDtrLTDrLTDbrLTDbl
-
To decide where is the object?
V
p
p
p
p
p
p
j
j
j
-
Caculate TDtl BTDtBTDtrBTDblBTDbBTDbr
LTDmin
-
>,
BMB=Min. (MAD(Mxt)), Mxtfrom N to N
-
(2)
(1)
(3)
(4)
(5)
-
,BMB=Min. (MAD(Mxb)), Mxb from N to N
-
>P-frame
ft-1(Mv),IDCT(Fuv) >Block Deviation()
BDTL
BDT
BDTR
BDBL
BDB
BDBR
Corrupted MB
Correctly decoded MB
BDx:Block Deviation for xMB
-
coeff_P
-
>I-frame
D45,D135,V1AH
-
BV(Block Variance)
SIlost
-
MDlost(Motion Distance)
MF(Moving Feature)
coeff_I
-
I or P
pCMBBD
BDlost=BDlostmax
BDlost=BDlost
plBDlost
BDlost>BDlostmax
coeff_P
MDiff>MDiffmax
P-frameAVX
MDiff
H~Bz
pCMBAH
pCMBBV
plAHlost
plBVlost
yes
P
I
No
No
yes
-
AHlost=AHlost
BVlost>BVlostmax
BVlost=BVlostmax
AHlost=AHlostmax
AHlost>AHlostmax
BVlost=BVlost
plMDlost
MDlost>MDlostmax
MDlost=MDlostmax
MDlost=MDlost
MF
coeff_I
coeff_I
-
P-frameI-frameComplex Video
-
Peak Signal to Noise Ratio(PSNR)Io : original imageIr : result imageMSE : mean square errorH : height of imageW : width of image
-
,
-
,
-
,PSNR=40.7dB
-
bilinearPSNR=39.72dB
-
4CIF (704x576) GOBMB12~MB22 10(49) for P or I frame 1(271) for Complex Video
CIF1
CIF2
CIF3
CIF4
GOB1
GOB2
GOB3
GOB4
GOB5
GOB6
GOB7
GOB8
GOB9
GOB10
GOB11
GOB12
MB2
MB3
MB4
MB5
MB6
MB7
MB8
MB9
MB10
MB11
MB1
MB13
MB14
MB15
MB16
MB17
MB18
MB19
MB20
MB21
MB22
MB12
MB24
MB25
MB26
MB27
MB28
MB29
MB30
MB31
MB32
MB33
MB23
704
576
176
48
-
P-frameOur approach-1(median) Our approach-2LTD_threshold=6LVD_threshold=20BDlostmax=2048
-
PSNR=11.24dB Frame number=6
-
bilinearPSNR=26.6dB
-
DMVE,PSNR=24.24dB
-
our approach-1,PSNR=26.94dB
-
Frame number=6PSNR=15.69dB
-
bilinearPSNR=33.65dB
-
DMVE, PSNR=34.46dB
-
our approach-2 PSNR=35.98dB
-
Frame number=6PSNR=14.3dB
-
bilinearPSNR=29.62dB
-
DMVE, PSNR=26.85dB
-
our approach-1 PSNR=29.88dB
-
I-frameOur approach(median)MDiffmax=16384BVlostmax=512AHlostmax=2048MDlostmax=16C1=0.4C2=0.6
-
Frame number=12PSNR=12.37dB
-
bilinearPSNR=37.28dB
-
DMVE,PSNR=28.33dB
-
our approach,PSNR=36.78dB
-
Frame number=12PSNR=9.54dB
-
bilinearPSNR=21.33dB
-
DMVE,PSNR=32.97dB
-
our approach,PSNR=31.98dB
-
Frame number=12PSNR=15.49dB
-
bilinearPSNR=41.14dB
-
DMVE,PSNR=31.18dB
-
our approach,PSNR=39.69dB
-
Complex VideoOur approach-1P or I frame(median) Our approach-2P frameI frame(median)
-
Frame number=192(I-frame)PSNR=13.28dB
-
bilinearPSNR=25.57dB
-
DMVE,PSNR=20.28dB
-
our approach-1,PSNR=25.45dB
-
(median)bilinearBDlostmax=2048BVlostmax=512AHlostmax=2048MDlostmax=16C1=0.40625=(0.01101)bC2=0.59375=(0.10011)b
GOB1
GOB2
GOB3
KK
704
16
MB1
MB2
MB3
MB4
MB44
KK
-
AVX
~
Atkp
~
Recovery motion vector
Motion vector input
DCTY
H
Clk,reset,Frametype,errordet,gobstart,mbdatastart,mba,cbp
frame memoryJ
From frame memory
MUX
X
-
Spatial MVy Storage
Temporal MVy Storage
mba,cbp,mbdatastartgobdatastart,frametype
AHlost,BDlost,BVlost,MF,SIlost decision ,coeff_I and coeff_P computation
Head processing
VLC/RLC
IQ
IDCT
Buffer
Frame memory
Line buffer704x8
Line buffer16x8
Bilinear Interpolation
Adaptive Algorithm for I or P Frame,
latch
MUX
Error Detection
Moving Feature Computation(MDlost)
Spatial MVx Storage
Temporal MVx Storage
MUX
Block Deviation or Block Variance computation and storage
H component computation and storage
Zig-Zag
Line Bufferread/writecontrol
Frame memoryaccess control
Control signalgenerater
MUX
MV storage access control
Median Vector and MDlost computation Control
Buffer
Line buffer32x8
MB Boundry
Continue two GOBs error
frametype
ErrorCorrect
bitstream
errordet
-
(Moving Feature)
FF
MUX
D
Q
MUX
FF
D
Q
0
6
KK
KK
MV Buffer
MDlost
From MV storage
0
FF
D
Q
MUX
MUX
0
0
-
BD or BV or AH
MUX
FF
D
Q
j
dctY
To Storage
MUX
0
-
MB1
MB2
MB3
MB4
MB5
MB45
MB46
MB47
MB48
MB49
MB89
MB90
MB91
MB92
MB93
Corrupted MB
MB44
MB88
MB132
KK
KK
KK
MB47~I
MB47~AVXI
-
1.MB472.3.errorcountdown=42 (88-47+1=42)
-
4.errorcountdown=05.GOB,MB6.MB89cbp=607. countpix=255errorflag
-
1.MB922.3.Buffer4.Recmvx=Median(1320323)=2Recmvy=Median(4321656)=4
-
bilinear
MB2
MB3
MB4
MB5
MB46
MB47
MB48
MB49
MB90
MB91
MB92
MB93
Write in top line buffer
Read out from top line buffer and write in top buffer
Write in bottom buffer
bilinearI
-
top line buffer top buffer 1.MB42.MB3 bottom line line bufferWAwrite addressline buffer
-
3.MB915.line bufferMB3 bottom line 4. line bufferMB3bottom lineread address6.MB937.top bufferbilinear
-
bottom buffer 1.MB922.MB91top linebottom buffer3.MB934.MB(256 clocks)MB91top line
-
Bilinear1.MB932.16clocks3.
-
BD or BV1.MB32.3. MB3dctBV=1886
-
4.MB935.MB93BV(BVt)
-
AH 1.MB32.MB3AH3. MB3dctAH=174
-
4.MB935. MB93AH(AHt)
-
coeff_I , coeff_PMB93
-
MB5
MB2BV=1809AH=239
MB3BV=1886AH=174
MB4BV=1777AH=182
MB49
MB47BVT=1886BVB=0AHT=174AHB=0MDlost=5BVlost=512;AHlost=174
MB48
MB93
MB90BV=1892AH=247
MB91BV=0AH=0
MB92BV=0AH=0
-
MB93
-
BLPs->LBwrite Bottom Line Pixels into Line BufferLB->topBread from Line Buffer and write into top BufferRAThe Read address of top line bufferWAThe Wirte address of top line buffer
MB1 DCTX
MB2 DCTX
pMB2AHMBV
256 clocks
GOB1 start signal
256 clocks
MB44 DCTX
pMB44AHMBV
256 clocks
pMB1AHMBV
16 clocks
NMB1BLPs->LB (WA=0~15)
1 clocks
16 clocks
MB1 DCTX
NMB43BLPs->LB (WA=672~687)
GOB2 start signal
MB3 DCTX
RA=0~15
3 clocks
pMB1AHMBV
256 clocks
1 clocks
MB2 DCTX
pMB2AHMBV
256 clocks
NMB1BLPs->LB (WA=0~15)
16 clocks
errordet
257 clocks
RA=16~31
256 clocks
1 clocks
NMB44BLPs->LB (WA=688~703)
16 clocks
2 clocks
RA=0
BV=0AH=0
LB->topB
LB->topB
2 cks
257 clocks
1 cks
256 clocks
42 clocks
1 clocks
GOB3 start signal
1 clocks
WA=0
RA=0~15
LB->topB
2 clocks
-
MB92MB93MB46MB47
RA=0~15
MB1 DCTX
pMB1AHMBV
WA=0
LB->topB
255 clocks
254 clocks
256 clocks
MB2 DCTX
pMB2AHMBV
256 clocks
NMB1BLPs->LB (WA=0~15)
16 clocks
RA=16~31
2 cks
256 clocks
LB->topB
256 clocks
MB3 DCTX
pMB3AHMBV
256 clocks
NMB2BLPs->LB (WA=16~31)
16 clocks
RA=32~47
256 clocks
1 ck
LB->topB
256 clocks
1 ck
MB4 DCTX
pMB4AHMBV
256 clocks
NMB3BLPs->LB (WA=32~47)
16 clocks
RA=48~63
256 clocks
LB->topB
256 clocks
1 ck
2 cks
2 cks
Load MV
7 clocks
GOB2_MB3_output=adaptive_algorithm_output
Median vector
21 clocks
bilinear
MB5 DCTX
256 clocks
1 ck
bilinear
pMB5AHMBV
256 clocks
NMB4BLPs->LB (WA=48~63)
16 clocks
RA=64~79
256 clocks
LB->topB
256 clocks
256 clocks
1 ck
bilinear
1 ck
2 cks
2 cks
RA=80~95
256 clocks
LB->topB
256 clocks
1 ck
MB6 DCTX
pMB6AHMBV
256 clocks
1 ck
256 clocks
Load MV
7 clocks
GOB2_MB2_output=frame_memory_output
Median vector
21 clocks
Load MV
7 clocks
Median vector
21 clocks
GOB2_MB4_output=adaptive_algorithm_output
NMB5BLPs->LB (WA=64~79)
16 clocks
-
Cell_based
-
Cell_based RTL-HDL (Gate-Level) (Place & Route) CCadence Verilog-XLSynopsys Design AnalyzerRTL-HDL Cadence Verilog-XLCadence Silicon EnsembleCadence DraculaTimeMill/PowerMill
-
RTL Level Simulation , Gate Level Simulation 50MHzPost Simulation (TIMEMILL )TT Corner :253.3V 50MHz FF Corner 253.3V 50MHz 03.6V 50MHz SS Corner 253.3V 50MHz 1253.0V 50MHz FS Corner 253.3V 50MHz SF Corner 253.3V 50MHz
-
bilinearout[7:0]Memin[7:0] Frametype[1:0]
-
error_occure_nextgob
nextmb
recmvx[4:0]
recmvy[4:0]
EXTVDD
out[7:0]
preset
errordet
gobstart
mbdatastart
Frametypr[0:1]
bilinearout[7:0]
clk2
INTVSS
Memin[7:0]
mba[3:0]
mba[4:5]
dctin[11:0]
EXTVSS
cbp[2:5]
mvyin[4:0]
cbp[1:0]
mvxin[4:0]
INTVDD
rambottomin[7:0]
ramtopin[7:0]
-
100 pins99 pins
-
45~135I or P framePSNRDMVE0.3~0.4dB PSNR2dBVLSI(CIC)(S35-91F)