a hardware-friendly wavelet entropy codec for scalable video hendrik eeckhaut elis-paris ghent...

A hardware-Friendly Wavelet Entropy Codec for Scalable video

Hendrik EeckhautELIS-PARIS

Ghent UniversityBelgium

2

Scalable video

Encodeonce

Decode according torequired QoS or

available hardware resources.

Quality ~ deployed hardware resources

3

Overview• Scalable video codec

• Scalable Wavelet Entropy Encoding :• New algorithm

• Memory footprint

• Compression results

• Conclusions

4

Overview video codec

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

originalframes

Exploittemporal

redundancy

Exploitspatial

redundancy

Exploitstatisticalproperties

5

Motion estimation

vectors + error frame

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

6

Motion compensationReference framecurrent GOP

Reference framenext GOP

timeScalability in time

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

7

Wavelet transform

Scalability in resolution

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

8

Wavelet Entropy Coder

• Scalability:• Quality

• Resolution

• Good compression

• Economical with memory

• High degree of parallelism

• Sweet and simple

Wavelet Entropy Codec

• Actual compression of frames• Tight coupling of pixel characterization and

Arithmetic coding (modified arithmetic encoder of CABAC)

ModelSelector

ArithmeticEncoder

Entropy Encoding

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

9

Scalability in quality

Scalability in quality

10

Parallelism

In parallelinParallel

Easy to omit subbands → resolution scalability

11

Algorithm

Bitplane

Significance bitmap

Sign bitmap

Contextcharacterization Model Selection

Aritmetic Encoder

0.3 0.7

0.4 0.6

0.45 0.55

0.5 0.5

0.6 0.4

0.7 0.3

Significance bitmap

Sign bitmap

Current pixel

0.3 0.70.80.2

12

Algorithm

Bitplane

Significance bitmap

Sign bitmap

13

Compression gain

Context/Arithmetic models64 different models: each model needs only a 9 bit state

• 1 data model

• 27 sign models

• 27 significance models

• 8 special highest bitlayer significance models

• 1 refinement model

64 for each type of subband:480 subband models

[LL,HL,LH,HH] [resolution layer][color channel] [temporal level]

~> 30.720 different models in total

0

1

1/2

real probability

time

unitialised

trained probability

• Models are initialised (warmed up)with values obtained from training a number of representative sequences

14

Memory requirements

• Arithmetic Decoder:• State Lookup table: 256B

• Range Lookup table: 2048B

• Model state table: 64B

• Buffer

• Model Selector• Significance and sign bitmap: 2 x 3168B

• A few buffers• (1 MRAM block is large enough to contain multiple

significance and sign bitmaps)

Arithmetic DecoderModel selector

Name Number Size (byte + parity)MRAM 2 64kM4K 138 512M512 224 64

Registers 25660 1/8214kiB

Altera Stratix S25

~7 M4K block

~13 M4K blocks

15

Results

0 5000 10000 15000 20000 250000

5

10

15

20

25

30

35

40

45

50

55

60

Average PSNRMobile CIF, 289 frames, 30Hz

WEEQTL

bitrate (kibps)

PS

NR

(d

B)

QTL: QuadTree Limited (comparable compression as EBCOT of jpeg2000)

16

Conclusion• Algorithm is simple

• No recursive behaviour

• Small memory footprint[ 2 bitmaps of size(wavelet subband) ]

• Memory access is very regular

• Arithmetic coder must process a lot symbols, but is simple and fast

• Compression is better than QTL-algorithm for all bitrates

TODO: We are working hard on aneffective FPGA-implementation of the decoder

17

18

Arithmetic Coding

Contextcharacterization Model Selection

Aritmetic Encoder

0.3 0.7

0.4 0.6

0.45 0.55

0.5 0.5

0.6 0.4

0.7 0.3

Significance bitmap

Sign bitmap

Current pixel

0.3 0.70.80.2

19

Scalable video (animated)

Encodeonce

Decode according torequired QoS or

available hardware resources.

Quality ~ deployed hardware resources

20

Overview video codec

MotionEstimation P

ack

EntropyEncoding

WaveletTransform

Motion VectorEncoding

MotionComp.

EntropyDecoding

Motion VectorDecoding

Unpack

Pullbitstream

InverseWavelet T.

originalframes

decodedframes