a critique of jpeg decoding_2

7/30/2019 A Critique of JPEG Decoding_2

1/3

A Critique of JPEG Decoding

INTRODUCTION

In todays world JPEG is the most common format for storing and transmitting images on the World

Wide Web, which is why any device supporting internet usage needs to be capable of JPEG

Decoding. The name "JPEG" stands for Joint Photographic Experts Group, the name of the

committee that created the JPEG standard and also other still picture coding standards. The JPEG is a

DCT-based standard where the encoding process comprises of Forward DCT of image data followed

by quantization and then by Entropy encoding. The Decoding of JPEG being the reverse process.

The Reconstructed image data obtained after decoding is in YCbCr format which still needs to be

color converted to RGB format before being used by the display.

BITSTREAM SYNTAX AND DECODING PROCESS

There are four distinct modes of operation under which the various coding processes are defined:

sequential DCT-based, progressive DCT-based, lossless, and hierarchical, where sequential DCT-

based mode is the most common one and is the subject of this papers discussion.

A jpeg file typically consists of Markers followed by the encoded data, where the encoded data may

be present in one scan (as in sequential mode) or in several scans (as in progressive mode). Markers

serve to identify the various structural parts of the compressed data formats such as Huffman tables,

quantization tables and frame data. The initial task of any decoder would be to parse the markers for

getting a summary of the encoded data that follows such as height, width, subsampling information

for allocating the various resources needed for the actual decoding. The JPEG decoding process is aninverse transformation where the encoded data is first decoded and restored to 8 pixels by 8 pixel


2/3

data blocks using a preliminary Decoding stage. This stage includes both Huffman Decoding and

Run-Length decoding two distinct coding schemes. Next the Quantization table specification is used

to approximately regain the spectral components of the image block, while low frequency

components may be fully restored the high frequency components may be severely distorted

however this distortion is barely perceptible. Finally the Inverse Discrete Cosine Transform

approximately recovers the original 8x8 data block. Now all the previous computations are constant

time operations i.e. for a particular implementation (hardware and software configuration) they take

the same machine instructions for any 8x8 encoded input. The process of color conversion is also

similar in nature, where a particular implementation takes constant time for any kind of input.

The total time T1 for decoding a jpeg file is hence

Where,

b is the time taken for decoding a macroblock ,

N is the total number of Minimum coded Units [MCU][1],

is the total time taken for parsing the various markers and frame data.

The Color space conversion is a time consuming operation. It has been reported that it consumes up

to 40% of the processing time of a highly optimized decoder [3, 2]. The total time T2 taken for color

space conversion by any decoder is

Where, c is the totaltime taken for converting one MCU.

COMMERCIALLY AVAILABLE MULTIMEDIA HARDWARES

The commercially available multimedia hardwares now a days have hardware blocks for IDCT,

Entropy Decoding and other video computations. These blocks are clubbed together in a subsystem

where they can be pipelined together. For Color conversion a separate hardware is provided, which

can either be pipelined with the video computation blocks or is present in a separate subsystem

where it cannot be directly pipelined with the video computational blocks. In such cases where the

direct pipelining is not possible, the host processor needs to configure the Color conversion

hardware to access the output of the decoder from external memory.

DECODING MODES

The interaction between the decoder and color convertor can be in two modes namely, Image mode

and Segment/Slice mode. In Image mode the complete image is first decoded to YCbCr format,

which is then color converted, hence the total time taken for the whole use-case is T1 + T2 and


3/3

memory ( proportional to the dimensions of the image) needs to be reserved for holding YCbCr and

RGB images. Alternatively the decoder can decode a segment of the image and while it is decoding

the next segment, the previous segment already decoded can be fed for color conversion. This mode

clearly has its advantages in terms of memory and execution time, since fixed size segments need to

be accommodated in memory and time can be saved due to the parallel execution of image

decoding and color conversion.

Assuming the exchanged segments between the decoder and color convertor, have a size ofk MCUs

(hence total segments) and is the time taken for configuring and starting the color convertorfor operating on a segment then, the time taken for decoding a segment is (excluding the timetaken for header parsing) and the time taken for color conversion of the segment is . Now,

if () ,

if

(

) ,

if ( ) .

EXTENT OF PIPELINING

Assuming in equations (3), (4) and (5) values and are optimized (to the maximum possiblelevel), then Total time depends on the choice of

and on the value of

. The value

heavily

depends on the placement of the color conversion hardware since if pipelining (at the hardware

level) is possible, would be minimum whereas if the host processor intervention is required, then shall include the latency of the host processor and external memory read time which is usuallygreater than the hardware pipelining. The value depends on the minimum size of the decoderoutput and color convertor input, if both hardwares can operate at the MCU level then the value of

can be brought to 1. Hence the total time is least, when color conversion block and decoder arehardware pipelined and interact with individual MCUs.

REFERENCES

[1] jpegstandard

[2] M. Bartkowiak. Optimisations of Color Transformation for Real Time Video Decoding. In Digital

Signal Processing for Multimedia Communications and Services, September 2001.

[3] F. Bensaali and A. Amira. Accelerating Colour Space Conversion on Reconfigurable Hardware.Image and Vision Computing, 23:935942, 2005.

a critique of jpeg decoding_2

Documents