human visual system inspired color space transform in...

Download Human Visual System Inspired Color Space Transform in ·

Post on 27-Feb-2019




0 download

Embed Size (px)


Human Visual System Inspired Color SpaceTransform in Lossy JPEG 2000and JPEG XR Compression

Roman Starosolski

Institute of Informatics, Silesian University of Technology,Akademicka 16, 44-100 Gliwice, Poland

Abstract. In this paper, we present a very simple color space trans-form HVSCT inspired by an actual analog transform performed by thehuman visual system. We evaluate the applicability of the transform tolossy image compression by comparing it, in the cases of JPEG 2000 andJPEG-XR coding, to the ICT/YCbCr and YCoCg transforms for 3 setsof test images. The presented transform is competitive, especially forhigh-quality or near-lossless compression. In general, while the HVSCTtransform results in PSNR close to YCoCg and better than the mostcommonly used YCbCr transform, at the highest bitrates it is in manycases the best among the tested transforms. The HVSCT applicabilityreaches beyond the compressed image storage; as its components arecloser to the components transmitted to the human brain via the opticnerve than the components of traditional transforms, it may be effec-tive for algorithms aimed at mimicking the effects of processing done bythe human visual system, e.g., for image recognition, retrieval, or imageanalysis for data mining.

Keywords: image processing, color space transform, human visual sys-tem, bio-inspired computations, lossy image compression, ICT, YCbCr,YCoCg, LDgEb, image compression standards, JPEG 2000, JPEG XR

1 Introduction

For natural images, the correlation of the RGB color space primary color com-ponents red (R), green (G), and blue (B) is high [18]. Correlation results fromthe typical characteristic of RGB images and reflects that the same informa-tion is contained in two or all three components. For example, an image areawhich is bright in one component usually is also bright in others. Computergenerated images also share such characteristic, since artificial images mostlyare made to resemble natural ones. Recent image compression standards: JPEG2000 [28,9] (as well as the DICOM incorporating JPEG 2000 [15]) and JPEG

NOTICE: this is the authors version of a work that was subsequently published inS. Kozielski et al. (Eds.): BDAS 2017, CCIS 716, pp. 564-575, 2017. The final publi-cation is available at Springer via 44.


2 Roman Starosolski

XR [4,10] compress independently the components obtained from an RGB imageby using a transform to a less correlated color space. Although the independentcompression of transformed components is not the only method for color imagecompression, it is the most frequently used one. It allows to construct a colorimage compression algorithm based on a simpler grayscale image compressionalgorithm. As compared to compressing the untransformed components, by ap-plying the color space transform we improve the image reconstruction qualityor the lossless compression ratio (for lossy and lossless algorithms, respectively),since without the transform the same information would be independently en-coded more than one time. However, alternative approaches are known that takeadvantage of inter-component correlations while encoding of untransformed ortransformed components [1,6,7].

In this paper we present a human visual system inspired color space transform(HVSCT) for lossy image compression. We evaluate this transform by comparingit for 3 sets of test images and 2 image compression standards (JPEG 2000 andJPEG-XR) to transforms ICT/YCbCr and YCoCg.

The reminder of this paper is organized as follows. In Section 2 we discussproperties of irreversible color space transforms and present the ICT/YCbCrand YCoCg transforms used then for comparison with HVSCT. Section 3 in-troduces the new transform. Section 4 contains experimental procedure, results,and discussion; Section 5 summarizes the research.

2 Color space transforms

The Karhunen-Loeve transform (KLT) is an image-dependent transform thatfor a specific image is constructed by using the Principal Component Analysis(PCA), it optimally decorrelates the image [16,18]. The computational time com-plexity of PCA/KLT is in practice too high to compute it each time an imagegets compressed. Instead, fixed transforms are constructed based on PCA/KLTby performing PCA on a set of typical images. Then, assuming that the set is suf-ficiently representative also for other images, which were not included in the set,we use the obtained fixed KLT transform variant for all images. The frequentlyused color space transforms, for example, the YCbCr color space transform de-scribed below, are fixed transforms constructed based on PCA/KLT; however,there are algorithms constructing a color space transform for the specific image.An adaptive selection of the transform from a large family of 60 simple trans-forms was proposed by Strutz [26]; performing the selection slightly increasesthe overall cost of the lossless color image compression algorithm. In [27], aneven larger family of 108 simple transforms is presented; adaptive transform se-lection is performed for the entire image or for separate image regions, however,the latter approach leads to only a small further ratio improvement. Singh andKumar [19] presented an image adaptive method of constructing a color spacetransform based on the Singular Value Decomposition. Although this methodis of significantly greater computational time complexity, than a method whichdirectly selects a transform from a family of simple transforms, it is still simplerthan computing PCA/KLT for a given image.

Human Visual System Inspired Color Space Transform 3

The probably most commonly used color space, but the RGB space, isYCbCr. It was constructed using PCA/KLT, but with an additional requirement:the transform should contain a component that approximates the luminance per-ception of the human visual system [14]. YCbCr contains the Y component thatrepresents the luminance and two chrominance components: Cb and Cr. YCbCrwas constructed decades ago for video data and nowadays is used both for videoand for still image compression. There are many variants of the transform be-tween RGB and YCbCr (resulting in respective variants of the YCbCr colorspace). Below we present one of them, ICT (Eq. 1), with inverse (Eq. 2):YCb


= 0.29900 0.58700 0.114000.16875 0.33126 0.50000

0.50000 0.41869 0.08131


, (1)RGB

=1.00000 0.00000 1.402001.00000 0.34413 0.71414

1.00000 1.77200 0.00000


. (2)ICT is defined in the JPEG 2000 standard for lossy compression [10].

Note, that if the transformed components are to be stored using integer num-bers, then the transform is not exactly reversiblewe say that it is irreversibleor not integer-reversible. It is not a problem in a typical case of lossy coding,where distortions introduced by forward and inverse transform are much smallerthan distortions caused by lossy compression and decompression. However, inthe case of the very high quality coding, the color space transform may limit theobtainable reconstruction quality. The integer-reversible variants of ICT and ofother transforms are constructed using the lifting scheme [2]. The reversibil-ity is obtained at the cost of the dynamic range expansion of the transformedchrominance components by 1 bit (the dynamic range of a component is de-fined as a number of bits required to store pixel intensities of this component).The dynamic range expansion affects the transform applicability, since certainalgorithms and implementations either do not allow or do not process efficientlyimages of depths greater than, e.g., 8 bits per component. Expansion may beavoided by the use of modular arithmetic (as in the RCT transform in the JPEG-LS extended standard [8]), however, such transform introduces sharp edges totransformed components, that worsen the lossy compression effects. In this re-search we focus on typical transforms for lossy codingnot using the modulararithmetic and not expanding the dynamic range of transformed components.

A recent YCoCg transform is an another interesting transform (forward inEq. 3 and inverse in Eq. 4):YCo


= 1/4 1/2 1/41/2 0 1/21/4 1/2 1/4


, (3)RGB

=1 1 11 0 1

1 1 1


. (4)

4 Roman Starosolski

It was obtained based on PCA/KLT constructed for a Kodak image-set (seesection 4.1 for the Kodak set description); YCoCg is an irreversible variantof a YCoCg-R transform included in the JPEG-XR standard [10,14]. The YCoCgtransform is significantly simpler to compute, than ICT. The former requires 15simple floating point operations (additions, subtractions, multiplications) forforward and 8 operations for inverse transform. The YCoCg forward transformmay be computed in 6 integer operations (add, subtract, and bit-shift; the latterdenoted by >>):

t = (R + B)>>1; Y = (G + t)>>1; Co = R t; Cg = Y t;

inverse in 4 additions and subtractions only:

G = Y + Cg; t = Y Cg; R = t + Co; B = t Co;

3 New transform inspired by human visual system

We described in detail previously [21] the following interesting fact. A color spacetransform that results in a single luminance and 2 chrominance components isperformed by our (i.e., human) visual system. There are three types of conecells in our retinas that are most sensitive to three light wavelengths, theseare S-cones (short wavelength with sensitivity peak in violet), M-cones (middlewavelength, sensitivity peak in green), and L-cones (long wavelength, peak inyellow). According to the common opinion, the cones simply respond to blue(S-cones), green (M-cones), and red (


View more >