transaction / regular paper titleor.nsfc.gov.cn/bitstream/00001903-5/486836/1/10000143… · web...

IEEE TRANSACTIONS ON JOURNAL NAME, MANUSCRIPT ID 1

A 3D Image Quality Assessment Method based on Vector Information and SVD of

quaternion matrix under Cloud Computing Environment

Xingang Liu, Member, IEEE, Lan Zhangand Kaixuan Lu

Abstract—With the increasing demands of end-users to the visual perception in three-dimension (3D) image, quality assessment for 3D imageis dominantly required as the feedback information for multimedia transmission systems. In this paper, a novel full-reference quality assessment method by considering the depth and integral color information of 3D image under cloud computing environment is proposed. Based on the property of the depth information in 3D image, the depth map is firstly separated into different planes according to the perception of human visual system (HVS). Then, after express the image pixels of every separated plane through quaternions, the structural and energy information are separated by quaternion singular value decomposition (QSVD). The distortion of structural and energy in every plane are calculated in various formulas respectively. The final result is calculated in terms of the global score, which synthesizes the structural and energy distortion scores in every individual depth plane. It should be pointed out that the chrominance information is employed in our mechanism to evaluate the color image quality because of its useful characteristic for 3D color image quality assessment, and its spatial correlation is used for calculating structural distortion through vector cross-product. Our experimental results confirm that the proposed method has achieves better performance under cloud computing environments compared with other existing 3D image quality assessment methods.

Index Terms—three dimension (3D) image; depth information; quaternion singular value decomposition (QSVD); structural information; color image quality assessment; cloud computing

————————————————————

1 INTRODUCTIONITHthe rapid multimedia technology devel-opment, the research on 3D image (3D) is

becoming more and more multiple. 3D image quality assessmentbecomes one of the fundamen-tal research points in the multimedia communica-tion systems, since image quality is a crucial in-dex for evaluating the algorithms in image pro-cessing. In general, image quality assessment methods consist of subjective quality assessment (SQA) methods and objective quality assessment (OQA) methods [1].Compared with OQA, SQA is more in agreement with human visual system (HVS) on account of it evaluates the image quality by human beings directly [2]. However, the partic-ipation of observers in the SQA procedure make-sitnot only time-cost and complicated for opera-tion but also cannot be incorporated into auto-matic image system because it needs number of observers view and mark the evaluated score in

W terms of their visual perception for each image. On the other hand, OQA measures the image qual-ity without observers participating but utilizes the mathematical methods to computeimage quality scores. For OQA, although it is not as accurate as SQA, its simpler implementation means it can pre-dict image quality automatically which makes it much more popular. Traditionally, OQAis catego-rized into full reference (FR), reduced reference (RR) and no reference (NR) according to the em-ployed information [3] described as follows:

Full reference (FR): assess the image quality by making a comparison between the original and distorted images;

Reduced reference (RR): not all but some infor-mation of the original and distorted images are compared to establish the IQA function;

Non-Reference-Free (NR): only the information of the distorted images is utilized in the IQA function.

It is confirmed that FR is the most accurate method among the three categories since it uses

xxxx-xxxx/0x/$xx.00 © 200x IEEE Published by the IEEE Computer Society

————————————————

X. Liu, L. Zhangand K. Lu are with the school of Electronic Engineering, University of Electronic Science andTechnol-ogy of China, Chengdu811731, China.

E-mail: [email protected].

Please note that all acknowledgments should be placed at the end of the paper, before the bibliography(note that corresponding authorship is not noted in affiliation box, but in acknowledgment section).

2 IEEE TRANSACTIONS ON XXXXXXXXXXXXXXXXXXXX, VOL. #, NO. #, MMMMMMMM 1996

enough information both from the original and distorted images. And the proposed method in this paper is a FR method.

Among the numerous FR OQA methods, Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) [4] are most widely known because of their simple formulations which calculate the distortion between the independent reference im-age pixels and corresponding distorted image pix-els. Many previous tests had been demonstrated their limitations in 2D images, which will also oc-cur while applying to 3D images [5]. Besides, MSE and PSNR do not take the perception of hu-man visual perception into consideration, so that they do not have satisfied correlation with the 3D images subjective scores distinctly. To further op-timize the objective quality assessment algo-rithms, many other objective metrics have been proposed afterwards, such as: Universal image Quality Index (UQI) [6], Structural Similarity In-dex measurement System (SSIM) [7], SVD-based Measure for local and global assessment (SVD) [8], discrete wavelet transformation (DWT) ap-proach [9], and discrete cosine transformation (DCT) measurement [10] and so on. Both UQI and SSIM metrics measure image quality through structural distortion and SVD-based, DCT-based, DWT-based approaches all attempt to model the people’s neuron responses with decompose an im-age into several basis images equivalently by transforming the image information into different domains. The above five methods are all in allu-sion to 2D image OQA and had been proved that they have better performance than the MSE and PSNR on 2D image databases.

Since the main function of the HVS is extract-ing structural information from the viewing field, the OQA method named as SSIM proposed by Wang et al. has been widely accepted [7]. It takes image structural information into consideration instead of the error sensitivity-based measure-ment for quality assessment. Besides, its block-based distortion measurement can effectively eliminate the blocking artifact.However, SSIM fails to measure blurred images whose texture is not obvious or severe fuzzy. In [8], Eskicioglu et al. proposed a SVD-based method for grayscale image quality assessment which based on M-block singular value decomposition. The singular values are superior in stability and capable of character-izing the energy information of image, so it can display the change of image quality commendably. However, it is not HVS-based and ignores the cor-responding unitary matrix which denotes the spa-tial distribution of the image energy. Moreover, itonly concentrates on people’s sensitivity to lumi-nance information but ignores the chrominance information. Besides, this method aims to evalu-

ate 2D image quality and does not take 3D image propertiesinto consideration. Many similar pro-posals and related improvement based on SVD were released afterwards [11-16], but all of them are just for 2D images and have a poor perfor-mance when apply to 3D image quality assess-ment.

With the stereoscopic technology developed, 3D image replaces 2D image and becomes a re-search point in the modern word. It is the fact that the stereoscopic image is one of the popular 3D image formats, which consists of two images (left and right views) captured by the closely lo-cated two cameras where the visible difference between the two images is regarded as the dis-parity [17]. By making use of the two different perspective images, people can perceive the depth information, which is the principle factor that produces the stereo effect. Therefore, one straightforward OQA for 3D image is measuring the quality of the left and right views with 2D im-age quality assessment methods separately, and then the average value of the above two scores is served as the global measure [18]. However, this kind of method is ineffective because it does not consider the stereoscopic distortion and the inter-action between the two views when people ob-serve 3D images. Another research direction is applying 2D image quality assessment methods to evaluate each view on the basis of considering the depth or disparity information of 3D image ac-cording to HVS [19], [20], [21]. Since this kind of method takes global stereoscopic distortion into account fully and improves the accuracy of OQA for 3D images substantially, it becomes one of the popular research focuses.

Recently, with the processing ability and power support of some processing terminal are very lim-ited, which are not suitable for real-time signal processing, cloud computing network becomes very popular. In the past several years, the num-ber of applications over cloud computing network has increased amazingly where image processing is among them. There are some cloud-based qual-ity assessment for medical image in the field of image processing [22-24], and refer to these methods we can research the 3D OQA methods under the cloud computing environment.

To provide a more efficient OQA method for 3D color image under cloud computing environments, this paper proposed a novel method, which based on depth perceptual property of HVS with consid-ering the spatial correlation of the basis images generated by singular value decomposition of quaternion matrixes (QSVD). The organization of this paper is as following. In section II, the theo-ries of the related algorithms employed in this pa-

AUTHOR: TITLE 3

per are introduced. Then, the description of the proposed method is presented in section III. To confirm the accepted performance of our pro-posal, section IV releases the experimental results and performance analysis. Finally, the conclusions are drawn in section V.

2 BACKGROUNDMost of the previous IQA methods focus on the

mathematicalmodeling by using luminance infor-mation even while evaluating the quality of color image.But in [25], the authors proved that com-bined luminance and chrominance information into the IQA method can make it perform better, which implies that the chrominance information is supposed to be taken into consideration when evaluate color image quality. To describe the chrominance information adequately, the quater-nions are employed in this paper.With the employ-ment of quaternion, not only the distortion of lu-minance but also the distortion of chrominance in-formation of color pixels can be integrated into the measurement.Meanwhile, the singular value decomposition of quaternion matrix is exploited and the products of QSVD (singular values and singular vectors) are all used to quantify the per-ceptual quality. In this section, the introduced background including the essential attribute of quaternion matrix, the property of quaternion-mathematic calculation, the QSVD and the origi-nal M block SVD-based method are introduced.

2.1 QuaternionQuaternion is one simple type of hyper complex

numbers, which is an expansion of complex num-bers.A complex number consists of a imaginary number, and a quaternion extends the expression of imaginary number from one-dimension to three-dimension space. Therefore, a quaternion consists of one real number and three imaginary numbers which can be described as following equation show.

(1)

Where r, a, b, and c are all real numbers, , , and are the imaginary numbers of the quater-nion. The interrelationships among the three imaginary numbers are shown as following.

(2)

A pure quaternion is the case with the first real number equals to zero (r=0), which is utilized to represent a color pixel for image processing fre-quently. The three parameters (red, green and

blue) used to describe the information of color im-age pixels can be exploited as the coefficient of the imaginary numbers in a quaternion [26]. Therefore, a color image pixel is expressed with the pure quaternion as following.

(3)

Where r(x,y),g(x,y),b(x,y) mean the red, green and blue of the pixel (x, y) of the color image, respectively. A mxncolor image could be presented with a mxn-matrix whose elements are all pure quaternions as Eq.(3). Let , be pure quaternion numbers, their multiplication as shown in the following equation can be obtained by using the laws given in Eq. (2).

(4)

Here, are all pure quaternions. and are the standard dot and vector cross prod-

ucts of the two vectors respectively [27]. Eq.(4) shows that any multiplication of two pure quater-nions consists of two parts: scalar part (standard dot product) and vector part (vector cross prod-uct). It is the fact that the vector part represents spatial rotation and much more spatial transfor-mation, which indicates that the production of quaternions can be exploited to calculate the structural distortion [28].

2.2 The SVD-based methodSingular value decomposition is a famous trans-

formation in linear algebra with expanding the conventional diagonalization of unitary matrix to the field of matrix analysis. Through the decompo-sition, we can decompose the images into several basis images and abandon the unimportant infor-mation to reduce the dimensionality and complex-ity in the calculation. The SVD of a matrix A can be defined as Eq.(5).

(5)

Where,S is a diagonal matrix which elements on diagonal are singular values of matrix A,andU, Vareunitary matrixes.SVD is widely applied in im-age processing such as image coding, filter de-signing, watermarking, IQA and so on.


There are many IQA methods based on SVD, and among them the most classical method is mentioned in [8] which will be briefly introduced as following. The method proposed in [8] is a block-based (block size is 8x8) algorithm which mainly concentrated on processing singular val-ues generated from SVD of image blocks. The dis-tance between the singular values of the original image block and distorted image block indicates the local distortion of image block. Finally, the lo-cal distortions of every image block are integrated into a global distortion which is used as the nu-merical index for objective measurement.

The local distortion obtained as following Eq.(6).

(6)

Where,si and delegate the singular values of pixel block in the original and distorted image re-spectively. Dt representsthe distortion degree that the tth block of the distorted image deviated from the corresponding original image block.

The numerical measurement of global distor-tion is computed as Eq.(7).

(7)

Where Dmid delegates the median of Dt, where t represents the index of the blocks with value range from 1 to N. Here, denotes the total number of the blocks in the image.

2.3 The quaternion singular value decompositionThe original SVD method is concentrated on grayscale images and could not be apply into color images without the quaternion model. Based on the definition of SVD on the complex adjoint matrix, SVD can be extended to quaternion case which is named quaternion singular value decom-position (QSVD). As [16] mentioned every quater-nion matrix can be decomposed into three ma-trixes as following Eq.(8).

(8)

Where UHmxm, VHnxn, are quaternion unitary matrixes and elements of these two matrixes are all quaternions. The columns vector of U and V are the left and right singular vectors of quater-nion matrix Q. r is a real diagonal matrix and its diagonal elements 1,2,…,r are singular values of quaternion matrix. The singular values of Q are all real numbers and r is the rank of Q. The Eq.(8) is the QSVD of quaternion Q and it can be applied to color image coding, color image classification, im-age transmission and color IQA. From Eq.(8), we can know that the QSVD can decompose the color image into two parts: one is color information (singular value) and the other is structure infor-mation (singular vector). The property of separat-ing structure and color synchronously is the pri-mary reason that QSVD is exploited in the pro-posed method.

3 THE DESCRIPTION OF THE PROPOSED METHODAs HVS is the final receiver of images, many OQA methods have considered the property of HVS and utilized features of image to characterize the per-ception of HVS. Besides, many previous surveys had been conducted to investigate the influence that different factors impact on the performance of OQA methods. It had been found that two prin-cipal factors can influence the validity of the OQA methods: 1) Feature detection and extraction; 2) Effectivity of feature pooling method. In addition, people also verified that HVS is sensitive to image structural changes at edges. Therefore, the key

Fig. 1. The system diagram of our proposed method

AUTHOR: TITLE 5

point in OQA is searching for an effective repre-sentation for structural feature. For the purpose of comprehensive detecting structural feature, many OQA methods transform the original image pixel blocks into other domains [8-10]. Compared with other domain transform methods, SVD is su-perior in representing image structural informa-tion with its basis images is sensitive in capturing any variations of images. The products generated from SVD can represent complete information of image blocks:U and V in Eq.(5) are capable of rep-resenting the image structural distortion and S can denote the activity level of image. Meanwhile, the productions of SVD can have a general prefer-able view on any changes of image quality. There-fore, the adhibition of SVD can qualify the distor-tion of images preferably.

Based on the SVD, the proposed method ex-tends the application of SVD to color space through QSVD. In this paper, the productions of QSVD are all reserved and employed into calcula-tion which means it is convenient to fuse the structural distortion and energy (color) distortion together as an overall measurement. In spite of the huge computational cost of structural compar-ison, we can make our proposed method be suit-able for real-time system through calculating the structural comparison under the cloud computing environment.

The system diagram of our proposed OQA model is shown in Fig.1. Qr and Qdrefer to image pixel blocks expressed with the pure quaternion matrix in reference and distorted 3D color im-ages. In this paper, the image blocks are sepa-rated into different layers with different levels of importanceaccording to the depth values of blocks. The distortion of every depth layer is mea-sured by a unitary measurementMQSi, which in-cludes two comparisons: M_QSVD and QS. The first comparison is M_QSVD: energy distortion of pixel blocks between reference and distorted im-ages; the second is QS: structural distortion of pixel blocks between the reference and distorted images. They are calculated with different ap-proaches – the first is calculated with singular val-ues Sr, Sd and the last is calculated with singular vectors Ur, Ud, Vr, Vd. Finally, the numerical mea-surementsofdistortiondefined on different layers are integratedinto a unitary measurement MQS through optimize operation.Compared with other previous work the proposed method has several peculiarities as follow shows:

1)In allusion to 3D IQA method, the HVS percep-tual characteristics to 3D scene are studied in our research process, which shows that the depth information is important in HVS percep-tion. Thus, the depth information is capitalized

on to partition the 3D image blocks into differ-ent groups in the proposed method.

2) The image visual distortion is quantified by both the singular values and singular vectors generated by QSVD which can detect more fea-tures to represent image quality. This makes the proposed method more comprehensive and rea-sonable.

3) Structural degradation represented by singular vector in this paper is calculated with vector cross production, which contains much more structural information (such as spatial correla-tion) than dot operation.

4) The proposed algorithm is realized under the cloud computing environment, which makes the overall system is automatic.

In this section, the proposed system will be intro-duced in detail including: how partition the image according to depth information, how calculate the energy and structural distortion.

3.1 Partition images according to depthinformationIt is experimentally proved that 3D image depth information makes an important role in human be-ings’ perception. In 3D image, the distance be-tween the surface of an object and the viewpoint represents depth value, which indicates the visual ability of perceiving the world in three dimen-sions. Many researches on human physical and mental perception had verified that human visual perception to the depth information is variational along with the location of the pixel changed. When observe a 3D image, the larger the percep-tual depth value, the farther the object in an im-age away fromthe observer. This means it is harder to see the texture of object distinctly and the object is easier to be regarded as a back-ground. Conversely, the smaller the distance be-tween the object and observer, which signifies that the texture of object is easier to recognize and be regarded as a foreground.


Fig. 2. The block based disposal of depth information

To design an objective algorithm that is consis-tent with subjective human evaluation, the per-ceptual characteristics to 3D scene of HVS are taken into account for the sake of improving the accuracy of 3D IQA method. When people observe a 3D image, HVS divides proper visual regions into different depth layers with different sensitiv-ity according to its perceptual depth value. HVS can highly optimize and obtain the structural in-formation of the different depth layers in an im-age, which can be applied to IQA models. Based on this property, image pixel blocks are seg-mented into different layers in accordance with the image’s depth histogram, and their distortion on every layer is calculated disjunctively.

In 3D computer graphics, the information relat-ing to distance between the surfaces of objects in a scene and the viewpoint is stored in the depth map. Hence, according the depth values in depth map, the image blocks are separated into differ-ent layers with different weight factors. The layer with the highest weight factor is the region that human most interested in, which means distortion of this region has the greatest influence on HVS. Accurate partition of pixel blocks can help im-prove the precision of the proposed 3D OQA method with objectively precise distinguish the in-teresting regions of HVS.

With considering that the classical manipula-tionof singular values is based on 8x8block, the image also be disposed with 8x8block as the basic unit,which is evitable to destroy the image struc-ture and the relationship between the adjacent pixels.The image blocks are split into different layers in light of their corresponding average depth values of 8x8 blocks (shown in Fig.2)through the depth thresholds which are de-cided in accordance with the histogram of depth values.The thresholds located at the appeared troughs in the depth histogram. To illustrate this, an example is enumerated in Fig.3 which shows how determine the depth thresholds according to

the depth histogram. In Fig.3, (a) is the original image named “im3_l”, (b) shows the depth his-togram of “im3_l”. From it we can acknowledge that there are two depth thresholds at the troughs in the depth histogram and according to them we partition the original image into three regions. Fig.4 shows the human visual images of the differ-ent divided regions in different depth layers.For (a)-(c) in Fig.4, pixel blocks located at the present depth layer are marked with black and on other depth layers are marked with highlight. And for (d) in Fig.(4), the darker regions are closer to viewpoints than brighter regions.

(a). The original image “im3_l”(distortion type is blur)

0 50 100 1500

20

40

60

80

100

120

140

160

180

200

Depth value

Freq

uenc

y

Depth plane 1 Depth plane 2

Depth plane 3

Threshold 2

Threshold1

(b) The depth histogram of “im3_l”

3.2 Quaternion image processing3.2.1 Quaternion modelBased on the previous researches, people found that HVS is much more sensitive to luminance changes than chrominance changes. This is the primary reason that a multitude of OQA methods for color images are concentrated on the process-ing of luminance information. However, adding chrominance information into the process can produce a better effect, since chrominance can re-veal a portion of color image information. This pa-per expresses the color image pixels through quaternion which is first proposed to describe complex numbers in 3D space [29]. With the em-ployment of quaternion model, it is capable of ex-panding the calculation of grayscale image lumi-nance to color vector. Among the color vector methods, treating every color pixel as a quater-nion is a convenient approach compared with other approaches. Hence, in 3D color image, ev-ery color pixel is expressed with a pure quater-nion as Eq.(3) due to its effectivityin express-ingthe information of both luminance and chromi-nance.

AUTHOR: TITLE 7

100 200 300 400 500 600

50

100

150

200

250

300

350100 200 300 400 500 600

50

100

150

200

250

300

350

(a) depth plane 1 of “im3_l” (b) depth plane 2 of “im3_l”

100 200 300 400 500 600

50

100

150

200

250

300

350100 200 300 400 500 600

50

100

150

200

250

300

350

(c) depth plane 3 of “im3_l” (d) the overall depth visual image

Fig. 4. The perceptive vision of different depth planes of “im3_l”. (a)-(c) are the different planes of the original image. (d) the overall visual depth image of “im3_l”

3.2.2 Structure and color information segregation

(a) The original image

(b) The structure image denoted by UV from SVD

It is the fact that different types of distortion de-creasing the quality of image are all in the same way with influencing the image structure. Such image structural distortioncan be accountedwith the utilization of SVD by its unitary matrixes UandV in Eq.(5). The Ui and Vi in Eq.(5) are veri-fied can commendably represent the spatial distri-bution of the energy of images and be sensitive to perturbation. Any introduced distortion would change the unitary matrixes U, V apparently which makes unitary matrixes can be an effective measurement for IQA. Unitary matrixes repre-

senting the texture information of an image is shown in Fig.5. It comprehensively revealed that the singular vectors of SVD can represent the tex-ture information of a color image.

A large proportion of previous IQA methods with quaternion model only take advantage of in-formation expressed by the dot product of quater-nion and it results that the previous IQA methods cannot completely represent the image informa-tion [30]. However, as aforementioned Eq.(4) shows: the multiplication of quaternionsshould consider vector operation, which means the vec-tor operation ought to be involved in the proposed OQA method. It is possible to apply the vector op-eration into the calculation of quaternion matrix through QSVD. The productions of QSVD shown in Eq.(8), include singular values and singular vectors, which denote the image energy and structural informationrespectively. Energy infor-mation expressed by singular values (elements of r in Eq.(8)) represents the significance variation along with the frequency components change. It results that the singular values arecalculated only with dot operation according to the classical method in [8] since considering spatial vector op-eration is pointless. On the other hand, since the spatial information expressed through the vector cross product operation of quaternion singular vectors is profitable for calculating structural dis-tortion. Structural distortion expressed by singu-lar vectors is calculated through integrating the vector cross multiplication and dot operation of quaternion vectors instead of utilizing dot opera-tion barely. In this paper, the novel method dis-poses the unitary matrix with both vector and dot product included, which will be introduced in Sec-tion Ⅲ-4.

3.2.3 Processing of color informationAs aforementioned, the manipulation of singular values generated from QSVD includes only stan-dard dot product operation.Considering the well performance of the classicalSVD-based method in [8] which consists of dot operation merely; we de-veloped the processing of singular values gener-ated by QSVD as the SVD-based algorithm pro-posed in [8]. Singular values generated by QSVD supersede the singular values generated by SVD of grayscale image employed in the original method in [8]. Finally, the energy (color) degrada-tion is calculated with submitting the singular val-ues of QSVD into Eq.(6) and Eq.(7).

3.2.4 Processing of structural informationIn [25], the researchers defined two expressions: the dot and vector portion. Dot portion is the sum-mation of average numbers in each color channel over the image region. Whereas, the vector por-tion can be regarded as object in color image

(a) The left-eye images of the test images

(b) The right-eye images of the test images


which is the rest subtracting the dot portion from the pixel. In this paper, with considering the cal-culating limitation of quaternion vector cross product operation, the vector portion expressed by quaternion model for MxN image is defined in Eq. (9), (10).

(9)

(10)

, are the vector portions with representing the structural elements in reference and distortion images. , are the elements of quaternion unitary matrix generated from QSVD in reference and distortion images respectively. v(·) means ex-tracting vector components from a quaternion.

The classic method of calculating structural in-formation is the structural similarity index (SSIM) [7]. It calculates the structural distortion through scalar cross-correlation between two images x and y, which is defined as following.

(11)

The SSIM visual quality matrix includes three pa-

rameters: μis the average luminance of a grayscale image, σis the standard deviation of lu-minance, and σxy is the covariance of two images. In SSIM, authors mentioned the structural com-parison measure is defined as following:

(12)

Where C is a constant used for making the visual quality overcome much more stable. The above Eq.(12) is proposed for grayscale images and can be extended into color space. The structural infor-

mation of color image generally refers to some specialcharacteristic; therefore, the calculation of structural information involves spatial vector op-eration. Referring to Eq.(12), the measurement of structural comparison in color space is calculated as Eq.(13) shown in the proposed method.The main nonlinear comparison between reference and distortion images is represented withQS(x,y)

which includes both standard dot product and vector cross product of quaternion vectors gener-ated from QSVD.

(13)

, mean standard dot and vector cross product of quaternion vectors respectively. Eq.(13) is used for calculating the structural simi-larity between the reference and distorted color images, whereas Eq.(7) is used for calculating color degradation between distortion and refer-ence images. The two equations which have con-trary trends are used for calculating two different type numerical measurements. To integrate the

AUTHOR: TITLE 9

two numeral measurements, in this paper, we de-fines the pooling method that generates an overall measure index for every independent depth layer as the following Eq.(14). MQSi, i=1, 2,…, n, indi-cates the numerical measure index in different layers i by where i delegates the depth layer.

(14)

Where QSi is the structural measurement of ith

layer and M_QSVDi is the energy distortion of ith

layer from Eq.(7). The opposite tendency of the two parameters leads to their symbols are oppo-site which is shown in Eq.(14) that the sign of M_QSVDi is -1 when the sign of QSi is 1. And the global numerical measurement of the global dis-tortion is described with MQSas shown in Eq.(15).

(15)

Eq.(15) defines the pooling method that inte-grated the measure indexes of different layers into a global measurement. The weight factors αi is de-cided by the optimization function of the correlation coefficient between the objective scores and the subjective evaluation scores. The summation of αiis 1 (αi=1, i=1,2,..,n) and the values ofαi vary along with different distorted types of image.

3.3 Cloud environment for image processingThe introduction of quaternion vector cross prod-uct operation results in the computational process is miscellaneous, which means we should try to retrench the processing procedure. Besides, our proposed algorithm for the information process-ing and analysis to calculate the image quality scores has a high demand on computing power and memory. These requirements arise secondary demands with respect to data and computing availability and reliability. By the usage of cloud infrastructure these demands are specially ad-dressed, and the solution builds adaptive systems with on demand resources. Cloud computing is a model for enabling ubiquitous, convenient on-de-mand network access to a shared pool of config-urable computing resources that can be rapidly provisioned and released with minimal manage-ment effort or service provider interaction. This established cloud computing environment is focus on processing large data sets in our proposed al-gorithm. The computational resource in cloud en-vironment is generally organized as groups of vir-tual machines (VM), each of which is assumed to be able to compute independently. In this paper, we employ 10 VMs to operate our proposed method parallelly.With the employment of them, the system can be run in real-time.

TABLE 1PLCC COMPARED WITH OTHER METHODS

PLCC PSNR SSIM MSSIM VIFP SVD FSIM 3D_DCT PQM MQS

JPEG -0.1205

-0.4700

-0.5049

-0.4645 -0.277 -

0.6188 -0.4700 -0.5628

-0.779

2

JP2K -0.7728

-0.8611

-0.8832

-0.8749

-0.7905

-0.9088 -0.8365 -

0.8172

-0.928

8

FF -0.6607

-0.6529

-0.6829

-0.788

1

-0.5684 -0.687 -0.5890 -

0.6136-

0.7406

Blur -0.7750 -0.904 -

0.9196-

0.9266

-0.8820

7

-0.8231 -0.8140 -

0.8880

-0.949

9

WN -0.9245

-0.9326

-0.935

4

-0.8949

-0.9088

-0.9171 -0.4676 -

0.9253-

0.9276

ALL -0.6971

-0.8538

-0.8669

-0.8352

-0.7296

-0.8752 -0.6558 -

0.7269

-0.929

8

TABLE 2SROCC COMPARED WITH OTHER METHODS

SROCC PSNR SSIM MSSIM VIFP SVD FSIM 3D_DCT PQM MQS

JPEG -0.1183 -0.4311 -0.4558 -0.4494 -0.3629 -0.5756 -0.5271 -0.5531 -0.7493

JP2K -0.8013 -0.8537 -0.8756 -0.8848 -0.8190 -0.9038 -0.8657 -0.8085 -0.9168


4 RESULTS AND ANALYSISTo verify the performance of proposed novel algo-rithm, this paper makes a comparison between

the experimental result of proposed method and other OQA methods. Theexperiment database is the LIVE database phase I [31], which consists of five distortion types at different distortion levels: JPEG 2000 compression (JPEG 2000), white noise (WN), Gaussian blurring (BLUR), Fast Fading (FF), and JPEG compression (JPEG). Eight se-lected reference images (includes four left and four right images) exited in the database are shown in Fig.6. The subjective quality evaluation tests of the database are accomplished with a sin-gle-stimulus continuous quality evaluation (SS-CQE) [32] and the subjective scores are made public in the website [31].

30 35 40 45 50 55 60 65

-5

0

5

10

15

20

25

30

35

40

45

MQS

DM

OS

DMOS_vs. blurfitted curve

10 15 20 25 30 35 40 45 50

0

10

20

30

40

50

MQS

DM

OS

DMOS_vs. fffitted curve

48 50 52 54 56 58 60 62 64 66

-5

0

5

10

15

20

MQS

DM

OS

DMOS vs. jpegfitted curve

(a) blur (b) ff (c) jpeg

30 35 40 45 50 55 60 65

-5

0

5

10

15

20

25

30

35

40

MQS

DM

OS

DMOS_vs. jp2kfitted curve

15 20 25 30 35 40 45 50 55 60 65

0

10

20

30

40

50

60

MQS

DM

OS

DMOS vs. wnfitted curve

(d) jp2k (e) wn

Fig. 7. The plots of DMOS vs. MQS with different distortion types. (a)-(e) represents the plot of Blur, FF, JP2K, JPEG and WN distorted images respectively.

AUTHOR: TITLE 11

There are seven compared methods in our ex-periments among them three are 2D IQA methods (PSNR, SSIM[7], MSSIM [33], M_SVD[8], VIFP [34], FSIM[35]) and the other two are 3D IQA methods (3D_DCT[36], PQM[37]). According to the recommendation of VQEG, five criteria includ-ing PLCC, SROCC, KROCC, RMSE and OR are used for the comparison between the proposed method and other methods. Pearson linear corre-lation coefficient (PLCC)is the correlation coeffi-cient between the objective/subjective scores af-ter variance-weighted regression analysis with providing an evaluation of the prediction accuracy of OQA methods. Another one adopted to measure the performance of the proposed method through evaluating the prediction monotonicityis Spear-man rank order correlation coefficient (SROCC).Kendall rank order correlation coeffi-cient (KROCC) is a statistic used to measure the association between two measured quantities. It is a measure of rank correlation: the similarity of the orderings of the data when ranked by each of the quantities. For PLCC, SROCC andKROCC, the absolute values of them equal to 1 means the ob-

jective overcome has an exact consistency with the subjective perception.The other numerical in-dicator is root mean square error (RMSE) to mea-sure the predictive precision of objective over-come comprehensively. The smaller is the value of RMSE, the final objective scores perform better. The final indicator called Outlier Ratio (OR) is the ratio of “false” scores that lie outside the interval,

and defined as follows.

TABLE

4RMSE COMPARED WITH OTHER METHODS

RMSE PSNR SSIM MSSIM VIFP SVD FSIM 3D_DCT PQM MQS

JPEG 6.613 5.839 5.726 5.820 6.173 5.189 5.420 5.471 4.174

JP2K 8.083 6.728 6.027 5.474 7.827 5.177 6.660 7.510 4.870

FF 9.493 9.044 8.545 7.796 10.05 8.095 10.22 9.753 7.763

Blur 7.275 5.954 5.445 4.516 7.939 4.986 7.042 6.897 4.076

WN 6.017 5.919 5.900 6.322 5.375 6.228 6.017 14.38 6.052

ALL 10.000 8.296 7.792 6.953 10.09 6.519 11.54 10.82 6.044

TABLE 5OR COMPARED WITH OTHER METHODS

OR PSNR SSIM MSSIM VIFP SVD FSIM 3D_DCT PQM MQS

JPEG 0.025 0.0125 0.0125 0.0125 0.0375 0 0 0 0

JP2K 0 0 0 0 0 0 0 0 0


(16)

Where N is the total number of scores, and Nout is the total number of scores lied inside the interval,

. Si is the mean opinion scores (MOS), Oiis the objective score and is the stan-dard deviation of MOS. For the value of OR, the closer to 0 means the superior of the proposed OQA method.

AUTHOR: TITLE 13

The performance comparisons of PLCC, SROCC, KROCC, RMSE and OR with other eight methods are shown in table 1, 2, 3, 4 and 5. From the re-sults of these tables, we can conclude that the proposed MQS approach has an enhanced consis-tency with objective scores (difference mean opin-ion scores, DMOS) towards all the different com-pressiontypes of images. It has the best and most stable overall performance, and also has the most precise prediction precision. From table 1 we can see the proposed method has highest PLCCfor im-ages with distortion JPEG, JP2K and Blur. The value of PLCC is the highest when aggregate all data, thereforethe proposed methodpossess preferable applicability.

Results in Table 2 and Table 3 show that the proposed method has highest SROCC and KROC-Cexcept for images whose distortion types are WN. The proposed method decreases the perfor-mance of WN images compared with its original

SVD-based method. This is because the white noise uniformly distributed on images has a rela-tively minimal impact on the structural informa-tion than the energy information. It is easy to con-firm that the proposed method has the highest consistency with objective perception. And the RMSE of the proposed method is smallest com-pared with other methods, which shows that it has the highest predictive precision. The compari-son of OR shows that the scatter plots of proposed method have highest concentration. The values of above mentioned criteria indicate that the original SVD-based method is extremely suitable for the distortion of WN. The proposed method’s best performances for the whole criteria without dis-tinguishing the distortion type of images confirm its universal applicability once again.

Besides, in order to demonstrate the meliority of the proposed method more intuitively, as shown in Fig.7, a nonlinear logistic mapping between the

10 20 30 40 50 60 70 80 90

0

10

20

30

40

50

60

PSNR

DM

OS

DMOS vs. blurDMOS vs. ffDMOS vs. jp2kDMOS vs. jpegDMOS vs. wn

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

SSIM

DM

OS


0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

MSSIM

DM

OS


(a) PSNR (b)SSIM (c) MSSIM

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

VIFP

DM

OS


0 50 100 150 200

0

10

20

30

40

50

60

MSVD

DM

OS


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

FSIM

DM

OS


(d) VIF (e) MSVD (f) FSIM

10 20 30 40 50 60

0

10

20

30

40

50

60

DCT

DM

OS


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

PQM

DM

OS


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

10

20

30

40

50

60

MQS

DM

OS


(g) DCT (h) PQM (i) proposed MQS

Fig. 8 The performance of different OQA methods with testing on LIVEPhase I database. (a) – (i) represent SSIM, PSNR, MSSIM, VIF, MSVD, FSIM, 3D-DCT, PQM, and the proposed method MQS overcome vs. DMOS in sequence.


DMOS and the objective scores are applied to evaluate the proposed method as the VQEG sug-gested. Fig.7represents the visual prediction be-tween the objective scores and DMOS of the pro-posed method for different distortion types of im-ages. The vertical axis means subjective scores namely DMOS, and the horizontal axis means the objective scores of different distortion types. The figure shows that the concentration of the scatter plots is best for images with Blur and JP2K distor-tion, and this conclusion can be verified by the above tables. Fig.8presents the visual predictions of the proposed method and other eight compared methods through scatter plots (the subjective scoresDMOSvs. objective scores). As demon-strated in this figure, the concentration of the scatter plots is best when applying the proposed method to evaluate the distorted images. In Fig.8, (e) and (f) show that the quality assesses abilities of SVD model are improved statistically signifi-cant by the proposed metric. In summary, the pro-posed method not only performs the best consis-tency with HVS but also has the best prediction accuracy and optimal stability compared with other listed methods.

The primary cause is that PSNR, SSIM, MSSIM, VIFP, FSIM and M_svd are algorithms for 2D im-age; they did not take the properties of 3D image and HVS into account. MSSIM has a high correla-tion with DMOS, but the scatter plots are too dis-perseas Fig.8 shows. Although the DCT and PQM are all algorithms for 3D image, their perfor-mance is poorer than the proposed method since the original methods of PQM for 2D images are not performed so well as our original method. The 3D-DCT method mainly qualify the image quality depend on the stereo matching algorithm, but its measurement of disparity is not in keeping with the human vision of disparity. That is the reason why 3D-DCT method cannot performs as well as our method. In addition, color information and structural information are integratedinto our cal-culation through vector cross product operation, which aggrandizes its prediction accuracy. The other eight methods only take advantage of the lu-minance information with letting the chrominance information alone. The vector cross production embodies spatial vector information which repre-sents the structural changement of color images. With the vector cross production appended into the global numerical index calculation, we take property of elements in 3D color space into con-sideration.

From the experimental results, we can know that the depth information of 3D image plays an important role in evaluating the stereoscopic im-age quality. And the color information processing is also helpful in accessing color image quality on

account that the distortion of color information affects both luminance and chrominance in color space. Moreover, the vector special multiplication of quaternion vectors consists of two parts: scalar production and cross production, in whichthe lat-ter one includescertain spatial alteration informa-tion that the scalar production does not convey. Comprehensive utilization of the two factors is much more reasonable than only utilizing one fac-tor solely.The experimental indicators on the ex-periment database show that the proposed method preforms the best and has a highest valid-ity compared with other seven methods.

5 CONCLUSION AND FUTURE WORKIn this paper, a novel method that provides a nu-merical measurement is proposed to assess the quality of 3D color image under cloud computing environment. It takes full advantages of the depth sensation of HVS and property of QSVD theory with making the SVD-based method apply to 3D color image. And the experiment results show that it outperforms other 2D/3D OQA methods. The proposed method not only has the best stabil-ity with minimizing the amount of deviation but also has the highest prediction accuracy by virtue of its performance more consistent with the sub-jective scores than other OQA methods.

In conclusion, this proposed method simply combined the vector special multiplication into singular value decomposition method through the quaternion matrix. Its procedure is easy to realize and it performs very well on 3D image data-base.But because of the introduction of quater-nion vector product operation, the computational process is complex. Therefore, the further work is researching how make the algorithm be computa-tionally efficient while keeping the existing valid-ity.

ACKNOWLEDGMENTThe work was supported by the Natural Science Foundation of China on the grant G0501020161301268, and also supported by China Postdoctoral Science Foundation funded Project on the grant 2013M530396

REFERENCES

[1] Z. Wang and A. C. Bovik, Modern Image Quality Assessment. San Rafael, CA: Morgan & Claypool Publishers, Mar. 2006.

[2] A. KrishnaMoorthy, C. Su, A. Mittal, A. C. Bovik, “Subjective evaluation of stereoscopic image quality,” Signal Processing: Image Communication, vol. 28, no. 8, pp. 870-883, Sep. 2013.

AUTHOR: TITLE 15

[3] X. Liu, M. Chen, W. Tang, and C. Yu, “Hybrid no reference video quality assessment focusing on codec effects,” KSII Transactions on Internet and Information Systems, vol.3, iss. 2, pp.592-606, Mar. 2011.

[4] K. Seshadrinathan, R. Soundararajan, A. C. Bovik, and L. K. Cormack, “Study of Subjective and Objective Quality Assess-ment of Video,” IEEE Trans. on Image Processing, vol. 19, no. 6,pp. 1427-1441 Jun. 2010.

[5] X. Fei, L. Xiao, Y. Sun, Z. Wei, “Perceptual image quality as-sessment based on structural similarity and visual masking,” Signal Processing: Image Communication, vol. 27, no. 7, pp. 772-783, Aug. 2012.

[6] Z. Wang and A. C. Bovik, “A universal image quality index,” IEEE Signal Processing Letters, vol. 9, pp. 81 -84, Mar. 2002.

[7] Z. Wang, L. Lu, and A. C. Bovik, “Video quality assessment based on structural distortion measurement,” Signal Pro-cessing: Image Communication, vol. 19, no. 2, pp. 121-132, Feb. 2004.

[8] A. Shnayderman, A. Gusev, and A.M. Eskicioglu, “An svd-based grayscale image quality measure for local and global assessment”, IEEE Trans. of Image Processing, vol. 15, no. 2, pp. 422-429, Feb. 2006.

[9] C. Yang, W. Gao and L. Po, “Discrete wavelet transform-based structural similarity for image quality assessment,” in Processing of IEEE International Conference on Image Pro-cessing, San Diego, pp. 377 -380, Oct. 2008.

[10] A. C. Bovik and S. Liu,“DCT-domain blind measurement of blockingartifacts in DCT-coded images,” at the Inf. Conf. Acoustics,Speech, Signal Processing, Salt Lake City, UT, May 2001.

[11] A. Mansouri, A. M. Aznaveh, F. Torkamani-Azar, and J. A. Ja-hanshahi, “Image quality assessment using the singular value decomposition theorem,” Optical Review, vol. 16, no. 2, pp.49-53, Jan. 2009.

[12] C. Y. Wee, X. D. Jiang, A. C. Kot, and R. Paramesran, “Quanti-fying image distortions using SVD,” in 2010 5th IEEE Con-ference on Industrial Electronics and Applicationsis (ICIEA), Taichung, pp. 381-386, Jun. 2010.

[13] F. Q Zhang, J. L. Li, G. Chen, and J. J. Man, “Assessment of color video quality based on quaternion singular value de-composition,” in 2009 6th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Tianjin, Aug. 2009, pp. 7-10.

[14] A. Saha, G. Bhatnagar, and Q. M. J. Wu, “SVD filter based multiscale approach for image quality assessment,” in 2012 IEEE International Conference on Multimedia and Expo Workshop (ICMEW), Melbourne, pp. 43-38, Jul. 2012.

[15] M. Narwaria and W. S. Lin, “SVD-Based Quality Metric for Image and Video Using Machine Learning,” IEEE Trans. on systems and cybernetics – part B: cybernetics, vol. 42, no. 2,

pp. 347-364, Apr. 2012

[16] R. Wang, Y. Cui and Y. Yuan, “Image quality assessment using full-parameter singular value decomposition,” Optical Engi-neering, vol. 50, no.5, pp.1-8, May 2011.

[17] Z. M. P. Sazzad, S. Yamanaka, Y. Kawayoke, and Y. Horita, “Stereoscopic image quality prediction,” in Porc. IEEE QoMEX, San Diego, CA, USA, pp. 180-185, Jul. 2009.

[18] S. L. P. Yasakethu, C. T. E. R. Hewage, W. A. C. Fernando, and A. M. Kondoz, “Quality analysis for 3D video using 2D video quality models,” IEEE Trans. on Consumer Electronics, vol. 54,no. 4, pp.1969-1976, Nov. 2008.

[19] H. Shao, X. Cao, and G. Er, “Objective quality assessment of depth image based rendering in 3DTV system,” in 3DTV Conference: The Ture Vision - Capture, Transmission and Display of 3D video, Potsdam, pp.1-4, May 2009.

[20] F. Shao, S. Gu, G. Jiang, and M. Yu, “Stereoscopic images quality assessment by jointly evaluating image quality and depth perception,” in 2012 9th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD 2012),

Chongqing, pp. 1963-1966, May 2012.

[21] S. L. P. Yasakethu, D. V. S. X. De Silva, W. A. C. Fernando, and A. Kondoz, “Pre-dicting sensation of depth in 3D video,” Electronics Letters, vol. 46, no. 12, pp. 837-839, Jun. 2010.

[22] G. Antoniu, B. Thirion “A-brain using the cloud to understand the impact of genetic variability on the brain,” INRIA-MSR forum, Apr. 2011

[23] J. Wu, C. Jansen, M. Beier, M. Witt and D. Krefting “Extend-ing XNAT towards a cloud-based Quality Assessment Plat-form for Retinal Optical Coherence Tomographies,”2014 14th IEEE/ACM Internaitonal Symposium on Cluster, Cloud and Grid Computing, Chicago,pp. 764 – 773, May 2014.

[24] R. Alonso-Calvo, J. Crespo, V. Maojo, A. Muñoz, M. García-Remesal, and D. Pérez-Rey, “Cloud computing service for managing large medical image data-sets using balanced col-laborative agents,” in Advances on Practical Applications of Agents and Multiagent Systems, vol. 88, pp. 265–270, 2011.

[25] A. Kolaman, and O. Yadid-Pecht, “Quaternion Structure Simi-larity: A New Qualtiy Index for Color images,” IEEE Trans. of Image Processing, vol. 21, no. 4, pp.1526-1536, Apr. 2012.

[26] S. J. Sangwine, “Fourier transforms of colour images using quaternion, or hypercomplex, numbers,” Electronics letters, vol. 32, no. 21, pp. 1979-1980, Oct. 1996.

[27] R. Hunt and E. Carter, The Reproduction of colour. London, U.K.: Fountain Press, 1995.

[28] P. Denis, P. Carre, and C. F. Malogine, “Spatial snd spectral quaternionic approaches for colour images,” Computer. Vi-sion. And Image Understanding, vol. 107, no. 1-2, pp. 74-87,


Jul./Aug. 2007.

[29] W. R. Hamilton, “On quaternion,” Proceeding of the Royal Irish Academy, vol. 3, pp.1-16, 1846.

[30] C. Moxey, S. Sangwine and T. Ell, “Hypercomplex correlation techniques for vector images,” IEEE Trans. on Signal Pro-cessing. Vol. 51, no. 7, pp.1941-1953, Jul. 2003.

[31] Live 3D Image Quality Database – Phase I, 2012. Available: http: //live.ece.utexas.edu/research/quality/live_3dimage_phase1.html

[32] A. K. Moorthy, C. C. Su, A. Mittal, and A. C. Bovik, “Subjec-tive evaluation of stereoscopic image quality,” Signal Pro-cessing: Image Communication, SPIC, accepted, Sep. 2012.

[33] Z. Wang, E. P. Simoncelli and A. C. Bovik,“Multi-scale struc-tural similarity for image quality assessment,”37th IEEE Asilomar Conference on Signals, Systems and Computers, Pacific Grove, pp. 1398-1402, 2003

[34] H. R. Sheikh and A. C. Bovik,“Image information and visual quality”,IEEE Transactions on Image Processing,vol. 15, no. 2,pp. 430-444, 2006.

[35] L. Zhang, L. Zhang, X. Mou, and D. Zhang,”FSIM: A Feature Similarity Index for ImageQuality Assessment “,IEEE Trans-actions on Image Processing, vol. 20, no. 8, pp. 2378-2386, 2011

[36] L. Jin, A. Boev, A. Gotchev, and K. Egiazarian, “3D-DCT based perceptual quality assessment of stereo video,” in 18th IEEE International Conference on Image Processing (ICIP2011), pp. 2521-2524, Brussels, Sep. 2011.

[37] P. Joveluro, H. Malekmohamadi, W.A.C. Fernando, A.M. Kon-doz, “Perceptual video quality metric for 3D video quality as-sessment,” in Proc. of 3DTV Conference, pp. 1-4, Jul. 2010.

Xingang Liu (M’10) received the B.S. degree from the School of Electronic Engineering (EE), University of Electronic Science and Technology ofChina (UESTC), Chengdu, China, in 2000 and theM.S. and Ph.D. degrees from Yeungnam University,Gyeongsan, Korea, in 2005 and 2010, respectively.From 2010 to 2011, he was a BK21 Re-searchFellow with the School of Electrical and ElectronicEngineer-ing, Yonsei University, Seoul, Korea. He is currently an Professor with the School ofEE, UESTC, and also an Adjunct Professor with DonggukUniversity, Seoul. His research interests are multimedia-sig-nal-communicationrelated topics, such as heterogeneous/homoge-neous video transcoding, videoquality measurement, video signal er-ror concealment, mode decision algorithm,2-D/3-D video codec, and so on.

Dr. Liu is a member of Korean Information and Communications So-ciety (KICS) and Korean Society for Internet Information (KSII).

Lan Zhang is currently working toward the M.S. degree in the School of Electronic Engi-neering, University of Electronic Science and Technology of China, Chengdu, China.

Her research interests include 2D/3D image and video quality as-sessment.

Kaixuan Luis currently working toward the M.S. degree in the School of Electronic Engi-neering, University of Electronic Science and Technology of China,Chengdu, China.

His research interests include 2D/3D image and video quality assessment.

transaction / regular paper titleor.nsfc.gov.cn/bitstream/00001903-5/486836/1/10000143… · web...

Documents