optimal subset-division based discrimination and its kernelization

Pattern Recognition 45 (2012) 3590–3602

Contents lists available at SciVerse ScienceDirect

Pattern Recognition

0031-32

http://d

n Corr

Wuhan

E-m

journal homepage: www.elsevier.com/locate/pr

Optimal subset-division based discrimination and its kernelizationfor face and palmprint recognition

Xiaoyuan Jing a,b,c,n, Sheng Li b, David Zhang d, Chao Lan b, Jingyu Yang e

a State Key Laboratory of Software Engineering, Wuhan University, Wuhan, Chinab College of Automation, Nanjing University of Posts and Telecommunications, Chinac State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, Chinad Department of Computing, Hong Kong Polytechnic University, Kowloon, Hong Kong, Chinae College of Computer Science, Nanjing University of Science and Technology, China

a r t i c l e i n f o

Article history:

Received 16 August 2010

Received in revised form

18 February 2012

Accepted 1 April 2012Available online 14 April 2012

Keywords:

Optimal subset-division

Discriminant analysis

Improved stability criterion

Kernel method

Face recognition

Palmprint recognition

03/$ - see front matter & 2012 Elsevier Ltd. A

x.doi.org/10.1016/j.patcog.2012.04.001

esponding author at: State Key Laborator

University, Wuhan, China.

ail address: [email protected] (X. Jing).

a b s t r a c t

Discriminant analysis is effective in extracting discriminative features and reducing dimensionality. In

this paper, we propose an optimal subset-division based discrimination (OSDD) approach to enhance

the classification performance of discriminant analysis technique. OSDD first divides the sample set

into several subsets by using an improved stability criterion and K-means algorithm. We separately

calculate the optimal discriminant vectors from each subset. Then we construct the projection

transformation by combining the discriminant vectors derived from all subsets. Furthermore, we

provide a nonlinear extension of OSDD, that is, the optimal subset-division based kernel discrimination

(OSKD) approach. It employs the kernel K-means algorithm to divide the sample set in the kernel space

and obtains the nonlinear projection transformation. The proposed approaches are applied to face and

palmprint recognition, and are examined using the AR and FERET face databases and the PolyU

palmprint database. The experimental results demonstrate that the proposed approaches outperform

several related linear and nonlinear discriminant analysis methods.

& 2012 Elsevier Ltd. All rights reserved.

1. Introduction

Discriminant analysis is an important research topic in the fieldof pattern recognition and computer vision [1], especially in facerecognition [2,3] and palmprint recognition [4]. Linear discriminantanalysis (LDA) is a popular and widely used supervised discriminantanalysis method [5]. LDA calculates the discriminant vectors bymaximizing the between-class scatter and minimizing the within-class scatter simultaneously. It is effective in extracting discrimina-tive features and reducing dimensionality. Many methods have beendeveloped to improve the performance of LDA, such as direct LDA[6], enhanced Fisher linear discriminant model (EFM) [7], regular-ized discriminant analysis [8], uncorrelated optimal discriminantvectors (UODV) [9], 2D-LDA [10], LDA/QR [11], discriminant com-mon vectors (DCV) [12], improved LDA [13], incremental LDA [14],semi-supervised discriminant analysis (SSDA) [15], local uncorre-lated discriminant transform (LUDT) [16], tensor rank one discrimi-nant analysis (TR1DA) [17], Perturbation LDA (P-LDA) [18], andsubclass discriminant analysis (SDA) [19,20]. SDA approximates the

ll rights reserved.

y of Software Engineering,

underlying distribution with mixture of Gaussians by dividing aclass into suitable subclasses. It reformulates the between-classscatter matrix and employs a stability criterion to determine thenumber of subclasses. Sugiyama put forward a local Fisher discri-minant analysis (LFDA) method [21], which uses local neighborhoodinformation to construct the weighted between-class and within-class scatter matrices, and then performs discriminant analysis.

Unlike LDA, LFDA and SDA that extract discriminative featuresfrom the whole dataset, two discriminant methods, i.e., MultipleLDA (MLDA) [22] and Random sampling LDA (RLDA) [23,24], havebeen recently proposed to perform discriminant analysis by split-ting the sample set into several subsets. Multiple LDA (MLDA)randomly splits the original dataset into several subsets, constructsthe LDA classifier for each subset and selects an appropriateclassifier to do recognition [22]. Random sampling LDA (RLDA)randomly chooses training samples to construct some subsets andrandomly chooses principal components (e.g., eigenfaces or eigen-palms) to construct other subsets, then produces multiple weakclassifiers and integrates them by using a fusion rule [23,24]. BothRLDA and MLDA employ the random sampling strategies.

In recent years, many kernel discriminant methods have beenpresented to extract nonlinear discriminative features andenhance the classification performance of linear discriminationtechniques, such as kernel discriminant analysis (KDA) [25,26],

X. Jing et al. / Pattern Recognition 45 (2012) 3590–3602 3591

kernel direct discriminant analysis (KDDA) [27], improved kernelFisher discriminant analysis [28], complete kernel Fisher discri-minant (CKFD) [29], kernel discriminant common vectors (KDCV)[30], kernel subclass discriminant analysis (KSDA) [31], kernellocal Fisher discriminant analysis (KLFDA) [32], kernel uncorre-lated adjacent-class discriminant analysis (KUADA) [33], andmapped virtual samples (MVS) based kernel discriminant frame-work [34].

1.1. Motivation and contribution

In this paper, we propose an optimal subset-division baseddiscrimination (OSDD) approach. According to the distribution ofsamples, OSDD divides the sample set into subsets by using theK-means algorithm and attains the discriminant vectors fromeach subset. We then construct the projection transformation bycombining the discriminant vectors derived from each subset.How to find the appropriate number of subsets. Inspired by thestability criterion used in subclass discriminant analysis (SDA),which determines the number of subclasses, we put forward animproved stability criterion to determine the optimal number ofsubsets. This criterion first computes the between-class scattermatrices and the within-class scatter matrices of all subsets, andthen measures the average conflict degree of eigenvectors ofbetween-class and within-class scatter matrices. The less averagedegree is, the better separability within each subset and the lowercorrelation between all subsets are. OSDD determines the optimalnumber of subsets with the least average conflict degree.

The differences between OSDD and two related methods (i.e.,RLDA and MLDA) is that (1) OSDD employs the improved stabilitycriterion to get the optimal number of subsets, while RLDA andMLDA empirically select the number of subsets and (2) OSDDdivides the sample set according to the distribution of samples byusing the K-means algorithm, while RLDA and MLDA use therandom sampling strategies.

OSDD is expected to handle non-linearly separable datasetsbetter than many linear approaches, since it divides the wholedataset into several subsets and consequently simplifies datasetsdistributions. Nevertheless, it is still worthy to explore the non-linear discriminant capability of OSDD, in particular, when someobtained subsets still show complicated and non-linearly separabledata distribution. Hence, in this paper, we further extend OSDD inthe kernel space and present an optimal subset-division basedkernel discrimination (OSKD) approach to extract nonlinear dis-criminative features. It maps the sample set from the input spaceto the kernel space, and employs the kernel K-means algorithm todivide the sample set. The improved stability criterion can be usedto find the optimal number of subsets in the kernel space. Thenonlinear discriminant vectors are calculated from each subset,and combined as a whole projection transformation.

Using the AR and FERET face databases and the PolyU palm-print database, the experimental results show that the proposedapproaches outperform several related linear and nonlinear dis-criminant analysis methods. The discriminative features achievedby our approaches are demonstrated to have favorable localdiscriminabilities, and also have good generalization abilities forthe whole sample set.

1.2. Organization

The rest of this paper is organized as follows: in Section 2, weoutline some related work. In Section 3 we describe our OSDDapproach and propose an improved stability criterion to deter-mine the optimal number of subsets. We describe the nonlinearextension of OSDD in Section 4. Experimental results on the faceand palmprint databases are presented in Section 5. In Section 6,

we provide the further experimental analysis, before drawingconclusions in Section 7.

2. Related work

2.1. Linear discriminant analysis and kernel discriminant analysis

Assume that the original sample set X ¼ fx1,x2,. . .,xNg is com-posed of c classes, and there are ni training samples in the ithclass. Linear discriminant analysis (LDA) tries to find a set ofoptimal projections W ¼ ½w1,w2,. . .,wm�, such that the ratio of thedeterminant of between-class scatter to the determinant ofwithin-class scatter is maximized, i.e.,

JðWÞ ¼ argmaxW

9WT SbW9

9WT SwW9, ð1Þ

where Sb and Sw are the between-class scatter matrix and thewithin-class scatter matrix, respectively [5]. Generally, W isconstructed by the eigenvectors of (Sw)�1Sb.

Kernel discriminant analysis (KDA) is based on a conceptualtransformation from original input space into a nonlinear high-dimensional kernel space [25,26]. For a given nonlinear mappingfunction j, the input data space Rd can be mapped into the kernelspace F:

j : Rd-F, x-jðxÞ: ð2Þ

KDA reformulates the scatter matrices Sjb and Sjw. In the kernelspace, the Fisher criterion is defined as

JðWjÞ ¼ argmax

Wj

9WjT Sjb Wj9

9WjT SjwWj9, ð3Þ

Wj is constructed by the eigenvectors of ðSjwÞ�1Sjb .

2.2. Subclass discriminant analysis and its kernelization

Subclass discriminant analysis (SDA) was presented to adapt toa large variety of data distributions [19,20]. It employs a simplecriterion that could provide a good estimate for the number ofsubclasses needed to represent each class. SDA divides classes intothe optimal number of subclasses. The extracted discriminantvectors can maximize the distance between the class means andthe means of the subclasses that correspond to the same class.

SDA redefines the between-class scatter SB as

SB ¼XC�1

i ¼ 1

XHi

j ¼ 1

XC

k ¼ iþ1

XHk

l ¼ 1

pijpklðmij�mklÞðmij�mklÞT , ð4Þ

where Hi is the number of subclass divisions in class i, pij ¼ nij=n,and mij is the mean of the jth subclass in class i.

Kernel subclass discriminant analysis (KSDA) is the kernelversion of SDA [31]. KSDA adopts nonlinear clustering techniqueto find the underlying distributions of datasets in the kernelspace. It reformulates Eq. (4) in the kernel space and thenperforms SDA.

2.3. Random sampling LDA

Random sampling LDA (RLDA) [23,24] constructs two kinds ofweak classifiers. First, it randomly selects the samples whosedimensions are reduced by principal component analysis (PCA)[1], and constructs a set of null-space LDA classifiers. Second, itrandomly selects a number of principal components (e.g., eigen-faces or eigenpalms) derived by PCA transform to generaterandom subspaces and constructs multiple LDA classifiers. Finally,

Table 1Comparison of OSDD and related methods.

Method Subset division strategy Fusion method Kernel extension

LDA N/A N/A YesSDA N/Aa N/A YesRLDA Random division Classifiers fusion NoMLDA Random division Classifiers fusion NoOSDD Improved stability criterion

and K-means clusteringbased division

Discriminantfeatures fusion

Yes

a Subclass division using a stability criterion.

X. Jing et al. / Pattern Recognition 45 (2012) 3590–36023592

these two kinds of complementary classifiers are integrated usingthe majority-voting fusion rule.

2.4. Multiple LDA subspaces

Multiple LDA (MLDA) [22] randomly splits the original datasetinto multiple clusters and computes a single LDA subspace foreach cluster. In MLDA, each cluster contains the same number ofclasses. For a new unknown test sample, it is projected onto allsubspaces, and the class label for this sample can be obtained bysearching for the closest class center over all projected spaces.

Table 1 shows the comparison of LDA, SDA, RLDA, MLDA andOSDD in the aspects of subset’s division strategy, the fusionmethod of subset’s discriminant information, and whether per-forming the kernel extension.

3. Optimal subset-division based discrimination

In this section, we first provide the description of OSDDapproach. Then we introduce how to determine the optimalnumber of subsets and divide the sample set into several subsets.

3.1. OSDD approach

Assume that we have divided the original sample setX ¼ fx1,x2,. . .,xNg into Lo subsets, X ¼ fX1,X2,. . .,XLo

g. The jth subsetXj contains Nj samples that belong to cj classes,

PLo

j ¼ 1 Nj¼N, andPLo

j ¼ 1 cj ¼ c, where c is the number of classes in the sample set X.Here, Lo is the optimal number of subsets determined by theoptimal subsets division strategy described in Section 3.2. Werealize OSDD by the following three steps.

3.1.1. Perform PCA for each subset

For high-dimensional data such as images, discriminant ana-lysis methods always suffer from the so-called singularity pro-blems, that is, the scatter matrices are always singular. Toovercome the singularity problems, PCA can be used to reducethe dimensionality of the data [5].

For the jth subset, we define the total scatter matrix as

Sjt ¼

1

Nj

Xcj

i ¼ 1

Xnji

l ¼ 1

ðxil�mjÞðxil�mjÞT , ð5Þ

where cj, nji, Nj and mj are the number of classes, the number ofsamples of ith class, the total number of samples and the mean ofall the samples in the jth subset, respectively.

The PCA projection transformation wjpca ¼ ½w

jpca1,wj

pca2,. . .,

wj

pcaðNj�cjÞ� is chosen to maximize Jðwj

pcaÞ ¼wjpca

T Sjtw

jpca. To ensure

the non-singularity of Sjw in the following procedures, we select at

most Nj�cj principal components for each subset. Then we

project the samples of the jth subset onto the subspace spanned

by wjpca and obtain a low-dimensional sample set Zj

pca ¼wjpcaX.

3.1.2. Calculate discriminant vectors in the PCA transformed spaces

For the jth subset, the between-class scatter matrix Sjb and the

within-class scatter matrix Sjw are constructed in the PCA trans-

formed space as follows:

Sjb ¼

Xcj

i ¼ 1

ðmji�mjÞðm

ji�mjÞ

T , ð6Þ

Sjw ¼

1

Nj

Xcj

i ¼ 1

Xnji

l ¼ 1

ðzjil�mj

iÞðzjil�mj

iÞT , ð7Þ

where zjil is a sample in Zj

pca, mji is the mean of the ith class in Zj

pca

and mj is the global mean of Zjpca.

We calculate the discriminant vectors derived from differentsubsets. For each subset, we obtain a set of discriminant vectorsthat maximize the following objective function:

Jðwjf ldÞ ¼

9wj Tf ld Sj

bwjf ld9

9wj Tf ld Sj

wwjf ld9

ð8Þ

It is easy to prove that the optimal solutions wjf ld ¼ ½w

jf ld1,

wjf ld2,. . .,wj

f ldðlÞ� are the eigenvectors of ðSjwÞ�1Sj

b corresponding tothe l largest eigenvalues. According to Eq. (6), the rank of Sj

b is atmost cj�1. Hence, there are at most cj�1 discriminant vectors forthe jth subset. Generally, the discriminant vectors calculated froma subset have the better local discriminabilities than the discri-minant vectors extracted from the whole sample set.

Finally, we obtain c�Lo discriminant vectors from all subsets,where Lo is the number of subsets.

3.1.3. Construct the linear projection transformation and do

classification

The dimensions of the c�Lo discriminant vectors are the same.We integrate the discriminant vectors derived from all subsetsand construct the projection transformation W as

W ¼ ½w11,w1

2, � � � ,wLo

ck�1�

¼ ½w1 Tf ld1 w1 T

pca ,w1 Tf ld2 w1 T

pca � � � ,wLo Tf ldðck�1Þw

Lo Tpca � ð9Þ

After projecting original samples on W, we employ the nearestneighbor classifier with cosine distance to classify the projectedsamples. Fig. 1 shows the flowchart of our optimal subset-divisionbased discrimination (OSDD) approach.

In Section 3.2, we will show that according to the optimalsubsets division strategy, each subset does not overlap othersubsets, and it holds a low conflict degree with other subsets.Therefore, the discriminant vectors calculated from each subsethave good generalization abilities for other subsets. We cancombine the discriminant vectors derived from all subsets as awhole projection transformation.

3.2. Optimal subsets division

3.2.1. Improved stability criterion

A stability criterion was proposed to judge whether the linearfeature extraction methods are applicable [35]. In the case of LDA,when the smallest angle between the ith eigenvector given by thebetween-class scatter matrix (Sb) to be maximized and the first i

eigenvectors given by the within-class scatter matrix (Sw) to beminimized is close to zero, the results of LDA are not guaranteed

······

Training dataset X

1X

Calculate discriminant vectors W

W = [W1 W2 …WLo]

Y = WT X

······

Z = WTXtest

Testing dataset X

Using the improved stability criterion to determine the optimal

number of subsets o

test

L

LX2X

PCA PCA PCA ······



Nearest neighbor classifier Recognition results

Fig. 1. Flowchart of optimal subset-division based discrimination (OSDD)

approach.


to be correct. The stability criterion was formulated as [20]

K ¼1

c�1

Xc�1

i ¼ 1

max8jr iðuT

j viÞ2: ð10Þ

where vi is the ith eigenvector of Sb and uj is the jth eigenvector of Sw.In Eq. (10), the value of K measures the average conflict degree

between the eigenvectors of Sb and Sw. In practice, the value of K

is desired to be as small as possible. A smaller value of K means alower average conflict degree, which indicates a better linearseparability of the sample set.

However, the stability criterion defined in Eq. (10) just reflectsthe average conflict degree between eigenvectors of a singledataset. In OSDD, the sample set is split into several subsets.We need to measure the conflict degree of eigenvectors bothwithin the subsets and between the subsets. Therefore, theoriginal stability criterion of SDA is not suitable for OSDD. Wedefine an improved stability criterion described as follows.

Let vji denote the ith eigenvector of Sjb, ujm and unm denote the

mth eigenvector of Sjw and Sn

w, respectively. Assume that thesample set is divided into L subsets, the within-sets averageconflict value bK 1ðLÞ is formulated as

bK 1ðLÞ ¼1

l1

XL

j ¼ 1

Xcj�1

i ¼ 1

ck1

ji , ð11Þ

where

ck1

ji ¼max8mr cj�1

ðuTjmvjiÞ

2 if max8mr cj�1

ðuTjmvjiÞ

2Zb

0 otherwise,

8<:

b is a nonnegative constant, and l1 is the number ofck1

ji whosevalues are unequal to zero.

And the between-sets average conflict value cK2 ðLÞ is formu-lated as

cK2 ðLÞ ¼1

l2

XL

j ¼ 1

Xcj�1

i ¼ 1

ck2

ji , ð12Þ

where

ck2

ji ¼

max8mrcj�1,

8nrL,na j

ðuTnmvjiÞ

2 if max8mrcj�1,

8nrL,na j

ðuTnmvjiÞ

2Zb

0 otherwise

,

8><>:

b is a nonnegative constant, and l2 is the number ofck2

ji whosevalues are unequal to zero.

Then, we define the improved stability criterion as

bK ðLÞ ¼bK 1ðLÞ if L¼ 1

bK 1ðLÞþbK 2ðLÞ2 otherwise

,

8<: ð13Þ

where bK ðLÞ indicates the overall average conflict value.In Eq. (13), bK 1ðLÞ represents the conflict degree within each

subset, bK 2ðLÞ indicates the conflict degree between all subsets,and bK ðLÞ represents the overall average conflict degree bothwithin the subsets and between the subsets. In OSDD, thediscriminant vectors derived from each subset contribute to thewhole sample set. Therefore, the generalization of the discrimi-nant vectors should be considered, and bK 2ðLÞ may give a properestimation. In Eqs. (11) and (12), the threshold value b is a smallnonnegative constant (in the experiments, b¼0.05). When theconflict value of eigenvectors is smaller than b, we consider thatthe relationship between these eigenvectors is an agreementrather than a conflict. Therefore, it is not appropriate to take thisvalue into account when calculating bK 1ðLÞ or bK 2ðLÞ.

3.2.2. Dividing sample set into subsets

To use the improved stability criterion and determine theoptimal number of subsets, we need to calculate the values of bK ðLÞfor each of the possible values of L. Then we select the numbercorresponding to the minimum value.

Therefore, the optimal number of subsets Lo is given by

Lo ¼ argminL

bK ðLÞ: ð14Þ

Then, we use the K-means algorithm to divide the originalsample set X into Lo disjoint subsets X ¼ fX1,X2,. . .,XLo

g. TheK-means algorithm is widely used for clustering. It can divide thesample set into several disjoint clusters according to its spatialdistribution. This is different from other procedures used to dividethe dataset like the random sampling procedures employed byRandom sampling LDA (RLDA) and Multiple LDA (MLDA).

In this paper, we use the mean samples of all classes instead ofall samples. If the mean vector mj is assigned to the 4th subset, allthe samples of the jth class are assigned to the 4th subset. That isto say, the subsets division does not change the class attributes ofthe samples. The procedures of optimal subsets division aresummarized in Algorithm 1. Fig. 2 shows the distribution ofsubsets on three face and palmprint image databases, where thenumber of subsets in Fig. 2a, b and c are 3, 8 and 7, respectively. Itshows that, by using the improved stability criterion and K-meansalgorithm, the datasets could be divided into several disjointsubsets according to their spatial distributions.

Algorithm 1. Optimal Subsets Division

for L¼2 to_Ly do

Divide the sample set into L subsets using the K-meansalgorithm and mean samples.Calculate Sj , j¼ 1, � � � ,L.
b

−1.5 −1 −0.5 0 0.5 1 1.5x 105

−8

−6

−4

−2

0

2

4

6

8

10

Distribution of subsets on the AR face database−8 −6 −4 −2 0 2 4 6

x 104

−10

−5

0

5

Distribution of subsets on the FERET face database

−10 −8 −6 −4 −2 0 2 4 6 8x 104

x 104

x 104x 104

−8

−6

−4

−2

0

2

4

6

Distribution of subsets on the PolyU palmprint database

Fig. 2. Distribution of subsets on the AR, FERET face and PolyU palmprint databases.


Compute SjbVj ¼ ljV j, Vj ¼ ½vj1, � � � ,vjðcj�1Þ�, j¼ 1, � � � ,L.

Calculate Sjw, j¼ 1, � � � ,L.

Compute SjwUj ¼ ljUj, Uj ¼ ½uj1, � � � ,ujðcj�1Þ�, j¼ 1, � � � ,L.

Calculate bK Lð Þ using Eq. (13).end for

Lo ¼ argminL

bK ðLÞ:Divide the sample set into Lo subsets and calculate theprojection transformation according to Section 3.1.yThe value of

_L can be specified for different datasets.

In addition, the proposed OSDD approach is well suited for thelarge-scale classification problems. By using the optimal subsetsdivision strategy, experimental results show that the number ofsubsets is small, and each set contains many classes. It’s a rare casethat one class is considered as one cluster. Moreover, if only one classis assigned into a subset, this class is well-separated, and it cannotcontribute effective discriminant information according to the LDAtechnique. Since this class is well-separated, it could be easilyseparated by the discriminant vectors derived from other subsets.

3.2.3. Difference of stability criterions between OSDD and SDA

It’s important to note that the stability criterion used in OSDDis essentially different from that of subclass discriminant analysis

(SDA). SDA uses a stability criterion to divide each class intoseveral subclasses and constructs overall scatter matrices, whileOSDD splits the sample set into several subsets based on theimproved stability criterion, and constructs scatter matrices foreach subset. That is, SDA treats the sample set as a single dataset,whereas OSDD treats the sample set as multiple datasets (sub-sets) and each subset contains lots of classes. Moreover, since thestability criterion of SDA could not measure the conflict degree ofmultiple datasets, it is not suitable for OSDD.

3.3. Computational complexities

Table 2 shows the computational complexities of LDA, SDA,MLDA, RLDA and the proposed OSDD approach. Ye and Li [11]pointed out that the computational complexity of LDA is O(N2d),where N is the total number of training samples and d is thedimension of sample. In SDA, l denotes the times of attempting todivide the subclasses. The value of l can be specified by the user orbe set so as to guarantee that the minimum number of samplesper subclass is sufficiently large. Therefore, SDA solves the eigen-decomposition problem l times, and the computational complex-ity of SDA is O(N2dþ lN3). The complexity of LFDA is O(cN2d).RLDA first costs O(N2d) to perform PCA on the dataset, and then itconstructs h LDA or N-LDA (Null space LDA) subspaces that costsOðhN3

i Þ, where Ni is the dimension of the subspace. Hence, thecomplexity of RLDA is OðN2dþhN3

i Þ. MLDA randomly splits the


original dataset into k clusters, and computes a single LDAsubspace for each cluster that costs Oð ~N

2dÞ, where ~N is the

number of samples in one cluster. Meanwhile, MLDA has to makes attempts to divide the datasets and get the proper number ofclusters [22]. Therefore, the total computational complexity ofMLDA is Oðsk ~N

2dÞ ¼Oððs=kÞN2dÞ.

In our OSDD approach, the computational complexity causedby the improved stability criterion is

P_LL ¼ 2 N2

L d, where L denotesthe number of subsets and NL �N=L is the number of samples inone subset. In practice, the value of

_L is fairly small, and it’s easy

to see thatP_

LL ¼ 2 N2

L d¼P_

LL ¼ 2ðN

2=L2Þd. Since

P1L ¼ 1 1=L2

¼ p2=6[36], we have

P_LL ¼ 2ðN

2=L2Þdo ððp2=6Þ�1ÞN2d� ð3=5ÞN2d. There-

fore, the complexity of OSDD is O(N2d). It’s obvious that OSDD,MLDA and LDA have lower computational complexities thanother compared methods.

4. Optimal subset-division based kernel discrimination(OSKD)

In this section, the OSDD approach is performed in a highdimension space by using the kernel trick. We realize the OSKD inthe following four steps:

4.1. Nonlinear mapping and subsets division in the kernel space

Let f: Rd-F denote a nonlinear mapping. The original sampleset X is injected into F by f: xi-f(xi). We obtain a set of mappedsamples C¼ ffðx1Þ,fðx2Þ,. . .,fðxNÞg.

The improved stability criterion described in Section 3.2 canalso be reformulated to determine the optimal number of subsets

Table 2Computational complexities of OSDD and related methods.

Method LDA SDA LFDA

Complexity O(N2d) O(N2dþ lN3) O(cN2d

Fig. 3. Demo images of one subje

Table 3

Recognition rates and the values of bK varying with the different number of subsets L o

L 1 2 3 4 5

Rates (%) 78.23 85.79 86.75 86.53 85.84

bK 0.1654 0.1527 0.1461 0.1493 0.157

in the kernel space. We use the kernel K-means algorithm [37] todivide the mapped samples into Lo subsets, C¼ fC1,C2,. . .,CLo

g.

4.2. Perform KPCA for each subset

For the jth subset, we perform KPCA by maximizing thefollowing equation:

Jðwjf

kpcaÞ ¼wjf Tkpca Sjf

t wjf

kpca, ð15Þ

where Sjft ¼

PNj

i ¼ 1ðfðxjiÞ�mfj ÞðfðxjiÞ�mf

j ÞT , mf

j is the global meanof the jth subset in the kernel space.

According to the kernel reproducing theory [37], the projectiontransformation wjf

kpca in F can be linearly expressed by using allthe mapped samples:

wjf

kpca ¼XN

i ¼ 1

ajifðxiÞ ¼Caj, ð16Þ

where aj ¼ ðaj1,aj

2,. . .,ajNÞ

T is a coefficient matrix.Substituting Eq. (16) into Eq. (15), we have

Jðwjf Þ ¼ ajT

CTCjCjTCaj ¼ ajT

KjKjTaj, ð17Þ

where Kj¼CTCj, which indicates an N�Nj non-symmetric kernel

matrix whose element is Kjm,n ¼/fðxmÞ,fðxjnÞS.

The solution of Eq. (17) is equivalent to the eigenvalueproblem:

ljaj ¼ KjKjTaj: ð18Þ

The optimal solutions aj ¼ ðaj1,aj2,. . .,ajðNj�cjÞÞT are the eigen-

vectors corresponding to Nj�cj largest eigenvalues of KjKjT. We

RLDA MLDA OSDD

) OðN2dþhN3i Þ Oððs=kÞN2dÞ O(N2d)

ct from the AR face database.

n the AR face database.

6 7 8 9 10

85.50 85.42 85.16 84.63 83.56

7 0.1551 0.1536 0.1562 0.1555 0.1634


project the mapped training sample set Cj on wjf

kpca by

Zjf

KPCA ¼wjfTkpcaC

j¼ ajT

CTCj¼ ajT

Kj: ð19Þ

4.3. Calculate kernel discriminant vectors in the KPCA

transformed space

By using the KPCA transformed sample set Zjf

KPCA, we reformu-late Eqs. (6) and (7) as

Sjf

b ¼Xcj

i ¼ 1

ðmjfj �mf

j Þðmjfj �mf

j ÞT , ð20Þ

Sjf

w ¼1

Nj

Xcj

i ¼ 1

Xnji

l ¼ 1

ðzjfil �mjf

j Þðzjfil �mjf

j ÞT , ð21Þ

where zjfil , mjf

j and mfj are the sample, the mean of the ith class

and the mean of all samples in Zjf

KPCA, respectively.

1 5 10 15 20 25 3060

65

70

75

80

85

90

95

100

Random Testing No.

Rec

ogni

tion

Rat

es (%

)

LDA SDA LFDA MLDA RLDA OSDD

1 5 10 15 20 25 3060

65

70

75

80

85

90

95

100

Random Testing No.

Rec

ogni

tion

Rat

es (%

)

KDA KSDA KLFDA OSKD

Fig. 4. Recognition rates of all compared methods on the AR face database.

Table 4Average recognition rates and standard deviations of compared methods on the AR fac

Method LDA SDA LFDA

Average rates (%) 78.3577.28 78.8776.86 78.667Method KDA KSDA KLFDA

Average rates (%) 82.3378.16 80.1277.54 81.787

Similar to the OSDD approach, we generate a set of nonlineardiscriminant vectors for the jth subset, wjf

kf d ¼ ½wjf

kf d1,wjf

kf d2,. . .,wjf

kf dðcj�1Þ�, by maximizing the following criterion:

Jðwjf

kf dÞ ¼9wjfT

kf dSjf

b wjf

kf d9

9wjfTkf dSjf

w wjf

kf d9: ð22Þ

The number of discriminant vectors for the jth subset is atmost cj�1. After the computations for all subsets, we obtain c�Lo

nonlinear discriminant vectors.

4.4. Construct the nonlinear projection transformation and do

classification

We construct the nonlinear projection transformation Wj as

Wf¼ ½w1f

1 ,w1f2 ,. . .,wLo f

cL�1 �

¼ ½w1 fTkf d1 w1 fT

kpca ,w1 fTkf d2 w1 fT

kpca � � � ,wLo fTkf dðcL�1Þw

Lo fTkpca �: ð23Þ

Then, the OSKD features can be generated by

Zf¼WfC¼

w1 fTkf d1 w1 fT

kpca

w1 fTkf d2 w1 fT

kpca

^

wLo fTkf dðcL�1Þw

Lo fTkpca

26666664

37777775

,C¼

w1 fTkf d1 a1T

CT

w1 fTkf d2 a1T

CT

^

wLo fTkf dðcL�1Þa

LToCT

26666664

37777775

C¼

w1 fTkf d1 a1T

w1 fTkf d2 a1T

^

wLo fTkf dðcL�1Þa

LTo

26666664

37777775

K , ð24Þ

where K¼CTC indicates an N�N symmetric kernel matrix whoseelement is Ki,j¼/f(xi),f(xj)S.

Finally, the nearest neighbor classifier with cosine distance isemployed to perform classification.

In nonlinear methods, the construction of kernel matrix costsO(N2d), and the eigen-decomposition of kernel matrix costs O(N3).Thus, the computational complexity of KDA, KSDA and KLFDA areO(N2dþN3) [38], O(N2dþ lN3) and O(cN2dþN3), respectively.OSKD employs the improved stability criterion to obtain theoptimal number of subsets in the kernel space, which costsð3=5ÞN2d. And OSKD costs LUN3

L � LUðN=LÞ3 ¼N3=L2 to solve the L

eigen-problems for L subsets, where NL �N=L is the number ofsamples in one subset. The computational complexity of OSKD isOðN2dþN3=L2

Þ, which is lower than the complexities of KDA,KSDA and KLFDA.

5. Experiments

In this section, we compare the proposed approach withseveral representative discriminant methods on the public ARand FERET face databases and the PolyU palmprint database. Forall compared methods, we use the nearest neighbor classifier withthe cosine distance to do classification. The cosine distance

e database.

MLDA RLDA OSDD

8.87 81.3977.47 81.5878.45 84.3678.19OSKD

8.22 85.4078.16

1 5 10 15 20 25 3040

50

60

70

80

9095

Random Testing No.

Rec

ogni

tion

Rat

es (%

)


90

95KDA KSDA KLFDA OSKD


between two vectors x and y is defined as follows:

dcosðx,yÞ ¼xT y

:x:U:y:,

where :U: is the notation of Euclidean norm.

5.1. Experiments using the AR face database

The AR face database [39] contains over 4000 color face imagesof 126 people (70 men and 56 women), including frontal views offaces with different facial expressions, under different lightingconditions and with various occlusions. Most of the pictures weretaken in two sessions (separated by two weeks). Each sessionyielded 13 color images, with 119 individuals (65men and 54women) participating in each session. We selected images from119 individuals for use in our experiment for a total number of3094 (¼119�26) samples. All color images are transformed intogray images and each image was scaled to 60�60 with 256 Gylevels. Fig. 3 illustrates all of the samples of one subject, where(a)–(m) are from Session 1 and (n)–(z) are from Session 2. Thedetails of the images are (a) and (n), neutral expression; (b) and(o), smile; (c) and (p), anger; (d) and (q), scream; (e) and (r), leftlight on; (f) and (s), right light on; (g) and (t), all sides light on;(h) and (u), wearing sun glasses; (i) and (v), wearing sun glassesand left light on; (j) and (w), wearing sun glasses and right lighton; (k) and (x), wearing scarf; (l) and (y) wearing scarf and leftlight on; (m) and (z), wearing scarf and right light on.

In order to evaluate the impact of different variations to therecognition results, we run each compared methods 30 times. Ineach time, we randomly choose 6 images of every subject astraining samples. The rest is for testing.

By using the improved stability criterion described in Eq. (13)and Eq. (14), we find that the optimal number of subsets (i.e., thevalue of Lo) is 3. Table 3 shows the recognition rates and theoverall conflict values (bK ) varying with the different number ofsubsets L. It shows that the recognition rates are almost on theopposite trend compared to the overall conflict values.

Fig. 4 shows the recognition rates of 30 random tests of ourapproaches and other compared methods: (a) OSDD, LDA, SDA,LFDA, MLDA (Multiple LDA) and RLDA (Random sampling LDA);(b) OSKD, KDA, KSDA and KLFDA. To get the best recognitionresults, RLDA constructs 10 LDA subspaces and 10 N-LDA (Null

Fig. 5. Demo images of one subject from the FERET face database.

Table 5


L 1 2 3 4 5

Rates (%) 51.57 69.93 77.43 79.86 79.86bK 0.2477 0.2145 0.2183 0.2231 0.22

space LDA) subspaces, and MLDA divides the dataset into 2 clus-ters. The average recognition results of related methods areshown in Table 4. In this paper, we consider the Gaussian kernelkðx,yÞ ¼ expð�:x�y:2

=2d2i Þ for the compared kernel methods, and

set the parameter di ¼ i� d, iA1,. . .,20, where d is the standarddeviation of training dataset. For each compared kernel method,the parameter i was selected such that the best classificationperformance was obtained.

Table 4 shows that OSDD and OSKD perform better than othercompared linear and nonlinear methods. Compared with LDA,SDA, LFDA, MLDA and RLDA, OSDD improves the average recogni-tion rate at least by 2.78% (¼84.36%�81.58%). And the averagerecognition rate of OSKD is at least 3.07% (¼85.40%�82.33%)higher than KDA, KSDA and KLFDA.

5.2. Experiments using the FERET face database

The FERET database [40] contains 2200 facial images from 200individuals with each person contributing 11 images. The images,named with two-character strings ranging from ‘‘ba’’ to ‘‘bk’’, were

n the FERET face database.

6 7 8 9 10

80.93 81.50 81.87 81.06 80.9403 0.2127 0.2074 0.2062 0.2151 0.2193

1 5 10 15 20 25 3055

60

65

70

75

80

85

Random Testing No.

Rec

ogni

tion

Rat

es (%

)

Fig. 6. Recognition rates of all compared methods on the FERET face database.


captured under various illuminations and display a variety of facialexpressions and poses. Each image is 384n256 with 256 gray levels.Since many images in this database include the background and thebody chest region, we automatically cropped every image sample.That is, we crop the image and preserve the rows from the 40th to the340th in the original image, producing an image size of 300�256.We scaled the intercepted images to 60�50 to preserve the aspectratio. Fig. 5 shows all of the samples of one subject. The details of theimages are as follows: (a) regular facial status; (b) þ151 pose angle;(c) �151 pose angle; (d) þ401 pose angle; (e) �401 pose angle; (f)þ251 pose angle; (g) �251 pose angle; (h) alternative expression;(i) different illumination; (j) þ601 pose angle; (k) �601 pose angle.We run each compared method 30 times. In each time, 4 images ofeach person are randomly selected as training samples. And theremainder 7 images are regarded as testing samples.

The number of subsets in this experiment is set as 8 accordingto the improved stability criterion. Table 5 shows the recognitionrates and the values of bK varying with the different number ofsubsets L.

Fig. 6a and b show the recognition rates across 30 runs of OSDD,OSKD and related linear and nonlinear discriminant methods,respectively. To obtain the best recognition performance, RLDAconstructs 10 random subspaces, and the optimal number ofclusters in MLDA is 10. The average recognition results are listedin Table 6. OSDD boosts the average recognition rate at least by

Fig. 7. Demonstration images of one su

Table 7


L 1 2 3 4 5

Rates (%) 89.45 97.59 97.06 97.16 98.64

bK 0.1788 0.1774 0.1657 0.1746 0.156

Table 6Average recognition rates and standard deviations of compared methods on the FERET

Method LDA SDA LFDA


Average rates (%) 74.2578.55 76.6175.16 77.627

3.6% (¼83.32%�79.72%) than other linear methods, and OSKDboosts the average rate at least by 6.81% (¼84.43%�77.62%) thanother kernel methods.

5.3. Experiments using the PolyU palmprint database

We use a palmprint database provided by the Hong KongPolytechnic University (HK PolyU). This database collected palm-print images from 189 individuals. The subjects mainly consistedof volunteers from the students and staff at HK PolyU. The subjectwas asked to provide about 10 images, each of the left palm andthe right palm. Therefore, each person provided 20 images so thatthe database contains a total of 3780 images from 189 indivi-duals. The size of all the original palmprint images is 384�284pixels with 75 dpi resolution. Using the processing method in[41], the sub-images with a fixed size (128�128) are extractedfrom the original images. To reduce the computational cost, eachsub-image is compressed to 64�64. We use these sub-images torepresent the original palmprint images and to conduct ourexperiments. Fig. 7 shows all the image samples of one subject.The major changes are in illumination, position including shiftand rotation, and texture details. Similar to the kinds of changesencountered in facial expressions, the image may also be slightlyaffected by the way the hand is posed, shrunk, or stretched.

bject from the palmprint database.

n the PolyU palmprint database.

6 7 8 9 10

98.68 99.04 98.97 98.11 98.24

1 0.1579 0.1507 0.1531 0.1609 0.1685

face database.

MLDA RLDA OSDD

5.39 68.7377.48 79.7275.92 83.3275.18

OSKD

5.64 84.4375.17

Table 9Computational complexities and average computing time of OSDD and other

compared linear methods on the AR, FERET and PolyU databases.

Method Complexity Average computing time (s)

AR face

database

FERET face

database

PolyU palmprint

database


We compare the proposed methods with other comparedmethods 30 times. In each time, we randomly choose 4 imagesof each person as training samples. The rest is for testing. In thisexperiment, the optimal number of subsets is 7. Table 7 shows therecognition rates and the values of bK varying with the differentnumber of subsets L.

Fig. 8a and b separately show the recognition rates of 30random tests of OSDD, OSKD and their compared methods. RLDAconstructs 6 random subspaces, and MLDA divides the datasetinto 4 clusters. Our proposed OSDD and OSKD approaches out-perform other compared linear and nonlinear methods in allcases. Table 8 shows the average recognition results of OSDD,OSKD and related methods. OSDD improves the average recogni-tion rate at least by 2.71% (¼97.56%�94.85%) than other linearmethods, and OSKD improves the average rate at least by 2.40%(¼98.77%�96.37%) than other kernel methods.

1 5 10 15 20 25 30

75

80

85

90

95

100

Random Testing No.

Rec

ogni

tion

Rat

es (%

)


1 5 10 15 20 25 3091

92

93

94

95

96

97

98

99

100

Random Testing No.

Rec

ogni

tion

Rat

es (%

)

KDA KSDA KLFDA OSKD

Fig. 8. Recognition rates of all compared methods on the PolyU palmprint

database.

Table 8Average recognition rates and standard deviations of compared methods on the PolyU

Method LDA SDA LFDA


Average rates (%) 96.2470.60 96.3771.48 94.457

6. Further experimental analysis

In this section, we analyze the performance of our OSDDapproach and other compared methods on the aspects of compu-tational complexities, robustness, and stability.

palmprint database.

MLDA RLDA OSDD

2.29 94.5372.88 94.8572.32 97.5672.13

OSKD

1.36 98.7770.45

LDA O(N2d) 9.39 14.57 16.23

SDA O(N2dþ lN3) 25.61 48.48 59.41

LFDA O(cN2d) 24.66 24.05 28.84

MLDA Oððs=kÞN2dÞ 6.86 9.05 10.48

RLDA OðN2dþhN3i Þ

40.84 29.52 22.24

OSDD O(N2d) 7.31 7.82 9.54

Table 10Computational complexities and average computing time of OSKD and other

compared nonlinear methods on the AR, FERET and PolyU databases.

Method Complexity Average computing time (s)

AR face

database

FERET face

database

PolyU palmprint

database

KDA O(N2dþN3) 10.97 16.43 21.33

KSDA O(N2dþ lN3) 27.05 63.60 96.82

KLFDA O(cN2dþN3) 21.78 29.32 43.22

OSKD OðN2dþðN3=L2ÞÞ 9.16 10.36 15.07

LDA SDA LFDA MLDA RLDA OSDD KDA KSDA KLFDA OSKD0

10

20

30

40

50

60

70

Ave

rage

com

putin

g tim

e (s

)

Fig. 9. Total average computing time (s) of all compared methods on three

databases.


6.1. Comparison of computing time

Table 9 shows the computational complexities and averagecomputing time of OSDD and other linear compared methods onthree databases, and Table 10 shows those results of OSKD andrelated nonlinear methods. Fig. 9 shows the total average com-puting time of all methods on three databases. Table 9 and Fig. 9show that OSDD and MLDA have lower computational complex-ities than other methods. And OSDD spends less computing timethan other compared methods on the FERET face and PolyU

OSDD NOSDD−1 NOSDD−2 NOSDD−3 NOSDD−4 NOSDD−578

79

80

81

82

83

84

85

AR Face Database

Rec

ogni

tion

Rat

es (%

)


79

80

81

82

83

84

FERET Face Database

Rec

ogni

tion

Rat

es (%

)


93

94

95

96

97

98

PolyU Palmprint Database

Rec

ogni

tion

Rat

es (%

)

Fig. 10. Recognition rates of OSDD and NOSDD (random sampling 5 times) on

three databases.

palmprint databases. Table 10 shows that OSKD consumes lesscomputing time than related nonlinear methods on threedatabases.

6.2. Robustness analysis of the OSDD approach

To give a sense of the robustness of the proposed OSDDapproach, we compare the recognition results based on optimalsubset-division (i.e., OSDD) and non-optimal subset-division (i.e.,NOSDD). OSDD adopts the optimal subset-division by using theimproved stability criterion. And NOSDD employs a non-optimalsubset-division strategy using random sampling method. Fig. 10shows the average recognition rates of OSDD and NOSDD (5different random sampling realizations) across 30 random testson the AR, FERET face and PolyU palmprint databases. It showsthat the optimal subset-division is superior to the non-optimalsubset-division, which demonstrates the robustness of our OSDDapproach.

6.3. McNemar’s test

To statistically analyze the classification results given inTables 4, 6 and 8, we conduct a statistical test, McNemar’s test,which gives statistical significance of OSDD and OSKD comparedwith related methods. In our experiments, the McNemar’s testuses a significance level of 0.05. If the p-value is below thesignificance level 0.05, the performance difference between twocompared methods is considered to be statistically significant.Table 11 shows the p-values between OSDD and other comparedlinear methods, and Table 12 shows the p-values between OSKDand related nonlinear methods. According to Tables 11 and 12,the following conclusions are reached:

(1)

Tablp-va

Me

A

OS

OS

OS

OS

OS

a

sign

Tablp-va

Me

A

OS

OS

OS

a

sign

The proposed OSDD approach statistically significantly out-performs other compared linear methods, including LDA, SDA,LFDA, MLDA and RLDA, on all the three databases.

(2)
The proposed OSKD approach is significantly better thanrelated nonlinear methods including KDA, KSDA and KLFDAon the AR and FERET face databases.
e 11lues between OSDD and other compared linear methods on three databases.

thod Method

B

AR face

database

FERET face

database

PolyU palmprint

database

DD LDA 3.8671�10�8a 3.5309�10�8a 8.6825�10�8a

DD SDA 3.3145�10�5a 2.5395�10�8a 2.0966�10�7a

DD LFDA 1.8759�10�8a 2.8052�10�8a 0.0280a

DD MLDA 0.0297a 2.1881�10�8a 9.7824�10�5a

DD RLDA 0.0417a 7.4826�10�7a 4.1930�10�8a

Statistically significant difference between Method A and Method B at a

ificance level of 0.05.

e 12lues between OSKD and related nonlinear methods on three databases.

thod Method

B

AR face

database

FERET face

database

PolyU palmprint

database

KD KDA 3.7666�10�5a 7.0114�10�4a 0.0397a

KD KSDA 0.0356a 3.1638�10�5a 0.1506

KD KLFDA 9.4147�10�8a 1.1058�10�8a 1.3940�10�8a

Statistically significant difference between Method A and Method B at a

ificance level of 0.05.


(3)
On the PolyU palmprint database, OSKD significantly outper-forms KDA and KLFDA. Although the recognition rates ofOSKD is better than KSDA (see Table 8), the performancedifference between OSKD and KSDA on the PolyU database isnot statistically significant.
7. Conclusions

In this paper, we propose an optimal subset-division baseddiscrimination (OSDD) approach and its kernel extension (OSKD)for feature extraction and classification. OSDD first divides thesample set into several subsets by using the optimal subsets divisionstrategy. It then calculates the discriminant vectors from all subsets,and combines them to construct the projection transformation.OSKD acquires nonlinear projection transformation from all subsetsin the kernel space. Experimental results on the AR and FERET facedatabases and the PolyU palmprint database show that the proposedapproaches outperform several related discriminant analysis meth-ods. OSDD improves the average recognition rate at least by 2.71% incontrast with linear discriminant methods including LDA, SDA,LFDA, MLDA and RLDA, and OSKD improves average rate at leastby 2.40% in contrast with KDA, KSDA and KLFDA. Experimentalresults also demonstrate the higher computational efficiency,robustness and statistically significance of our approaches. Hence,the proposed approaches enhance the classification performance ofdiscriminant analysis effectively.

Acknowledgment

The work described in this paper was fully supported by theNSFC under Project nos. 61073113 and 60772059, the NewCentury Excellent Talents of Education Ministry under Projectno. NCET-09-0162, the Doctoral Foundation of Education Ministryunder Project no. 20093223110001, the Qing-Lan EngineeringAcademic Leader of Jiangsu Province, and the Foundation ofJiangsu Province Universities under Project no. 09KJB510011.

References

[1] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition,Academic Press, Inc., 1990.

[2] K. Etemad, R. Chellapa, Discriminant analysis for recognition of human faceimages, Journal of the Optical Society of America A 14 (8) (1997) 1724–1733.

[3] P.K. Mohanty, S. Sarkar, P.J. Phillips, R. Kasturi, Subspace approximation offace recognition algorithms: an empirical study, IEEE Transactions onInformation Forensics and Security 3 (4) (2008) 734–748.

[4] A. Kong, D. Zhang, M. Kamel, A survey of palmprint recognition, PatternRecognition 42 (7) (2009) 1408–1418.

[5] P.N. Belhumeur, J.P. Hespanha, D.J. Kriegman, Eigenfaces vs. Fisherfaces:recognition using class specific linear projection, IEEE Transactions on PatternAnalysis and Machine Intelligence 19 (7) (1997) 711–720.

[6] H. Yu, J. Yang, A direct LDA algorithm for high-dimensional data with applicationto face recognition, Pattern Recognition 34 (10) (2001) 2067–2070.

[7] C.J. Liu, H. Wechsler, Gabor feature based classification using the enhancedFisher linear discriminant model for face recognition, IEEE Transactions onImage Processing 11 (4) (2002) 467–476.

[8] D.Q. Dai, P.C. Yuen, Regularized discriminant analysis and its application toface recognition, Pattern Recognition 36 (3) (2003) 845–847.

[9] X.Y. Jing, D. Zhang, Z. Jin, UODV: improved algorithm and generalized theory,Pattern Recognition 36 (11) (2003) 2593–2602.

[10] M. Li, B.Z. Yuan, 2D-LDA: a statistical linear discriminant analysis for imagematrix, Pattern Recognition Letters 26 (5) (2005) 527–532.

[11] J. Ye, Q. Li, A two-stage linear discriminant analysis via QR decomposition,IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (6) (2005)929–941.

[12] H. Cevikalp, M. Neamtu, M. Wilkes, A. Barkana, Discriminative commonvectors for face recognition, IEEE Transactions on Pattern Analysis andMachine Intelligence 27 (1) (2005) 4–13.

[13] X.Y. Jing, D. Zhang, Y.Y. Tang, An improved LDA approach, IEEE Transactionson Systems, Man and Cybernetics Part B 34 (5) (2004) 1942–1951.

[14] T.K. Kim, S.F. Wong, B. Stenger, J. Kittler, R. Cipolla, Incremental lineardiscriminant analysis using sufficient spanning set approximations, in:Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni-tion (CVPR), 2007, pp. 1–8.

[15] Y. Zhang, D.Y. Yeung, Semi-supervised discriminant analysis using robustpath-based similarity, in: Proceedings of the IEEE Conference on ComputerVision and Pattern Recognition (CVPR), 2008, pp. 1–8.

[16] X.Y. Jing, S. Li, D. Zhang, J.Y. Yang, Face recognition based on local uncorre-lated and weighted global uncorrelated discriminant transforms, in: Proceed-ings of the IEEE International Conference on Image Processing (ICIP 2011),2011, pp. 3110–3113.

[17] D.C. Tao, X.L. Li, X.D. Wu, S.J. Maybank, Tensor rank one discriminantanalysis— a convergent method for discriminative multilinear subspaceselection, Neurocomputing 71 (10–12) (2008) 1866–1882.

[18] W.S. Zheng, J.H. Lai, P.C. Yuen, S.Z. Li, Perturbation LDA: learning thedifference between the class empirical mean and its expectation, PatternRecognition 42 (5) (2009) 764–779.

[19] M. Zhu, A.M. Martinez, Optimal subclass discovery for discriminant analysis,in: Proceedings of the IEEE Workshop Learning in Computer Vision andPattern Recognition (LCVPR), 2004, p. 97.

[20] M. Zhu, A.M. Martinez, Subclass discriminant analysis, IEEE Transactions onPattern Analysis and Machine Intelligence 28 (8) (2008) 1274–1286.

[21] M. Sugiyama, Local Fisher discriminant analysis for supervised dimension-ality reduction, in: Proceedings of the 23rd International Conference onMachine Learning (ICML 2006), Pittsburgh, PA, 2006, pp. 905–912.

[22] M. Uray, P. Roth, H. Bischof, Efficient classification for large-scale problems bymultiple LDA subspaces, in: Proceedings of the Fourth International Conferenceon Computer Vision Theory and Applications VISAPP, 2009, pp. 299–306.

[23] X. Wang, X. Tang, Random sampling LDA for face recognition, in: Proceedingsof the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),vol. 2, 2004, pp. 259–265.

[24] X. Wang, X. Tang, Random sampling for subspace face recognition, Interna-tional Journal of Computer Vision 70 (1) (2006) 91–104.

[25] S. Mika, G. Ratsch, J. Weston, B. Scholkopf, K.R. Muller, Fisher discriminantanalysis with kernels, in: Proceeding of the IEEE Neural Networks for SignalProcessing Workshop, 1999, pp. 41–48.

[26] G. Baudat, F. Anouar, Generalized discriminant analysis using a kernelapproach, Neural Computation 12 (2000) 2385–2404.

[27] J. Lu, K.N. Plataniotis, A.N. Venetsanopoulos, Face recognition using kerneldirect discriminant analysis algorithms, IEEE Transactions on Neural Net-works 14 (1) (2003) 117–126.

[28] Q.S. Liu, H.Q. Lu, S.D. Ma, Improving kernel Fisher discriminant analysis forface recognition, IEEE Transactions on Circuits and Systems for VideoTechnology 14 (1) (2004) 42–49.

[29] J. Yang, A.F. Frangi, D. Zhang, J.Y. Yang, Z. Jin, KPCA plus LDA: a completekernel Fisher discriminant framework for feature extraction and recognition,IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (2) (2005)230–244.

[30] X.Y. Jing, Y.F. Yao, D. Zhang, J.Y. Yang, M. Li, Face and palmprint pixel levelfusion and KDCV-RBF classifier for small sample biometric recognition,Pattern Recognition 40 (11) (2007) 3209–3224.

[31] B. Chen, L. Yuan, H. Liu, Z. Bao, Kernel subclass discriminant analysis,Neurocomputing 72 (1–3) (2007) 455–458.

[32] M. Sugiyama, Dimensionality reduction of multimodal labeled data by localFisher discriminant analysis, Journal of Machine Learning Research 8 (2007)1027–1061.

[33] X.Y. Jing, S. Li, Y.F. Yao, L.S. Bian, J.Y. Yang, Kernel uncorrelated adjacent-classdiscriminant analysis, in: Proceedings of the International Conference onPattern Recognition (ICPR), 2010, pp. 706–709.

[34] S. Li, X.Y. Jing, D. Zhang, Y.F. Yao, L.S. Bian, A novel kernel discriminant featureextraction framework based on mapped virtual samples for face recognition,in: Proceedings of the IEEE International Conference on Image Processing(ICIP 2011), 2011, pp. 3066–3069.

[35] A.M. Martinez, M. Zhu, Where are linear feature extraction methods applic-able? IEEE Transactions on Pattern Analysis and Machine Intelligence 27 (12)(2005) 1934–1944.

[36] H.M. Srivastava, Some simple algorithms for the evaluations and representa-tions of the Riemann Zeta function at positive integer arguments, Journal ofMathematical Analysis and Applications 246 (2000) 331–351.

[37] J. Shawe-Taylor, N. Cristianini, Kernel Methods for Pattern Analysis, Cam-bridge University Press, Cambridge, 2004.

[38] X. Tao, J. Ye, Q. Li, R. Janardan, V. Cherkassky, Efficient kernel discriminantanalysis via QR decomposition. in: Proceedings of the Eighteenth AnnualConference on Neural Information Processing Systems (NIPS 2004), 2004,pp. 1529–1536.

[39] A.M. Martinez, R. Benavente, The AR Face Database, CVC Technical Report 24,1998.

[40] P.J. Phillips, H. Moon, P. Rauss, S.A. Rizvi, The FERET evaluation methodologyfor face recognition algorithms, IEEE Transactions on Pattern Analysis andMachine Intelligence 22 (10) (2000) 1090–1104.

[41] D. Zhang, W.K. Kong, J. You, M. Wong, On-line palmprint identification, IEEETransactions on Pattern Analysis and Machine Intelligence 25 (9) (2003)1041–1150.


Xiao-Yuan Jing received his M.Sc. and Ph.D. in Pattern Recognition from Nanjing
University of Science and Technology (1995 and 1998, respectively), China. He was aProfessor at the Bio-computing Research Center of Shenzhen Graduate School, Harbin Institute of Technology, China. He is now a Professor at the State Key Laboratory ofSoftware Engineering, Wuhan University, and at the College of Automation, Nanjing University of Posts and Telecommunications, China. His research interest includesbiometrics, pattern recognition, image processing, neural networks, machine learning and artificial intelligence.
Sheng Li received his B.Sc. degree in Computer Science from the Nanjing University of Posts and Telecommunications in 2010. He is now reading for the M.Sc. degree inInformation Security at the Nanjing University of Posts and Telecommunications, China. His research interest includes pattern recognition, biometrics and machinelearning.

David Zhang received his M.Sc. and Ph.D. in Computer Science from the Harbin Institute of Technology (1982 and 1985, respectively). In 1994, he received his second Ph.D.in Electrical and Computer Engineering from the University of Waterloo, Canada. Currently he is a professor at the Hong Kong Polytechnic University, where he is thefounding director of the Biometrics Technology Center supported by Hong Kong SAR government. His research interest includes biometrics, image processing and patternrecognition.

Chao Lan received his M.Sc. degree in Pattern Recognition from the Nanjing University of Posts and Telecommunications in 2011. He is now reading for the Ph.D. inComputer Science at the University of Kansas, USA. His research interest includes pattern recognition and machine learning.

Jing-Yu Yang is now a Professor at the College of Computer Science, Nanjing University of Science and Technology, China, where he is the founder of Research Institute ofPatten Recognition and Intelligent Systems. His research interest includes pattern recognition, artificial intelligence, biometrics and robotics.

optimal subset-division based discrimination and its kernelization

Documents