jenn-jier james lien (連震杰)
DESCRIPTION
Motivation Challenges of face recognition Facial variations Face recognition using image sets Surveillance Video retrieval illumination pose facial expressionTRANSCRIPT
Jenn-Jier James Lien ()
Kernel Discriminant Analysis Based on Canonical Difference for Face
Recognition in Image Sets Wen-Sheng Chu () Ju-Chin Chen ()
Jenn-Jier James Lien () Robotics Lab, CSIE NCKU CVGIP 2007
Motivation Challenges of face recognition
Facial variations Face recognition using image sets Surveillance
Video retrieval illumination pose facial expression Why Multi-view
Image Sets?
Multiple facial images contain more information than a single
image. Person A Person A Person B Person B A or B? Single input
pattern (Single-to-many) Multiple input patterns (Many-to-many)
Training/Testing Data: Facial Expression
For subjecti Image Set 1 Training Image Set 2 Image Set 3 Image Set
4 Testing Image Set 5 More Training/Testing Data: Illumination
(Yale B)
For subjectj Image Set 1 Training Image Set 2 Image Set 3 Image Set
4 Testing Image Set 5 System Overview T Pi Testing Data Testing
image set Xtest
Training Data Testing Data Subject N Xm-2 Xm ... Xm-1 Subject 1 X3
X1 X2 Training image sets {X1,,Xm} Testing image set Xtest ...
Testing Process Kernel Subspace Generation Ptest Training Process
Kernel Subspace Generation (Total m subspaces) Kernel Discriminant
Transformation (KDT) Pi Reference Subspace Reftest X T Reference
Subspace: Refi=TTPi Output Identification result Training Process
Pi Testing image set Xtest Training Data
Xi={ , ,, } 32 1 2 ni ni ~= 100 Training Data Subject N Xm-2 Xm ...
Xm-1 Subject 1 X3 X1 X2 Training image sets {X1,,Xm} Testing image
set Xtest ... Testing Process Kernel Subspace Generation Ptest
Training Process Kernel Subspace Generation Pi (Total m subspaces)
Kernel Discriminant Transformation (KDT) Pi Pi={ei1,,eid} Reference
Subspace Reftest X Reference Subspace: Refi=TTPi Identification
result Kernel Subspace Generation (KSG)
h ni Nonlinearmappingfunction 32 x 32 ni Kernel subspace of Xi
(dImage Set Xi Kernel Matrix Kii Kernel Subspace Pi X1={ , ,, } 1 2
n1 K11 P1= ,d < ni Xm={ , ,, } nm 2 1 Kmm Pm= From the theory of
reproducing kernels, Dimensionality may be ! i-th image set where
SVD s-th image of i-th image set KPCA: B Scholkopf, A Smola, KR
Muller - Advances in Kernel Methods-Support Vector Learning, 1999
Training Process: Kernel Discriminant Transformation (KDT)
Subject N Xm-2 Xm ... Xm-1 Subject 1 X3 X1 X2 Training image sets
{X1,,Xm} Testing image set Xtest ... Testing Process Kernel
Subspace Generation Ptest Kernel Subspace Generation (Total m
subspaces) Kernel Discriminant Transformation (KDT) Pi Reference
Subspace Reftest X T Reference Subspace: Refi=TTPi Identification
result How to measure similarity?
KDT: Main Idea Based on the concept of LDA, KDT is derived to find
a transformation matrix T. We proposed an iterative process to
optimize T. Dimensionality of T is assumed to be w. d x w
transformation matrix (KDT matrix) Within Subjects Subspace
32x32-dim d-dim w-dim KPCA 1 1 1 T KPCA How to measuresimilarity?
Between Subjects 2 2 2 KDT: Canonical Difference (CD) Similarity
Measurement
Kernel subspace P1 Canonical subspace C1 Kernel subspace P2
Canonical subspace C2 d1 d2 u1 u2 v1 v2 Capture more common views
and illumination than eigenvectors. KDT: CD Canonical Vector v.s.
Eigenvector (cont.)
eigenvectors B1 B2 B1B2 canonical vectors C1 C2 A similarity
measurement of two subspaces C1C2 KDT: CD Canonical Subspace
(cont.)
Consider SVD on orthonormal basis matrices B1 and B2: d-dimensinoal
orthonormal basis matrices eigenvector SVD 0 Eigenvalue = cos2i 1
Similarity measurement canonical subspaces (also orthonormal) T.K.
Kim, J. Kittler and R. Cipolla, Discriminative Learning and
Recognition of Image Set Classes Using Canonical Correlations, IEEE
Trans. on PAMI, 2007 KDT: KDT Matrix Optimization
Kernel Subspace Reference Subspace Canonical Subspace T Iterative
learning Canonical Difference Based on LDA Kernel subspace
Orthonormal basis matrices are required to obtain canonical
subspaces Ci. Is Refi normalized? Usually not! KDT: Kernel Subspace
Normalization
QR-decomposition is performed to obtain two orthonormal basis
matrices. d d invertible upper triangular matrix w d orthonormal
matrix SVD KDT: Formulation Canonical Subspace Derivation Qi =
TTPiRi-1 SB, Sw
Form of LDA T KDT: Solution T={t1,,tq,,tw}
Contain the info of Dimensionality may be! T T={t1,,tq,,tw} Replace
using kernel trick
Derivation Using the theory of reproducing kernels again: Replace
using kernel trick Following similar steps, we can obtain KDT:
Numerical Issues is solved by simply computing the leading
eigenvectors of U-1V. To make sure that U is positive-definite, we
regularize U by U (=0.001) where Training Process Refi = TTPi where
each element is given by T Pi
Subject N Xm-2 Xm ... Xm-1 Subject 1 X3 X1 X2 Training image sets
{X1,,Xm} Testing image set Xtest ... Testing Process Kernel
Subspace Generation Ptest Kernel Subspace Generation (Total m
subspaces) Kernel Discriminant Transformation (KDT) Refi = TTPi
where each element is given by Pi Reference Subspace Reftest X T
Reference Subspace: Refi=TTPi Identification result Testing Process
T T Pi Testing image set Xtest
Subject N Xm-2 Xm ... Xm-1 Subject 1 X3 X1 X2 Training image sets
{X1,,Xm} Testing image set Xtest ... Testing Process Kernel
Subspace Generation Ptest Kernel Subspace Generation (Total m
subspaces) T Kernel Discriminant Transformation (KDT) Pi X X
Reference Subspace Reftest=TTPtest T Reference Subspace: Refi=TTPi
Identification result Training List #individual (N) 32 #image
set/individual 3
#image/set (ni) ~100 size of normalized template 32x32
dimensionality KMSM 30 KCMSM DCC 20 KDT of Gaussian kernel function
0.05 for regularization 10-3 Training: Convergence of Jacobian
Value
J() tends to converge to a specified value under different
initializations. Testing: Comparison with Other Methods
The proposed KDT is compared to 3 related methods under 10 randomly
chosen experiments. KMSM (avg=0.837) KCMSM (0.862) DCC (0.889) KDT
(0.911) Conclusions Canonical differences is provided as a
similarity measurement between two subspaces. Based on canonical
difference, we derived a KDT and applied it to a proposed face
recognition system. Our system is capable of recognizing faces
using image sets against facial variations. Thanks for your
attention Related Works Mutual subspace method (MSM) Constrained
MSM (CMSM)
Subspace V Subspace U project project Constrained Subspace c Uc Vc
Discriminantive canonical correlation (DCC) Kernel MSM (KMSM),
Kernel CMSM (KCMSM) Mutual Subspace Method (MSM)
Utilize the canonical angles for similarity. Subspace B1 Subspace
B2 u1 Eigenvectors u2 v1 u 2 v2 v 1 K. Fukui and O. Yamaguchi, Face
Recognition Using Multi-viewpoint Patterns for Robot Vision, ISRR
2003 Perform KDT on Subspace?
By KPCA, we can obtain s.t. Multiply T to both sides of equal sign,
It can be observed that the kernel subspace of transformed mapped
image sets is equivalent to applying T to the original kernel
subspace. Using the theory of reproducing kernels again:
KDT Optimization Using the theory of reproducing kernels again:
Following similar steps, we can obtain That is, T={t1,,tq,,tw}
where Training: Dimensionality w of KDT V.S. Identification
Rate
The identification rate is guaranteed to be greater than 90% after
w > 2,200. Training: Similarity Matrix
32 1 Similarity matrix behaves better after 10-times iterative
learning. 1 1 32 Similarity Id Number 1st iteration 10th iteration
Kij KSG: Kernel Matrix Gaussian kernel function:
Kernel matrix Kij: the correlation between i-th image set and j-th
image set. j-th image set ... r 1 2 nj i-th image set Kij ni nj
Kernel trick s 1 2 ni