biometrics face recognition - cedar.buffalo.edu · • train pca • during face recognition see if...
TRANSCRIPT
http://www.cubs.buffalo.edu
Face Recognition
University at Buffalo CSE666 Lecture Slides
Resources:http://www.face-rec.org/algorithms/
http://www.cubs.buffalo.edu
Overview of face recognition algorithms
• Correlation- Pixel based correspondence between two face images
• Structural- Based on identifying landmark points
• Linear models- PCA (Principal Component Analysis)- LDA (Linear Discriminant Analysis)- ICA (Independent Component Analysis)- Combinations of above
• Non-linear models-Kernel mapping (and using PCA, LDA, ICA)-Active shape/appearance models-Manifold mapping
• 2D Model vs. 3D Model• Matching vs. Classification
http://www.cubs.buffalo.edu
Correlation MatchingTest Image Test Image captured captured by by cameracamera
Correlation Correlation Filter H Filter H (Template)(Template)
Frequency Frequency Domain arrayDomain array
FFTFFT
N x N pixelsN x N pixels
N x N pixelsN x N pixelsResulting Resulting Frequency Frequency Domain arrayDomain array
IFFTIFFT
PSRPSR
*B.V.K. Vijaya Kumar, Marios Savvides, C. Xie, K. Venkataramani, J. Thornton and A. Mahalanobis, “Biometric Verification using Correlation Filters”, Applied Optics, 2003*B.V.K. Vijaya Kumar, M. Savvides, K. Venkataramani, C. Xie, "Spatial frequency domain image processing for biometric recognition," IEEE Proc. of International Conference on Image Processing (ICIP), Vol. I, 53-56, 2002
http://www.cubs.buffalo.edu
Elastic Bunch Graph Matching
L. Wiskott, J.-M. Fellous, N. Krueuger, C. von der Malsburg, Face Recognition by Elastic Bunch GraphMatching, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, 1997
http://www.cubs.buffalo.edu
Elastic Bunch Graph Matching
L. Wiskott, J.-M. Fellous, N. Krueuger, C. von der Malsburg, Face Recognition by Elastic Bunch GraphMatching, IEEE Trans. on Pattern Analysis and Machine Intelligence, Vol. 19, No. 7, 1997
• Heuristic matching algorithm
http://www.cubs.buffalo.edu
Principal Component Analysis (PCA)
Previous discussion: PCA as a feature extraction method
( )iaA =
∑=
=n
iiiy
1
ax
∑=
=m
iiiy
1
ˆ ax
- representation of original vector in
KL basis vectors
- the projection on subspace spanned by eigenvectors with largest eigenvalues
∑+=
=−n
miiE
1
2 ])ˆ[( λxx
- Karhunen-Loeve transform
ia - eigenvectors of matrix ][ Tx E xxR =
x
m
Axy = - projection of original vector to KL basisx
xAy mm = - first PCA coefficients –‘best’ features
m
Features Representation
- sum of the smallest eigenvaluesmn −
http://www.cubs.buffalo.edu
PCA as appearance model
xAy mm = - first PCA coefficients –
‘best’ features - should provide a good approximation to all the samples (for which PCA was trained)
• Distance from feature space (DFFS) should be small:
m
|ˆ| xx −=DFFS
x
x̂
• PCA provides a model on typical faces• DFFS can be used it to separate faces from non-faces
• DIFS (distance in feature space) can be used to match two faces
mTm yAx =ˆ
|ˆˆ| 21 xx −=DIFS
http://www.cubs.buffalo.edu
PCA for face recognition
• Collect a set of face images for training
• Train PCA
• During face recognition see if the distance between test and enrolled PCA feature vectors is less than threshold:
M. Turk, A. Pentland, Eigenfaces for Recognition, Journal of Cognitive Neurosicence, Vol. 3, No. 1, 1991
θ<−=− |||ˆˆ| 2121 yyxx
x̂ - reconstructed face (hyperplane in the original feature space)- projection into PCA feature spacey
http://www.cubs.buffalo.edu
If has dimension , then has dimension
PCA – subspace on training samples
M. Turk, A. Pentland, Eigenfaces for Recognition, Journal of Cognitive Neurosicence, Vol. 3, No. 1, 1991
][ Tx E xxR =x
2N 22 NN ×
Suppose we have training samplesM ][ 21 MxxxX …=
Tx XXR = matrix of dimension 22 NN ×
Instead consider of dimensionXXL T= MM ×
If is an eigenvector of then is an eigenvector of : v L Xv xR
XvXvXXvXvX λλ =⇒= TT
• The number of training samples is usually less than the dimension of image vectors , so this procedure makes sense2NM <
http://www.cubs.buffalo.edu
Linear Discriminant Analysis
Covariance matrix for class :
]))([( Tiii E µxµxS −−=
i
Within-class scatter matrix: ∑=
=M
iiiw P
1
SS
Between-class scatter matrix: ∑=
−−=M
i
Tiiib P
100 ))(( µµµµS
∑=
=M
iiiP
10 µµwhere
)det()det(
)det( 1bw
w
b SSSS −=Optimization criteria: maximize
http://www.cubs.buffalo.edu
Linear Discriminant Analysis
bw SS 1−Solution: projection is determined by the eigenvectors of
• Usually has better performance than PCA• Requires samples of the same class (same person face) to train• Need to make sure that is non-singular
wSwS
Solution:apply PCA first to reduce the number of dimensions, then perform LDA.
http://www.cubs.buffalo.edu
Independent Component Analysis
• Whereas PCA makes uncorrelated features, there might be still dependent• ICA tries to reduce higher order dependence• Search for proper projection is more difficult• Use approximationapproaches
Bartlett et al. ”Face Recognition by Independent Component Analysis”, 2002
http://www.cubs.buffalo.edu
Comparisons of subspace methods
Delac et al. “Independent Comparative Study of PCA, ICA and LDA on the FERET Data Set”, 2006
http://www.cubs.buffalo.edu
Comparisons of subspace methods
Delac et al. “Independent Comparative Study of PCA, ICA and LDA on the FERET Data Set”, 2006
http://www.cubs.buffalo.edu
Kernel Methods• Kernel mapping:
-map original image feature vectors into higher dimensional spaceusing some kernel functions:
- Covariance matrix in kernel space has elements
- Kernel trick: there exists a function
- No need to explicitly calculate - use some choosen kernel function:
- The number of eigenvectors is still limited by the number of training samples as in regular PCA
NfRR fN >>→Φ ,:
)()( ji xx Φ⋅Φ
ijjiji Kk =Φ⋅Φ= )()(),( xxxx
)( ixΦ
2
2
2),( σji
ek ji
xx
xx−
−=
http://www.cubs.buffalo.edu
Kernel Methods
M.-H. Yang, “Face Recognition Using Kernel Methods”, 2002
http://www.cubs.buffalo.edu
Local Linear Embedding
Roweis S., Saul L. “Nonlinear dimensionality reduction by locally linear embedding”, Science, 2000
http://www.cubs.buffalo.edu
• Step 1: find optimal local linear representation of point as a combination of its neighbors
• Step 2: find lower dimensional representation of
Local Linear Embedding
Roweis S., Saul L. “Nonlinear dimensionality reduction by locally linear embedding”, Science, 2000
min)( →−= ∑ ∑i j
jiji XWXW��
ε
iX�
jX�
1=∑j
ijW
0=ijW if j is not a neighbor of i
min)( →−=Φ ∑ ∑i j
jiji YWYY��
iY�
iX�
http://www.cubs.buffalo.edu
Local Linear Embedding
Roweis S., Saul L. “Nonlinear dimensionality reduction by locally linear embedding”, Science, 2000
http://www.cubs.buffalo.edu
Local Linear Embedding
http://www.cubs.buffalo.edu
3D Model Based Recognition
Blanz, Vetter “Face recognition based on fitting a 3D morphable model” PAMI 200303
http://www.cubs.buffalo.edu
3D Model Based Recognition
Blanz, Vetter “Face recognition based on fitting a 3D morphable model” PAMI 200303
http://www.cubs.buffalo.edu
3D Model Based Recognition
Blanz, Vetter “Face recognition based on fitting a 3D morphable model” PAMI 200303