face recognition and biometric systems eigenfaces (3)

Face Recognition and Biometric Systems

Eigenfaces (3)


Plan of the lecture

Eigenfaces-based methods Fisherfaces Bayesian Matching Local PCA

Face relevance mapsError function minimisation Eigenfaces – feature extraction definition of recognition error optimal masks and weights


Eigenfaces – drawbacks

Main drawbacks: holistic method face topology not taken into

account statistical analysis of differences

between images in the training set character of differences not taken

into account


Example


Example: PCA


Example: PCA not helpful


Example: Linear Discriminant Analysis


Fisherfaces

PCA finds main directions of variance class identity not utilised

Methods based on PCA which utilise class identity: Linear Discriminant Analysis (LDA) Fisherfaces


Fisherfaces

Principal Component Analysis: training set covariance matrix

Linear Discriminant Analysis: classified training set two covar.

matrices within-class covariance matrix between-class covariance matrix

orthogonal basis from two matrices


Fisherfaces

Between-class matrix

CB – between-class covariance matrix

C – number of classesMi – number of images in i-th class

– average imagei – average image of i-th class

C

iiiiB M

1)(( μμμμC


Fisherfaces

Within-class covariance matrix

CW – within-class covariance matrix

C – number of classesXi – set of images which belong to i-th class

xk – k-th image which belongs to i-th class

i – average image of i-th class

C

i XikikW

ik1)((

xμxμxC


Fisherfaces

PCA:

- eigenvectors matrix (vectors in columns)

LDA:

vv C

||maxarg TCψψψψ

||

||maxarg

ψCψ

ψCψψ

ψW

TB

T

vv WB CC vvBW CC 1


Fisherfaces

LDA – hard to find inverse matrixFisherfaces – improved approach: PCA for dimensionality reduction LDA for finding optimal orthogonal

basis

|''|

|''|maxarg

ψψCψψ

ψψCψψψ

ψPCAW

TPCA

TPCAB

TPCA

T


Fisherfaces

Feature extraction in the Fisherfaces:

1. Feature vector calculated by PCA normalised image as an input dimensionality reduction

2. Feature vector calculated by LDA PCA feature vector as an input rotation of feature vector no dimensionality reduction


Bayesian Matching

Vectors similarity based on probability of their difference classification

I – set of intra-personal pairs E – set of extra-personal pairs

)|()(),( 21 II PPIIS

21 II


Bayesian Matching

)()|()()|(

)()|()|(

EEII

III PPPP

PPP

P(|) – probability of observing a given difference in a defined set of differences function of PCA back projection error –

() 2)(~)|( eP


Bayesian Matching

Two classes of image pairs intra- and extra-personal

Differences generated from pairs two classes of pairs

PCA used for both classes separately two image spaces


Bayesian Matching

Image difference recognition Dual Eigenfaces

Difference distance from two image spacesBayesian Matching – a slow method image difference calculated for every

comparison possibility of applying other method for

selecting candidates (n most similar images)


Local PCA

Based on detected features eyes, nose, mouth

PCA for features small part of face image analysis of small images (eigeneyes,

eigennoses, etc.)

Less dimensional spacesLower effectiveness, but supports the Eigenfaces


Local PCA

K1

K2

K3

K4


Other methods

Local Feature Analysis2D PCA, 2D LDAIndependent Component Analysis


Face relevance map

Face topology eyes & nose – extra-personal

differences mouth & cheeks – intra-personal

differences

Nature of features concerned with location


Face relevance map

Face relevance map enhance influence of extra-personal

features decrease influence of intra-personal

features

Feature extraction with a map (m)

N

jjjiji mxuw

1x ii uw


Face relevance map

„T” map artificial map for eyes and nose binary values

Results: FeretA: 423 -> 445 (3,7%)

Conclusion: good approach, need for better map generation methods


Face relevance mapDifference map – statistical analysisPairs of images: intra-personal extra-personal

Average differences between images: average intra-personal difference average extra-personal difference

Map obtained by subtracting intra-personal difference from extra-personal oneResults for FeretA: 423 -> 462 (6,4%)


Face relevance map

Colour data information lost during conversion

to GS low distinctiveness can be used for map generation

Colour used for detection eye and mouth map masks based on detection maps


Face relevance map

Desired effect: higher values around eyes and nose lower values in the area of mouth

Maps deliver information about features locationTwo possible approaches: image -> feature maps -> face relevance

map image -> feature maps -> features -> f.r.m.


Face relevance map

Maps from pointsNose location derived from eye & mouth weighted mean

eye(R): (15, 24)eye(L): (49, 24)mouth: (32, 58)


Face relevance map

Single point influence

r – radius, mmax – maximal map value

Map – summed influence of the points eye, nose – positive weights mouth – negative weights

20

20

),(max

)()(),(

),(

yydxxdyxD

emyxm

yx

yxDr


Face relevance map

Maps from colour improvement comparable to

difference maps colour data carry information

concerning nature of face areas generated for every image

Map may be imposed during normalisation


Face relevance map

Maps from colour - examples


Face relevance map

Back-projection based dynamic map dynamic – created for every image

Back projection: map of local projection error higher error = lower importance map should be smoothed

Good for occluded images


Face relevance map

Back-projection based dynamic map examples of occluded face images


Recognition errorMaps take into account difference nature basing on face topologyDifferences not concerned with location lighting

Eigenfaces – appearance interpretation various types of information some responsible for lighting

Weights assigned to eigenvectors:

N

jjjijii mxuw

1


Recognition error

Eigenvectors weights lower values for intra-personal

directions of variance

How to obtain the weights? visual assessment – may be

incorrect the same procedure as in the case

of difference masks


Recognition error

A better method for obtaining maps and eigenvector weights: error function minimisation


Recognition error

Definition of recognition problem: M vectors, C classes and C base vectors

(ui1) Mi vectors in i-th class (uij) classification of non-base vectors (j > 1)

Single comparison similarity to home class and foreign class classes represented by base vectors


Recognition error

Single comparison error:

uij – a vector which is being recognised

ui1 – home class base vector

uk1 – foreign class base vectorS – similarity between vectors

),(

),(),(

1

11

iij

kijkij S

Sd

uu

uuuu


Recognition error

Single comparison:

correct if

incorrect if

1),( 1 kijd uu

1),( 1 kijd uu


Recognition error

Error for comparison with all classes:

Error for the whole set:

C

i

M

j

C

ikk

ij

i

kdD1 2 1

),( 1uu

C

ikk

ijij kdD1

),()( 1uuu


...

Scalar products betweennormalised image and

eigenvectors

...

K1

K2

K3

Feature vector

Eigenfaces: feature extraction


Feature vector element ( ):

- dimensionality of feature vector

- normalised face image- i-th eigenvector

Improvements to the Eigenfaces face relevance masks eigenvector weights

xv Tiiu


li ,1lx

iv

iu



Improved feature extraction:

- i-th eigenvector weight- j-th element (pixel) of the mask- j-th element of the i-th

eigenvector- i-th element of the feature

vector

N

jjjijii mxvu

1 li ,1

ijmijviu



Similarity based on Euclidean distance:

l

iii uu

S

1

221

21

)(1

1),( uu

l

ppkijp

l

ppiijp

kij

uu

uud

1

21

1

21

1

)(1

)(1),( uu


Error minimisation

Recognition error is a function of mask and eigenvector weightsThe function may be minimised optimal mask optimal eigenvector weights

Example of mask optimisation...


Error minimisation

Optimised dataset problem of overfitting

How to avoid overfitting? large datasets optimisation can be stopped

Advantages of overfitting overfitting to a group of people

40%45%50%55%60%65%70%75%80%85%90%95%

100%

-1 9 19 29

Iteration

Cla

ss

ific

ati

on

eff

ec

tiv

en

es

s Optimised set

FeretA

FeretC

Notre-Dame -connected images

Notre-Dame - imagesnot connected


Summary

There are many methods derived from the EigenfacesError is a function of masks and eigenvectors weightsClassification parameters can be optimisedImprovement aims at: including face topology feature analysis difference classification


Thank you for your attention!

Next time:

Face detection

face recognition and biometric systems eigenfaces (3)

Documents

face recognition

pca slide

biometric systems example

th class slide

biometric systems plan

account slide

helpful slide

dimensionality reduction