recognition of human faces: from biological to artifical vision massimo tistarelli computer vision...
Post on 24-Jan-2016
220 Views
Preview:
TRANSCRIPT
Recognition of Human Recognition of Human Faces: from Biological Faces: from Biological
to Artifical Visionto Artifical Vision
Massimo TistarelliMassimo TistarelliComputer Vision Laboratory
University of Sassari – Italytista@uniss.it
October 30 2007 - IstanbulMassimo Tistarelli 2
The Computer Vision Lab
Started in 1990 at University of Genova
Moved to Alghero, Sardinia in 2002
October 30 2007 - IstanbulMassimo Tistarelli 3
The Computer Vision Lab
October 30 2007 - IstanbulMassimo Tistarelli 4
The Computer Vision Lab
Second European Conference on Computer Vision – May 1992 – ECCV 1992 – S. Margherita Ligure, Italy
Int.l Workshop on Facial Image Analyisis and Recognition Technology – May 1998 – Freiburg, Germany
Int.l Workshop on Biometric Authentication – June 2002 Copenhagen, Denmark
Int.l Summer School for Advanced Studies on Biometrics:– 1st Authentication and Recognition – June 2-6 2003– 2nd Multimodality and System Integration – June 6-10 2005– 3rd New Sensors, Databases and Evaluation – June 12-16 2006– 4th New Technologies and Embedded Systems – June 11-15 2007– 5th New Technologies for Security and Privacy – June 9-13 2008
Fifth IEEE AutoId Workshop – June 7-8 2007, Alghero, Italy
Third IAPR/IEEE Int.l Conference on Biometrics – June 2-5 2009, Alghero, Italy
October 30 2007 - IstanbulMassimo Tistarelli 5
Credits
The laboratory staff:
– Manuele Bicego
– Gavin Brelstaff
– Linda Brodo
– Enrico Grosso
– Dakshina Kisku (on leave from IITK)
– Andrea Lagorio
– Ajita Rattani (joint Phd with UNICA)
– Elif Surer
October 30 2007 - IstanbulMassimo Tistarelli 6
Concise outline
Recognition of human faces and Biometrics
Issues from the Human Visual System
Applied technologies for face recognition
Exploiting knowledge from biological systems
New “trends and hints” ?
October 30 2007 - IstanbulMassimo Tistarelli 7
Biometrics
•Biometrics (as in statistics): “Statistical study
of biological observations and phenomena”
(Webster)
•Biometrics (as in security industry): “A
measurable, physical characteristic or personal
behavioral trait used to recognize the identity, or
verify the claimed identity, of an enrollee”
•Biometric recognition: Personal recognition
based on “who you are or what you do” as
opposed to “what you know” (password) or
“what you have” (ID card)
October 30 2007 - IstanbulMassimo Tistarelli 8
Most natural for humans
Highly acceptable and non-intrusive
Highly applicable:
– Static identity verification
– Uncontrolled face detection and identification from video
Medium to Low performances
Not unique (twins)
Aging and time effects
Why face recognition?
October 30 2007 - IstanbulMassimo Tistarelli 9
Are both from the same Are both from the same subject?subject?
How do we recognize faces?
October 30 2007 - IstanbulMassimo Tistarelli 10
Are both from the same Are both from the same subject?subject?
How do we recognize faces?
October 30 2007 - IstanbulMassimo Tistarelli 11
Score
Automated face recognition
Face Detection and Selection
Facial featuresextraction
Registrationand
Representation
Matching(comparison)
Input (grabbed) sequence
Individual face model
October 30 2007 - IstanbulMassimo Tistarelli 12
Automatic classification of faces
A class separation problem:
Fal
se
Raj
ecti
on
False Acceptance
?
October 30 2007 - IstanbulMassimo Tistarelli 13
Fk
Fh
L1 L2
S1
S3
S2
= (Fh,Fk)
Identification from human faces
A class separation problem:
• Choice of optimal representation • Inter-class similarity vs Intra-class variability
October 30 2007 - IstanbulMassimo Tistarelli 14
• Two different people may have very similar appearance
Twins Father and son
www.marykateandashley.com news.bbc.co.uk/hi/english/in_depth/americas/2000/us_elections
Inter-class SIMILARITY
October 30 2007 - IstanbulMassimo Tistarelli 15
Intra-class VARIABILITY
October 30 2007 - IstanbulMassimo Tistarelli 16
Recognition/Identification and Authentication/Verification
Many facts come into play:
– The dimensionality of the data and feature space
– The noise in the data
– The data alignment/localizazion
– The acquisition device
– The algorithmic classification
– The time variability of the data
– MORE….
October 30 2007 - IstanbulMassimo Tistarelli 17
Data dimensionality
... Not many ... (20x14)
It’s more a question of spatial distribution and …
proper frequency tuning
How many pixels should be processed to reliably How many pixels should be processed to reliably recognize a face?recognize a face?
October 30 2007 - IstanbulMassimo Tistarelli 18
Local analysis (Receptive fields)
The density and size of the cones decreases linearly from the retina center (fovea) to the periphery
October 30 2007 - IstanbulMassimo Tistarelli 19
Contextual analysis (Visual attention)
Eye movements while watching a girl’s face (A.L. Yarbus, “Eye Movements and Vision”, Plenum Press, 1967)
October 30 2007 - IstanbulMassimo Tistarelli 20
Space-variant representations
J. Bigun et al. 1997
M. Tistarelli et al. 1996
October 30 2007 - IstanbulMassimo Tistarelli 21
Where is the “Mom’s face” neuron?
October 30 2007 - IstanbulMassimo Tistarelli 22
Brain anatomy
Coronal anatomical image of a brain:STS, Superior temporal sulcus
MTG, middle temporal gyrus
ITS, inferior temporal sulcus
ITG, inferior temporal gyrus
FG, fusiform gyrus
FFrom rom fMRIfMRI, f, face-selective neurons have been found in the ace-selective neurons have been found in the inferior inferior temporal areastemporal areas, the , the superior temporal sensory areasuperior temporal sensory area, the , the amygdalaamygdala, the , the ventral striatumventral striatum ( (receivingreceiving input from the amygdala) and the input from the amygdala) and the inferior inferior convexityconvexity..
Specific regions have been reported also in the fusiform gyrus responding Specific regions have been reported also in the fusiform gyrus responding significantly to viewing faces.significantly to viewing faces. A. R. Damasio, J. Damasio and G. W. Van Hoesen. “Prosopagnosia: “Prosopagnosia:
Anatomic basis and behavioral mechanisms”. Anatomic basis and behavioral mechanisms”. Neurology, vol. 32, 331–341, 1982.
October 30 2007 - IstanbulMassimo Tistarelli 23
Brain activation
• Parts of the Parts of the inferiorinferior and and medialmedial temporal cortextemporal cortex may work may work together to process faces,together to process faces, the the amygdalaamygdala being being responsible for responsible for assigning significance to faces, assigning significance to faces, and thus and thus affecting both affecting both attention and mnemonic attention and mnemonic aspects of face processingaspects of face processing..
• Autism and Asperger syndromAutism and Asperger syndrom
C. A. Nelson. “The development and neural bases of face recognition”. “The development and neural bases of face recognition”. Infant and Child Development, vol. 10, 3-18, 2001.J. P. Aggleton, M. J. Burton and R. E. Passingham. “Cortical and subcortical afferents to the amygdala of the rhesus monkey “Cortical and subcortical afferents to the amygdala of the rhesus monkey (Macaca mulatta)”. (Macaca mulatta)”. Brain Research, vol. 190, 347–368, 1980.
October 30 2007 - IstanbulMassimo Tistarelli 24
Brain activation – fMRI maps
C. L. Leveroni et al. “Neural Systems Underlying the Recognition of Familiar and Newly Learned Faces”, The Journal of Neuroscience, January 15, 2000, 20(2):878–886
Recognition of 50 Familiar Faces (FF) vs 50 Newly Learned Faces (NL) and compared to rejection of 50 Foil (FO -False Objective) faces.Encoding (EN) session for learning new faces.
October 30 2007 - IstanbulMassimo Tistarelli 25
Brain activation areas
C. L. Leveroni et al. “Neural Systems Underlying the Recognition of Familiar and Newly Learned Faces”, The Journal of Neuroscience, January 15, 2000, 20(2):878–886
October 30 2007 - IstanbulMassimo Tistarelli 26
Facial features and brain activation
Aylward et al. Brain Activation during Face Perception: Evidence of a Developmental Change. J. Cogn. Neurosci..2005; 17: 308-319.
October 30 2007 - IstanbulMassimo Tistarelli 27
Aylward et al. Brain Activation during Face Perception: Evidence of a Developmental Change. J. Cogn. Neurosci..2005; 17: 308-319.
Eyes vs radialAll stimuli
Facial features and brain activation
October 30 2007 - IstanbulMassimo Tistarelli 28
Vaina, L.M., Solomon, J., Chowdhury, S., Sinha, P., Belliveau, J.W., “Functional Neuroanatomy of Biological Motion Perception in Humans”. Proc. of the National Academy of Sciences of the United States of America, Vol. 98, No. 20 (Sep. 25, 2001) , pp. 11656-11661
Face and motion perception
Biological Motion
Non Rigid Motion
October 30 2007 - IstanbulMassimo Tistarelli 29
Neural architecture of face perception - 1
J.V. Haxby, E.A. Hoffman, and M.I. Gobbini. “The distributed human neural system for face perception”. Trends in Cognitive Sciences, vol. 4, No. 6, 223--233, June 2000
Motion
Static
October 30 2007 - IstanbulMassimo Tistarelli 30
A. O’Toole, D.A. Roark, and H. Abdi. “ Recognizing moving faces: A psychological and neural synthesis”.Trends in Cognitive Science, 6:261--266, 2002.
Neural architecture of face perception - 2
October 30 2007 - IstanbulMassimo Tistarelli 31
Cooperative/competitive visual tracking AND recognition
Evidence
Face trackingprocess
Feature extractionregistration process
Incrementalmatching process
Expectation
Biological face recognition
October 30 2007 - IstanbulMassimo Tistarelli 32
Face representation
It is not clear how faces are represented in the HVS, but the representation is formed by a dynamically updated collection of visual fixations.– Both foveal and peripheral vision are involved, the former
responsible for a more accurate representation.J.M. Henderson et al. “Eye movements are functional during face learning”, Memory & Cognition
2005, 33 (1), 98-106.D.R. Melmoth et al. “The Effect of Contrast and Size Scaling on Face Perception in Foveal and
Extrafoveal Vision”, Investigative Ophthalmology and Visual Science. 2000;41:2811-2819.
This representation includes both iconic data as well as information about the spatial relationship among face elements.
– What about a “face template”?
October 30 2007 - IstanbulMassimo Tistarelli 33
Pose-dependent
Algorithms
Pose-invariant
Pose-dependency
Matching features
Appearance-based (Holistic)PCA, LDA, ICA, LPPKernel . . .
Feature-based (Analytic)Gabor setsLBP . . .
HybridLFAEGBM/JETsAAM . . .
Viewer-centered Images
Object-centered Models
Face representation
A step back:Taxonomy of face recognition Algorithms
• Gordon et al., 1995 3D from 2D
• Lengagne et al., 1996 3D from stereo
• Atick et al., 1996 LFA
• Yan et al., 1996 Geometric modeling
• Zhao et al., 2000 Shape from Shading • Zhang et al., 2000 3D from video
October 30 2007 - IstanbulMassimo Tistarelli 34
Automatic face recognition
Face recognition involves several sequential processes:
Face detection
Facial featureslocalization
Registration/Representation
Classification
Normalization
October 30 2007 - IstanbulMassimo Tistarelli 35
Face detection
Sebastian Marcel - http://www.idiap.ch/~marcel/demos/comparison/opencv/alt/image_66-detect.jpg
October 30 2007 - IstanbulMassimo Tistarelli 36
The Viola-Jones face detector
• At present the most efficient face detector
implemented and largely adopted
• Publicly available within the Open-CV library
• Several improvements are being developed
P. Viola, M. J. Jones, "Robust Real-Time Face Detection", International Journal Computer Vision, 57(2), pp. 137-154, 2004.
October 30 2007 - IstanbulMassimo Tistarelli 37
Some improvements…
Improve localization accuracy by post-filtering the face region
October 30 2007 - IstanbulMassimo Tistarelli 38
Automatic face recognition
Face recognition involves several sequential processes:
Face detection
Facial featureslocalization
Registration/Representation
Classification
Normalization
October 30 2007 - IstanbulMassimo Tistarelli 39
THE Problem
Illumination compensation
Original ImageHistogram equalization
October 30 2007 - IstanbulMassimo Tistarelli 40
Image re-lighting
),(),(),( yxLyxRyxI ),(
),(),(
yxL
yxIyxR
dxdyLLdxdyyxIyxLyxLF yx )()),(),()(,()( 222
L(x,y)
(1)
?
Isotropic diffusion (Gaussian filtering)
Anisotropic diffusion (Lagrange solution of (1))
R. Gross and V. Brajovic, “An Image Preprocessing Algorithm for Illumination Invariant Face Recognition”, International Conference on Audio- and Video-Based Biometric Person Authentication, 2003.
D. Jobson, Z. Rahmann and G. Woodell, “A Multiscale Retinex for Bridging the Gap Between Color Images and the Human Observations of Scenes”, IEEE Transanctions on Image Processing, volume 6, Issue 7, 1997.
October 30 2007 - IstanbulMassimo Tistarelli 41
Automatic face recognition
Face recognition involves several sequential processes:
Face detection
Facial featureslocalization
Registration/Representation
Classification
Normalization
October 30 2007 - IstanbulMassimo Tistarelli 42
Understanding Facial features
Gray level oriented patterns/photometric properties
Physical Landmarks
October 30 2007 - IstanbulMassimo Tistarelli 43
Human face structure
Cranial circumferenceMax. cranial breadthMin. frontal breadthBigonial breadthUpper facial heightBasion-Prosthion lengthNasal breadth (max.)Lower nasal breadthOrbital breadthBiorbital breadthForamen magnum breadthCranial heightMax. cranial lengthBizygomatic breadthTotal facial heightBasion-Nasion lengthBasal heightUpper nasal breadthOrbital heightInterorbital breadthPalate-external breadth & lengthPalate-internal breadth & length…. (more)
Craniometric measurements (1)
October 30 2007 - IstanbulMassimo Tistarelli 44
Facial Region
L_eye R_eye Nose Mth L_ear R_ear Chn L_chk R_chk F_head
Me
an
Pe
rce
nt
En
tere
d
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
Mean percentage of times (averaged across viewers) that each facial region was fixated at least once.
How to define facial features?
J.H. Henderson et al. “Gaze Control for Face Learning and Recognition by Humans and Machine”; in T. Shipley and P. Kellman (Eds.), From Fragments to Objects: Segmentation and Grouping in Vision
October 30 2007 - IstanbulMassimo Tistarelli 45
Facial features as 2D/3D landmarks
• 2D landmarks can be defined and tracked on face images
• Simple 2D vs complex 3D representations
October 30 2007 - IstanbulMassimo Tistarelli 46
• The LFA-based approach (Local Feature Analyisis) uses localized kernels, which are constructed from PCA-based eigenvectors, for extracting pre-defined topographic facial features (e.g., eyebrows, cheek, mouth, etc.)
Marked average face image
•Five topographic kernels are shown in the top row
•Five corresponding residual correlations (response) in the bottom row.
Facial features as 2D/3D landmarks
October 30 2007 - IstanbulMassimo Tistarelli 47
Facial features as 2D patterns
Gabor wavelets• Provide a description of the local
structure of the facial patterns
• Convolution with a bank of frequency-tuned filters
October 30 2007 - IstanbulMassimo Tistarelli 48
Local Binary Patterns (LBP)
• Pixels are labeled by thresholding the 3x3neighbourhood with the center value and considering the result as a binary number.
• The histogram of the labels is used as a texture descriptor.
Facial features as 2D patterns
October 30 2007 - IstanbulMassimo Tistarelli 49
Automatic face recognition
Face recognition involves several sequential processes:
Face detection
Facial featureslocalization
Registration/Representation
Classification
Normalization
October 30 2007 - IstanbulMassimo Tistarelli 50
Face displacement and perception
… … But the data is still thereBut the data is still there
October 30 2007 - IstanbulMassimo Tistarelli 51
In cognitive psychology it is called “perceptual organization”
The “registration” problem
October 30 2007 - IstanbulMassimo Tistarelli 52
The “registration” problem
S. Arca, P. Campadelli, and R. Lanzarotti. A face recognition system based on automatically determined facial fiducial points. Pattern Recognition, 39(3):432–443, 2006.
October 30 2007 - IstanbulMassimo Tistarelli 53
Automatic face recognition
Face recognition involves several sequential processes:
Face detection
Facial featureslocalization
Registration/Representation
Classification
Normalization
October 30 2007 - IstanbulMassimo Tistarelli 54
Linear Linear algorithmalgorithm
SVM, MPM, PCA, CCA, FDA…
DataData Embed dataEmbed data
x1
xn
if data described by numerical vectors: embedding ~ (non-linear) transformation
Kernel methods and learning
October 30 2007 - IstanbulMassimo Tistarelli 55
Support Vector Machines
Support Vector Machines are Support Vector Machines are binarybinary classifiers classifiers
Class 1Class 1
Class 2Class 2V. Vapnik, S.E. Golowich, A.J. Smola: Support Vector Method for Function Approximation, Regression Estimation and Signal Processing. Neural Information Processing Systems 1996: 281-287
October 30 2007 - IstanbulMassimo Tistarelli 56
The separating hyperplane can be very complex
Problems may occur with outliers
The shape of the hyperplane depends on the population of the classes
Need for an “impostor” class
Fk
Fh
ScSi
Support Vector Machines
October 30 2007 - IstanbulMassimo Tistarelli 57
One-Class Support Vector Machines are One-Class Support Vector Machines are unaryunary classifiers classifiers
One-Class Support Vector Machines
Class 1Class 1
ImpostorsImpostorsBen-Hur, A., Horn, D., Siegelmann, H., , Vapnik, V.: « Support vector clustering ». Journal of Machine Learning Research 2 (2001) 125–137
October 30 2007 - IstanbulMassimo Tistarelli 58
The separating surface is a hyperspehere
Selectivity can be adjusted by two parameters
No need for direct “impostor” training
Fk
Fh
ScSi
...122 iRaxi
One-Class Support Vector Machines
October 30 2007 - IstanbulMassimo Tistarelli 59
5 .5 0 %N O N EO n e - C la s s S V M
6 .1 9 %M i x e d d a ta s e t
6 .0 0 %Y a l e
5 .5 0 %S ti r li n g
6 .0 0 %B e rnB in a r y S V M
E q u a l E r r o r R a te
(E E R )I m p o s to r s e tM e t h o d
5 .5 0 %N O N EO n e - C la s s S V M
6 .1 9 %M i x e d d a ta s e t
6 .0 0 %Y a l e
5 .5 0 %S ti r li n g
6 .0 0 %B e rnB in a r y S V M
E q u a l E r r o r R a te
(E E R )I m p o s to r s e tM e t h o d
0 . 5 6 %O n e th r e s h o l d p e r
s u b j e c t
3 . 3 8 %
1 2 2 5 2 02 2 7 5
S in g l e th r e s h o l d
E q u a l E r r o r R a t e
( E E R )I m p o s t o r t e s t sC l ie n t t e s t sT h r e s h o ld
0 . 5 6 %O n e th r e s h o l d p e r
s u b j e c t
3 . 3 8 %
1 2 2 5 2 02 2 7 5
S in g l e th r e s h o l d
E q u a l E r r o r R a t e
( E E R )I m p o s t o r t e s t sC l ie n t t e s t sT h r e s h o ld
Poor Poor representationrepresentation
(10 frames/sbj)(10 frames/sbj)
Rich Rich representationrepresentation
(100 (100 frames/sbj)frames/sbj)
One-Class Support Vector Machines
October 30 2007 - IstanbulMassimo Tistarelli 60
Test Test ParameterFalse Reject
Rate False Accept
Rate
Fingerprint
FVC[2004]
Exaggerated distortion 2% 2%
FpVTE[2003]
US govt. operational data 0.1% 1%
Face
FRVT[2002]
Varied lighting, outdoor/indoor
10% 1%
FRGC[2006]
Time lapse, varied lighting/expression,
outdoor/indoor10% 0.1%
IrisITIRT
[2005] Indoor environment,
multiple visits0.99% 0.94%
VoiceNIST
[2004]Text independent, multi-
lingual5-10% 2-5%
Face recognition performances
October 30 2007 - IstanbulMassimo Tistarelli 61
Human vs Machine face recognition performances
A. O’Toole, J. Phillips, F. Jiang, Ayyad, Pénard & A. O’Toole, J. Phillips, F. Jiang, Ayyad, Pénard & AbdiAbdi, “Benchmarking Algorithms Against Humans” , “Benchmarking Algorithms Against Humans” IEEE:T-PAMI, 2007 IEEE:T-PAMI, 2007 (in press)(in press) Identity Matching for Easy Face Pairs
0
0.2
0.4
0.6
0.8
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
False Accept Rate
Veri
ficati
on
Rate
CMU
NJIT
Algorithm B
Algorithm D
Viisage
Algorithm C
Algorithm A
Human Performance
Chance Performance
Identity Matching for Difficult Face Pairs
0
0.2
0.4
0.6
0.8
1
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
False Accept Rate
Veri
ficati
on
Rate
NJIT
CMU
Viisage
Human Performance
Algorithm A
Algorithm B
Algorithm C
Algorithm D
Chance Performance
October 30 2007 - IstanbulMassimo Tistarelli 62
We are still HERE!
…Where is the “Mom’s face” neuron?
October 30 2007 - IstanbulMassimo Tistarelli 63
Automatic face recognition
WHERE DO WE GO NEXT?
Face detection
Facial featureslocalization
Registration/Representation
Classification
Normalization
October 30 2007 - IstanbulMassimo Tistarelli 64
Where do we go next?
ATTENTIVE AND DYNAMIC
PROCESSES
October 30 2007 - IstanbulMassimo Tistarelli 65
Are both from the same Are both from the same subject?subject?
What was missing?What was missing?
A simple recognition experiment…
October 30 2007 - IstanbulMassimo Tistarelli 66
Visual attention
Eye movements while watching a girl’s face (A.L. Yarbus, “Eye Movements and Vision”, Plenum Press, 1967)
October 30 2007 - IstanbulMassimo Tistarelli 67
The analysis of visual tracking patterns is sometimes misleading…
R.C. Miall, University Laboratory of Physiology, Oxford, UKJohn Tchalenko, Camberwell College of Arts, London, UK
Visual tracking
October 30 2007 - IstanbulMassimo Tistarelli 68
Subject-specific representation
For recognition we are not interested on what faces have in common but rather what differentiate one face from another.
For localization and tracking we are interested on what every face has in common (to tell a face from “non-faces”)
October 30 2007 - IstanbulMassimo Tistarelli 69
SVMfeature space
Distinctive patches
Face 1Face 2
Confusion: most similar patches
Distinctive patches
Fk
Fh
Bicego M., Brelstaff G., Brodo L., Grosso E., Lagorio A. and Tistarelli M. (2007) “Distinctiveness of faces: a computational approach”, ACM Transactions on Applied Perception, Vol. 5, n. 2, 2008.(in press)
Subject-specific representation
October 30 2007 - IstanbulMassimo Tistarelli 70
Three examples of differences extracted from pairs of images of the same person. The 100 most weighted patches are
shown
Subject-specific representation
October 30 2007 - IstanbulMassimo Tistarelli 71
A
A B
B
(A) perceptual and (B) computational test results of saliency
of local facial features. Red and green points in (A) represent the two positions selected by each of the 45 subjects (13 male, 32 female).
Subject-specific representation
October 30 2007 - IstanbulMassimo Tistarelli 72
(A) perceptual and (B) computational test results of saliency
of local facial features. Red and green points in (A) represent the two positions selected by each of the 45 subjects (13 male, 32 female).
A B
A B
Subject-specific representation
October 30 2007 - IstanbulMassimo Tistarelli 73
52 subjects from the BANCA database (26 M and 26 F).
Matched Controlled (MC) BANCA protocol.
The Weighted Error Rate (WER) is computed from the threshold from G1 on G2 (26 subjects each).
Image areas are selectively deleted from the face image.
Authentication test
October 30 2007 - IstanbulMassimo Tistarelli 74
Authentication test
Quantitative results by selectively masking image areas
October 30 2007 - IstanbulMassimo Tistarelli 75
Comparative tests
Lowe [2004] (SIFT),
Salah et al. [2002],
Walther [2006],
Ullman et al. [2002].
Authentication test
October 30 2007 - IstanbulMassimo Tistarelli 76
Hidden Markov Models
Statistical analysis of sequences of patterns
… …
October 30 2007 - IstanbulMassimo Tistarelli 77
Hidden Markov Models
Modelling (learning) sequences of symbols
– capture the underlying structure of a set of symbol strings
– stochastic generalisation of finite-state automata, where both transitions between states and generation of output symbols are governed by probabilistic distributions
Markovian Models, where states are not directly observable
– a probability function is associated to each state describing the probability that a symbol is emitted from this state.
October 30 2007 - IstanbulMassimo Tistarelli 78
Coding– the image is divided into a sequence of T sub-images
– for each sub-image the DCT coefficients are computed
– only the most important D coefficients are retained
– the final sequence is composed by DxT symbols
Learning : train one HMM for each subject:– the number of states is fixed a priori
– at the end we have one HMM for each subject
Recognition : – Bayesian scheme: assuming a priori equi probable classes, the object
is assigned to the highest likelihood class
Hidden Markov Models
October 30 2007 - IstanbulMassimo Tistarelli 79
Recognition and selective attention
Starting point: HMM based classification of faces
“Walking on the face” for obtaining HMM sequences
Standard raster scan-path Saliency-based scan-path
Attention drives face
scanning
October 30 2007 - IstanbulMassimo Tistarelli 80
Model of attention-based classification
Inhibition of Return
Face sampleSaliency Master Map Feature
MapsNormalization and Integration
Hidden Markov Model
Fovea
'What'Information
Feature Extraction
content
location 'Where'Information
Associative Level
Attentive Level
classification
October 30 2007 - IstanbulMassimo Tistarelli 81
Experiments on BANCA protocol MC
Gabor wavelets for saliency map construction
Employed features: gray levels, DCT coefficients, Haar
wavelets
A. A. Salah, M. Bicego, L. Akarun, E. Grosso, M. Tistarelli: "Hidden Markov model-based face recognition using selective attention", Human Vision and Electronic Imaging XII, Proc. of SPIE, vol. 6492, (2007)
Authentication test
October 30 2007 - IstanbulMassimo Tistarelli 82
About 40% of the saccades are sufficient to obtain the same result of raster scan
In some cases further saccades may decrease the classification performances
The system can be improved by exploiting also the where information
Authentication test
October 30 2007 - IstanbulMassimo Tistarelli 83
Face recognition from video
Dynamics in a video stream conveys far more Dynamics in a video stream conveys far more information than a collection of single snapshotsinformation than a collection of single snapshots
October 30 2007 - IstanbulMassimo Tistarelli 84
Individual motion can be highly distinctiveIndividual motion can be highly distinctive
Face recognition from video
October 30 2007 - IstanbulMassimo Tistarelli 85
… … and even more distinctiveand even more distinctive
Face recognition from video
October 30 2007 - IstanbulMassimo Tistarelli 86
Face recognition from video
Advantages from video:
• More data available• Temporal integration• Behavioral cues• Spatial and temporal sampling
October 30 2007 - IstanbulMassimo Tistarelli 87
Effects of motion on representation
Facerotation
Facescaling
Facetranslation
Face sub-spaces manifold
October 30 2007 - IstanbulMassimo Tistarelli 88
The curse of dimensionality…
…the risk is to have too much data to be processed
How to exploit the added information in video?How to exploit the added information in video?
Face recognition from video
Standard VGA (640x480)
1 frame: 300 KByte30 frames: 1 MByte
Standard video: ~1 MByte/Sec
October 30 2007 - IstanbulMassimo Tistarelli 89
Not just more data to be processed:
Data selection (pose, expression, illumination, noise…)
Multi-data fusion (decision/score/feature level)
3D reconstruction/virtual views
Resolution enhancement
Expression and emotion analysis
Behavioral analysisBehavioral analysis
DynamicDynamic video templates video templates…?
Face recognition from video
October 30 2007 - IstanbulMassimo Tistarelli 90
A. O’Toole, D.A. Roark, and H. Abdi. “ Recognizing moving faces: A psychological and neural synthesis”.Trends in Cognitive Science, 6:261--266, 2002.
Neural architecture of face perception - 2
October 30 2007 - IstanbulMassimo Tistarelli 91
How to represent photometric and dynamic How to represent photometric and dynamic information at the same time?information at the same time?
Face recognition from video
October 30 2007 - IstanbulMassimo Tistarelli 92
Hidden Markov Models
Statistical analysis of sequences of patterns
… …
October 30 2007 - IstanbulMassimo Tistarelli 93
Standard HMMs can be extended to multi-dimensional patterns and sequences … in many ways
1. Each image is modeled as a single HMM and the sequence of images as a sequence of HMMs (A. Hadid and M. Pietikainen. “An experimental investigation about the integration of facial dynamics in video-based face recognition”. Electronic Letters on Computer Vision and Image Analysis, 5(1):1-13, 2005.)
2. The entire video is modeled as a single HMM
(X. Liu and T. Chen. “Video-based face recognition using adaptive hidden Markov models”. In Proc. Int. Conf.
on Computer Vision and Pattern Recognition, 2003.)
3. The images and the sequence itself are modeled as a complex, hierarchical HMM-based structure (M. Bicego, E.Grosso, M. Tistarelli. “Person authentication from video of faces: a behavioral and physiological approach using Pseudo Hierarchical Hidden Markov Models”, Int.l Conference on Biometric Authentication 2006, Hong Kong, China, January 2006. )
Dynamic Hidden Markov Models
October 30 2007 - IstanbulMassimo Tistarelli 94
Physiological and behavioral featuresPhysiological and behavioral features
HMMHMM
PH-HMMPH-HMM
Pseudo Hyerarchical HMM
October 30 2007 - IstanbulMassimo Tistarelli 95
Pseudo-Hierarchical Pseudo-Hierarchical Hidden Markov ModelHidden Markov Model
The image stream is represented as a collection of sub-streams, each corresponding to a different facial expression or motion.
The change in expression is capturede by the transition matrix of the PH-HMM
Pseudo Hyerarchical HMM
October 30 2007 - IstanbulMassimo Tistarelli 96
Recogniton from video: performances
Classification Results (recognition rate)Still images: one HMM for class 11/21 (52.38%)
Still images: one HMM for cluster of class 14/21 (66.67%)
Still images: one HMM for image 12/21 (57.14%)
Video: Pseudo Hierarchical HMM 21/21 (100%)_________________________________________
Verification Results (EER)Client tests: 20 - Impostor tests: 420
Still images: one HMM for class 20.24%
Still images: one HMM for cluster of class 10.60%
Still images: one HMM for image 13.81%
Video: Pseudo Hierarchical HMM 6.07 %
M. Bicego, E. Grosso, M. Tistarelli: "Person Authentication from video of faces: a behavioral and physiological approach using Pseudo Hierarchical Hidden Markov Models" . Int. Conf. On Biometrics (ICBA06) , LNCS 3832, D. Zhang, A.K. Jain Eds, pp 113-120, (2006)
October 30 2007 - IstanbulMassimo Tistarelli 97
Industrial FR systemsA4Vision, Inc. (3D scanner)
AcSys Biometrics Corp.
Animetrics Inc. (3D shape)
C-VIS Computer Vision und Automation GmbH (Neural Networks)
Cognitec Systems GmbH (LFA)
Cybula Ltd. (3D shape/2D texture)
DreamMirh Co., Ltd. (2D ??? )
Geometrix, Inc. (3D shape)
Iconquest (2D Fractal-based ??? )
Identix Inc. (LFA)
Imagis Technologies Inc. (VISIPHOR Advanced Face Recognition ???)
Neven Vision, Inc. (2D Gabor wavelets; feature-based )
Takumi Vision Technologies, Inc. (Hi-Tech algorithms ??? )
Viisage (Template matching ??? )
VisionSphere Technologies Inc. (2D features with Holistic Feature Code)
Source: www.face-rec.org
October 30 2007 - IstanbulMassimo Tistarelli 98
Near-infra red imaging
Authentimetric F1
October 30 2007 - IstanbulMassimo Tistarelli 99
Conclusion
• Many algorithmic and system solutions have been proposed.
• More investments needed.
• More basic research on open issues.
• Need for “universal” databases for evaluation and testing on key applications.
October 30 2007 - IstanbulMassimo Tistarelli 100
New technologies for Security and Privacy
Contact: tista@uniss.it
http://biometrics.uniss.it
5th Int.l Summer School for Advanced Studies on
Biometrics for secure authentication:
Alghero, Italy - June, 9 – 13 2008
October 30 2007 - IstanbulMassimo Tistarelli 101
Contact: tista@uniss.it
http://icb07.uniss.it
3rd IAPR/IEEE Int.l Conference onBiometrics
Alghero, Italy - June, 2 – 5 2009
top related