Mo Guo
For CS519
Face Recognition with
Deep Learning
Outline1. Introduction
2. Related works
3. DeepFace
4. Alignment
5. Learning
6. Training
7. Results
• Classical Face recognition pipeline
Introduction
• Shallow Learning
SIFT, LBP, HOG, etc.Features = handpicked Works well on small datasets but fails on large datasetsAlso ails on illumination variations and facial expressions
Face patterns lie on a complex nonlinear and non‐convex manifold in the high‐dimensional space.
Related Works
• Deep LearningConvolutional Neural Networks (CNNs)DeepFaceDeepID SeriesFacenetVGGFaceetc.
Related Works
DeepFace (Taigman and Wolf 2014)
2D/3D face modeling and alignment
using affine transformations
9 layer deep neural network
120 million parameters
Alignment（Frontalization）(a) The detected face, with 6 initial fidu- cial points.
(b) The induced 2D-aligned crop.
(c) 67 fiducial points on the 2D-aligned crop with their
corresponding Delaunay triangulation, we added
triangles on the contour to avoid discontinuities.
(d) The reference 3D shape transformed to the 2D-
aligned crop image-plane.
(e) Triangle visibility w.r.t. to the fitted 3D-2D camera;
darker triangles are less visible.
(f) The 67 fiducial points induced by the 3D model that
are used to direct the piece-wise affine warping.
(g) The final frontalized crop.
(h) A new view generated by the 3D model.
Deep Learning
• Input: 3D aligned 3 channel (RGB) face image
152x152 pixels
• 9 layer deep neural network architecture
• Performs softmax for minimizing cross entropy
loss
• Uses SGD(stochastic), Dropout, ReLU
• Outputs k-Class prediction
Architecture
Layer 1-3 :
• Convolution layers - extract low-level features (e.g. simple edges and
texture)
• ReLU after each conv. layer, making the whole cascade produce
highly non-linear and sparse features
• Max-pooling: make convolution network more robust to local
translations.
Architecture
Layer 4-6:
• Apply filters to different locations on the feature map
• Similar to a conv. layer but spatially dependent
Architecture
Layer F7
• Fully connected and generates 4096d vector
• Sparse representation of face descriptor
• 75% of outputs are zero
• mainly due to ReLU and Dropout
Architecture
• F8 calculates probability with softmax
• softmax produces a distribution over the class labels
• Cross-entropy loss function: for each training sample
• Computed using SGD and performs backpropagation
Layer F8
• Fully connected and generates 4030d vector
Training
• Trained on SFC 4M faces (4030 identities, 800-1200 images per person)
• Focus on Labeled Faces in the Wild (LFW) evaluation
• Used SGD with momentum of 0.9
• Learning rate 0.01 with manual decreasing, final rate was 0.0001
• Random weight initializing
• 15 epochs of training
• 3 days total on a GPU-based engine
Training on SFC
• Experimented with different depths of networks
• Removed C3/L4,L5/C3, L4, L5
• Compared error rate to number of classes K
– Deeper is better
Result on LFW
