a computer aided system to identify the face shape of a person

A Computer Aided System to Identify the Face Shape of a Person using Machine Learning and Image

Processing Techniques

Department of Computer Science Faculty of Applied Sciences

University of Sri Jayewardenepura, Sri Lanka

T. G. I. Fernando H. N. Gunasinghe

H.N.Gunasinghe | SCA-162

Content

Introduction Objectives Methodology Results and Discussion Conclusions Future Work References

05/31/2016


Introduction (1/2) People spend days and hours

to select best hair styles, hair cuts and eye glass frames.

Sometimes they end up choosing unmatching products.

Because they don’t know that…

Eyeglass Frame shapes, Hair Styles and Hair Cuts are defined according to each face shape.

However knowing the face shape is much more difficult than we think.

05/31/2016


Introduction (2/2) The common practice used by experts is

guessing the face shape by directly looking at the face.

Identifying the correct face shape is much difficult by looking at the face Shape is 2D Face is 3D

Facial images can be used to detect the face shape Image is a 2D projection of the 3D object

Machine learning based classifiers can be used to automate the shape identification process

05/31/2016


Objectives The goal

To develop a computer aided system that can help people to identify their face shape

System overview

Original image Preprocessed image Pre-trained Model Predicted face shape

05/31/2016


Literature Review (1/2) Digital photographs of faces have been

used to identify face shape Online face shape identification

Principles of shallow learning have been widely used for the classifying images Shape Classification using Local and Global

Features [2] An AAM-based Face Shape Classification Method

used for Facial Expression Recognition [3] Irregular shapes classification by back-

propagation [4] Object Recognition: A Shape-Based Approach

using Artificial Neural Networks [5] 05/31/2016


Literature Review (2/2) Principles of deep learning have been widely

used for the classify images Face Image Analysis With Convolutional Neural

Networks [6] ImageNet Classification with Deep Convolutional

Neural Networks [7]

05/31/2016

https://www.google.lk/url?sa=i&rct=j&q=&esrc=s&source=images&cd=&cad=rja&uact=8&ved=0ahUKEwjOtJvP_LPMAhXYBY4KHa0HBKsQjRwIBw&url=https://nivdul.wordpress.com/2015/11/17/exploring-deep-learning-with-li-zhe/&bvm=bv.120853415,d.c2E&psig=AFQjCNHQoXwpMs0B5MZf-Sl3wwhiS_Eygg&ust=1462023456484138


Methodology Design Data

Collection

Apply image processing techniques

Train the neural networksMultilayer perceptron

Convolutional neural network

Generate Results

Compare results and select the best model

Predict face shapes for new images

05/31/2016


Preparing Dataset Collect facial images from the internet

Google image search Labeled, straight frontal Faces

Applying image processing techniques Cropping – Capture face Noise removal – Background, hair, ears Resizing image – Multi size to 256 x 256 Color space transformation – RGB to Grayscale Edge detection – Photoshop edge detection Rotation – clockwise, anticlockwise Resizing face – Multi size to 180 x 180 and 220 x

22005/31/2016


Image Classification Dataset

Dataset - 4200

Training set

315075%

Validation set63015%

Test Set42010%

Number of Categories = Number of face shapes = 7 Oval, Oblong, Round,

Square, Triangle, Inverted Triangle, Diamond

Equal number of images for each category

Randomly selected images for each training, validation and test set 05/31/2016


Tools and Technologies Ubuntu 14.04 with NVIDIA GTX 980 GPU

System development environment Caffe

A deep learning framework made with expression, speed, and modularity in mind

developed by the Berkeley Vision and Learning Center (BVLC) and by community contributors

Open Source NVIDIA DIGITS 2

The NVIDIA Deep Learning GPU Training System Uses Caffe modules Locally hosted – remotely accessible

05/31/2016


Multilayer Perceptron - Design

Three hidden layers Fully connected ReLU 1048-256-7 neurons

per layer Softmax output layer SGD training algorithm Batch size = 16

05/31/2016


Convolution neural network - Design AlexNet with fine tuning Five convolution layers with

ReLU 96-256-384-384-256 neurons

per layer Three Fully connected ReLU

4096-4096-7 neurons per layer Softmax output layer SGD training algorithm Batch size = 100 Crop size = 255

05/31/2016


Training the networks

MLP CNN

Graph of Learning rate vs. epoch for each network

Learning rate = 0.0001Epochs = 50Step down = 10, 20%Step size = 0.6

Learning rate = 0.01Epochs = 500Step down = 167, 33%Step size = 0.1

05/31/2016


Results and discussion (1/2)

MLP CNN

The graph of validation accuracy vs. training and validation loss

05/31/2016


Results and discussion (2/2)

Model Epochs Training time Validation accuracy

Convolutional NN 500 1 hour, 3 minutes 98.5%

MLP 50 18 minutes, 46 seconds 58.3%

The table of results including validation accuracy, training time and #of epochs

05/31/2016


Conclusion The goal of the study was to develop a prototype

system that can identify face shape by using image processing and machine learning.

Multi-layer perceptron and convolutional neural network architectures were considered.

Each system was trained, validated and tested. The accuracy of validation dataset for each model

was compared to select the best model. The results indicate that the convolutional neural

network is more useful in identification of shapes than multilayer perceptron.

The implemented system can identify face shape in good accuracy rate.

05/31/2016


Future Work Limitations

Only 3150 Face images were used to train the system

System was trained using a single GPU Future Works

Try the system with many machine learning techniques and architectures

Use larger images with minimum preprocessing Train the system with multi GPUs Apply these techniques for a Larger dataset Improve the GUI of the system Implement mobile/online/standalone applications

05/31/2016


References (1/2)1. M. B. Gantt, How to wear your hair, Tate Publishing and

Enterprises LLC, USA, 2010.2. K. L. Lim and K. H. Galoogahi, Shape Classification Using Local

and Global Features, Fourth Pacific-Rim Symposium on Image and Video Technology, 2010.

3. J. So, S. Han, H. C. Shin and Y. Han, An AAM-based Face Shape Classification Method Used for Facial Expression Recognition, International Journal of Research in Engineering and Technology (IJRET), 2013, 2-4, 164-168.

4. S. W. Lin, S. Y. Chou and S. C. Chen, Irregular shapes classification by back-propagation, Springer, 2006.

5. J. D. Vries, Object Recognition:A Shape-Based Approach using Artificial Neural Networks, Dissertation.

6. S. Duffner, Face Image Analysis with Convolutional Neural Networks, Dissertaion, 2007.

05/31/2016


References (2/2)7. A. Krizhevsky, I. Sutskever, and G. E. Hinton, ImageNet Classification

with Deep Convolutional Neural Networks, Neural Information Processing Systems, 2012.

8. NVIDIA DIGITS [Computer Software], Retrieved from https://developer.nvidia.com/digits.

9. A.Y. Jia, E.Shelhamer, J. Donahue, S.Karayev, J. Long, R.Girshick, S.Guadarrama, and T. Darrell, Caffe: Convolutional architecture for fast feature embedding, Proceedings of the ACM International Conference on Multimedia, 2014, 675-678.

10. B. E. Boser, I. M. Guyon and V. N. Vapnik, "A Training Algorithm for Optimal Margin Classiers, Proceedings of the Fifth Annual Workshop on Computational Learning, 1992,144-152.

11. K. Jarrett, K. Kavukcuoglu, M. A. Ranzato and Y. LeCun, What is the Best Multi-Stage Architecture for Object Recognition?,The Courant Institute of Mathematical Sciences, New York University.

12. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever and R. Salakhutdinov, Dropout: A Simple Way to Prevent Neural Networks from Overfitting, Journal of Machine Learning Research, 2014, 15, 1929-1958.

05/31/2016

https://developer.nvidia.com/digits

Thank you

Q&A

[email protected]