![Page 1: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/1.jpg)
Develop A Face Recognition System on ARM Board Using OpenCV
WU JiaOpenCV China Team
![Page 2: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/2.jpg)
Outline
• Face recognition in brief•Build a FR system on ARM board using OpenCV•Assignment
1
![Page 3: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/3.jpg)
Face recognition is to identify or verify a person from an image or a video frame.
2
Face
Reco
gniti
on
A
B
C
Who is this person, 1:N Is this person X, 1:1
Face
Reco
gniti
on
Is A
Not A
Not A
![Page 4: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/4.jpg)
Face Recognition Pipeline
3
input images face detection face alignment
calculate similarity
face representation
same person
different person
![Page 5: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/5.jpg)
Face Detection • Traditional methods
Viola-Jones face detector, DPM etc.
• Deep learning based
4
Hang Du et al., The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advancesthe development of representa0ve face detec0on methods
multi-stagesingle-stage
anchor-basedanchor-freeother
![Page 6: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/6.jpg)
Face Alignment To obtain a canonical layout of the unconstrained face.
• Landmark-based methods• Landmark-free methods• Face frontalization
5
coordinate regressionheatmap regression3D model fittinglandmark-freeface frontalization
Hang Du et al., The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances
![Page 7: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/7.jpg)
Face Representation To extract discrimina6ve features from aligned face images, which is key to a face recogni6on system.• Tradi:onal methods
Eigenface, Gabor, LBP etc.
• Deep learning based
6Hang Du et al., The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances
classificationfeature embedding
semi-supervisedhybrid
![Page 8: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/8.jpg)
Outline
• Face recogniCon in brief•Build a FR system on ARM board using OpenCV•Assignment
7
![Page 9: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/9.jpg)
8
usb cameraARM dev board
face detection face preprocessing
featuredatabase
face detec<on face preprocessing
feature extraction
Registration Image
Camera Image result
Registration
Recognition
feature extrac<on
calculatesimilarity
Face Recognition System Architecture
![Page 10: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/10.jpg)
9
EAIDK-310
EXT_IO RECOVER RK3228H MASKROM
USB2.0X2
USB3.0X1USB2.0X1
RJ45Ethernet
WIFI/BT
LPDDR3
RESET
DC5V IN HDMI OUT SPEAKER
![Page 11: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/11.jpg)
OpenCV DNN Module
• Inference framework• Compatible to many popular deep learning frameworks
• Run on multiple devices: CPU, GPU, VPU, FPGA • Run on different OS: Linux, Windows, Android, MacOS• Backend framework to leverage different acceleration libs
10
Lightness
Convenience
UniversalityAccelerated
![Page 12: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/12.jpg)
well tested networks
11
Image Classification Object DetectionSemantic
SegmentationPose
EstimationImage Processing Person
Identification
Text Detection and
Recognition
Depth Estimation
CaffeAlexNetGoogLeNetVGGResNetSqueezeNetDenseNetShuffleNet
TensorFlowInceptionMobileNetEfficientNet
Darknetdarknet19
ONNXAlexNetGoogleNetCaffeNetRCNN_ILSVRC13ZFNet512VGG16VGG16_bnResNet-18v1ResNet-50v1CNN MnistMobileNetv2LResNet100E-IREmotion FERPlusSqueezenetDenseNet121Inception v1, v2Shufflenet
CaffeSSD VGGMobileNet-SSDFaster-RCNNR-FCNOpenCV Face Detector
TensorFlowSSDFaster-RCNNMask-RCNNEASTEfficientDet
DarknetYOLOv2tiny YOLO YOLOv3tiny YOLOv3YOLOv4tiny YOLOv4
ONNXTinyYolov2PyTorchYOLOv3PyTorchYOLOv3-tinyPyTorch SSD VGG
CaffeFCN
TorchENet
ONNXResNet101_DUC_HDC
TensorFlowDeepLab
PytorchUNetDeepLabV3FPNUNetPlusBiSeNet
CaffeOpenPose
PyTrochAlphaPose
CaffeColorization
TorchFast-Neural-Style
ONNXStyle Transfer
TorchOpenFace
PyTorchTorchreid
TensorFlowMoibleFaceNet
PyTorchEasyOCRCRNN
PyTorchMonodepth2
For more detail: https://github.com/opencv/opencv/wiki/Deep-Learning-in-OpenCV
![Page 13: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/13.jpg)
12
https://www.learnopencv.com/cpu-performance-comparison-of-opencv-and-other-deep-learning-frameworks/
85.4 182.8
3008.3
53.9 70.1
1184.7
0500
100015002000250030003500
Squeeze Net 1.1 MobileNet SSD 1.0 VGG16
Inference time(ms) @Cortex-A72x2
OpenCV OpenCV+Tengine-lite
Tengine: • edge AI inference framework. • Surports CPU, GPU, DSP, NPU. • Surports TensorFlow, Caffe, MxNet,
PyThorch, MegEngine, DarkNet, ONNX, ncnn.
AWS t2.large instance (2 vCPUs and 8 GB of RAM, no GPU), Ubuntu 16.04 LTS
Speed comparison
![Page 14: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/14.jpg)
13
Easy-to-useload dl modelchoose accelerating backendset target device
inferencing
![Page 15: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/15.jpg)
Now let’s try to build the system
14
landmark detection
face detection
feature extractionOpenFace
model
libfacedetection
similarity l2 distance
Speed can reach 1000FPS.https://github.com/ShiqiYu/libfacedetection
face detection face preprocessing
feature extraction
Input Image
![Page 16: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/16.jpg)
face and landmark detection
15
![Page 17: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/17.jpg)
face alignment
16
![Page 18: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/18.jpg)
feature extraction
17
calculate similarity
![Page 19: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/19.jpg)
Outline
• Face recognition in brief•Build a FR system on ARM board using OpenCV•Assignment
18
![Page 20: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/20.jpg)
Assignment
A. Build a complete face recognition system on ARM board using OpenCV. Write a report in English about the system.
B. Build any other kind of biometric recognition system on ARM board using OpenCV. Write a report in English about the system.
• Submit to: [email protected]• Deadline: 23:59 Jan. 27, 2021• Top 3 teams will be awarded the advanced EAIDK-610 Kit.
19
![Page 21: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/21.jpg)
Issues
20
• Training‒ use deep learning frameworks
•Registration
•UI
face detec<on face preprocessing
featuredatabase
Registration Image
feature extraction
![Page 22: Develop A Face Recognition System on ARM Board Using OpenCV](https://reader030.vdocuments.net/reader030/viewer/2022020621/61eab670006312738951487c/html5/thumbnails/22.jpg)
Thank You!
www.opencv.org.cn