develop a face recognition system on arm board using opencv
TRANSCRIPT
Develop A Face Recognition System on ARM Board Using OpenCV
WU JiaOpenCV China Team
Outline
• Face recognition in brief•Build a FR system on ARM board using OpenCV•Assignment
1
Face recognition is to identify or verify a person from an image or a video frame.
2
Face
Reco
gniti
on
A
B
C
Who is this person, 1:N Is this person X, 1:1
Face
Reco
gniti
on
Is A
Not A
Not A
Face Recognition Pipeline
3
input images face detection face alignment
calculate similarity
face representation
same person
different person
Face Detection • Traditional methods
Viola-Jones face detector, DPM etc.
• Deep learning based
4
Hang Du et al., The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advancesthe development of representa0ve face detec0on methods
multi-stagesingle-stage
anchor-basedanchor-freeother
Face Alignment To obtain a canonical layout of the unconstrained face.
• Landmark-based methods• Landmark-free methods• Face frontalization
5
coordinate regressionheatmap regression3D model fittinglandmark-freeface frontalization
Hang Du et al., The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances
Face Representation To extract discrimina6ve features from aligned face images, which is key to a face recogni6on system.• Tradi:onal methods
Eigenface, Gabor, LBP etc.
• Deep learning based
6Hang Du et al., The Elements of End-to-end Deep Face Recognition: A Survey of Recent Advances
classificationfeature embedding
semi-supervisedhybrid
Outline
• Face recogniCon in brief•Build a FR system on ARM board using OpenCV•Assignment
7
8
usb cameraARM dev board
face detection face preprocessing
featuredatabase
face detec<on face preprocessing
feature extraction
Registration Image
Camera Image result
Registration
Recognition
feature extrac<on
calculatesimilarity
Face Recognition System Architecture
9
EAIDK-310
EXT_IO RECOVER RK3228H MASKROM
USB2.0X2
USB3.0X1USB2.0X1
RJ45Ethernet
WIFI/BT
LPDDR3
RESET
DC5V IN HDMI OUT SPEAKER
OpenCV DNN Module
• Inference framework• Compatible to many popular deep learning frameworks
• Run on multiple devices: CPU, GPU, VPU, FPGA • Run on different OS: Linux, Windows, Android, MacOS• Backend framework to leverage different acceleration libs
10
Lightness
Convenience
UniversalityAccelerated
well tested networks
11
Image Classification Object DetectionSemantic
SegmentationPose
EstimationImage Processing Person
Identification
Text Detection and
Recognition
Depth Estimation
CaffeAlexNetGoogLeNetVGGResNetSqueezeNetDenseNetShuffleNet
TensorFlowInceptionMobileNetEfficientNet
Darknetdarknet19
ONNXAlexNetGoogleNetCaffeNetRCNN_ILSVRC13ZFNet512VGG16VGG16_bnResNet-18v1ResNet-50v1CNN MnistMobileNetv2LResNet100E-IREmotion FERPlusSqueezenetDenseNet121Inception v1, v2Shufflenet
CaffeSSD VGGMobileNet-SSDFaster-RCNNR-FCNOpenCV Face Detector
TensorFlowSSDFaster-RCNNMask-RCNNEASTEfficientDet
DarknetYOLOv2tiny YOLO YOLOv3tiny YOLOv3YOLOv4tiny YOLOv4
ONNXTinyYolov2PyTorchYOLOv3PyTorchYOLOv3-tinyPyTorch SSD VGG
CaffeFCN
TorchENet
ONNXResNet101_DUC_HDC
TensorFlowDeepLab
PytorchUNetDeepLabV3FPNUNetPlusBiSeNet
CaffeOpenPose
PyTrochAlphaPose
CaffeColorization
TorchFast-Neural-Style
ONNXStyle Transfer
TorchOpenFace
PyTorchTorchreid
TensorFlowMoibleFaceNet
PyTorchEasyOCRCRNN
PyTorchMonodepth2
For more detail: https://github.com/opencv/opencv/wiki/Deep-Learning-in-OpenCV
12
https://www.learnopencv.com/cpu-performance-comparison-of-opencv-and-other-deep-learning-frameworks/
85.4 182.8
3008.3
53.9 70.1
1184.7
0500
100015002000250030003500
Squeeze Net 1.1 MobileNet SSD 1.0 VGG16
Inference time(ms) @Cortex-A72x2
OpenCV OpenCV+Tengine-lite
Tengine: • edge AI inference framework. • Surports CPU, GPU, DSP, NPU. • Surports TensorFlow, Caffe, MxNet,
PyThorch, MegEngine, DarkNet, ONNX, ncnn.
AWS t2.large instance (2 vCPUs and 8 GB of RAM, no GPU), Ubuntu 16.04 LTS
Speed comparison
13
Easy-to-useload dl modelchoose accelerating backendset target device
inferencing
Now let’s try to build the system
14
landmark detection
face detection
feature extractionOpenFace
model
libfacedetection
similarity l2 distance
Speed can reach 1000FPS.https://github.com/ShiqiYu/libfacedetection
face detection face preprocessing
feature extraction
Input Image
face and landmark detection
15
face alignment
16
feature extraction
17
calculate similarity
Outline
• Face recognition in brief•Build a FR system on ARM board using OpenCV•Assignment
18
Assignment
A. Build a complete face recognition system on ARM board using OpenCV. Write a report in English about the system.
B. Build any other kind of biometric recognition system on ARM board using OpenCV. Write a report in English about the system.
• Submit to: [email protected]• Deadline: 23:59 Jan. 27, 2021• Top 3 teams will be awarded the advanced EAIDK-610 Kit.
19
Issues
20
• Training‒ use deep learning frameworks
•Registration
•UI
face detec<on face preprocessing
featuredatabase
Registration Image
feature extraction
Thank You!
www.opencv.org.cn