accelerating face-in-the-crowd recognition with gpu technology · • face detection is the largest...
TRANSCRIPT
![Page 1: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/1.jpg)
Accelerating Face-in-the-Crowd Recognition with GPU Technology Brian Lovell Steve Brain CTO Lead Software Engineer
nVidia GTC 5-Nov-13
www.imagus.com.au
![Page 2: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/2.jpg)
GeoInt Accelerator • Platform to start developing with GPUs • GPU access, GPU-accelerated apps and libraries • Register to learn more: http://www.nvidia.com/geoint
Webinar Feedback Submit your feedback for a chance to win Tesla K20 GPU* https://www.surveymonkey.com/s/imagus * Offer is valid until December 1st 2013
![Page 3: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/3.jpg)
Outline • Forensic Face Search • Face in the Crowd Recognition (FCR) • Parallel Distributed FCR • Embedded Face Recognition • Software Tools for CVPR • What We Have Learned
Airport Railway Station Seaport
![Page 4: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/4.jpg)
Forensic Face Search
![Page 5: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/5.jpg)
Recognising Faces from CCTV
• “The major problem with CCTV biometric evidence compared to DNA and Fingerprints is the inability to match faces to names.” - Graeme Gerrard, Former DCC and Co-Chair CCTV Strategy Committee
• Humans can do this task if they are familiar with the person under surveillance, but it is extremely difficult to automate this matching with any degree of reliability.
• This is largely due to the uncontrolled nature of covert photography exacerbated by poor resolution.
• Computer face recognition works remarkably well for passport quality images, but CCTV is much, much harder.
• Need a technology to shortlist suspects from CCTV images
• Aim is for high rank rather than a perfect match
![Page 6: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/6.jpg)
Best Practice Police and Defence Technology
![Page 7: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/7.jpg)
![Page 8: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/8.jpg)
Personal Example: Could not find any of my 3 daughters in School Photo of
1000 students
Tess
Can recognise with eyecontact iPhone app from LCD screen image!
![Page 9: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/9.jpg)
Our Approach to Face Matching
• Dictionary approach from field of speech recognition
• Very fast approximations, small templates, DCT features, insensitive to alignment, pose, illumination, expression
• Currently robustly matches faces at 12 pixels eye to eye whereas most commercial systems need a minimum of 64 pixels.
![Page 10: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/10.jpg)
Face Recognition Landscape
Ours
Others
Low High
Perfo
rman
ce
Misaligned faces and poor resolution images is an Imagus iFaceRec strength.
Quality of Image Aligned frontal, 100 pixels eye to eye Partially aligned, non-frontal, 12 pixels eye to eye
Resolution Limit for Human
Recognition
![Page 11: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/11.jpg)
Forensic Face Match System
The LFW Database •Images gathered off the web
•Variable •Camera •Photographer •Lighting •Pose •Expression •Age •Race •Gender •Resolution •Number of photos (1-245)
• Uncontrolled Gallery •4000 Identities •10000 Images
•Uncontrolled Probe Images •2000 Identities •4000 images
•Good proxy for CCTV surveillance task •We have also benchmarked on railway CCTV images with fair results
![Page 12: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/12.jpg)
Results Face Features Only Full Images
Over 4000 persons in database
Who?
![Page 13: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/13.jpg)
LFW Benchmarks show 80% confidence in just 0.5% of the Database
In 80% of cases the probe is recognised in the top 0.5% of the entire database of 4000 identities and 10000 images.
The system does not do a one-to-one matching, instead it rank orders the entire face database against the probe image
Make the needle in the haystack easier to find
Now achieves almost 50% Rank 1 matches
![Page 14: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/14.jpg)
Commercial Deployments
• Singapore, Bangkok, Indonesia, Abu Dhabi, … • Working on smart phone deployments for
Military, Police, and Banking Sector
![Page 15: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/15.jpg)
Face in the Crowd Recognition
![Page 16: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/16.jpg)
From Still to Video • Need to track individuals and assemble tracklets of
face images • Need to determine how to process tracklets to get best
recognition – All frames? – Best frame? – Some frames?
• Want true video enrolment rather than still image • Integrate and improve identity over multiple frames • Need to automatically enrol individuals from CCTV
as well as to recognise them
![Page 17: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/17.jpg)
IFSEC2011 Award • IFSEC is the World’s largest
security show • It is held in Birmingham
each year • 25,000 visitors • 700 companies • 700 guests at award dinners • Our technology won the
Major Category of CCTV System of the Year for Face Recognition in a crowd
![Page 18: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/18.jpg)
FOR VIDEOS PLEASE SEE WWW.IMAGUS.COM.AU
Parallel Distributed Face in the Crowd Recognition
![Page 19: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/19.jpg)
Parallel Distributed Face Search Across a Camera Network
• Recoded in Linux • Uses Robot Operating System (ROS) from
Willow-Garage as distributed middleware • Scalable to multiple CPUs and GPUs • Unlimited Cameras and Servers • Scalable to airports and large infrastructure • Easily add additional biometrics
and other information
![Page 20: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/20.jpg)
ROS: Person Identification in a Crowd
![Page 21: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/21.jpg)
ROS: Cross-Camera Face Match
![Page 22: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/22.jpg)
Embedded Face Recognition
![Page 23: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/23.jpg)
Previous Work in this Area
Dilbert et al., 2005
![Page 24: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/24.jpg)
Sagem Handheld Unit for QPS (Police)
Single handed Operation Fingerprint Reader We have added Face Rec Server Client Operation Windows CE OS
![Page 25: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/25.jpg)
IPhone Mobile Application for Field Operations
• Recognise persons in the field • Local or remote face database • Recognition direct from identity
cards or LCD monitor
![Page 26: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/26.jpg)
Recent Android Implementation for Defence
![Page 27: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/27.jpg)
Half Lit Face
![Page 28: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/28.jpg)
Half Lit Face Example
![Page 29: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/29.jpg)
Even ….
The Cat Burglar?
![Page 30: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/30.jpg)
Method also works for Dingos
![Page 31: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/31.jpg)
And Gender
![Page 32: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/32.jpg)
iFace Demo Release 5 Nov 2013
iFace Demo Version 1.0.0.1111 (beta) Windows 64 (CUDA enabled)
![Page 33: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/33.jpg)
Software Tools for CVPR Applications
![Page 34: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/34.jpg)
Armadillo
• Written in my Advanced Surveillance Research Group to develop CVPR applications
• Easy Conversion of Matlab to fast commercial C++ code
• Distributed in Ubuntu
34
Armadillo C++ linear algebra library
Cited in Google Tech talks 42,000 hits per annum http://arma.sourceforge.net/
![Page 35: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/35.jpg)
OpenCV
• C++ Library of Standard Computer Vision algorithms
• Some functions are GPU-enabled including face detection
• Face detection rather than recognition is the face-in-the crowd recognition bottleneck
• We are working on GPU-enabled best practice face detector
![Page 36: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/36.jpg)
OpenCV
• Face detection is the largest processing load in our system – 95% of CPU load
• We specialise in recognition not detection. • Indeed, our recogniser works on many faces
that cannot be detected by our detector • OpenCV has face detectors based on Haar and
LBP cascades.
![Page 37: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/37.jpg)
ROS
• Designed to create robot applications • Parallel Distributed Middleware • Transparently Distribute face recognition load
to multiple CPUs and GPUs • Only on Ubuntu Unix, but Windows version is
under development
![Page 38: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/38.jpg)
Video Coding 101 • Don’t copy video memory • Reuse memory segments where possible. • Use separate heaps / allocators for video
frames and small objects. • Fragmentation will cause more problems than
using a buffer bigger than you require.
![Page 39: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/39.jpg)
Stride
• Understand stride – this allows for image cropping without copying memory
• We can pull lots of faces from a single source image without copying any image data, we need only keep a reference to the base frame.
![Page 40: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/40.jpg)
x
y
s
x
y
s
x2
y2
s
Stride (step), used to allow matrices to use aligned memory.
When passing an image that allows stride, remember you do not have to copy memory to pass a cropped image. Used in ROI functions in OpenCV.
![Page 41: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/41.jpg)
GPU Acceleration of Detection
• Open CV already has OpenCL and CUDA accelerated Haar cascades, and also CUDA accelerated LBP.
• Enabling CUDA took ~2 lines of changes. • 1) convert cv::Mat to cv::gpu::GPUMat • 2) instantiate cv::gpu::CascadeClassifier_GPU • From 89% cpu usage to 12%! • Up to 7 video streams per CPU
![Page 42: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/42.jpg)
Multi-Camera Face Recognition Appliance
+ GPU
![Page 43: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/43.jpg)
What We Have Learned • Database trials are useful but they are
only a small part of the story • You cannot fake bad data by degrading
good data – H/T Jim Wayman • Find problems that match the performance
levels you can attain and work in the sweet spot
• Collect your own datasets • UI design is crucial to user acceptance
![Page 44: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/44.jpg)
Questions
![Page 45: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/45.jpg)
GeoInt Accelerator • Platform to start developing with GPUs • GPU access, GPU-accelerated apps and libraries • Register to learn more: http://www.nvidia.com/geoint
Webinar Feedback Submit your feedback for a chance to win Tesla K20 GPU* https://www.surveymonkey.com/s/imagus * Offer is valid until December 1st 2013
![Page 46: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/46.jpg)
Upcoming GTC Express Webinars
Register at www.gputechconf.com/gtcexpress
November 6 - Bright Cluster Manager: A CUDA-ready Management Solution for GPU-based HPC
November 13 - Innovative Solutions for Energy Exploration and Data Processing
November 20 - Improving Performance using the CUDA Memory Model and Features of the Kepler Architecture
November 21 - Speeding Up Financial Risk Management Cost Efficiently for Intra-day and Pre-deal CVA Calculations
![Page 47: Accelerating Face-in-the-Crowd Recognition with GPU Technology · • Face detection is the largest processing load in our system – 95% of CPU load • We specialise in recognition](https://reader034.vdocuments.net/reader034/viewer/2022051322/600feae56e59b937462bcb05/html5/thumbnails/47.jpg)
GTC 2014 Call for Posters Open
Posters should describe novel or interesting topics in
Science and research
Professional graphics
Mobile computing
Automotive applications
Game development
Cloud computing
Submit for chance to win Best Poster Award
www.gputechconf.com