accelerating computer vision with opencv and...
TRANSCRIPT
![Page 1: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/1.jpg)
ACCELERATING COMPUTER VISION WITH OPENCV AND CUDA
Kirill Kornyakov (Itseez)
![Page 2: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/2.jpg)
AGENDA
1. Slides: OpenCV with CUDA
2. Lab: Video Stabilization
3. QA
![Page 3: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/3.jpg)
OPENCV
Open Source Computer Vision Library
1. 2500+ algorithms
2. More than 8M downloads, active community
3. Permissive BSD license
4. Cross-platform
5. Real-time performance
![Page 4: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/4.jpg)
Your application
Dependencies: Eigen, IPP, JPEG, PNG, Jasper, multimedia TBB
x86, x64 ARM GPU
Windows Linux MacOSX
MIPS
CUDA OpenCL SSE, AVX
Android iOS WinRT
NEON
GCD Concurrency
Python Java
C
cv::parallel_for_
OpenCV
Acceleration API
Hardware
Operating System
Threading API
Bindings
Library
OPENCV ARCHITECTURE
Tegra K1
![Page 5: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/5.jpg)
OPENCV CUDA API
![Page 6: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/6.jpg)
OPENCV AND CUDA
![Page 7: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/7.jpg)
WHY CUDA IS SO GOOD FOR VISION?
Two analogies:
§ Computer Vision vs Computer Graphics
§ Human Vision
![Page 8: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/8.jpg)
COMPUTER VISION AND GPU
High-level information
about a scene Raster image
Computer Vision on GPU
Computer Vision
Computer Graphics
The same hardware boosts both! Red ball Human face
High-level information
about a scene Raster image
Computer Vision on GPU
Computer Vision
Computer Graphics
The same hardware boosts both! Red ball Human face
High-level information
about a scene Raster image
Computer Vision on GPU
Computer Vision
Computer Graphics
The same hardware boosts both! Red ball Human face
High-level information
about a scene Raster image
Computer Vision on GPU
Computer Vision
Computer Graphics
The same hardware boosts both! Red ball Human face
High-level information
about a scene Raster image
Computer Vision on GPU
Computer Vision
Computer Graphics
The same hardware boosts both! Red ball Human face
![Page 9: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/9.jpg)
HUMAN VISION Norbert Kruger et al. Deep Hierarchies in the Primate Visual Cortex: What Can We Learn For Computer Vision?
• 50% of neocortex is about vision (color area)
• 60% of it is for primitive image processing (yellow area)
So, ~30% of our brainpower is spent on simple image processing!
![Page 10: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/10.jpg)
TEGRA K1
![Page 11: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/11.jpg)
ADDITIONAL INFORMATION GTC talks § Joe Stam (NVIDIA) - Extending OpenCV with GPU Acceleration
§ Anton Obukhov (NVIDIA) - Computer Vision on GPU with OpenCV
§ James Fung (NVIDIA) - Computer Vision on GPU with OpenCV
NVIDIA webinars § Shalini Gupta (NVIDIA) - OpenCV - Accelerated Computer Vision using GPUs
§ Anatoly Baksheev (Itseez) - Getting Started with GPU-accelerated Computer Vision using OpenCV and CUDA
Documentation § Homepage: http://opencv.org/platforms/cuda.html
§ Reference manual: http://docs.opencv.org/modules/gpu/doc/gpu.html
§ Tutorials: http://docs.opencv.org/doc/tutorials/gpu/table_of_content_gpu/table_of_content_gpu.html#table-of-content-gpu
![Page 12: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/12.jpg)
![Page 13: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/13.jpg)
![Page 14: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/14.jpg)
![Page 15: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/15.jpg)
LAB
![Page 16: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/16.jpg)
VIDEO STABILIZATION
![Page 17: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/17.jpg)
APPLICATION LIFECYCLE
POC (x86)
Porting
Profiling
Optimize bottlenecks
Fine Tuning
Productization
Regression Tests
Performance Tests
![Page 18: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/18.jpg)
TIMINGS
CPU version (ms)
Naive GPU version (ms)
GPU w/ buffer reuse (ms)
Feature Detection 43 90 20
Feature Tracking 51 64 11.8
Image Warping 24 7.5 0.45
TOTAL 125 (118) 312 (160) 46 (32)
![Page 19: Accelerating Computer Vision with OpenCV and CUDAon-demand.gputechconf.com/gtc/2014/presentations/HANDS... · 2014-04-18 · Title: Accelerating Computer Vision with OpenCV and CUDA](https://reader030.vdocuments.net/reader030/viewer/2022041017/5eca94bc71ecc1733000cec5/html5/thumbnails/19.jpg)
CONCLUSION § GPU greatly helps in real-time Computer Vision apps
§ It helps even better on mobile devices,
where we are usually power/performance bound