video analytics survey - higher school of economics...2017/03/25 · image processing applications...
TRANSCRIPT
![Page 1: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/1.jpg)
Video Analytics SurveyAuthor: Dmitry Khizbullin
Intellivision
1
![Page 2: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/2.jpg)
Areas of video analytics (VA) application
1. Surveillance / security2. Retail (shops, stores, parks)3. Automobile: driver assistance / autopilot4. Mobile apps5. Military6. Medical
2
![Page 3: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/3.jpg)
Video analytics applications1. Object detection (images): person/pedestrian, car,
License Plate Detection, tumor, Frontal Collision Warning2. Object recognition (images): face, tumor, age, gender,
License Plate Recognition3. Object tracking (video): single-camera, multi-camera,
moving-camera, Lane Departure Warning4. Special metrics/events extraction: heartbeat, goods theft,
people counting for retail, parking lot occupancy, crossroad occupancy, left/abandoned, slip/fall, fever
3
![Page 4: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/4.jpg)
Examples
4
Pedestrian detection License plate detection/recognition
![Page 5: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/5.jpg)
Examples
5
Frontal collision warning Lane departure warning
![Page 6: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/6.jpg)
Examples
6
Object tracking Multicamera
![Page 7: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/7.jpg)
Examples
7
Parking lot occupancy
Point of service
![Page 8: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/8.jpg)
Examples
8
Abandoned objectFever detection
![Page 9: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/9.jpg)
Image processing applications with ML
1. Image search, similar images, image annotation2. Style transfer: Prisma3. Face enhancement: FacePlus by itseez4. Fun overlays: iSwap Faces, Funny Camera
9
![Page 10: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/10.jpg)
Eras of ML (my view)
1. Expert systems (past, few ML)2. Single-knowledge-area systems (nowadays)3. Future systems (multi-knowledge-area)
A DARPA Perspective on Artificial Intelligence
https://www.youtube.com/watch?v=-O01G3tSYpU
10
![Page 11: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/11.jpg)
Handcraft vs Machine Learning
1. Pure handcrafted solution2. Hybrid handcrafted-ML solution3. End-to-end ML solution
11
Pre-process handcrafted
featuresML stage
Post-process handcrafted
decision logic
![Page 12: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/12.jpg)
Handcraft
Handcrafted features:
LBP, HOG, Optical flow, edges, frame difference
Handcrafted post-processing:
Spatial filtering (clusterization), temporal filtering
12
![Page 13: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/13.jpg)
Machine learning
1. Conventional: statistics, SVM, decision trees, cascades of classifiers
2. Deep learning: classic fully-connected, convolutional (CNN), full-convolutional (FCN), complex architectures (Google Inception, MS Residual/Resnet, Darknet/YOLO etc), LSTMs
13
![Page 14: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/14.jpg)
Image public datasets
1. Imagenet (1000 class, 1.2M images)2. MS COCO (80 class, 120k images)3. Pascal VOC (20-class, 25k images)
14
![Page 15: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/15.jpg)
Transfer techniques
1. Transfer data (use the same dataset in a different task)2. Transfer learning (use the same leaned model in a
different task)3. Transfer intuition (use your knowledge of what is
feasible, what is the maximal possible accuracy and what approach is the best)
15
![Page 16: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/16.jpg)
Points to care about
1. Source data: public & private2. Runtime: offline / low-latency / real-time3. How critical the accuracy is: advisory /
statistically-accurate / never-fail4. Servers for training models
16
![Page 17: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/17.jpg)
Legitimacy
Public data Private data
Allowed for commercial use Very limited Must collect by your own, usually small
Prohibited for commercial use
Rather big, take care to not disclose that you use it
Big, but take care to not disclose occasionally
17
![Page 18: Video Analytics Survey - Higher School of Economics...2017/03/25 · Image processing applications with ML 1. Image search, similar images, image annotation 2. Style transfer: Prisma](https://reader034.vdocuments.net/reader034/viewer/2022042122/5e9c5850444f462d0a5239e9/html5/thumbnails/18.jpg)
Thanks for your attention
Questions?
18