rob laber, lead cv engineer hristo bojinov, cto...cloud instance selection for your gpu fleet...
TRANSCRIPT
Cloud Instance Selection for Your GPU FleetLessons from Developing Smart Kitchen Technology
Hristo Bojinov, CTORob Laber, Lead CV Engineer
Motivation
Motivation
Motivation
The Thesis
The Problem
Food ↔ People disconnect
Not-so-smart “smart kitchen”
Food info not available, not actionable
The Vision
What We Do
Food, personalization, technology
“The Way You Eat”
Icons made by Madebyoliver, Popcorn Arts, Freepik from www.flaticon.com are licensed by CC 3.0 BY
What We Do
Food, personalization, technology
“The Way You Eat”
(⇒ Computer Vision is essential)
Icons made by Madebyoliver, Popcorn Arts, Freepik from www.flaticon.com are licensed by CC 3.0 BY
Pushing the Envelope on CV
Go from here...
Pushing the Envelope on CV
… to here.
Computer Vision at Innit
Helps us understand users
❖ Inventory, behaviors, multi-sensor fusion, market analytics❖ And, build a delightful user experience
Applications in storage and processing
❖ Recognize and act on food state❖ Visible light, depth, IR
Multi-site program (HQ, academia)
CV Research
Food Recognition Service (AWS)
❖ G2 & G3 instance backend (details coming up)❖ Frontend orchestrates auto and manual processing❖ Service API for 3rd party use
CV Tech: Food Recognition System
Service Layer
Computer Vision
Backend
Offline ProcessingData
StoreWeb
Portal
CV Tech: Food Recognition System
CV Tech: Object Detection Stage
CV Tech: Object Detection Stage
CV Tech: Object Detection Stage
CV Tech: Object Detection Stage
CV Tech: Classification Stage
CV Tech: Classification Stage
CV Tech: Classification Stage
CV Tech: Classification Stage
Controlled scene layout ⇒ precision
In-house data collection and tools
CNN based
CV Tech: Product DB Image Retrieval
CV Tech: Product DB Image Retrieval
❖ Exact product (or attribute) matching
❖ KAZE descriptors (GPU acceleration WIP, stay tuned)
➢ Current need to balance CPU/GPU
➢ Order-of-magnitude acceleration
❖ Hierarchical analysis in the pipeline
❖ + Precise matching using CNN
Summary: The Challenge
Complex Scenes
❖ Segmentation / object detection required❖ Must handle occlusions, uncertainty❖ Highly variable lighting conditions❖ Product DB not 100% accurate❖ Non-image feeds?
Cloud instances: types, plans, sizes...
So what is all this?
Service Layer
Computer Vision
Backend
Offline ProcessingData
StoreWeb
Portal
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
Offline ProcessingData
StoreWeb
Portal
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Single responsibility
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Single responsibility
❖ Stateless GPUs
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Single responsibility
❖ Stateless GPUs
❖
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Non-vision tasks
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Non-vision tasks
❖ Horizontal scaling
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
❖ Non-vision tasks
❖ Horizontal scaling
❖ Reserved instances
Decoupled Service Architecture
Decoupled Service Architecture
WHY?
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
Service Layer
Service Layer
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
Service Layer
Service Layer
Cost-effective buffer
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
To summarize...
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
To summarize...
❖ Reserved instances
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
To summarize...
❖ Reserved instances
❖ Optimize hardware usage
Decoupled Service Architecture
Service Layer
Computer Vision
Backend
To summarize...
❖ Reserved instances
❖ Optimize hardware usage
Delegate tasks to save money!
Isolated Background Jobs
Service Layer
Computer Vision
Backend
Offline ProcessingData
StoreWeb
Portal
Isolated Background Jobs
Clustering
Isolated Background Jobs
Big Data
Isolated Background Jobs
Neural Net Training
Isolated Background Jobs
Offline ProcessingData
Store
Isolated Background Jobs
Offline ProcessingData
Store
❖ One-off or periodic
Isolated Background Jobs
Offline ProcessingData
Store
❖ One-off or periodic
❖ Resource intensive
Isolated Background Jobs
Offline ProcessingData
Store
❖ One-off or periodic
❖ Resource intensive
❖ Need it now?
Spot Pricing?
Spot Pricing?
❖ Auctioned resources
Spot Pricing?
❖ Auctioned resources
❖ Price changes with resource availability
Spot Pricing?
❖ Auctioned resources
❖ Price changes with resource availability
❖ Interrupted jobs
Isolated Background Jobs
Offline ProcessingData
Store
❖ One-off or periodic
❖ Resource intensive
❖ Need it now?
Spot pricing saves money!
Isolated Background Jobs
Trade-Offs
Trade-Offs
❖ Cost vs. Performance
Trade-Offs
❖ Cost vs. Performance
❖ Local vs. Cloud
Trade-Offs
❖ Cost vs. Performance
❖ Local vs. Cloud
❖ Cost vs. Complexity
Trade-Offs
❖ Cost vs. Performance
❖ Local vs. Cloud
❖ Cost vs. Complexity
❖ Now vs. Later
Takeaways
❖ Complex workload → use differentiated EC2 instances
❖ Profile GPU-dependent tasks, choose carefully
❖ Consider online / offline loads, spot instances
Join the party! [email protected]
About Innit❖ Inform and elevate the interaction between people and food❖ 5+ years in the making, substantial funding, IP & tech❖ Partnerships (appliances, FMCG, culinary)
About the Presenters❖ R&D and Technology at Innit