introducing - opencv...introducing... open source (bsd licensed) machine learning and computer...
TRANSCRIPT
OpenVisionCapsulesIntroducing...
Open source (BSD licensed) machine learning and computer vision encapsulation
system
OpenVisionCapsules maintainerTyler Compton1/7/2020
What is OpenVisionCapsules?
Format for encapsulating vision algorithms
detector_vehicle.cap
Format for representing analysis results
Application
Tools for encapsulating your algorithm
detector_vehicle.cap recognizer_make_model.cap
What is OpenVisionCapsules?
Capsules are self-describing, single-file algorithm packages
Input: Video FrameOutput: Car, motorcycle, and truck detections
Name: “detector_vehicle”Version: 1Description: “Finds vehicles in video”
Why is this important?
“Using the framework of technical debt, we note that it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning.”
- Machine Learning: The High Interest Credit Card of Technical Debt
Why is this important?
Vehicle Detector Make/Model
Recognizer
Video Quality
ClassifierVehicle/Person
Tracker
Person Detector
Application
Why is this important?
Vehicle Detector
Make/Model Recognizer
Video Quality
Classifier
Vehicle/Person Tracker
Person Detector
Algorithm developers want to write this:
Why is this important?
Application
Application developers want to write this:
Why is this important?
Nobody wants to write this:
The solution!
detector_vehicle.cap
recognizer_make_model.cap
classifier_video_quality.cap
tracker_person_vehicle.cap
detector_person.cap
Algorithm developers write this:
The solution!Application developers write this:
The solution!
detector_vehicle.cap
recognizer_make_model.cap
classifier_video_quality.cap
tracker_person_vehicle.cap
detector_person.cap
OpenVisionCapsules hooks it all up:
Support for OpenVisionCapsules
● BrainFrame will support OpenVisionCapsules starting January 16th
● The OpenCV Hardware Partnership will work with hardware providers to
add support for OpenVisionCapsules
● Consider using OpenVisionCapsules in your computer vision project!
Available now as a developer preview!github.com/opencv/open_vision_capsules
Available now as a developer preview!github.com/opencv/open_vision_capsules
Q & AHW
Items Existing Model Formats:Tensorflow - .pb/.meta/.pbtxt, .tfliteKeras - .h5, .kerasCaffe - .caffemodel, .prototxtTorch - .pt, .pth, t7ONNX -.onnx
OpenVisionCapsules Format .cap
(Portable format for all others)
Design goal DNN training DNN deployment:Self-contained, Executable
DNN model Computing
AI computing target support:CPU, GPU, or other AI chip
✔ ✔
Batch-deploy DNN models onto multiple computing targets
X ✔
Separately batchable inference from parallelizable operations
X ✔
Self-contained DNN model & processing information
DNN weights & model in a single file
X / ✔Separate files in most formats
✔
Executable logic for pre/post processing
X ✔
Expose standardized APIs for runtime algorithm configuration
X ✔
DNN Model Interconnect, Compatibility
Describes the interconnection to other DNN models
X ✔
Compatible with other formats X ✔
OpenVisionCapsules FormatA portable format for all others, e.g. Tensorflow, Torch, ONNX, etc.