"blending cloud and edge machine learning to deliver real-time video monitoring," a...

Copyright © 2017 Camio 1

real-time video search

Carter Maslan

May 2017

Blending Cloud and Edge Machine Learning to Deliver Real-time

Video Monitoring


In deluge of video, less is more

https://www.camio.com/app/#search;[email protected]


sport utility vehicle

Camio makes video simple & useful

cameras that learn

cloud service

client firmware

works with any camera on any network


• Compute Power

deep learning +

inexpensive cameras +

cloud-connected compute capacity

• Social Expectation

transparency & accountability

+ security threats +

always-connected mobility

Why now? First time feasible & expected


Instant answers, not hours of video


Historical challenges

Problem

Limited edge compute power

Expensive cloud compute

Limited Internet bandwidth

False positives

Diverse workloads

Mitigation

RTSP streams via Camio Box

Real-time event ranking

Metadata-first pipeline

Adaptive motion filters

Hooks for callback extensions


Context is important

Precision & recall aren’t just about Edge vs. Cloud


Metadata-first video pipeline


• Uploads metadata [and video]

• Modular cloud / edge compatible components

• General-purpose task managers

• Validated firmware updates

• Remote logging

Plumbing is important

Effort for Edge + Cloud is 90% plumbing


The Camio Box and Cloud Pipelines

Removal of

spurious motion

Motion

Detection

Events, time

compression

Box

Cloud

Indexing,

storage,

search

ML:

important for

user?

ML: scene

recognition

ML: object

recognition

Current



Removal of

spurious motion

Motion

Detection

Events, time

compression

Box

Cloud

Indexing,

storage,

search

ML:

important for

user?

ML: scene

recognition

ML: object

recognition

Next release



Removal of

spurious motion

Motion

Detection

Events, time

compression

Box

Cloud

Indexing,

storage,

search

ML:

important for

user?

ML: scene

recognition

ML: object

recognition

~ June



Removal of

spurious motion

Motion

Detection

Events, time

compression

Box

Cloud

Indexing,

storage,

search

ML:

important for

user?

ML: scene

recognition

ML: object

recognition

~ Summer


• Edge

• Fat content ML

• High frame rate ML

• Sequential operations

• Use-it-or-lose it state

• Counters (and math)

How to decide what’s Edge vs. Cloud?

• Cloud

• Parallelizable spikes

• Long async operations

• Context & feedback loops

• Shared ML models

• Services & hooks

Driven by available hardware, quality, cost and bandwidth


• User-specific query parsers

• Parser is augmented by labels observed in user’s own history.

• Precedence and boolean logic inferred from known cameras, zones, colors, direction

of movement, etc.

• Unrecognized query terms still applied as filters (e.g., OCR text).

• Natural Language Parser is used for dates, times, relative ranges.

• Minimizes need for search operators (only “v:{{view_token}}”).

• /api/search

• Text-focused search is voice-input friendly.

• E.g., https://camio.com/api/search?text=people+8am+to+10am

• JSON supports flexible metadata/media responses.

How Camio Search Works

https://camio.com/api/search?text=people+8am+to+10am


Example hardware

VM

Box

● 3 1080p cams

● ARM A53

● Quad-core

● 2 GHz

● 2 GB RAM

● Gbit Ethernet

Box Pro

● 16 1080p cams

● Intel i5-5200U

● Dual core (8 equiv)

● 2.2 GHz

● 8 GB RAM

● Gbit Ethernet

Box Virtual

● Linux VM

● Deploy on local

hardware

● Grow with the need

● Download and go!


• https://www.embedded-vision.com/platinum-

members/embedded-vision-alliance/embedded-vision-

training/videos/pages/feb-2017-member-meeting-camio

• https://camio.com/metadata

• https://api.camio.com

Resources

https://www.embedded-vision.com/platinum-members/embedded-vision-alliance/embedded-vision-training/videos/pages/feb-2017-member-meeting-camio

https://camio.com/metadata

https://api.camio.com


real-time video search

Thank You

Making real-time information from real-world events useful and accessible to people,

apps, and services via intelligent video monitoring of any video source.

"blending cloud and edge machine learning to deliver real-time video monitoring," a...

Technology