Video Recognition Systems of NRC

http://iit-iti.nrc-cnrc.gc.ca/about-sujet/cv-vi_e.html

http://synapse.vit.iit.nrc.ca (www.videorecognition.com)

Leader: Dr. Dmitry Gorodnichy

DNI IARPA VACE, Phase III Meeting Washington DC,

1 November 2006

2. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

NRC - IIT

• From Space Shuttle (Canadarm &

Canadarm2), to Hollywood, to Fine

Arts, to Security applications

3. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

VRS

• Part of Computational Video Group (formed in 2001)

• Our Mandate: Develop Video Recognition

technologies for Canadian companies and OGD’s

• Higher mission:

Computer Vision allows computers to see.

Perceptual Vision allows computers to understand what they see.™

Test-bed and Criteria:

If you are able to recognize something, why computer can’t ?..

4. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

A bit of history

First: Canadarm2

Then: Nouse™, ACE Surveilance

• Canada Borders Services Agency (CBSA)

• Canadian Police Research Center (CPRC, RCMP)

• Ottawa Health Center

• U of Ottawa Music Dept.

• Several Canadian/Ontario private companies 5. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

Key applications

and partners

Domain 1: Security, Surveillance and Monitoring.

Domain 2: Visually-enabled computer-human

interaction (inc. hand-free user interfaces)

Domain 3: Visually-enhanced communication and

Intelligent video processing (eg annotation of

video)

(And partners)

6. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

Technologies developed

1. Neuro-associative memorization/ recognition

Face Recognition from (low-res) Video

(Better than correlogram) Object learning

2. (Multiple) object detection and tracking

Automated tele-operating, …

3. Nose (convex-shape) tracking

Hands-free interfaces

4. Critical Evidence Snapshot extraction

Intelligent Surveillance

5. Fingers, hands detection tracking

Piano playing annotation

7. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

1. Neuro-associative memorization/ recognition

Face Recognition from (low-res) Video

(Better than correlogram) Object learning

2. (Multiple) object detection and tracking

Automated tele-operating, …

3. Nose (convex-shape) tracking

Hands-free interfaces

4. Critical Evidence Snapshot extraction

Intelligent Surveillance

5. Fingers, hands detection tracking

Piano playing annotation

Research & Results

Technologies developed

(X,Y)

name

8. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

What we do

• Lots of research and … coding

– Working with companies: consulting, making

prototypes

– Working with OGD: consulting, joint proposals

• Lead IEEE-published workshops:

at CVPR, CRV, IJCNN

– on Face Processing in Video (2003-2005)

– on Video Recognition (VideoRec’07)

– Neuro-technology IJCNN demonstration workshop

9. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

#1:

Face Recognition in Video

Bad news: low-resolution (TV:320x260) data i.o.d = 12 pixels !

Good news: face detectors (Viola,CMU,MIT,Pitts) can detect such faces !

Solution: make use of temporal domain!

i.e. Learn & Retrieve face from sequences (using Recurrent Neural Networks)

10. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

#1. Efficient visual attention mechanisms (motion etc)

#2. Decision based on accumulating results over time.

#3. Three main principles of neuro-processing

1. non-linear processing,

2. distributed collective decision making

3. synaptic plasticity.

They allows the system:

a) to accumulate learning data in time by adjusting

synapses,

b) to associate a visual stimulus to a semantic

meaning based on the computed synaptic values

Keys principles

(from biological systems) (X,Y)

“Paul”

11. Video Recognition projects and interests (Dr. Dmitry Gorodnichy)

Demo

Solution:

- 12 pixel i.o.d. face model + accumulation over time while tracking (both in training and recognition)

- combination of neuro-biological and computer vision approaches

Tested: on TV shows, on computer user recognition