visual geometry group department of engineering science...

Image Classification

Dr. Ernesto CotoResearch Software Engineer

Visual Geometry GroupDepartment of Engineering Science, University of Oxford

Outline

2

BBC News Search

– Image Classification Engine

–Demo

Sharing our work

–VGG's Image Classification (VIC) engine

–Demo

– Features

–Availability

–Deployment

–Data Ingestion

BBC News Search

Powerful public web engine for visual search overthe BBC News dataset ( ~17K programmes, ~10Khours of video, ~5M keyframes, ~1.5 TB)

3Available at: http://zeus.robots.ox.ac.uk/bbc_search/

BBC News Search

Objects/Scenes (Image Classification): Searches of abroad nature: e.g. animals (horses, etc.), vehicles (cars, etc.),styles (gothic architecture, etc.)

Instances (Image Matching): Searches for a specificobject or building (e.g Big Ben)

Text: Searches for instances of a text string

People: Searches of people or face types (e.g. Tony Blair)

4

BBC News Search

The Image Classification engine can be input text. The textis used to downloaded training images from a searchengine. This starts a search for similar images in the dataset

Matching images in the dataset are ranked and displayed

If the input is an image, the image is used to train themodel

5

Sharing our work: VIC

You can use the same Image Classification engine forsearching on your own dataset !

VGG's Image Classification (VIC) engine

6

Sample dataset: COCO2014

Common Object in Context (COCO) – 2014 version

Complex everyday scenes containing common objects intheir natural context.

Includes 80 object categories

More than 300K images

– We use a subset that includes objects and text (~14Kimages)

http://mscoco.org/7

VIC: Features

Query by entering a text or an image

Training images automatically downloaded fromGoogle

Automatic training, classification and result ranking

User management

Query refining (Uber classifiers) and caching

Curated queries

Data ingestion:

– Use your own dataset and/or extend it later

– Define your own metadata

8

VIC: Availability

Supported platforms: Linux and Mac. Currentlyworking on the Windows deployment

Deployment options:

–Simple deployment in your computer via Dockertechnology

– For advanced users, native deployment from thesource code is also possible

9Images take from: https://www.docker.com

VIC: Simple Deployment

1) Install Docker (needs privileged access)

2) Install the VIC application (installer files provided)

3) Instruct VIC to ingest your data

10

111

2

3

VIC: Mac Deployment Demo

Install Docker for Mac

Install VIC using provided installer

BEFORE starting VIC, copy your own data to VIC folder

Start the VIC application and perform the data Ingestion

11

Admin tools with different options to add/remove data

from the Image Classification engine

VIC: Advanced Deployment

Native deployment successfully tested on Ubuntu 14 LTS,macOs Sierra. Installation scripts can be provided.

Frontend consists of a Django Application and the imagedownloading engine (Google web engine only)

– Could provide support for other downloading engines

– Source code: https://gitlab.com/vgg/vgg_frontend

Backend consists of a python wrapper and a native imageclassification component

– Source code: https://gitlab.com/vgg/vgg_classifier

Backend and Frontend communicate via an API. Newbackends can be incorporated.

Docker deployment requires Docker v17.03.1-ce.12

VIC: Stats

Data ingestion:

– Dataset: ~14K images, ~27 min (multi-threaded)

– Negatives: ~17K images, ~1 hour (single-thread)

Backend data footprint

– Dataset: ~2.4 GB

– Extracted dataset features: 7.6 MB

– Negative images: ~416.7 MB

– Extracted negative features: 9.5 MB

Backend memory footprint: 185 MB (reducible with PQ)

One cached query: 18 MB (16 MB of training images)

Docker image: 1.4 GB13

Image Classification

Dr. Ernesto CotoResearch Software Engineer

Visual Geometry GroupDepartment of Engineering Science, University of Oxford