© 2017 Arm Limited

An introduction to Machine Learning

silicon

November 28 2017

Title 44pt sentence case

Affiliations 24pt sentence case

20pt sentence case

Insight forTechnologyInvestors

© 2017 Arm Limited 3

AI/ML terminology

Additional terms

• Location

• Cloud – processing done in data farms

• Edge – processing done in local devices

• Types of machine learning

• Model – a mathematical approximation of a collection of input data

• Training – in deep learning, data-sets are used to create a ‘model’

• Inference – using a ‘model’ to check against new data

Artificial Intelligence

Machine Learning

Deep Learning

Algorithms: CNNs,

RNNs, etc.

© 2017 Arm Limited 4

Neural Networks (NNs) outperform humans

28%

26%

16%

12%

7.3% 6.7%

3.6% 3%

2010 2011 2012 2013 2014 2015 2016

shallow

AlexNet, 8 layers

Human error

deep

ZF, 8 layers

VGG, 19 layers

GoogleNet, 22 layers

ResNet, 152 layers

CUImage

Data for ImageNet Large Scale Visual Recognition Challenge

Deep networks, introduced in 2012, resulted in big improvements

Error rates have now stabilized at ~3%

Cla

ssif

icat

ion

err

or

(Image source: Synopsys)

© 2017 Arm Limited 5

Machine Learning training

Training data Model

For each piece of data used to train the model, millions of model parameters are adjusted.

The process is repeated many times until the model delivers satisfactory performance.

© 2017 Arm Limited 6

Machine Learning inference

✓

Input Model Output

96.4% confidence

97.4% confidence

When new data is presented to the trained model, large numbers of multiply-add operations

are performed using the new data and the model parameters. The process is performed once.

© 2017 Arm Limited 7

Why is on-device ML driving AI to the Edge?

Bandwidth PrivacyLatencyCostPower

© 2017 Arm Limited 8

Inference everywhere

Robotics

Surveillance IoT Augmented reality

Mobile DronesAutomotive

Shipping & logistics

© 2017 Arm Limited 9

Processor options for Machine Learning workloads

© 2017 Arm Limited 10

A System-on-Chip contains multiple compute engines

Main processor (CPU)A versatile compute engine for running rich software. The main CPU runs device’s operating system, applications and user interface. It also manages the flow of data to specialist processors in the device.

Graphics processor (GPU)Used for generating 2D/3D images and executing highly-parallelised workloads such as neural network arithmetic

Digital signal processors (DSPs)A specialist form of CPU, optimised for analysing waveforms.Useful for radio control, sensor readings, audio and image processing

Accelerators Heavily-optimised data processors for frequently-used tasks,e.g. encryption, video, computer vision

© 2017 Arm Limited 11

Comparing processor options for Machine Learning

Training Inference Usability

Hardware cost Power efficiency Hardware cost Power efficiency Flexibility Programmability

CPU

DSP

GPU 1 2 3 1 2

Accelerator

FPGA

Weak, relative to alternatives

Good, relativeto alternatives

1 = High volume, evolving workload2 = High volume, stable workload3 = Low volume, evolving workload

1 = A client device that requires a GPU for graphics2 = A device that uses a GPU for ML work only

© 2017 Arm Limited 12

Processor options for various sizes of chip

Machine Learning demands (accuracy, response time) vary by use case

All use cases can default to a CPU

A GPU is often a good ‘all-rounder’ solution

Accelerators are useful when it is essential to either maximize response speed or minimize power consumption

Silicon area / power consumption

Perf

orm

ance

Keyword detection

Speech recognition

Visual object detection

Visual object recognition

Accelerator

Accelerator

Cortex-M

Cortex-A

(little CPU)

Cortex-A

(big CPU)

GPU

© 2017 Arm Limited 13

Arm’s ML computing platform

Arm DS-5 / Keil tools / compilers / drivers

AI Applications: ML, CV, speech recognition etc. Applications

Edge devices

Stable SW interfaces

Neural network frameworks(e.g. Tensorflow, Caffe, AndroidNN)

SpiritComputer Vision

Compute librarySpirit metadata

library

Optional Spirit libraries

& model sets

SVE

CPU CPU GPU Partner IP: DSPs, FPGAs, accelerators Provided

by Arm

Provided by third-party

© 2017 Arm Limited 14

Machine Learning is driving all of Arm’s technology roadmap

Processor design Software support Computer vision

1515 © 2017 Arm Limited

The Arm trademarks featured in this presentation are registered trademarks or trademarks of Arm Limited (or its subsidiaries) in the US and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners.

www.arm.com/company/policies/trademarks