deep learning in real world @deep learning tokyo
TRANSCRIPT
-
Deep Learning in real world:Automobile, Robotics, Bio Science
Daisuke [email protected]
Preferred Networks, Inc.
Preferred Infrastructure, Inc.
March 20, 2016 @ DLT: Deep Learning Tokyo
-
l Preferred Networks
Founded in March 2014, Offices: Tokyo and San mateo
40 engineers & researchers
Investors : NTT, FANUC, Toyota
l Transportation
Autonomous driving, and its application
Joint work with Toyota, Panasonic
Manufacturing
Intelligent Robot, Industrial equipment
Joint work with FANUC
Healthcare
Genomic analysis
2
-
AutomotiveHumanoid Robot
Preferred Networks positioning in AI: Industrial IoT
3
Consumer Industrial
Cloud
Device
PhotoGameText
Speech
Infrastructure
Factory Robot
Automotive
Healthcare
Smart City
Industry4.0
Industrial IoT
-
Parking detection
4
l Segmentation + Edge detection
-
Anomaly Detection from Sensors
5
Deep Learning detects abnormal parts
Detect abnormal signals
Normal Abnormal
Sensor data from decelerators in robots
-
6
Deep learning based method can detect symptoms of failure much earlier
Proposed Method
Detect 40 days before the failure
Threshold
Previous Method
Elapsted Time
Detect just before the failure
Robotfailure
Robotfailure 15
-
Massive prediction of (QSAR)
l Drug discovery
Deep learning can predict cross-reactive, side effect, and toxicity from theirstructures and known experimental result.
7
Drugs
Assays
-
Deep Learning for HealthCare
Prediction Model
Genome Drug Assay
Multi-ModalUse different types of data Genome, Drug, Assay, Clinical dataset
Multi-TaskLearn similar different tasks at the same time to enhance their capabilities
PersonalizedMedicine
DrugDiscovery
Diagnosis
Clinical data
-
chainer-DCGANAnime Image Generation from scratchhttps://github.com/mattya/chainer-DCGAN
9
-
2 hours later
10
-
1 day later
11
-
Future direction
-
1. We need non-supervised learning
l Supervised learning is very successful
However, its annotation cost is large
Even human cannot annotate for difficult problems
E.g. Segmentation of video, massive people tracking, cars orientation
l We have unlimited data but its usage is still limited
Image, video, sound and other sensor (LIDAR) + Context information
l Semi-supervised, weakly supervised, one-shot learning is promising
Use unlimited unlabeled data and very few labeled data
l Reinforcement learning is also promising
Self training (human just design reward rules)
13
-
2. Machine teaches other Machines
l Humans learn much faster than machines
Better than a thousand days of diligent study is one day with a great teacher (Japanese proverb)
l Trained machine can teach other machines in several ways
Distillation (Hinton+ 2015)
u Imitating teachers behavior even including their how to mistake
u E.g. Convert a large ensemble model into a small model
Privileged Information Vapnik+ 2014, 2015
u Teacher gives hints at training time, and student use them to learn faster
u E.g. Image with annotation (this is the image of 20 male.)
l Mixing different knowledge from multiple machines
Gathering compressed knowledge from edge devices (car, robotics)
14
-
3. Other objectives for learning
l We need another training objective for unsupervised training
Maximum likelihood estimation is not good strategy for high-dimensional data
u The estimation of gradients of the partition function is very noisy
GAN (generative adversarial network) works quite well for image generation task. By analyzing GAN, we can find what is important for generative model
l Humans and animals seem to use several signals for learning that todays machines cannot use yet.
Predictability seems very fundamental for unsupervised learning but not all
Brain seems to do some inference (unconsciously) so that it explains the world, that makes learning signals for training deep layers [Bengio+ 2016]
How to model curiosity (or incentive to know unknown information) in machines ?
Thanks !
Thanks !
15