knowledgeable independent focused - amazon web …… · knowledgeable independent ... phd...
Post on 11-Apr-2018
222 Views
Preview:
TRANSCRIPT
ATONRÂ PARTNERS SA 12, rue Pierra Fatio - 1204 GENEVA – SWITZERLAND - Tel: + 41 22 310 15 01 www.atonra.ch
ARTIFICIAL INTELLIGENCEThe Technology Of The Future
Knowledgeable
Independent
Focused
15 June 2017
Artificial Intelligence – The Technology of The Future
➢ 2004: AtonRâ Partners finds its roots in fundamental equity research
➢ 2014: AtonRâ Partners’ business shifts toward asset management
➢ Research implemented into thematic investments
➢ Focus on growth, innovation and technology
➢ Scientific research at the core of our DNA
➢ CHF 200 million in assets under management
2
Who We Are
Artificial Intelligence – The Technology of The Future 3
Mobile Payments
Now entering mass adoption phase
Biotechnology
New drug technologies for today’s
diseases
Innovation
Technologies transforming economy
Artificial Intelligence and Robotics
The third step of technological evolution
Global Defense and Security
Strong defense for enduring peace
Bionics
Making human enhancement a reality
Our Investment Themes
Artificial Intelligence – The Technology of The Future 4
AtonRâ’s scientific advisors
Christoph Sinhart
➢ Education
✓ 2013 - 2015: Master of Science in Artificial Intelligence - Maastricht University
✓ 2010 - 2013: Bachelor of Science in Knowledge Engineering - Maastricht University
➢ Working Experience
✓ 2016: Cofounding Stainly (Switzerland) together with AtonRâ Partners
• Natural Language Processing algorithms for the financial sector
✓ 2015: Start of collaboration with AtonRâ Partners
• Developing of internal ERP System for AtonRâ Partners and trying to automate tasks
of Financial Advisors/Analysts
• IT and AI Consulting for AtonRâ PartnersChristopher Wittlinger
Artificial Intelligence – The Technology of The Future 5
Sjoerd van Steenkiste
➢ Education
✓ PhD student, Artificial Intelligence, Swiss AI Lab IDSIA (2016 - present)
✓ MSc, Artificial Intelligence, Maastricht University (2013-2016)
✓ MSc, Operations Research, Maastricht University (2013-2015)
✓ BSc, Knowledge Engineering, Maastricht University (2010 - 2013)
➢ Working Experience
✓ AtonRâ Partners, Scientific Advisor (2014 - present)
✓ NNAISENSE, Research Scientist (April – June 2016)
➢ Publications & pre-prints
✓ A Wavelet-based Encoding for Neuroevolution - Sjoerd van Steenkiste, Jan Koutník, Kurt Driessens,
Jürgen Schmidhuber - Genetic and Evolutionary Computation Conference (GECCO), 2016
✓ Neural Expectation Maximization - Klaus Greff, Sjoerd van Steenkiste, Jürgen Schmidhuber - International
Conference on Learning Representations (ICLR), 2017, workshop
Artificial Intelligence – The Technology of The Future 6
AI, A Game-Changer For Many Industries
➢ One of the most impactful technologies of the 21st
century
➢ The IT industry has already been transformed by AI over
the last decade
➢ Large investments in AI have led to new industries and
smarter applications
➢ Current technology already enables other industries to be
transformed
➢ As research advances and applications become smarter,
AI is expected to become essential in our everyday life
Sources: Toptal, get.com, Google Translate, hiteks.com, hardbaconmedia, extremetech.com
Recommendation /
Advertising Systems
Spam /
Fraud Detection
Customer
Interaction
Healthcare
Self-driving
Cars /
Robotics
Language
Translation
Artificial Intelligence – The Technology of The Future 7
What Is Artificial Intelligence?
➢ AI: Intelligence exhibited by machines
✓ A fundamental research area with potentially broad
practical impact
✓ Subfields correspond to goals that require intelligent
solutions
➢ Recent advances in AI are mainly the result of innovations in
Machine Learning (ML)
➢ Long-term goal is Artificial General Intelligence (AGI) or
“Strong”AI
✓ Progress in ML operates at a different scale than progress
in AGI
Learning
Planning
General
Intelligence
Reasoning
Perception
Motion &
Manipulation
Creativity
Natural
Language
Processing
Artificial Intelligence
Artificial Intelligence – The Technology of The Future 8
The Need For Machine Learning
➢ Most of the human knowledge and skills can not be described in
an explicit form
✓ Large variability of inputs
✓ Abstract notion of concepts
➢ Algorithmic tasks are well-defined and can be implemented
efficiently
✓ Sorting lists, arithmetic operations
➢ Cognitive tasks can not be implemented efficiently
✓ How do we define a cat?
How to sort a list?
[4, 2, 1, 9, 6]
[2, 4, 1, 9, 6]
[1, 2, 4, 9, 6]
[1, 2, 4, 9, 6]
[1, 2, 4, 6, 9]
Which images contain a cat?
Source: image-net.org
Artificial Intelligence – The Technology of The Future 9
Supervised Learning At The Heart of AI
➢ Using Machine Learning (ML), we are able to learn these complex non-linear functions / programs
that are not algorithmic in nature
➢ Current ML applications make use of supervised learning
➢ A lot of labeled data need to be available in order to ensure generalization
✓ Generalization – the ability of the machine to produce sensible answers on new inputs that it
has never seen during training
Input Machine Learning Model Output
CatCat
DogCat
CatCat
Target
DogCat
CatDog
DogCat
?
Artificial Intelligence – The Technology of The Future 10
Deep Neural Networks Are A Powerful Method
➢ Researchers have succeeded in training deep neural networks
➢ They consist of many layers of differentiable non-linear transformations
✓ Learn a hierarchical feature representation along side the objective and can be trained
end-to-end
• Early layers learn low-level features such as edges and corners
• Later layers combine these features to obtain “face” features or “ear” features
• The last layer of the model then computes the output from this high-level
representation
Input Deep Learning Model Output
CatCat
DogCat
CatCat
Target
DogCat
CatDog
DogCat
Input LayerL1 L2 L3 Output
Source: Convolutional Deep Belief Networks for Scalable Unsupervised Learning of Hierarchical Representations, Honglak Lee, Roger Grosse, Rajesh Ranganath, Andrew Y. Ng, ICML 2009
Artificial Intelligence – The Technology of The Future 11
The Deep Learning Era
➢ Deep Learning (DL) models that can be trained end-to-end have successfully rivalled human
performance in several domains
➢ These recent advances are the product of several critical components:
✓ Increased computational efficiency
✓ Availability of large amounts of labeled data to train
✓ Large funding (from industry) in machine learning research
Go Speech Recognition
Object Recognition
Atari Games Skin Cancer Detection
Artificial Intelligence – The Technology of The Future 12
It’s All About Computing Power…
➢ Training Deep Learning (DL) models is computationally
expensive
✓ Optimization is iterative
✓ Learning complex functions requires models have large
numbers of parameters and big data
➢ The majority of the computations can be executed in parallel
✓ Graphic Processing Units (GPUs) are specialized at
massively parallelizing computation
✓ Chip performance has increased exponentially according to
Moore’s law
➢ GPUs have become essential for research and application in
deep learning
Source: NVIDIA
Artificial Intelligence – The Technology of The Future 13
… And Data
➢ Successful industrial applications of Deep Learning (DL) are learned
through feedback and require large amounts of labeled data
➢ More complex models require more data in order to ensure
generalization performance
✓ ~1M labeled images for a 1000-way discriminator / locator of objects
in a picture
✓ ~10M labeled images of face recognition
✓ ~1B sentence pairs for Machine Translation
➢ Labelled data is the most important resource for deep learning methods
at production scale
➢ Access to the right data yields a competitive advantage
Source: https://developers.google.com/recaptcha/old/docs/customization
Acquiring labeled data through CAPTCHA
ImageNet classification
Artificial Intelligence – The Technology of The Future
➢ Massive funding in Machine Learning/Deep Learning has sparked many breakthroughs since 2012
➢ Deep Learning is a young field and most problems benefit from an individual approach that requires some research
✓ Fundamental research can directly be incorporated in applications
✓ Tech companies have successfully acquired many researchers from academia and start-ups to perform
fundamental research
44
33
25
10
10
10
2
0
0 10 20 30 40 50
Microsoft
Deepmind
IBM
Amazon
Baidu
Apple
# papersAccepted papers at ICML 2017
Scientific Progress Continues Unabated
14
Artificial Intelligence – The Technology of The Future
Some Challenges Remain
And Need To Be Tackled
Artificial Intelligence – The Technology of The Future
➢ Current AI approaches are extremely good at learning complicated functions provided that enough data is available
✓ In domains where little labeled data is available, Deep Learning (DL) will likely fail
➢ DL models are mostly unable to transfer learned knowledge across different domains
➢ As a rule of thumb we can currently solve most of the problems that would require a second of thought by a human
✓ General understanding as we humans have is thus still far away
✓ There is neither a clear road to achieve this
How Powerful Are Current Methods?
16
Artificial Intelligence – The Technology of The Future
➢ Deep Networks are black-box architectures, which makes it difficult to evaluate how a model has
learned the task
Source: worldlife.org
✓ If the network has learned about the presence of ice instead of the color of the bear skin, it will wrongly
predict the right image to be of a brown bear
➢ Generalization performance helps evaluate the learned knowledge, but is always limited
➢ This may be problematic in which an incorrect decision may affect human lives
✓ Medical diagnoses and self-driving cars are clear examples of this
Polar Bear Polar Bear Brown Bear Brown Bear Polar Bear
Train Test
Current Limitations of Deep Networks
17
Artificial Intelligence – The Technology of The Future
➢ Machine Learning models do not learn concepts in the same way humans do
Source: Adversarial Examples for Semantic Image Segmentation, Volker Fischer, Mummadi Chaithanya Kumar, Jan Hendrik Metzen, Thomas Brox, ICLR
workshop poster. Explaining and Harnessing Adversarial Examples, Ian J. Goodfellow, Jonathon Shlens, Christian Szegedy, https://arxiv.org/abs/1412.6572
Fooling an image classifier
Fooling an image segmenter
✓ Adversarial methods can be used to target
trained deep neural networks to alter their
predictions
✓ These so-called adversarial attacks may
seem harmless at first, but could have
disastrous effects
Fooling Learned Models, Or Hacking In An AI World
18
Artificial Intelligence – The Technology of The Future
➢ New learning paradigms that rely less on labeled data can
overcome the limitations of supervised learning
✓ Reinforcement Learning
✓ Unsupervised Learning
➢ Increase robustness by separating out different parts of the
function to be learned
✓ Most modern self-driving car systems are not learned
end-to-end
➢ Inspecting how different parts of a model interact helps in
identifying what has been learned
➢ Many of these are active research areas
Source: Show, attend and tell: Neural image caption generation with visual attention, K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhutdinov, R.S. Zemel, Y. Bengio. arXiv preprint arXiv:1502.03044, Vol 2(3), pp. 5. CoRR. 2015.
input predictioninput
model
input predictionground truth
model
Unsupervised learning
Supervised learning
Reinforcement learning
Identifying what has been learned
Can We Overcome These Challenges?
input
rewardmodel worldaction
19
Artificial Intelligence – The Technology of The Future
➢ An agent (model) observes an environment and interacts with it by means of actions
✓ The environment provides feedback to the agent in the form of rewards
✓ The agent learns to act in order to maximize its reward
➢ Many problems can be modelled in this context, i.e. control, trading, interaction
✓ However the credit-assignment problem makes learning difficult: The agent does not know, which (sequence of)
action(s) caused a positive reward
➢ There is a lot of ongoing research in this domain and current techniques are increasingly being applied in practice
Source: https://deepmind.com/blog/deepmind-ai-reduces-google-data-centre-cooling-bill-40/
Reducing energy consumption of Google data centersReinforcement learning
Reinforcement Learning: Giving Rewards To AI Models
input
rewardmodel worldaction
20
Artificial Intelligence – The Technology of The Future
➢ Learning structured representations of data without supervision
✓ These representations contain useful knowledge in a condensed form
✓ Makes it easier to generalize information across domains
✓ Supervised learning methods can learn from these representations using fewer labeled
examples
Source: Klaus Greff, Sjoerd van Steenkiste, Jürgen Schmidhuber, Neural Expectation Maximization, ICLR workshop poster
Learning about structure in the world
inputprediction
input
model
Unsupervised learning
Unsupervised Learning Reduces The Need For Labeled Data
21
Artificial Intelligence – The Technology of The Future
➢ Learning generative models is useful for many application domains of AI: Feature learning, Planning
Source: StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks Han Zhang,
Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaolei Huang, Xiaogang Wang, Dimitris Metaxa https://github.com/luanfujun/deep-photo-styletransfer, https://github.com/SKTBrain/DiscoGAN
Text to image generation
Content Image Style Image Output Image
Photo Style Transfer
Media / eCommerce: Improved Creation & Design
➢ Also, interesting applications in creativity & design
✓ Generating images from text, shapes
✓ Image in-painting
✓ Cross-domain content transfer
23
Artificial Intelligence – The Technology of The Future
➢ AI offers many opportunities to modernize the health care industry
✓ Automated analysis of scans / treatments
✓ Automated patient diagnoses
✓ Precision medicine
➢ With the amount of labelled medical data that is already available, it is believed
that current technology could already solve specific tasks more efficiently
✓ Radiology / dermatology is the main focus of a handful of AI start-ups
➢ AI advances are also expected to aid in medical research (such as for a cure of
cancer) by predicting drug responses and searching for anomalies in large bulks
of data
Health Care: Improved Diagnostics And Personalized Medicine
24
Artificial Intelligence – The Technology of The Future
➢ Artificial Intelligence / Machine Learning (ML) techniques are important for advanced trading techniques
✓ The large amount of data that is easily labeled makes this an interesting place for Deep Learning (DL) methods
✓ However the nature of the financial market complicates the use of DL methods
✓ The ratio of hidden market variables to observed quantities is high and requires more data to be modelled
efficiently
✓ Decision making incorporates a lot of sentiment,
which is not present in the data
Finance: Enhancing Decision-Making
➢ Deep learning models will not be able to model the
entire market
✓ However they can be used alongside financial
experts to improve upon decision making,
feature identification or anomaly detection
25
Artificial Intelligence – The Technology of The Future
➢ Mobile devices are the primary consumer of many AI applications
➢ It is desirable to have models directly operate on the device
✓ User data can remain private
✓ Online learning from user experience yields a more personalized AI
➢ A rising number of AI applications increases the demand for hardware
➢ Cloud computing platforms that offer fast GPUs (Graphic Processing Units)
are an attractive alternative
➢ Major tech companies are developing specific AI hardware to optimize
inference
✓ Hosting this hardware in the cloud provides additional means of return on
investment
Source: https://research.googleblog.com/2017/04/federated-learning-collaborative.html
Federated Learning
Cloud Computing
Cloud operators and GPU vendors: The Picks And Shovels
of The AI Rush
26
Artificial Intelligence – The Technology of The Future
➢ Superintelligence could happen in decades or centuries, many of the top AI experts
disagree
✓ However recent advances in AI have revived the debate concerning AI safety
➢ Two common failure cases can be considered
✓ An AI is programmed to do harm
• Autonomous weapons - need not be robots
✓ An AI is programmed to do good but implements a destructive strategy in achieving its
goal
• Researchers / Practitioners could fail in aligning the goals of an AI with our
goals through misspecification (example: paperclip factory)
➢ The objective is to align the goals of an AI with ours before it becomes superintelligent
✓ Develop methods to evaluate what an AI has learned and be able to test its goals
✓ Learn how to raise an AI in order to have it adopt our goals and beliefs
Source: Portal 2, When Will AI Exceed Human Performance? Evidence from AI Experts Katja Grace, John Salvatier, Allan Dafoe, Baobao Zhang, Owain Evans.
When can we expect Human Level Machine Intelligence (HLMI) as predicted by AI experts
Superintelligence & AI Safety
27
Artificial Intelligence – The Technology of The Future
➢ A more serious and immediate concern than the superhuman intelligence scenario
➢ But we are not yet at a point where massive job losses are imminent
✓ However, AI is expected to replace a lot of educated jobs in the near future
➢ Which jobs are currently at risk?
✓ Jobs that are well-defined / have an exact outcome and for which a lot of labelled training data is
available
✓ For example translators, radiologists, taxi-drivers, planners / logistics managers
➢ The closer we get to Artificial General Intelligence, the more jobs are expected to be on the line
✓ Until we have reached that point, AI will also offer a lot of new job opportunities
Will AI Affect Jobs?
28
Artificial Intelligence – The Technology of The Future
➢ AI will have a large impact on humanity in the 21st century
➢ Innovations in AI are currently driven by supervised machine learning
✓ Success factors include access to human expertise, compute resources and data
➢ Current supervised learning approaches are black-box systems that have some practical
disadvantages
✓ Difficult to evaluate learned knowledge, susceptible to adversarial attacks and too much
dependent on labeled data
✓ Large scale research in deep learning will mitigate these concerns in the near future
➢ Artificial General Intelligence is far away and it is unclear how to get there
➢ Job losses as a result of new AI technology are a more immediate concern
Summary
29
Artificial Intelligence – The Technology of The Future 30
REINFORCEMENT
LEARNING
UNSUPERVISED
LEARNING
MACHINE
LEARNING
SUPERVISED
LEARNING
Rewards
Data
(raw data)
Labeled data
(data with tags,
e.g. dogs, cats…)
Q & A
Knowledgeable
Independent
Focused
ATONRÂ PARTNERS SA 12, rue Pierra Fatio - 1204 GENEVA – SWITZERLAND - Tel: + 41 22 310 15 01 www.atonra.ch
Artificial Intelligence – The Technology of The Future
THANK YOU FOR YOUR ATTENTION
top related