2015-07 12015-07

ARAKAWA, Naoya, Ph.D

Human-Level AI & Phenomenology

2015-07-11

2015-07 2

Today’s Topic

● Creating Human-Like AI

○ Background, Issues & Approaches

○ Its relation to Embodiment &

Phenomenology

○ My recent activities

2015-07 3

Abridged CV

• Education–Undergraduate ： Brain & Neural Nets

–Graduate (M.E.) ： Systems Science

–Ph.D ： Philosophy of Language“The Naturalization of Reference”

• Work: Natural Language Processing

– Machine Translation

– Dialog systems

– Semantic analysis, Ontology compiling

• Recent activities: Artificial General Intelligence

2015-07 4

Table of Contents

1.Background for Human-Level AI● AI with Human cognitive functions● Recent ‘AI boom’● Two contrapositions

2. Issues to be solved

3. How to create Human-Level AI

4. My recent activities

2015-07 5

Human-Like AI

● An aim/ambition of the AI discipline

○ 「 Agalmatophilia 」 ?

○ AI as 「 Cognitive Science 」

● Constructive (Make & Test) Understanding of

Human-beings

○ Build to understand

○ Difficulty in fully analytic understanding

2015-07 6

Recent “AI Boom”

● Media Coverage○ AI books for general public

○ TV programs on AI

○ New research centers

● Technological Background○ Computing Power○ Availability of “Big Data”○ Some notable results: Chess, Jeopardy!, Self-driving

cars, ...○ Advances in Machine Learning

Deep Learning! ⇒

2015-07 7

Advances in Machine Learning

● The Neural Net Strikes Back!

● Deep Learning○ Multi-Layered Neural Networks

○ Notable results in pattern recognition

○ Automatic concept formationGoogle Brain (Cat), Google Dreams (Inceptionism)

● Recurrent Neural Network (RNN)○ Learning time-series○ Captioning images with deep learning (Stanford U.)

● Reinforcement Learning○ Learning action sequences based on rewards

○ Deep Q Network: playing Atari games

2015-07 8

AGI vs. Narrow AI

● Artificial General Intelligence vs. Narrow AI○ Artificial General Intelligence

■ ‘General’ in the sense that it can learn various skills■ Human-Like AI AGI⊂■ Long hoped... but difficult to realize⇒

○ Narrow AI: to solve specific issues〜 the current main stream

● GOFAI vs. Emergentist AI○ Good Old-Fashined (Symbolic) AI

■ Criticized by thinkers such as Dreyfus & Lakoff■ Knowledge acquisition bottleneck

○ Emergentist AI■ Knowledge is not to be given but to learn■ Analog (statistic)※Advances in machine learning AGI sees the light here!?⇒

2015-07 9

Table of Contents

1.Background for Human-Level AI

2. Issues to be solved ● Knowledge Acquisition ＝ Learning ＝ Epistemology● Cognitive Functions

2. How to create Human-Level AI

3. My recent activities

2015-07 10

Issues to be solved

Knowledge Acquisition ＝ Learning＝ Epistemology● How do we get knowledge?

● How do machines get knowledge?

● More concretely:

○ Acquistion of concepts ←（ perception & motion ）

○ Knowledge acquisition on action（ praxis/pragmatics←motion & perception ）

○ Language Acquistion■ Acquistion of Vocabulary (the Symbol Grounding Problem)

■ Acquistion of Grammar

2015-07 11

Cognitive Functions to be realized

○ Human-Level AI⇔Inventory of Human Cognitive Functions○ Learning 〜 Knowledge Acquisition

■ Pattern Recognition (mostly supervised)■ Conceptual Learning (mostly unsupervised)

● ‘Clustering’● ‘Representation Learning’ in Deep Learning

■ Reinforcement Learning ： learning action sequences based on rewards■ Episodic Memory ： One-shot Learning

○ Planning & Execution■ Emergentist AI: trying to get inspiration from the prefrontal cortex?

○ Linguistic Functions■ Generativity （ Syntax ）■ Social aspects （ Pragmatics ）■ Grounding （ Semantics ）

2015-07 12

Table of Contents

1.Background for Human-LevelAI

2. Issues to be solved

3. How to create Human-Level AI● Three Pillars● Make & Test (Constructive) Approach

2. My recent activities

2015-07 13

How to Create Human-Level AI

1.Three Pillars （ IMHO ）•Cognitive Architecture: Overall Structural ModelsIntelligence has ‘structure’Traditional ones: symbolicYou can learn from the brain too.

•Machine LearningMathematical models for learning

•Cognitive Robotics (embodiment)Learning developmentally in the environment

2.The Constructive (Make & Test) Approach• Hypotheses robots/simulation to corroborate⇒• Cognitive Robotics• Artificial Brains

2015-07 14

Cognitive Robotics

• Robotics as Cognitive Science• Stance: cognition requires the body.• ‘Constructive’ understanding of cognition

Construct to understand!

• Genres– Cognitive Developmental Robotics

• Developing cognitive abilities like human children

– Robotics for Symbol Emergence• Learning language via interaction with the environment

– Robotics for Social Intelligence• Communicating robots

2015-07 15

Cognitive Developmental Robotics

• Developing cognitive abilities like human children• Robots learns from interaction with the

environment• To complement experiments with human infants

(which are difficult for ethical reasons)• Researches in Japan, e.g.:

–Asada Lab. @ Osaka U.–Kuniyoshi Lab. @ Tokyo U.–The Constructive Developmental Science @ MEXT

• Ref.– Cangelosi, A. et al.: Developmental Robotics

-- From Babies to Robots, MIT Press (2015).– Asada M. et al.: "Cognitive developmental robotics: a survey," in IEEE Transactions

on Autonomous Mental Development, Vol.1, No.1, pp.12--34 (2009)

2015-07 16

Robotics for Symbol Emergence

• Learning language via interaction with the environment• Human-beings ： no grammar, no vocabulary given• ref. Developmental Linguistics

– Tomasello, Meltzoff, Spelke, …– Chomskians （ the merge theory ）– cf. Evolutional Linguistics (animal cognitive functions)

• The Symbol Grounding Problem: mapping symbols to things in the world

• Machine learning methods– Non-parametiric bayes, Recursive Neural Net…

• Getting insights from developmental linguistics• Yet to succeed in language acquistion

2015-07 17

Robotics for Social Intelligence

● Communicatin study with robots

● Communication requiring the body

● Mimetics

● Joint attention

● Empathy

2015-07 18

Cognitive Robotics & Embodiment

• The interests of cognitive robotics researchers〜 the interests of embodiment researchers

• Common terms– Body Image & Body Scheme, etc.

2015-07 19

Artificial Brains

● Reproducing human cognitive functions by creating something similar to the brain

● Brain Simulation○ Physiological models

○ Blue Brain Project, Neurogrid Project, etc.

● Brain-Inspired Cognitive Architectures○ Examples

■Nengo/SPAUN (C. Eliasmith et al.)

■Leabra (O’Reilly et al.)

■The Whole Brain Architecture (to be mentioned later)

2015-07 20

脳研究の現状

● Advance in functional brain imaging (e.g., fMRI)● Cognitive Neuro-Scientists

○ A. Damasio ： Somatic Marker Hypothesis （ role of emotion ）

○ V.S. Ramachandran ： presenting cognitive disorders○ E. Kandel ： memory research

○ E. Goldberg ： cerebral hemispheres & prefrontal cortex

● Modeling cerebral organs○ Cerebral cortex & areas （ perception, motion, planning, …)

the uniform structure of cortex [Mountcastle]

○ Basal ganglia (striatum, etc.: reinforcement learning, WM…)

○ Limbic System (amygdala, etc.: emotion, reward,...)

○ Hypocampus (memory, space representation)

○ Cerebellum (motion control, higher-order cognitive functions)

⇒ To draw an integrated picture soon?

2015-07 21

The Brain and Cognitive Functions （ Figure ）

Prefrontal Cortex: Planning

Motor Area ： Motion Sequences

Basal Ganglia ：Reinforcement Learning

Cerebellum ： Feed-forward prediction ？

Hypocampus ： Episodic Memory（ Place Memory in Rodents ）

Where Path

What Path

Amigdalae, etc. ：Emotion

Language Areas

To think of an ‘architecture’ constituting of such functional modules to realize human-level intelligence

2015-07 22

Table of Contents

1.Background for Human-LevelAI

2. Issues to be solved

3. How to create Human-Level AI

4. My recent activities● Issue of Semantics● Overall Objectives● Phenomenology of Artefacts （ Manifesto ）● Phenomenology of Time● Language Acquistion by Artifacts● AGI related activities

2015-07 23

Semantic Issue ： doubts from my pre-history

• Creating an ontology for natural language• The problem of polysemy (ambiguity)

– How many senses?E.g., prepositions

– Border-line uses...

• How do humans acquire word senses?

• Keys in human developmental process

• Counsel by Lakoff, the Cognitive Linguists

Women, Fire, and Dangerous Things

It is impossible to deal with meaning with symbolic logic!

⇒ Radical readdressing is required!

2015-07 24

Overall Goal ： Explaining Cognition

● More precisely ： Grounding Semantics● But semantics requires epistemology.

○ No sense made without knowing the world.● By-product ： AGI/Human-Leval AI

○ But the by-product is the mean in the constructive method.

⇒ Methodological Loop

2015-07 25

Approach

● Learning from animals○ Modeling brains, comparative psychology, etc.

● Phenomenological & Developmental○ Knowledge acquisition from information given to

individuals ● Constructive (make & test)

○ Machine Learning

○ Robotics （ simulation ）● Language Acquistion

○ Language ： an essential component of cognition

○ Explanation with 1 〜 3 above

2015-07 26

Phenomenology of Artefact (2014-02)

• Husserlean phenomenology 〜 Grounding Epistemology• Epistemology from the first person view• Robots has the first person view

Video ： MIT Atlas robot - first person view sensor visualization ⇔

• Robots with kinesthetics • Developmental knowledge acquistion• Information processing with robots

– inspectable– methematically verifiable

• Time consciousness with machine learning? ⇒ Reconstructing phenomenology with artifacts (robots)?

2015-07 27

Phenomenology of Time

● Time Consciousness by Husserl: Urimpression, Protention, Retention

● Time-series Learning 〜 Time-series Prediction○ RNN (recurrent neural network)

○ Temporal Cerebral Models ： HTM, DeSTIN, etc. （ cf. akinestopsia @V5 ）○ PSI model by Dörner (cognitive psychologist)

Bach J.: Principles of Synthetic Intelligence -- PSI: An Architecture of Motivated Cognition , Oxford U.○ LLoyd, M.: “Time after Time -- Temporality in the dynamic brain,” Being Time: Dynamical Models of Phonomenal

Experience, John Benjamins Pub. Co. (2012)

● Time-series Learning & Phenomenology of Time○ Protention ： memory of the future (prediction)

○ Retention ： memory of the context (the internal state from the past input)

○ Urimpression⇔ contextualized (differential) present

● cf. Jun Tani, the roboticist○ RNN

○ Ref. to Husserlean phenomenology of time: longitudinal/transverse intentionality

2015-07 28

Towards Language Acquistion by Artifacts

• Developmental Robotics in the virtual world

• Learning from Infants’ language acquistion•Spelke

•Concepts of things: certain constraints–cf. Quine: “Gavagai”

–Seeing thing as a whole

cf. Husserl: looking around objects 3D object concept⇒

•Tomasello• Understanding reference by others requires understanding intention.

•Usage-based grammar learning (anti-generative grammar)

•Meltzoff

•Infants’ understanding of the intention of others

•Modeling own intentional motions first?

2015-07 29

Towards Language Acquistion by Artifacts (cont.)

• Acquistion of Verbs•Verbs are the crux of sentence structure•Acquired after object/nominal concepts•Modeling own intentional motions first (←Meltzoff)?

cf. sense of agency

Own intention is ‘given’

•Mapping to verbs• ‘Parental’ verb uses

•Pragmatic success/failure of own utterances

• Acquistion of syntax• Concatenating subsequent structures Merge?⇒

• Language acquistion with machine learning

2015-07 30

AGI-related Activities （ ads :- ）

❖Dwango AI Lab.● Brain/Cognitive Modeling, Language Acquistion, etc.

❖The Whole Brain Architecture Initiative (NPO)● Brain-inspired cognitive architecture

● Education, promotion

❖SIG AGI （ @ Japanese AI Society ）● a reading group

● planning to publish a textbook (in Japanese)…

❖Web site in Japanese●www.sig-agi.org

●Facebook Group

For more information, contact naoya.arakawa@nifty.com