Slide 1

Introduction to Developmental Learning 11 March 2014 Olivier.georgeon@liris.cnrs.fr http://www.oliviergeorgeon.com t 1/33oliviergeorgeon.com

Slide 2

Old dream of AI Instead of trying to produce a program to simulate the adult mind, why not rather try to produce one which simulates the child's? If this were then subjected to an appropriate course of education one would obtain the adult brain. Presumably, the child brain is something like a notebook []. Rather little mechanism, and lots of blank sheets. []. Our hope is that there is so little mechanism in the child brain that something like it can be easily programmed. The amount of work in the education we can assume, as a first approximation, to be much the same as for the human child. Computing machinery and intelligence (Alan Turing, 1950, Mind, philosophy journal). 2/33oliviergeorgeon.com

Slide 3

Is it even possible? Spiritualist vision of consciousness (it would require a soul). Causal openness of physical reality (quantum theory). Too complex. Materialist theory of consciousness (Julien Offray de La Mettrie, 1709-1751). Consciousness as a computational process (Chalmers 1994) http://consc.net/papers/computation.htmlhttp://consc.net/papers/computation.html No ? Yes? 3/33oliviergeorgeon.com

Slide 4

Outiline Example Demo of developmental learning. Theoretical bases Pose the problem. The question of self-programming. Exercise Implement your self-programming agent. 4/33oliviergeorgeon.com

Slide 5

6 Experiments The coupling agent/environment offers hierarchical sequential regularities of interactions, for example : After i 7, attempting i 1 or i 2 results more likely in i 1 than in i 2. After i 9, i 3, i 1, i 8 , i 4, i 7, i 1 can often be enacted. After i 8, sequence i 9, i 3, i 1 can often be enacted. After i 8, i 8 can often be enacted again. i 1 (5) i 2 (-10) i 3 (-3) i 7 (-1) i 8 (-1) i 5 (-1) i 6 (-1) i 9 (-1) i 10 (-1) i 4 (-3) 2 Results10 Interactions (value) 0101 0 0 1010 0101 0101 Example 1 5/28

Slide 6

Exemple 1: Bump: Touch: Move Forward or bump(5) (-10) Turn left / right (-3) Feel right/ front / left (-1) 6/28

Slide 7

Theoretical bases Philosophy of mind. Epistemology (theory of knowledge) Developmental psychology. Biology (autopoiesis, enaction). Neurosciences. 7/33oliviergeorgeon.com

Slide 8

Philosophy : is it possible? John Locke (1632 1704) Tabula Rasa La Mettrie (1709-1751). Matter can think David Chalmers A Computational Foundation for the Study of Cognition (1994) Daniel Dennett Consciousness explained (1991) Free will, individual choice, self-motivation, dterminism 8/33oliviergeorgeon.com

Slide 9

Key philosophical ideas for DIA cognition ascomputation in the broad sense. Causal structure Example: neural net with chemistry (neurotransmitters, hormones etc.). Determinisme does not contradict free will. Do not mistake determinism for predictibility. Herv Zwirn (Les systmes complexes, 2006) 9/33oliviergeorgeon.com

Slide 10

Epistmology (what can I know?) Concept of ontology Study of the nature of being Aristotle (384 322 BC). Onto: being , Logos: discourse. Discourse on the properties and categories of being. Reality as such is unknowable Emmanuel Kant, (1724 1804) 10/33oliviergeorgeon.com

Slide 11

Key epistemological ideas for DAI Implement learning mechanism with no ontological asumptions. Agnostic agents (Georgeon 2012). The agent will never know its environment as we see it. But with interactional assumptions Predefine the possibilities of interaction between the agent and its environment. Let the agent alone to construct its own ontology of the environment through its experience of interaction. 11/33oliviergeorgeon.com

Slide 12

Developmental psychology (How can I know?) Developmental learning Jean Piaget (1896 1980) Teleology / motivational principles the individual self-finalizes recursively. Do not separate perception and action a priori: Notion of sensorimoteur scheme Contructivist epistemology Jean-Louis Le Moigne (1931 - ) Ernst von Glasersfeld. Knowledge is an adaptation in the functional sense. 12/33oliviergeorgeon.com

Slide 13

Etapes dveloppementales indicatives Month 4: Bayesian prediction. Month 5: Models of hand movement. Month 6: Objects and face recognition. Month 7: Persistency of objects. Month 8: Dynamic models of objects. Month 9: Tool use (bring a cup to the mouth). Month 10: Gesture imitation, crawling. Month 11: Walk with the help of an adult. Month 15: Walk alone. 13/45oliviergeorgeon.com

Slide 14

Key psychological ideas for DAI Think in terms of interactions rather than separating perception and action a priori. Focus on an intermediary level of intelligence: Cognition smantique (Manzotti & Chella 2012) stimulus-response adaptation Semantic cognition Reasoning and language Low level High level Intermediary level 14/33oliviergeorgeon.com

Slide 15

Biology (why know?) Autopoiese auto: self, poise : creation Maturana (1972) Structural coupling agent/environment. Relational domain (the space of possibilities of interaction) Homeostasis Internal state regulation Self-motivation Theory of enaction Self-creation through interaction with the environment. Enactive Artificial Intelligence (Froeze and Ziemke 2009). 15/33oliviergeorgeon.com

Slide 16

Key ideas from biology for DAI Constitutive autonomy is necessary for sense- making. Evolution of possibilities of interaction during the systems life. Individuation. Design systems capable of programming themselves. The data that is learned is not merely parameter values but is executable data. 16/33oliviergeorgeon.com

Slide 17

Neurosciences Many levels of analysis A lot of plasticity AND a lot of pre-wiring 17/33oliviergeorgeon.com

Slide 18

Neuroscience Connectome of C. Elegans: 302 neurons. Entirely inborn connectome rather than acquired through experience 18/33oliviergeorgeon.com

Slide 19

Human connectome http://www.humanconnectomeproject.org 19/33oliviergeorgeon.com

Slide 20

Neurosciences Examples of mammalian brains No qualitative rupture : human cognitive functions (e.g., language reasoning) relies of brain structures that exist in other mammalian brains. (This does not mean there is no innate differences !). The brain serves at organizing behaviors in time and space. 20/33oliviergeorgeon.com

Slide 21

Key neuroscience ideas for DAI Renounce the hope that it will be simple. Maybe begin at an intermediary level and go down if it does not work? Biology can be source of inspiration Biologically Inspired Cognitive Architectures. Importance of the capacity to internally simulate courses of behaviors. 21/33oliviergeorgeon.com

Slide 22

Key ideas of the key ideas The objective is to learn (discover, organze and exploit) regularities of interaction in time and space to satisfy innate criteria (survival, curiosity, etc.). Without pre-encoded ontological knowledge Which allows a kind of constitutive autonomy (self-programming). 22/33oliviergeorgeon.com

Slide 23

Teaser for next course 23/5oliviergeorgeon.com

Slide 24

Exercice 24/33oliviergeorgeon.com

Slide 25

Exercice Two possible experiences E = {e 1,e 2 } Two possible results R = {r 1,r 2 } Four possible interactions E x R = {i 11, i 12, i 21, i 22 } Two environments env 1 : e 1 -> r 1, e 2 -> r 2 (i 12 et i 21 are never enacted) env 2 : e 1 -> r 2, e 2 -> r 1 (i 11 et i 22 are never enacted) Motivational systems: mot 1 : v(i 11 ) = v(i 12 ) = 1, v(i 21 ) = v(i 22 ) = -1 mot 2 : v(i 11 ) = v(i 12 ) = -1, v(i 21 ) = v(i 22 ) = 1 mot 2 : v(i 11 ) = v(i 21 ) = 1, v(i 12 ) = v(i 22 ) = -1 Implement un agent that learn to enact positive interactions without knowing its motivatins a priori (mot 1 or mot 2 ) neither its environnement (env 1 or env 2 ). Write a rapport of behavioral analysis based on activity traces. 25/33oliviergeorgeon.com

Slide 26

No hard-coded knowledge of the environment Agen{ public Experience chooseExperience(){ If (env == env 1 and mot == mot 1 ) or (env == env 2 and mot == mot 2 ) return e 1 ; else return e 2 ; } 26/33oliviergeorgeon.com

Slide 27

Implementation public static Experience e1 = new experience(); Experience e2 = new experience(); public static Result r1 = new result(); Result r2 = new result(); public static Interaction i11 = new Interaction(e1,r1, 1); etc. Public static void main() Agent agent = new Agent(); Environnement env = new Env1(); // Env2(); for(int i=0 ; i < 10 ; i++) e = agent.chooseExperience(r); r = env.giveResult(e); System.out.println(e, r, value); Class Agent public Experience chooseExperience(Result r) Class Environnement public Result giveResult(experience e) Class Env1 Class Env2 Class Experience Class Result Class Interaction(experience, result, value) public int getValue() 27/33oliviergeorgeon.com