global brain simulations włodzisław duch (google: duch) department of informatics, nicholaus...

28
Global Brain Global Brain Simulations Simulations Włodzisław Duch (Google: Duch) Włodzisław Duch (Google: Duch) Department of Informatics, Department of Informatics, Nicholaus Copernicus Nicholaus Copernicus University, University, Torun, Poland Torun, Poland School of Computer Engineering, School of Computer Engineering, Nanyang Technological Nanyang Technological University (NTU), University (NTU), Singapore Singapore , Singapore , Singapore

Post on 21-Dec-2015

218 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Global Brain SimulationsGlobal Brain Simulations

Włodzisław Duch (Google: Duch)Włodzisław Duch (Google: Duch)

Department of Informatics, Department of Informatics, Nicholaus Copernicus University, Nicholaus Copernicus University,

Torun, PolandTorun, Poland

School of Computer Engineering, School of Computer Engineering, Nanyang Technological University (NTU), Nanyang Technological University (NTU),

SingaporeSingapore, Singapore , Singapore

Page 2: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Attention-Based Artificial Cognitive Control Attention-Based Artificial Cognitive Control Understanding System (ABACCUS)Understanding System (ABACCUS)

Large EU integrated project (>150 pp), with 9 participants: Large EU integrated project (>150 pp), with 9 participants:

• King’s College London (John G. Taylor, coordinator), UK King’s College London (John G. Taylor, coordinator), UK • Centre for Brain & Cognitive Development, Berkbeck College, Centre for Brain & Cognitive Development, Berkbeck College,

University of London, UKUniversity of London, UK• Cognition and Brain Sciences Unit, Medical Research Council, UKCognition and Brain Sciences Unit, Medical Research Council, UK

• Robotics and Embedded Systems, Technical University of Munich, Robotics and Embedded Systems, Technical University of Munich, GG

• Institute of Neurophysiology and Pathophysiology, Institute of Neurophysiology and Pathophysiology, Universitätsklinikum Hamburg-Eppendorf, GUniversitätsklinikum Hamburg-Eppendorf, G

• Institute of Computer Science, Foundation for Research and Institute of Computer Science, Foundation for Research and Technology – Hellas, Heraklion, Crete, GRTechnology – Hellas, Heraklion, Crete, GR

• National Center for Scientific Research “Demokritos”, Athens, GRNational Center for Scientific Research “Demokritos”, Athens, GR

• Dipartimento di Informatica, Sistemistica, Telematica, Universita di Dipartimento di Informatica, Sistemistica, Telematica, Universita di Genova, IGenova, I

• Dep. of Informatics, Nicholaus Copernicus University, Torun, PLDep. of Informatics, Nicholaus Copernicus University, Torun, PL

Page 3: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

MotivationMotivation

AI and CI has not been able to create decent human-computer AI and CI has not been able to create decent human-computer interfaces, solve problems in computer vision, natural language interfaces, solve problems in computer vision, natural language understanding, cognitive search and data mining, or even reasoning in understanding, cognitive search and data mining, or even reasoning in theorem proving. theorem proving.

Practical: humanized, cognitive computer applications require a brain-Practical: humanized, cognitive computer applications require a brain-like architecture (either software or hardware) to deal with such like architecture (either software or hardware) to deal with such problems efficiently; it is at the center of cognitive robotics. problems efficiently; it is at the center of cognitive robotics.

Science: understand how high-level cognition arises from low-level Science: understand how high-level cognition arises from low-level interactions between neurons, build powerful research tool; to interactions between neurons, build powerful research tool; to understand complex systems is to be able to build them.understand complex systems is to be able to build them.

Computer speeds have just reached brain power (about 10Computer speeds have just reached brain power (about 101616 binop/s), binop/s), but computers are far from brain’s complexity and processing style. but computers are far from brain’s complexity and processing style.

Page 4: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Scheme of the brain ...Scheme of the brain ...

High-level sketch of the brain structures, with connections High-level sketch of the brain structures, with connections based on different types of neurotransmiters marked in based on different types of neurotransmiters marked in different colors.different colors.

Page 5: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

ABACCUS Goals• Assumption: gross neuroanatomical brain structure is critical for its Assumption: gross neuroanatomical brain structure is critical for its

function, therefore it should be preserved. function, therefore it should be preserved. • To demonstrate how fusion of the appropriate brain-based models, To demonstrate how fusion of the appropriate brain-based models,

guided by the overall architecture of the brain, and by its guided by the overall architecture of the brain, and by its developmental learning stages, can help attain high-level cognitive developmental learning stages, can help attain high-level cognitive processing capabilities. processing capabilities.

• Show basic language understanding and reasoning abilities for direct Show basic language understanding and reasoning abilities for direct human-machine communication, at the level of a pre-school child, human-machine communication, at the level of a pre-school child, mimicking solutions used by the human brain. mimicking solutions used by the human brain.

• Develop an attention control systems for focusing in sensory Develop an attention control systems for focusing in sensory surveillance tasks, and for image searchingsurveillance tasks, and for image searching..

• Development of control structures for autonomous machinesDevelopment of control structures for autonomous machines.. • Create its own goals in an autonomous fashionCreate its own goals in an autonomous fashion.. • Founded on neuro-scientific understanding of attention and the Founded on neuro-scientific understanding of attention and the

sensory and motor systems it controls, development in childrensensory and motor systems it controls, development in children,, simplified modeling, computer power. simplified modeling, computer power.

Page 6: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Special hardware?• Many have proposed the construction of brain-like computers, Many have proposed the construction of brain-like computers,

frequently using special hardware. frequently using special hardware.

• Connection Machines from Thinking Machines, Inc. (D. Hills, 1987) Connection Machines from Thinking Machines, Inc. (D. Hills, 1987) was almost successful, but never become massively parallel.was almost successful, but never become massively parallel.

• CAM Brain (ATR Kyoto) – failed attempt to evolve the large-scale CAM Brain (ATR Kyoto) – failed attempt to evolve the large-scale cellular neural network; to evolve one must know the function.cellular neural network; to evolve one must know the function.

• Needed: elements based on spiking biological neurons and the Needed: elements based on spiking biological neurons and the layered 2-D anatomy of mammalian cerebral cortex.layered 2-D anatomy of mammalian cerebral cortex.

• ALAVLSI (EU Consortium), a general architecture for perceptual ALAVLSI (EU Consortium), a general architecture for perceptual attention and learning based on neuromorphic VLSI technology. attention and learning based on neuromorphic VLSI technology. Coherent motion + speech categorization, project ends in 2005. Coherent motion + speech categorization, project ends in 2005.

Page 7: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Other attempts?Artificial Development (www.ad.com) is building CCortex™, Artificial Development (www.ad.com) is building CCortex™, a complete 20-billion neuron simulation of the Human a complete 20-billion neuron simulation of the Human Cortex and peripheral systems, on a cluster of 500 Cortex and peripheral systems, on a cluster of 500 computers - the largest neural network created to date.computers - the largest neural network created to date.

Artificial Development plans to deliver a wide range of commercial Artificial Development plans to deliver a wide range of commercial products based on artificial versions of the human brain that will products based on artificial versions of the human brain that will enhance business relationships globally. Rather unlikely? enhance business relationships globally. Rather unlikely?

The Ersatz Brain ProjectThe Ersatz Brain Project – James Anderson (Brown – James Anderson (Brown

University), based on modeling of intermediate level University), based on modeling of intermediate level

cerebral cortex structures - cortical columns of various cerebral cortex structures - cortical columns of various

sizes (mini ~10sizes (mini ~1022, plain ~10, plain ~1044, and hypercolumns ~10, and hypercolumns ~1055). ).

NofN, Network of Networks approximation, 2D BSB network.NofN, Network of Networks approximation, 2D BSB network.

Page 8: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

NomadsG. Edelman G. Edelman (Neurosciences Institute) & collaborators, created a series (Neurosciences Institute) & collaborators, created a series of Darwin automata, brain-based devices, “physical devices whose of Darwin automata, brain-based devices, “physical devices whose behavior is controlled by a simulated nervous system”.behavior is controlled by a simulated nervous system”.

(i)(i) The device must engage in a behavioral task. The device must engage in a behavioral task.

(ii)(ii) The device’s behavior must be controlled by a simulated The device’s behavior must be controlled by a simulated nervous system having a design that reflects the brain’s nervous system having a design that reflects the brain’s architecture and dynamics. architecture and dynamics.

(iii)(iii) The device’s behavior is modified by a reward or value system that The device’s behavior is modified by a reward or value system that signals the salience of environmental cues to its nervous system.signals the salience of environmental cues to its nervous system.

(iv)(iv) The device must be situated in the real world.The device must be situated in the real world.

Darwin VII consists of: a mobile base equipped with a CCD camera Darwin VII consists of: a mobile base equipped with a CCD camera and IR sensor for vision, microphones for hearing, conductivity sensors and IR sensor for vision, microphones for hearing, conductivity sensors for taste, and effectors for movement of its base, of its head, and of a for taste, and effectors for movement of its base, of its head, and of a gripping manipulator having one degree-of-freedom; 53K mean firing gripping manipulator having one degree-of-freedom; 53K mean firing +phase neurons, 1.7 M synapses, 28 brain areas.+phase neurons, 1.7 M synapses, 28 brain areas.

Page 9: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Sketch of the ABACCUS system

Rough sketch of the ABACCUS system, based on simplified Rough sketch of the ABACCUS system, based on simplified spiking neurons. spiking neurons.

Computational Platform, Simulation Environment and Integration

Neuroscience and Development

Vision

Memory System

Drive and Intrinsic reward system

Atomization system

Reasoning System

Feedback Attention

Control system

Motor Control

Speech TactileLearning of PFC goals

Working Memory

Value Maps Action/Object reward system

Page 10: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Natural perception

Spectrogram of speech: hearing a sentence.Spectrogram of speech: hearing a sentence.

Page 11: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Spiking vs. mean fieldBrain: 10Brain: 101111 Neurons Neurons

Neuron PoolsNeuron Pools

tM

tttntA spikes

t

),(lim)(

0Pool Activity:

Pool

M Neurons

neuron

123

M

spikes t

t

Linked Pools Linked Pools

A

C

B

...)(

extBB IAaFIIt

Mean-FieldModel:

F

neuron 1neuron 1neuron 2neuron 2

Networks of Spiking Networks of Spiking NeuronsNeurons

)())(()( tIVtVgtVdt

dsynLimim

)(2 tV

t

Integrate andFire Model:

Page 12: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

j

recGABAjijEirecGABArecGABA

j

recNMDAjij

i

EirecNMDArecNMDA

j

recAMPAjijEirecAMPArecAMPA

j

extAMPAjijEiextAMPAextAMPA

tswVtVgtI

tswtVMg

VtVgtI

tswVtVgtI

tswVtVgtI

)())(()(

)())57.3/)(062.0exp(][1(

))(()(

)())(()(

)())(()(

,,,

,2

,,

,,,

,,,

k

kj

GABA

GABAjGABA

j

k

kj

riseNMDA

NMDAjNMDA

j

NMDAjj

decayNMDA

NMDAjNMDA

j

k

kj

AMPA

AMPAjAMPA

j

ttts

tsdt

d

tttx

txdt

d

tstxts

tsdt

d

ttts

tsdt

d

)()(

)(

)()(

)(

))(1)(()(

)(

)()(

)(

,

,

Synaptic Dynamics

EPSP, IPSP)(tI syn

Spike

Spike

SomaSynapses

synCmC mR

synR

Page 13: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Primary objective 1

To develop linguistic powers of ABACCUS system. To develop linguistic powers of ABACCUS system.

Use training of single words by associating their representations to Use training of single words by associating their representations to internal representations of objects and actions. internal representations of objects and actions.

Use pair-wise associations to learn word pairs (like ‘kick ball’), Use pair-wise associations to learn word pairs (like ‘kick ball’), extend syntactically and functionally the use of function words.extend syntactically and functionally the use of function words.

Working memory modules, with associated phonological coding, Working memory modules, with associated phonological coding, will be created and fused in the language component, both for will be created and fused in the language component, both for speech understanding and generation. speech understanding and generation.

Extension for abstract concepts by tagging the associated words to Extension for abstract concepts by tagging the associated words to clusters of concrete object/action representations.clusters of concrete object/action representations.

Page 14: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Primary objective 2

To create a high-level cognitive system, able to solve problems To create a high-level cognitive system, able to solve problems requiring reasoning, thinking, imagination and creativity. requiring reasoning, thinking, imagination and creativity.

Based on the basic control concept of a forward model, acting as Based on the basic control concept of a forward model, acting as a predictor and working under attention control. a predictor and working under attention control.

Forward models include various semantic and goal networks.Forward models include various semantic and goal networks.

Sequences of activations of representations will be learnt and Sequences of activations of representations will be learnt and thereby used to achieve goals. thereby used to achieve goals.

Various of these forward models will be present and branch into Various of these forward models will be present and branch into each other during the running process by means of lateral each other during the running process by means of lateral connections. connections.

New routes will occur allowing complex goals to be achieved, or New routes will occur allowing complex goals to be achieved, or even new, previously unrecognized goals, to be arrived at even new, previously unrecognized goals, to be arrived at (required for creativity).(required for creativity).

Page 15: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Example of action internal models

Rough sketch of the ACTION Rough sketch of the ACTION subnetwork. subnetwork.

Sensory information enters the Sensory information enters the supplementary motor area (SMA) supplementary motor area (SMA)

(1) cortico-thalamo-cortical (the (1) cortico-thalamo-cortical (the ‘short’ loop);‘short’ loop);

(2) cortico-striatal-GPi-thalamo-(2) cortico-striatal-GPi-thalamo-cortical (the ‘long’ loop);cortical (the ‘long’ loop);

(3) cortico-(STN(3) cortico-(STNGPe)-GPi-GPe)-GPi-thalamo-cortical (‘first indirect’ loop);thalamo-cortical (‘first indirect’ loop);

(4) cortico-striatal-(STN(4) cortico-striatal-(STNGPe)-GPi-GPe)-GPi-thalamo-cortical (the ‘second thalamo-cortical (the ‘second indirect’ loop).indirect’ loop).

Page 16: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Primary objective 3 Extract a simplified architecture for the attention control system. Extract a simplified architecture for the attention control system.

Should involve both sensory and motor control, especially joint Should involve both sensory and motor control, especially joint sensory-motor control, systems whose creation is guided by sensory-motor control, systems whose creation is guided by neuroscientific knowledge about the brain.neuroscientific knowledge about the brain.

Approach based on infant development to train ABACCUS Approach based on infant development to train ABACCUS incrementally on the computer platform. incrementally on the computer platform.

Use neuroscientific data to guide the architecture of a large-scale Use neuroscientific data to guide the architecture of a large-scale neural simulation of the relevant components of the human brain. neural simulation of the relevant components of the human brain.

Use feed-forward/feedback training simulations based on the Use feed-forward/feedback training simulations based on the simplified brain architecture following developmental processes, simplified brain architecture following developmental processes, using the sensor and response signals of the robotic embodiment. using the sensor and response signals of the robotic embodiment.

Page 17: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Primary objective 4

Developing the ability to learn novel objects, both as stimuli and Developing the ability to learn novel objects, both as stimuli and as associated reward values. as associated reward values.

Use reward/penalty feedback training with attention control, with Use reward/penalty feedback training with attention control, with associated value maps constructed to learn to encode the values associated value maps constructed to learn to encode the values of stimuli and responses to them in the environment. of stimuli and responses to them in the environment.

Temporal sequence or schemata (as object/action sequences as Temporal sequence or schemata (as object/action sequences as well as the associated rewards involved) will be constructed to well as the associated rewards involved) will be constructed to function as predictors and so support reasoning, along with function as predictors and so support reasoning, along with automatic response learning.automatic response learning.

Neural codes of visual, auditory and tactile concepts will be learnt Neural codes of visual, auditory and tactile concepts will be learnt in a feedforward manner, without attention, as will codes for in a feedforward manner, without attention, as will codes for motor responses controlling the embodied robot.motor responses controlling the embodied robot.

Page 18: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Primary objective 5Develop a robotic embodiment; this involves: Develop a robotic embodiment; this involves:

Specification of robots available for ABACCUS control.Specification of robots available for ABACCUS control.

Development of sensor and response systems in the robot.Development of sensor and response systems in the robot. Sensory data mappings to the central computational platform.Sensory data mappings to the central computational platform.

An environmental task domain in which ABACCUS can develop its An environmental task domain in which ABACCUS can develop its concepts and cognitive powers. concepts and cognitive powers.

Simulation platform for the developmental training (on a Beowulf Simulation platform for the developmental training (on a Beowulf cluster or similar powerful computer system).cluster or similar powerful computer system).

Specifications for integration of various models on the central Specifications for integration of various models on the central computational platform.computational platform.

Compatibility of computational languages used by partners.Compatibility of computational languages used by partners.

Page 19: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Attention Control Attention control systems will be achieved by use of the early flow of Attention control systems will be achieved by use of the early flow of

activity from low-level cortex rapidly to prefrontal sites. activity from low-level cortex rapidly to prefrontal sites.

This initially uncoded activity will be used to bias feedback activity This initially uncoded activity will be used to bias feedback activity sent to parietal and/or temporal lobe, so as to bias the attention sent to parietal and/or temporal lobe, so as to bias the attention feedback from there down to the activated inputs encoded in the feedback from there down to the activated inputs encoded in the object representations in the temporal lobe. object representations in the temporal lobe.

This feedback loop will be trained by causal Hebbian learning, with This feedback loop will be trained by causal Hebbian learning, with the attention feedback represented by contrast gain amplification, so the attention feedback represented by contrast gain amplification, so by sigma-pi network quadratic nodes, onto neurons in hierarchically by sigma-pi network quadratic nodes, onto neurons in hierarchically lower modules; the goal representations will be learnt as part of this lower modules; the goal representations will be learnt as part of this processing. processing.

The manner in which a dual-control stereo camera can be used The manner in which a dual-control stereo camera can be used most effectively will be explored by modeling the manner in which most effectively will be explored by modeling the manner in which eye and attention control are separable.eye and attention control are separable.

Page 20: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Drives, Value Maps & Emotions Drives and reflexes will be enumerated and modeled as separate Drives and reflexes will be enumerated and modeled as separate

systems to be used as a launching pad for more controlled systems to be used as a launching pad for more controlled developmental sensory representations and responses. developmental sensory representations and responses.

The rewards system will also be developed as a separate The rewards system will also be developed as a separate component, based on the learning of reward/error prediction. component, based on the learning of reward/error prediction.

ABACCUS will be trained by reward error to develop value maps ABACCUS will be trained by reward error to develop value maps for objects and responses as goals, to bias learning of feedback for objects and responses as goals, to bias learning of feedback attention to filter out responses in complex environments. The attention to filter out responses in complex environments. The effect of the reward signal will be checked to determine if there is effect of the reward signal will be checked to determine if there is any modification of the earlier goal representations learnt under any modification of the earlier goal representations learnt under attention control. attention control.

Value maps associated to rewards attached to sequences of stimuli Value maps associated to rewards attached to sequences of stimuli (schemata) will give emotional bias in the cognitive decision-making (schemata) will give emotional bias in the cognitive decision-making it can perform. it can perform.

Page 21: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Short-term working memory

Two sorts of working memory are presently known: buffer or Two sorts of working memory are presently known: buffer or slave working memories, of a fixed and limited persistence, and slave working memories, of a fixed and limited persistence, and longer lasting prefrontal activities of duration determined by the longer lasting prefrontal activities of duration determined by the importance and difficulty of the goal they represent. importance and difficulty of the goal they represent.

Buffer memory is fed by the semantic representations, whose Buffer memory is fed by the semantic representations, whose activity they extend, and thereby allow the development of activity they extend, and thereby allow the development of attended state estimators in the various modalities for more attended state estimators in the various modalities for more general use across other cortical regions, especially goal sites. general use across other cortical regions, especially goal sites.

Buffer sites will be modeled by suitable recurrent circuits coupled Buffer sites will be modeled by suitable recurrent circuits coupled to the semantic maps from which they receive input. to the semantic maps from which they receive input.

Rehearsal of the working memory will be extended to various Rehearsal of the working memory will be extended to various transformations that are also known to occur in goal sites, such transformations that are also known to occur in goal sites, such as deletion, comparison and related functions. as deletion, comparison and related functions.

Page 22: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Concept Understanding and Abstraction

High–level concepts will be created by use of a hierarchy of High–level concepts will be created by use of a hierarchy of networks, each being more specific in its features to which it is networks, each being more specific in its features to which it is responsive than the next lower one. responsive than the next lower one.

The understanding based on such ontology allows to ‘understand’ The understanding based on such ontology allows to ‘understand’ various levels of concepts, by the associated super- or sub-various levels of concepts, by the associated super- or sub-ordinate concept levels activated by trained connectivity at the ordinate concept levels activated by trained connectivity at the levels of specificity allowed by the number of hierarchical layers.levels of specificity allowed by the number of hierarchical layers.

Abstraction will arise through creation of specific connections of a Abstraction will arise through creation of specific connections of a new word representation, describing the concept, to its various new word representation, describing the concept, to its various more concrete word and object/action representatives. more concrete word and object/action representatives.

Word representations will be linked to emotion state descriptors, Word representations will be linked to emotion state descriptors, that are connected to the associated reward value map that are connected to the associated reward value map representations of the associated system state. For example the representations of the associated system state. For example the word ‘anger’ would be associated with the underlying state arising word ‘anger’ would be associated with the underlying state arising from blockage, by the environment, of the gaining of a goal. from blockage, by the environment, of the gaining of a goal.

Page 23: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Basic tasks

2D domain, enclosed within four walls with obstacles on the floor 2D domain, enclosed within four walls with obstacles on the floor and 2D objects on the walls, of varying shapes (square, triangle, and 2D objects on the walls, of varying shapes (square, triangle, circle) and colors (red, green, blue), emitting various tones. circle) and colors (red, green, blue), emitting various tones. Objects are potential goals, to be learnt and responded to. Objects are potential goals, to be learnt and responded to.

Perceptions/Concepts: To develop internal representations of the Perceptions/Concepts: To develop internal representations of the 2D objects, based on shape/color feature maps created by 2D objects, based on shape/color feature maps created by learning, after relevant clustering on the feature map activations, learning, after relevant clustering on the feature map activations, to learn a representation for the objects.to learn a representation for the objects.

Actions/Goals: learn to touch/manipulate the 2D objects based on Actions/Goals: learn to touch/manipulate the 2D objects based on stimulus-action rewards and fused representation as a goal; to stimulus-action rewards and fused representation as a goal; to attentionally bias lower order perceptual representations just attentionally bias lower order perceptual representations just learnt (using dopamine-based reward learning).learnt (using dopamine-based reward learning).

Page 24: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Intermediate tasks 2D domain with walls, with 3D objects as potential goals; objects 2D domain with walls, with 3D objects as potential goals; objects

with simple shapes (sphere, box, ring), colors, emitting various with simple shapes (sphere, box, ring), colors, emitting various sounds, some in motion (to help develop tracking). sounds, some in motion (to help develop tracking).

Perception/Concepts/Goals: To develop internal representations of Perception/Concepts/Goals: To develop internal representations of the 3D objects, based on a set of their 2D views, clustered over the 3D objects, based on a set of their 2D views, clustered over time (by use of binocular representations of depth). Certain time (by use of binocular representations of depth). Certain examples of moving stimuli will be created so as to be learnt.examples of moving stimuli will be created so as to be learnt.

Actions/rewards: Motor responses to move specific objects to Actions/rewards: Motor responses to move specific objects to certain places, such as to stack an object on top of another, or certain places, such as to stack an object on top of another, or place one object inside another (object fusion observed developing place one object inside another (object fusion observed developing in both chimps and human infants). in both chimps and human infants).

Language: Word maps for actions and objects concepts will be Language: Word maps for actions and objects concepts will be created, using associations of the word maps to the relevant object created, using associations of the word maps to the relevant object maps; attention will be important in this learning process. maps; attention will be important in this learning process.

Page 25: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Advanced task domain Perception/Concepts/Goals: complex 3D objects present with Perception/Concepts/Goals: complex 3D objects present with

various features in various modalities, including animal shapes various features in various modalities, including animal shapes emitting sounds and smells. The internal concept representations emitting sounds and smells. The internal concept representations will be extended to hierarchies of concepts, so leading to will be extended to hierarchies of concepts, so leading to abstraction processes (where more abstract, basic level concepts abstraction processes (where more abstract, basic level concepts are activated, such as that for ‘animal’ by specific animal inputs, so are activated, such as that for ‘animal’ by specific animal inputs, so leading to spread of activity to related concepts and to associated leading to spread of activity to related concepts and to associated reasoning potentialities).reasoning potentialities).

Actions: with added complex tasks requiring reasoning to achieve Actions: with added complex tasks requiring reasoning to achieve fast/accurate responses: manipulating objects, putting objects into fast/accurate responses: manipulating objects, putting objects into slots/matching shapes, Tower of Hanoi (involving virtual slots/matching shapes, Tower of Hanoi (involving virtual movements of objects – translations and stacking – at a fixed set of movements of objects – translations and stacking – at a fixed set of different sites), Incident Detection, etc. AI-based approaches to different sites), Incident Detection, etc. AI-based approaches to such tasks exist (cf. SOAR), but ABACCUS will be based on the such tasks exist (cf. SOAR), but ABACCUS will be based on the flexible posterior representation transformation, goal creating and flexible posterior representation transformation, goal creating and resolving powers of a neural system.resolving powers of a neural system.

Page 26: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

More on advanced task domainLanguage powers will be expanded so that ABACCUS can be talked to, Language powers will be expanded so that ABACCUS can be talked to, using limited length word strings (initially of length two or three). These using limited length word strings (initially of length two or three). These strings will be understood in terms of the associated object/action maps strings will be understood in terms of the associated object/action maps and responses made by the system, either as actions or action and responses made by the system, either as actions or action sequences or suitably meaningful speech responses. Reasoning and sequences or suitably meaningful speech responses. Reasoning and creation of new types of responses will be done by use of lateral creation of new types of responses will be done by use of lateral spreading in memory to achieve new trajectories in the schemata space.spreading in memory to achieve new trajectories in the schemata space.

The ‘Incident Detection & Tracking Scenario’ involves a robot moving in The ‘Incident Detection & Tracking Scenario’ involves a robot moving in the internal environment, although with novel stimuli and obstacle layout. the internal environment, although with novel stimuli and obstacle layout. This includes ramps and stationary or slow moving obstacles. Candidate This includes ramps and stationary or slow moving obstacles. Candidate events of interest in this paradigm are:events of interest in this paradigm are:

a)a) Moving entity detection and classification (humans, animals).Moving entity detection and classification (humans, animals).

b)b) Static environment variation detection (unexpected objects).Static environment variation detection (unexpected objects).

c)c) Abnormal auditory events.Abnormal auditory events.

d)d) Abnormal visual events.Abnormal visual events.

Page 27: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Work packages

Summary of ABACCUS workpackages.Summary of ABACCUS workpackages.

Page 28: Global Brain Simulations Włodzisław Duch (Google: Duch) Department of Informatics, Nicholaus Copernicus University, Torun, Poland School of Computer Engineering,

Summary

ABACCUS is the most ambitious project formulated so far, ABACCUS is the most ambitious project formulated so far, an attempt to simulate functions of most brain structures.an attempt to simulate functions of most brain structures.

High order functions (including conscious-like behavior?) High order functions (including conscious-like behavior?) should result from elementary interactions in a system with should result from elementary interactions in a system with proper brain-like architecture. proper brain-like architecture.

Neuroscience and developmental science integrated in one Neuroscience and developmental science integrated in one computational model, useful for behavioral experiments and computational model, useful for behavioral experiments and computational psychiatry. computational psychiatry.

Wish us good luck ... Wish us good luck ...