ee141 ee690 design of embodied intelligence janusz starzyk eecs, ohio university

EE141

EE690 EE690 Design of Design of Embodied IntelligenceEmbodied Intelligence

Janusz StarzykJanusz StarzykEECS, Ohio UniversityEECS, Ohio University

EE141

“…Perhaps the last frontier of science – its ultimate challenge- is to understand the biological basis of consciousness and the mental process by which we perceive, act, learn and remember..” from Principles of Neural Science by E. R. Kandel et al. E. R. Kandel won Nobel Price in 2000 for his work on physiological basis of memory storage in neurons. His family roots are in Kolomyje – so he once told “as with all bright people, my roots are in Poland”

“…The question of intelligence is the last great terrestrial frontier of science...” from Jeff Hawkins On Intelligence. Jeff Hawkins founded the Redwood Neuroscience Institute devoted to brain research. He co-founded Palm Computing and Handspring Inc.

IntelligenceIntelligence

AI’s holy grailFrom Pattie Maes MIT Media Lab

EE141

Traditional Artificial Intelligence Embodied Intelligence (EI) Challenges of EI

We need to know how We need means to implement it We need resources to build and sustain its

operation Promises of EI

To economy To society

OutlineOutline

EE141

IntelligenceIntelligence

From http://www.indiana.edu/~intell/map.shtml

Mainstream Science on Intelligence December 13, 1994: An Editorial With 52 Signatories, History, and Bibliography by Linda S. Gottfredson, University of Delaware

Intelligence is a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience.

EE141

Various Definitions of Intelligence Various Definitions of Intelligence The American Heritage Dictionary:

The capacity to acquire and apply knowledge. The faculty of thought and reason.

Webster Dictionary: The act or state of knowing; the exercise of the understanding. The capacity to know or understand; readiness of

comprehension; Wikipedia – The Free Encyclopedia:

The capacity to reason, plan, solve problems, think abstractly, comprehend ideas and language, and learn.

The classical behavioral/biologists: The ability to adapt to new conditions and to successfully cope

with life situations. Dr. C. George Boeree, professor in the Psychology Department at

Shippensburg University: A person's capacity to (1) acquire knowledge (i.e. learn and

understand), (2) apply knowledge (solve problems), and (3) engage in abstract reasoning.

Stanford University Professor of Computer Science Dr. John McCarthy, a pioneer in AI:

The computational part of the ability to achieve goals in the world.

EE141

Traditional AITraditional AI Embodied Intelligence Embodied Intelligence Abstract intelligence

attempt to simulate “highest” human faculties:

– language, discursive reason, mathematics, abstract problem solving

Environment model Condition for problem

solving in abstract way “brain in a vat”

Embodiment knowledge is implicit in the

fact that we have a body– embodiment is a

foundation for brain development

Intelligence develops through interaction with environment Situated in a specific

environment Environment is its best

model

EE141

Design principles of intelligent systemsDesign principles of intelligent systemsfrom Rolf Pfeifer “Understanding of Intelligence”, 1999

Interaction with complex environment

cheap design ecological balance redundancy principle asynchronous parallel, loosely

coupled processes sensory-motor

coordination value principle

Agent

Drawing by Ciarán O’Leary- Dublin Institute of Technology

EE141

Embodied Intelligence Embodied Intelligence

Definition Embodied Intelligence (EI) is a mechanism that learns

how to survive in a hostile environment

– Mechanism: biological, mechanical or virtual agent

with embodied sensors and actuators– EI acts on environment and perceives its actions– Environment hostility is persistent and stimulates EI to act– Hostility: direct aggression, pain, anxiety or scarcity of resources– EI learns so it must have associative self-organizing memory– Knowledge is acquired by EI

EE141

Embodiment of MindEmbodiment of Mind

Definition Embodiment of a mind

is a mechanism

under the control of the intelligence core

that contains

sensors and actuators connected to the core through communication channels.

EnvironmentEnvironment

Intelligence core

Embodiment

Sensors

Actuators

EE141


Necessary for development of intelligence

Hosts brain’s interfaces that interact with environment

Not necessarily constant or in the form of a physical body

Boundary transforms modifying brain’s self-determination EnvironmentEnvironment

Intelligence core

Embodiment

Sensors

Actuators

EE141

Brain learns own body’s dynamic Self-awareness results from

identification with own embodiment Embodiment can be extended by

using tools and machines Successful operation depends on

correct perception of environment and own embodiment


EE141

Broca’sarea

Parsopercularis

Motor cortex Somatosensory cortex

Sensory associativecortex

PrimaryAuditory cortex

Wernicke’sarea

Visual associativecortex

Visualcortex

Brain OrganizationBrain Organization

While we learn its functionscan we emulate its operation?

EE141

How can we design intelligence?How can we design intelligence?

We need to know how We need means to

implement it We need resources to

build and sustain its operation

EE141

Requirements for Embodied IntelligenceRequirements for Embodied Intelligence

State oriented Learns spatio-temporal patterns

Situated in time and space

Learning Perpetual learning

Screening for novelty

Value driven Goal creation

Competing goals Emergence

Artificial evolution

Self-organization

EE141

INPUT OUTPUT

Simulation or

Real-World System

TaskEnvironment

Agent Architecture

Long-term Memory

Short-term Memory

Reason

ActPerceive

RETRIEVAL LEARNING

EI Interaction with EnvironmentEI Interaction with Environment

From Randolph M. Jones, P : www.soartech.com

EE141

Challenges of Embodied IntelligenceChallenges of Embodied Intelligence Development of sensory interfaces

Active vision Speech processing Tactile, smell, taste, temperature, pressure sensing Additional sensing

– Infrared, radar, lidar, ultrasound, GPS, etc.– Can too many senses be less useful?

Development of reinforcement interfaces Energy, temperature, pressure, acceleration level Teacher input

Development of motor interfaces Arms, legs, fingers, eye movement

EE141

Challenges of Embodied Intelligence (cont.)Challenges of Embodied Intelligence (cont.) Finding algorithmic solutions for

Association, memory, sequence learning, invariance building, representation, anticipation, value learning (pain reduction), goal creation, planning

Development of circuits for neural computing Determine organization of artificial minicolumn Self-organized hierarchy of minicolumns for

sensing and motor control Self-organization of goal creation pathway

EE141

V. Mountcastle argues that all regions of the brain perform the same computational algorithm

Groups of neurons (minicolumns) have the same structure and are connected in a pseudorandom way

Minicolumns organized in macrocolumns

VB Mountcastle (2003). Introduction [to a special issue of Cerebral Cortex on columns]. Cerebral Cortex, 13, 2-4.

Human Intelligence - Uniform StructureHuman Intelligence - Uniform Structure

EE141

Selective Processing in Minicolumns Selective Processing in Minicolumns

Sensory inputs are represented by gradually more abstract features in the sensory hierarchy

Use “sameness principle” of the observed objects to detect and learn feature invariance

Learn to store temporal sequences Use random wiring to preselect sensory

features Use feedback for input prediction and screen

input information for novelty Use redundant structures of sparsely connected

processing elements

EE141

Interactions between MinicolumnsInteractions between Minicolumns

Positive Reinforcement

Negative Reinforcement

Sensory Inputs Motor

Outputs

Valuecenter

Sensory processing

Motor processing

Pleasure Pain

Learn Control

Associate

Direct

EE141

Sensory neurons are responsible for representing environment receive inputs from sensors or sensory neurons on lower level represent the environment receive feedback input from motor and higher level sensory neurons help to activate motor and reinforcement neurons

Motor neurons are responsible for actions and skills are activated by reinforcement and sensory neurons activate actuators or provide an input to lower level motor neurons provide planning inputs to sensory neurons

Reinforcement neurons are responsible for building the value system, goal creation, learning, and exploration receive inputs from reinforcement neurons on the lower level receive inputs from sensory neurons provide inputs to motor neurons initiate learning and force exploration

Functions of Neurons in MinicolumnsFunctions of Neurons in Minicolumns

EE141

Selective Sensory-Motor PathwaysSelective Sensory-Motor Pathways

Hierarchical sensory and motor pathways Branching off from more general to more specific Easy to expand to higher levels Gradual loss of interconnect plasticity towards inputs

inputs

internal representations

EE141

Environment……………... … …

Sensory neurons in a minicolumn

Selective Sensory-Motor PathwaysSelective Sensory-Motor PathwaysIncreasing connection’s plasticity

106 neurons

1011 neurons

10 neurons

Sensory pathway

10 neurons

104 neuronsActivation pathway

EE141

General characteristics: Hierarchical structure Minicolumn processing Spatial and temporal association

via interconnections and dual neurons

Feedback connections Selective adaptation

Function: Invariant representation Anticipation Screening for novelty Value learning

... …

... …

... …

Organization of Sensory-motor PathwaysOrganization of Sensory-motor Pathways

EE141

Sensory-motor CoordinationSensory-motor Coordination

R: representationE: expectationA: associationD: directionP: planning Environment

…

…

…

…

D

R

E

A

Sensor path Motor path

Increasing connection’s adaptability’

Goal creation &Value system

EE141

Goal Driven BehaviorGoal Driven Behavior

Goal driven behavior is one of the required elements of intelligence

Perceptions and actions are activated selectively to serve the machine’s objectives

In the existing EI models, the goal is defined by designers and is given to the learning agent

Humans and animals create their own goals

The goal creation may be one of the most important elements of EI mechanism

EE141

Goal Creation Goal Creation

Goal creation ability is essential for developing intelligence

We will create goals based on simple structures interacting with sensory and motor pathways

Goals must be built and understood in a similar way to building perceptions

More complex goals can be understood only if representations for these goals are build

Goal creation should result from EI interaction with environment, by perceiving successes or failures of its actions

EE141

Pain-center and Goal CreationPain-center and Goal Creation

Simple Mechanism Creates hierarchy of

values Leads to formulation of

complex goals Reinforcement

neurotransmitter:• Pain increase• Pain decrease

Forces exploration

+

-

Environment

Sensor

MotorPain level

Dual pain levelPain increase

Pain decrease

(-)

(+)

Excitation

(-)

(-)

(+)

(+)

EE141

Abstract Goal CreationAbstract Goal Creation

The goal is to reduce the primitive pain level Abstract goals are created to satisfy the primitive goals

Expectation

AssociationInhibitionReinforcementConnectionPlanning

- +

PainDual pain

Food

refrigerator

- +

Stomach

Abstract pain(Delayed memory of pain)

“food” becomes a sensory input to

abstract pain center

Sensory pathway(perception, sense)

Motor pathway(action, reaction)

Primitive Level

Level I

Level II

Eat

Open

EE141

The Three Pathways CombinedThe Three Pathways Combined Goal creation, sensory

and motor pathways interact on different hierarchy levels

Pain driven goal creation sets goal priorities

The tree pathways emerge together from interaction with the environment

Pain tree IPain tree IIMotor pathwaySensory pathwayPain center to motor Sensor to motor Sensor to pain center

EE141



implement it Hardware self-organization Sparsity of interconnect Functionality requirements EI Design Efforts

We need resources to build and sustain its operation

EE141

We need means to implement itWe need means to implement it

What is needed is a self-organizing hardware Saves design work Increases hardware reliability Improves production yield Lowers design cost

Basic mechanism for self-organized hardware is pruning of random interconnections Nature does it

Activity triggered “growth” of new neurons Spare neurons used in the later stages of learning

EE141

Brain is self-organizing and sparse

Human Brain Human Brain at Birthat Birth 6 Years Old6 Years Old 14 Years Old14 Years Old

Rethinking the Brain, Families and Work Institute, Rima Shore, 1997.

We need means to implement itWe need means to implement it

EE141

Thompson, R. A., & Nelson, C. A. (2001). Developmental science and the media: Early brain development. American Psychologist, 56(1), 5-15.

Synaptic Density over the Lifespan Synaptic Density over the Lifespan

EE141

Sparse Connectivity Sparse Connectivity The brain is sparsely connected. A neuron in cortex may have on the order of

100,000 synapses. There are more than 1010 neurons in the brain. Fractional connectivity is very low: 0.001%.

Implications: Connections are expensive biologically. Short local connections are cheaper than

long ones.

EE141

Feature extraction – using WTA Generalization – using overlapping regions

… …

… …h+1

s

h

winner

…… … …

r-lower level region (single WTA in each)

Hierarchical Self-organizing Hierarchical Self-organizing Learning Memory (HSOLM)Learning Memory (HSOLM)

EE141

Hierarchical Self-organizing Learning MemoryHierarchical Self-organizing Learning Memory 3-layer R-net in hierarchical structure Reduced connection density The winner pathway replaces the WTA

Established using feed-back and local competition

…

…

…

… …

winner

Input pattern

h+1

s2

h

s1

number of output links from the lower level nodes

)),(... 2, 1,( 213

1 sshho nnninl

EE141

Self-Organizing Learning Arrays SOLARSelf-Organizing Learning Arrays SOLAR

EE141

Wiring in SOLARWiring in SOLAR

Initial wiring and final wiring selection for credit card approval problem

EE141

Associative SOLARAssociative SOLAR

EE141

IR

tim

Excitationlink

1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4

miss

sMUXTP MUX MUX MUX

mit

o1 2 3 4

MUX

momn1 n2 n3

PCN

LFN

LN

Multiple winnerDetection Neuron

FromESN

PreductionNeuron

PredictionMatchingNeuron

Sequence Learning Mechanism

EE141

Software or hardware?Software or hardware?

Sequential Error prone Require programming Low cost Well developed

programming methods

Concurrent Robust Require design Significant cost Hardware prototypes

hard to build

Software Hardware

EE141

Future software/hardware capabilitiesFuture software/hardware capabilities

Human brain complexity

2005 2010 2015 2020 2025 2030 2035 204010

4

105

106

107

108

109

1010

1011

Year

Num

ber

of n

euro

ns

Software Simulation (PC based) Hardware approach (FPGA)

Analog VLSI

100 FPGAs

10 FPGAs

EE141





EE141

Major Intelligent Systems Design EffortsMajor Intelligent Systems Design Efforts GOMS - Goals, Operators, Methods, and Selection Rules

Proposed by Card, Moran, and Newell (1983), for modeling and describing human task performance

SOAR - State, Operator, Application, Result Based on the book by Newell, A. (1990), Unified Theories of Cognition.

ACT-R (Adaptive Control of Thought - Rational ) Described by Anderson, J. (1993), Rules of the Mind

Artificial General Intelligence – Adaptive A.I. Inc. http://adaptiveai.com/

Hierarchical Temporal Memory – Numenta Inc.

Irvine Sensors Corporation (Costa Mesa, CA)

3DANN

“Brain On Silicon” will not be just a science fiction!

http://www.numenta.com/

EE141

GOMS - Goals, Operators, Methods, and Selection Rules Inspired by psychology of immediate action Explicit hierarchical task decompositions Explicit pairing of goals with tasks Belief, Goal and Plan representation and

selection

Primary Engineering Activities Cognitive task analysis Design of goal hierarchy Definition of plans and selection rules Definition of primitive operators and resource constraints

EI Design EffortsEI Design Efforts

Proposed by Card, Moran, and Newell (1983),

for modeling and describing human task performance From slide by Dan Glaser

EE141

SOAR - State, Operator, Application, Result Problem space and physical symbols Minimal set of rules to support intelligent behavior Belief, Goal and Plan representation and selection

Primary Engineering Activities Cognitive task analysis Identification of primitive operations Design of belief and goal hierarchies Design of situation interpretation hierarchies Interactions between knowledge units

http://sitemaker.umich.edu/soar

Newell, A. 1990. Unified Theories of Cognition. Cambridge, Massachusetts: Harvard University Press.


EE141

ACT-R (Adaptive Control of Thought - Rational ): a cognitive architecture inspired by psychological models of

memory, skills, and learning Optimization-oriented integrated memory, action, and learning Procedural knowledge units with explicit retrieval Uses if then production rules Belief, Goal and Plan representation and selection

Primary Engineering Activities Cognitive task analysis Identification of procedural and declarative knowledge Design of semantic associations and retrievable goal hierarchies Identification of primitive operations with logical and associative context

Anderson, John. Rules of the Mind, Lawrence Erlbaum Associates, Hillsdale, NJ (1993).


EE141

EI Design EffortsEI Design Efforts Hierarchical Temporal Memory – Numenta Inc.

Uses hierarchy of spatio-temporal associations Discover causes in the world Learns complex goal-oriented behavior Makes predictions while learning Combines unsupervised and supervised learning to make associations Aims at exceeding human level cognition

Primary Engineering Activities Cognitive structure analysis Pattern learning in hierarchical memories Temporal sequence learning mechanism Goal creation

http://www.numenta.com/

Die Stack


EE141

EI Design EffortsEI Design Efforts Artificial General Intelligence – Adaptive A.I. Inc.

Has general cognitive ability Acquires knowledge and skills Uses bidirectional, real-time interaction Selects information and uses an adaptive attention Unsupervised and self-supervised learning Adaptive and self-organizing data structure Processes dynamic patterns

Primary Engineering Activities Explicitly engineering functionality Conceptual design General proof of concept Animal level cognition

http://adaptiveai.com/


3DANN

“Brain On Silicon” will not be just a science fiction!

EE141





EE141From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

Resources – Evolution of ElectronicsResources – Evolution of Electronics

EE141 By Gordon E. MooreBy Gordon E. Moore


Clock Speed (doubles every 2.7 years)

EE141

Doubling (or Halving) timesDoubling (or Halving) times

Dynamic RAM Memory “Half Pitch” Feature Size 5.4 years Dynamic RAM Memory (bits per dollar) 1.5 years Average Transistor Price 1.6 years Microprocessor Cost per Transistor Cycle 1.1 years Total Bits Shipped 1.1 years Processor Performance in MIPS 1.8 years Transistors in Intel Microprocessors 2.0 years Microprocessor Clock Speed 2.7 years

From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

EE141

Why should we care?Why should we care?

Source: SEMATECHSource: SEMATECH

EE141

0%

20%

40%

60%

80%

100%

1999

2002

2005

2008

2011

2014

% Area Memory

% Area ReusedLogic

% Area New Logic

Percent of die area that must be occupied by memory to maintain SOC design productivity

Design Productivity Gap Design Productivity Gap Low-Value Designs? Low-Value Designs?

Source = Japanese system-LSI industry

EE141

2002 2010 2020 2030

Biomimetics and Bio-inspired SystemsBiomimetics and Bio-inspired SystemsImpact on Space Transportation, Space Science and Earth ScienceImpact on Space Transportation, Space Science and Earth Science

Mis

sio

n C

om

ple

xity

Biological Mimicking

Embryonics

Extremophiles

DNA Computing

Brain-like computing

Self Assembled Array

Artificial nanoporehigh resolution

Mars in situlife detector

Sensor Web

Biological nanoporelow resolution

Skin and Bone

Self healing structureand thermal protection

systems

Biologically inspired aero-space systems

Space Transportation

EE141

NanoEngineer-1 is an Open Source (GPL) project sponsored by Nanorex, Inc. It is an interactive 3D nanomechanical

engineering program

From Nanorex Inc.

Revolutions in electronics and computing will allow reconfigurable, autonomous,

"thinking" spacecraft

http://ipt.arc.nasa.gov/nanotechnology.html

Planetary Gear

Universal Joint

NanosystemsNanosystems

EE141

Promises of embodied intelligencePromises of embodied intelligence To society

Advanced use of technology– Robots– Tutors– Intelligent gadgets

Society of minds– Superhuman intelligence– Progress in science– Solution to societies’ ills

To industry Technological development New markets Economical growth

ISAC, a Two-Armed Humanoid RobotISAC, a Two-Armed Humanoid RobotVanderbilt UniversityVanderbilt University

EE141

Transhumanist Transhumanist valuesvalues

Nothing wrong about “tampering with nature” Individual choice in the use of enhancement technologies; Peace, international cooperation, anti-proliferation of WMDs Improving understanding (research and public debate; critical thinking;

open-mindedness; scientific progress; open discussion of the future) Getting smarter (individually; collectively; develop machine intelligence) Pragmatism; engineering and entrepreneur-spirit; “can-do” attitude Diversity (species, race, religious creed, sexual orientation, life style, etc.) Saving lives (life extension, anti-aging research, and cryonics)

Copyright Institute for Ethics and Emerging Technologies 2004-2005

EE141

Sounds like science fictionSounds like science fiction

If you’re trying to look far ahead, and what you see seems like science fiction, it might be wrong.

But if it doesn’t seem like science fiction, it’s definitely wrong.

From presentation by Feresight Institute

EE141

Embodied Artificial IntelligenceEmbodied Artificial IntelligenceBased on: [1] E. R. Kandel et al. Principles of Neural Science, McGraw-Hill/Appleton & Lange; 4 edition, 2000. [2] F. Inda, R. Pfeifer, L. Steels, Y. Kuniyoshi, “Embodied Artificial

Intelligence,” International seminar, Germany, July 2003.[3] R. Chrisley, “Embodied artificial intelligence, ” Artificial

Intelligence, vol. 149, pp.131-150, 2003.[4] R. Pfeifer and C. Scheier, Understanding Intelligence, MIT

Press, Cambridge, MA, 1999. [5] R. A. Brooks, “Intelligence without reason,” In Proc. IJCAI-91.

(1991) 569-595 .[6] R. A. Brooks, Flesh and Machines: How Robots Will Change

Us, (Pantheon, 2002).

[7] R. Kurzweil The Age of Spiritual Machines: When Computers

Exceed Human Intelligence, (Penguin, 2000).

ee141 ee690 design of embodied intelligence janusz starzyk eecs, ohio university

Documents

ee141 intelligence

question of intelligence

brain development intelligence

ohio university slide

intelligence ais holy

best model slide

abstract problem

mainstream science