1 consciousness and cognition janusz a. starzyk cognitive architectures

1

Consciousness and Consciousness and CCognitionognition

Janusz A. Starzyk

Cognitive ArchitecturesCognitive Architectures

Motivated LearningMotivated Learning

Various pains and external signals compete for attention. Attention switching results from competition. Cognitive perception is aided by winner of competition.

Definition: Motivated learning (ML) is pain based motivation, goal creation and learning in embodied agent. Machine creates abstract goals based on the

primitive pain signals. It receives internal rewards for satisfying its goals

(both primitive and abstract). ML applies to EI working in a hostile environment.

Primitive Goal CreationPrimitive Goal Creation

- +

Pain

Dry soilPrimitive

level

opentank

sit on garbage

refillfaucet

w. can water

Dual pain

Reinforcing a proper action

Abstract Goal HierarchyAbstract Goal Hierarchy

Abstract goals are created to reduce abstract pains and to satisfy the primitive goals A hierarchy of abstract goals is created to satisfy the lower level goals

ActivationStimulationInhibitionReinforcementEchoNeedExpectation

- +

+

Dry soilPrimitive Level

Level I

Level IIfaucet

-

w. can

open

water

+

Sensory pathway(perception, sense)

Motor pathway(action, reaction)

Level IIItank

-

refill

MOTOR

FUNCTION

SENSOR OBJECT REDUCES PAIN INCREASES

PAINEat Food Hunger Lack of Food

Buy Food at

Grocery Store Lack of Food Lack of Money

Withdraw from BankAccount

Lack of Money Overdrawn Account

Workin

The office Overdrawn Account Lack of job opportunities

Studyat

School Lack of job opportunities

-

Playwith

Toys - -

Goal Creation Experiment in MLGoal Creation Experiment in ML


Pain signals in CGS simulation

0 100 200 300 400 500 6000

1

Primitive Hunger

Pa

in

0 100 200 300 400 500 6000

0.5

Lack of Food

Pa

in

0 100 200 300 400 500 6000

0.5

Empty Gorcery

Pa

in

Discrete time


Action scatters in 5 CGS simulations

0 100 200 300 400 500 6000

5

10

15

20

25

30

35

40Goal Scatter Plot

Go

al I

D

Discrete time


The average pain signals in 100 CGS simulations

0 100 200 300 400 500 6000

0.5

Primitive Hunger

Pai

n

0 100 200 300 400 500 6000

0.10.2

Lack of FoodP

ain

0 100 200 300 400 500 6000

0.10.2

Empty Gorcery

Pai

n

0 100 200 300 400 500 6000

0.10.2

Lack of Money

Pai

n

0 100 200 300 400 500 6000

0.050.1

Lack of JobOpportunitites

Pai

n

Discrete time


Comparison between GCS and RL

Compare RL (TDF) and ML (GCS)Compare RL (TDF) and ML (GCS)

Mean primitive pain Pp value as a function of the number of iterations:

- green line for TDF -blue line for GCS.

Primitive pain ratio with pain threshold 0.1

Comparison of execution time on log-log scale TD-Falcon green GCS blue

Combined efficiency of GCS 1000 better than TDF

Compare RL (TDF) and ML (GCS)Compare RL (TDF) and ML (GCS)

Problem solved

Conclusion: embodied intelligence, with motivated learning based on goal creation system, effectively integrates environment modeling and decision making – thus it is poised to cross the chasm

Reinforcement LearningReinforcement Learning Motivated Learning Motivated Learning Single value function

Various objectives

Measurable rewards Predictable Objectives set by designer Maximizes the reward

Potentially unstable

Action depends on the state of the environment

Learning effort increases with complexity

Always active

Multiple value functions One for each goal

Internal rewards Unpredictable Sets its own objectives Solves minimax problem

Always stable

Action depends on the states of the environment and agent

Learns better in complex environment than RL

Acts when needed

http://www.bradfordvts.co.uk/images/goal.jpg

Drought

Reservoir

Irrigate

Thirsty

Water

Drink Water

Primitive Needs Dirty

Wash in Water

Abstract Needs

Primitive needsPrimitive needs

0 50 100 150 200 250 3000

0.02

0.04

0.06

0.08

0.1

0.12Competing need signals

Iterative step

Nee

d si

gnal

leve

l

DirtyThirstyDroughtThreshold

Drought

Reservoir

Public Money

Irrigate

Spend Money to Build

Thirsty

Water

Drink Water

Spend Money to Buy

Primitive Needs

Well

Draw own Water

Dirty

Wash in Water

Abstract Needs

Abstract needsAbstract needs

Drought

Reservoir

Public Money

Tourists' Attractions

Irrigate


Build Ecotourism

Thirsty

Water

Drink Water

Spend Money to Buy

Primitive Needs

Build Water Recreation

Wealthy Taxpayers

Rise Taxes

Well

Draw own Water

Dirty

Wash in Water

Well Building

Dig a Well

Abstract Needs

Ground Water

Water Supply


Drought

Reservoir

Public Money

Tourists' Attractions

Irrigate


Build Ecotourism

Thirsty

Water

Drink Water

Spend Money to Buy

Primitive Needs

Build Water Recreation

Policy

Develop Infrastructure

Wealthy Taxpayers

Rise Taxes

Well

Draw own Water

Dirty

Wash in Water

Well Building

Dig a Well

Abstract Needs

Employment Opportunities

Ground Water

Water Supply

Receive SalaryResource Management

and Planning

Management

Regulate Use

Planning


Definition of Machine ConsciousnessDefinition of Machine Consciousness

Consciousness is attention driven cognitive perception motivations,

thoughts, plans and action monitoring.

A machine is conscious IFF besides ability to perceive, act, learn and

remember, it has a central executive mechanism that controls all the

processes (conscious or subconscious) of the machine;

Photo: www.spectrum.ieee.org/.../biorobot11f-thumb.jpghttp://hplusmagazine.com/sites/default

Photo: http://eduspaces.net/csessums/weblog/11712.html

Consciousness: Consciousness: functional requirementsfunctional requirements

http://faculty.virginia.edu/consciousness

Intelligence

Central executive

Attention and attention switching

Mental saccades

Cognitive perception

Cognitive action control

Computational Model of Machine ConsciousnessComputational Model of Machine Consciousness

Semantic memory

Sensory processors

Data encoders/ decoders

Sensory units

Motor skills

Motor processors


Motor units

Emotions, rewards, and sub-cortical processing

Attention switching

Action monitoring

Motivation and goal processor

Planning and thinking

Episodic memory

Queuing and organization of episodes

Episodic Memory & Learning

Central Executive

Sensory-motor

Inspiration: human brainInspiration: human brainPhoto (brain): http://www.scholarpedia.org/article/Neuronal_correlates_of_consciousness

20

Sensory and Motor HierarchiesSensory and Motor Hierarchies Sensory and motor

systems appear to be arranged in hierarchies with information flowing between each level of the sensory and motor hierarchies.

Sensory- Motor BlockSensory- Motor Block

Semantic memory

Sensory processors


Sensory units

Motor skills

Motor processors


Motor units

Emotions, rewards, and sub-cortical processing

Sensory-motor

sensory processors integrated with semantic memorymotor processors integrated with motor skillssub-cortical processors integrated with emotions and rewards

http://www.ourbabynews.com/wp-content

Central ExecutiveCentral ExecutivePlatform for the emergence of consciousnessControls its conscious and subconscious processesIs driven by

attention switching learning mechanism creation and selection of

motivations and goals

ahsmail.uwaterloo.ca/kin356/cexec/cexec.htm http://www.unifesp.br/dpsicobio/eventos/workingmemory/

Attention switching

Action monitoring



Central Executive

Taskso cognitive perceptiono attentiono attention switchingo motivationo goal creation and selectiono thoughtso planningo learning, etc.

Central ExecutiveCentral Executive

http://prodinstres.pbworks.com

Interacts with other units for o performing its tasks o gathering data o giving directions to other units

No clearly identified decision centerDecisions are influenced by

o competing signals representing motivations, pains, desires, plans, and interrupt signals

• need not be cognitive or consciously realizedo competition can be interrupted by attention switching signal

Attention switching

Action monitoring



Central Executive

Central ExecutiveCentral Executive

http://www.resourceroom.net/

Attention is a selective process of cognitive

perception, action and other cognitive experiences like

thoughts, action planning,

expectations, dreams

Attention switching is needed to have a cognitive

experience

leads to sequences of cognitive experiences

Comic: http://lonewolflibrarian.wordpress.com/2009/08/05/attention-and-distraction-what-are-you-paying-attention-to-08-05-09/

Attention Switching !!!Attention Switching !!!

http://brandirons.com/

http://www.mukyaa.com

Attention Switching !!!Attention Switching !!!Dynamic process resulting from competition between

• representations related to motivations

• sensory inputs

• internal thoughts including spurious signals (like noise).

blog.gigoo.org/.../ http://www.cs.miami.edu

Attention Switching !!!Attention Switching !!!

Thus, while paying attention is a conscious experience,

switching attention does not have to be.

May be a result of :

•deliberate cognitive experience (and thus fully conscious signal)

• subconscious process (stimulated by internal or external signals)

Formulate episode

Saccade control

Changing perception

Changing environment

Advancement of a goal?

Yes No

Action control

Changing motivation

Write to episodic memory

Loop 1

Loop 2

Attention spotlight

Associative memory

From virtual game

Simplified Cognitive MachineSimplified Cognitive Machine

Input image

AB

C D

AB

C DA

B

C D

What Where

Visual SaccadesVisual Saccades

Mental SaccadesMental Saccades

This in turn activates memory traces in the global workspace area

that will be used for mental searches (mental saccades).

saccade

John

Input image

Episodic and associative memory network

his wife his house

his dogfriends

business

Spotlight on John

Frontal cortex

Mental saccade

wife house

dogfriends

business

Memory traces in frontal cortex

saccade

John

Input image

Episodic and associative memory network

his wife his house

his dogfriends

business

Spotlight on John

Frontal cortex

Mental saccade

wife house

dogfriends

business

Memory traces in frontal cortex

Selected part of the image resulting from an eye saccade.

Perceived input activates object recognition and associated areas of semantic and episodic memory.

Mental saccades in a conscious machineMental saccades in a conscious machine

Perceptual saccadesChanging perception


Associative memory

No

No

Action controlLoop 5

Loop 2

Perceptual saccadesChanging perception


Associative memory

No

No

Action control


Yes

Learning



Yes

Learning

Attention spotlight

Mental saccades

Continue search?

Yes

Loop 1

Attention spotlight

Mental saccades

Continue search?Continue search?

Yes

Loop 1

Plan action?

NoYes

Action?

Yes

No

Changing motivation

Loop 3

Loop 4

Plan action?Plan action?

NoYes

Action?Action?

Yes

No

Changing motivation

Loop 3

Loop 4

Loop 5

Loop 2

http://cdn-3.lifehack.org/wp-content

Comprehensive Cognitive ModelComprehensive Cognitive Model

Proposed cognitive system organization

Contains Semantic, episodic and

procedural memories. WTA attention switching Visual and mental

saccades Scene building Action planning And more…

Figure represents our top-level design model

Computational Model: SummaryComputational Model: SummarySelf-organizing mechanism of emerging motivations and other signals competing for attention is fundamental for conscious machines.A central executive controls conscious and subconscious processes driven by its attention switching mechanism.Attention switching is a dynamic process resulting from competition between representations, sensory inputs and internal thoughtsMental saccades of the working memory are fundamental for cognitive thinking, attention switching, planning, and action monitoring

Photo: http://www.prlog.org/10313829-homeless-man-earns-250000-after-viewing-prosperity-consciousness-video-subliminal-mind-training.html

Computational Model: ImplicationsComputational Model: ImplicationsMotivations for actions are physically distributed

o competing signals are generated in various parts of machine’s mind

Before a winner is selected, machine does not interpret the meaning of the competing signals Cognitive processing is predominantly sequential

o winner of the internal competition is an instantaneous director of the cognitive thought process, before it is replaced by another winner

Top down activation for perception, planning, internal thought or motor functions

o results in conscious experience• decision of what is observed and where is it • planning how to respond

o a train of such experiences constitutes consciousness

NeoAxis SimulationNeoAxis Simulation

Neoaxis Implementation VIDEO

ConclusionsConclusions1.Consciousness is computational2.Intelligent machines can be conscious

1 consciousness and cognition janusz a. starzyk cognitive architectures

Documents