1
Consciousness and Consciousness and CCognitionognition
Janusz A. Starzyk
Cognitive ArchitecturesCognitive Architectures
Motivated LearningMotivated Learning
Various pains and external signals compete for attention. Attention switching results from competition. Cognitive perception is aided by winner of competition.
Definition: Motivated learning (ML) is pain based motivation, goal creation and learning in embodied agent. Machine creates abstract goals based on the
primitive pain signals. It receives internal rewards for satisfying its goals
(both primitive and abstract). ML applies to EI working in a hostile environment.
Primitive Goal CreationPrimitive Goal Creation
- +
Pain
Dry soilPrimitive
level
opentank
sit on garbage
refillfaucet
w. can water
Dual pain
Reinforcing a proper action
Abstract Goal HierarchyAbstract Goal Hierarchy
Abstract goals are created to reduce abstract pains and to satisfy the primitive goals A hierarchy of abstract goals is created to satisfy the lower level goals
ActivationStimulationInhibitionReinforcementEchoNeedExpectation
- +
+
Dry soilPrimitive Level
Level I
Level IIfaucet
-
w. can
open
water
+
Sensory pathway(perception, sense)
Motor pathway(action, reaction)
Level IIItank
-
refill
MOTOR
FUNCTION
SENSOR OBJECT REDUCES PAIN INCREASES
PAINEat Food Hunger Lack of Food
Buy Food at
Grocery Store Lack of Food Lack of Money
Withdraw from BankAccount
Lack of Money Overdrawn Account
Workin
The office Overdrawn Account Lack of job opportunities
Studyat
School Lack of job opportunities
-
Playwith
Toys - -
Goal Creation Experiment in MLGoal Creation Experiment in ML
Goal Creation Experiment in MLGoal Creation Experiment in ML
Pain signals in CGS simulation
0 100 200 300 400 500 6000
1
Primitive Hunger
Pa
in
0 100 200 300 400 500 6000
0.5
Lack of Food
Pa
in
0 100 200 300 400 500 6000
0.5
Empty Gorcery
Pa
in
Discrete time
Goal Creation Experiment in MLGoal Creation Experiment in ML
Action scatters in 5 CGS simulations
0 100 200 300 400 500 6000
5
10
15
20
25
30
35
40Goal Scatter Plot
Go
al I
D
Discrete time
Goal Creation Experiment in MLGoal Creation Experiment in ML
The average pain signals in 100 CGS simulations
0 100 200 300 400 500 6000
0.5
Primitive Hunger
Pai
n
0 100 200 300 400 500 6000
0.10.2
Lack of FoodP
ain
0 100 200 300 400 500 6000
0.10.2
Empty Gorcery
Pai
n
0 100 200 300 400 500 6000
0.10.2
Lack of Money
Pai
n
0 100 200 300 400 500 6000
0.050.1
Lack of JobOpportunitites
Pai
n
Discrete time
Compare RL (TDF) and ML (GCS)Compare RL (TDF) and ML (GCS)
Mean primitive pain Pp value as a function of the number of iterations:
- green line for TDF -blue line for GCS.
Primitive pain ratio with pain threshold 0.1
Comparison of execution time on log-log scale TD-Falcon green GCS blue
Combined efficiency of GCS 1000 better than TDF
Compare RL (TDF) and ML (GCS)Compare RL (TDF) and ML (GCS)
Problem solved
Conclusion: embodied intelligence, with motivated learning based on goal creation system, effectively integrates environment modeling and decision making – thus it is poised to cross the chasm
Reinforcement LearningReinforcement Learning Motivated Learning Motivated Learning Single value function
Various objectives
Measurable rewards Predictable Objectives set by designer Maximizes the reward
Potentially unstable
Action depends on the state of the environment
Learning effort increases with complexity
Always active
Multiple value functions One for each goal
Internal rewards Unpredictable Sets its own objectives Solves minimax problem
Always stable
Action depends on the states of the environment and agent
Learns better in complex environment than RL
Acts when needed
http://www.bradfordvts.co.uk/images/goal.jpg
Drought
Reservoir
Irrigate
Thirsty
Water
Drink Water
Primitive Needs Dirty
Wash in Water
Abstract Needs
Primitive needsPrimitive needs
0 50 100 150 200 250 3000
0.02
0.04
0.06
0.08
0.1
0.12Competing need signals
Iterative step
Nee
d si
gnal
leve
l
DirtyThirstyDroughtThreshold
Drought
Reservoir
Public Money
Irrigate
Spend Money to Build
Thirsty
Water
Drink Water
Spend Money to Buy
Primitive Needs
Well
Draw own Water
Dirty
Wash in Water
Abstract Needs
Abstract needsAbstract needs
Drought
Reservoir
Public Money
Tourists' Attractions
Irrigate
Spend Money to Build
Build Ecotourism
Thirsty
Water
Drink Water
Spend Money to Buy
Primitive Needs
Build Water Recreation
Wealthy Taxpayers
Rise Taxes
Well
Draw own Water
Dirty
Wash in Water
Well Building
Dig a Well
Abstract Needs
Ground Water
Water Supply
Abstract needsAbstract needs
Drought
Reservoir
Public Money
Tourists' Attractions
Irrigate
Spend Money to Build
Build Ecotourism
Thirsty
Water
Drink Water
Spend Money to Buy
Primitive Needs
Build Water Recreation
Policy
Develop Infrastructure
Wealthy Taxpayers
Rise Taxes
Well
Draw own Water
Dirty
Wash in Water
Well Building
Dig a Well
Abstract Needs
Employment Opportunities
Ground Water
Water Supply
Receive SalaryResource Management
and Planning
Management
Regulate Use
Planning
Abstract needsAbstract needs
Definition of Machine ConsciousnessDefinition of Machine Consciousness
Consciousness is attention driven cognitive perception motivations,
thoughts, plans and action monitoring.
A machine is conscious IFF besides ability to perceive, act, learn and
remember, it has a central executive mechanism that controls all the
processes (conscious or subconscious) of the machine;
Photo: www.spectrum.ieee.org/.../biorobot11f-thumb.jpghttp://hplusmagazine.com/sites/default
Photo: http://eduspaces.net/csessums/weblog/11712.html
Consciousness: Consciousness: functional requirementsfunctional requirements
http://faculty.virginia.edu/consciousness
Intelligence
Central executive
Attention and attention switching
Mental saccades
Cognitive perception
Cognitive action control
Computational Model of Machine ConsciousnessComputational Model of Machine Consciousness
Semantic memory
Sensory processors
Data encoders/ decoders
Sensory units
Motor skills
Motor processors
Data encoders/ decoders
Motor units
Emotions, rewards, and sub-cortical processing
Attention switching
Action monitoring
Motivation and goal processor
Planning and thinking
Episodic memory
Queuing and organization of episodes
Episodic Memory & Learning
Central Executive
Sensory-motor
Inspiration: human brainInspiration: human brainPhoto (brain): http://www.scholarpedia.org/article/Neuronal_correlates_of_consciousness
20
Sensory and Motor HierarchiesSensory and Motor Hierarchies Sensory and motor
systems appear to be arranged in hierarchies with information flowing between each level of the sensory and motor hierarchies.
Sensory- Motor BlockSensory- Motor Block
Semantic memory
Sensory processors
Data encoders/ decoders
Sensory units
Motor skills
Motor processors
Data encoders/ decoders
Motor units
Emotions, rewards, and sub-cortical processing
Sensory-motor
sensory processors integrated with semantic memorymotor processors integrated with motor skillssub-cortical processors integrated with emotions and rewards
http://www.ourbabynews.com/wp-content
Central ExecutiveCentral ExecutivePlatform for the emergence of consciousnessControls its conscious and subconscious processesIs driven by
attention switching learning mechanism creation and selection of
motivations and goals
ahsmail.uwaterloo.ca/kin356/cexec/cexec.htm http://www.unifesp.br/dpsicobio/eventos/workingmemory/
Attention switching
Action monitoring
Motivation and goal processor
Planning and thinking
Central Executive
Taskso cognitive perceptiono attentiono attention switchingo motivationo goal creation and selectiono thoughtso planningo learning, etc.
Central ExecutiveCentral Executive
http://prodinstres.pbworks.com
Interacts with other units for o performing its tasks o gathering data o giving directions to other units
No clearly identified decision centerDecisions are influenced by
o competing signals representing motivations, pains, desires, plans, and interrupt signals
• need not be cognitive or consciously realizedo competition can be interrupted by attention switching signal
Attention switching
Action monitoring
Motivation and goal processor
Planning and thinking
Central Executive
Central ExecutiveCentral Executive
http://www.resourceroom.net/
Attention is a selective process of cognitive
perception, action and other cognitive experiences like
thoughts, action planning,
expectations, dreams
Attention switching is needed to have a cognitive
experience
leads to sequences of cognitive experiences
Comic: http://lonewolflibrarian.wordpress.com/2009/08/05/attention-and-distraction-what-are-you-paying-attention-to-08-05-09/
Attention Switching !!!Attention Switching !!!
http://brandirons.com/
http://www.mukyaa.com
Attention Switching !!!Attention Switching !!!Dynamic process resulting from competition between
• representations related to motivations
• sensory inputs
• internal thoughts including spurious signals (like noise).
blog.gigoo.org/.../ http://www.cs.miami.edu
Attention Switching !!!Attention Switching !!!
Thus, while paying attention is a conscious experience,
switching attention does not have to be.
May be a result of :
•deliberate cognitive experience (and thus fully conscious signal)
• subconscious process (stimulated by internal or external signals)
Formulate episode
Saccade control
Changing perception
Changing environment
Advancement of a goal?
Yes No
Action control
Changing motivation
Write to episodic memory
Loop 1
Loop 2
Attention spotlight
Associative memory
From virtual game
Simplified Cognitive MachineSimplified Cognitive Machine
Mental SaccadesMental Saccades
This in turn activates memory traces in the global workspace area
that will be used for mental searches (mental saccades).
saccade
John
Input image
Episodic and associative memory network
his wife his house
his dogfriends
business
Spotlight on John
Frontal cortex
Mental saccade
wife house
dogfriends
business
Memory traces in frontal cortex
saccade
John
Input image
Episodic and associative memory network
his wife his house
his dogfriends
business
Spotlight on John
Frontal cortex
Mental saccade
wife house
dogfriends
business
Memory traces in frontal cortex
Selected part of the image resulting from an eye saccade.
Perceived input activates object recognition and associated areas of semantic and episodic memory.
Mental saccades in a conscious machineMental saccades in a conscious machine
Perceptual saccadesChanging perception
Changing environment
Associative memory
No
No
Action controlLoop 5
Loop 2
Perceptual saccadesChanging perception
Changing environment
Associative memory
No
No
Action control
Advancement of a goal?
Yes
Learning
Advancement of a goal?
Advancement of a goal?
Yes
Learning
Attention spotlight
Mental saccades
Continue search?
Yes
Loop 1
Attention spotlight
Mental saccades
Continue search?Continue search?
Yes
Loop 1
Plan action?
NoYes
Action?
Yes
No
Changing motivation
Loop 3
Loop 4
Plan action?Plan action?
NoYes
Action?Action?
Yes
No
Changing motivation
Loop 3
Loop 4
Loop 5
Loop 2
http://cdn-3.lifehack.org/wp-content
Comprehensive Cognitive ModelComprehensive Cognitive Model
Proposed cognitive system organization
Contains Semantic, episodic and
procedural memories. WTA attention switching Visual and mental
saccades Scene building Action planning And more…
Figure represents our top-level design model
Computational Model: SummaryComputational Model: SummarySelf-organizing mechanism of emerging motivations and other signals competing for attention is fundamental for conscious machines.A central executive controls conscious and subconscious processes driven by its attention switching mechanism.Attention switching is a dynamic process resulting from competition between representations, sensory inputs and internal thoughtsMental saccades of the working memory are fundamental for cognitive thinking, attention switching, planning, and action monitoring
Photo: http://www.prlog.org/10313829-homeless-man-earns-250000-after-viewing-prosperity-consciousness-video-subliminal-mind-training.html
Computational Model: ImplicationsComputational Model: ImplicationsMotivations for actions are physically distributed
o competing signals are generated in various parts of machine’s mind
Before a winner is selected, machine does not interpret the meaning of the competing signals Cognitive processing is predominantly sequential
o winner of the internal competition is an instantaneous director of the cognitive thought process, before it is replaced by another winner
Top down activation for perception, planning, internal thought or motor functions
o results in conscious experience• decision of what is observed and where is it • planning how to respond
o a train of such experiences constitutes consciousness