A computational unification of cognitive behavior and emotion

Robert P. Marinier III, John E. Laird, Richard L. LewisCognitive Systems Researchvol. 10, no. 1, pp. 48-69, 2008

2008311760 최봉환

PEACTIDM

2 /24

Soar : cognitive architecture

• Cognitive architecture– Task-independent structure and subsystems

• Soar– For Cognitive modeling– For Real-world application

of knowledge-rich intelligent systems

– Long-term Memories• Procedural, semantic, episodic• Associative learning mechanisms

(working Memory)

3 /24

PEACTIDM in Soar

Motorhandled by simulation of the environment

Decode

send selected action to output system

Comprehend

implemented as a set of comprehend operators

Attend

implemented as an Attend operator by PEACTIDM ( only allow a stimuli at a time )

Encoding

matching rules in procedural memory generate domain-independent augmenta-tions

Perceive

reception of raw sensory inputs

Tasking

Create the goal in Short-term Memory

4 /24

appraisal theories

• What can emotion provide?– PEACTIDM and cognitive architectures

• Describe : processes, constraints and timescale • Do not describe : the specific knowledge structures

– Much of the information required by PEACTIDM• Structure of Encode generate, what information does Attend, informa-

tion by Comprehend generate, information of Intend use to generate a response

Emotion = the PEACTIDM operations• Appraisal theories

– Emotions result from the evaluation of the relations ship between goals and situations [Roseman & Smith, 2001]

• Ref) Parkinson (2009), Marsella and Gratch (2009), and Reisenzein (2009).

– Fit naturally into our immediate choice response task• Complex cognition = with complex emotion [Smith & Lazarus, 1990]

– Discrepancy from Expectation 전구를 끄려고 버튼을 눌렀지만 안 꺼진 경우

• Mismatch between the actual state and the expected state• Conflicts with the Outcome Probability

Feel Surprise

Emotion modeling : Introduce

5 /24

Scherer’s appraisal theory ( 2001)

• Features– 16 appraisal dimensions

• 4 groups : relevance, implication, coping potential, normative signifi-cance

– A continuous space of emotion• Provides a mapping from appraisal values to emotion labels• Labels modal emotions

– Appraisal are not generated simultaneously– Process model (abstract level)

Emotion modeling : in detail

6 /24

Integration : Theory

• How PEACTIDM + Scherer's appraisal theory

7 /24

Integration : Implementation(1)

• Appraisal values

• Computing the active appraisal frame– Pre-attentive appraisal frames[Gratch and Marsella,

2004]• Before Attend : one frame for each stimulus the agent perceives

– Attend = select a stimulus– Active frame : selected stimulus associated appraisal

frame

8 /24

Integration : Implementation(2)

• Sequences and time courses of appraisals– The appraisals are generated sequentially [Scherer,

2001]– The model implies avoid error and low efficiency

• Partially ordered sequences of appraisals• Varying time courses for the generation of those appraisals

• Determining the current emotion– Appraisal Detector [Smith & Kirby, 2001]

• processes the active frame to determine the current emotion– Supports one active appraisal frame at a time(=only one

emotion)– Categorical theories of emotion : fixed number of possi-

ble feelings• A unique appraisal frame a unique experience• segmenting the space of appraisal frames Categorical, linguistic la-

bels– Actual representation

• active appraisal frame: Suddenness = 1.0, Goal Relevance= 1.0, Out-come Probability = 1.0, Conduciveness = 1.0.

9 /24

Integration : Implementation(3)

• Calculating intensity– Summarizes the importance of the emotion– Intensity function [Marinier and Laird, 2007]

• Limited ranges : single value, should map to [0, 1]• No dominant appraisal : multiple values, should dominate the inten-

sity function, generally multiplication is used as combine method [Gratch and Marsella, 2004]

• Realization principle : expected stimuli should be less intense thant unexpected stimuli [Neal Reilly, 2006]

• OP : Outcome Probability, DE : Discrepancy from Expectation, S : Sud-dennesssUP : Unpredictability, IP : Intrinsic Pleasantness, GR : Goal Relevance, Cond : Conduciveness, Ctrl : Control, P : Power, num_dims : # of di-mension

10 /24

Integration : Implementation(4)

• Modeling the task

• The revised task

11 /24

Example : Eaters (Pacman) domain (1)

• Eaters Domain : an arbitrary # of cycle is required• New topic

– How previous emotions affect new emotions– The role of Tasking when the ongoing task may be viewed

as different subtasks

12 /24

Example : PEACTIDM (1)

• Perception & Encoding– Perception

• Per direction• by Symbolic data

– Encoding• 4 Cardinal direction : north/south/west/east• Each direction has passable, distance to goal• The distance to goal

– estimated on Manhattan distance

13 /24

Example : PEACTIDM (2)

• Attending– The selection of which stimulus : weighted random choice

• Weight : the values of the appraisals

14 /24

Example : PEACTIDM (3)

• Comprehension– Additional appraisal values to the active frame

• Conduciveness : if direction is passable and on the path to the goal then high

• Control and Power : if direction is passable then high– Specific stimuli determine

• "natural" Causal Agent• "chance" Causal Motive• "back out" : should not proceed, solve with heuristic method

( dynamic difference reduction ; Newell, Shaw, and Simon, 1960)– Comprehension operators

• Complete : when can act as stimuli• Ignore : control return to attend

• Tasking (in generelly Managing goals)– Abstracted goal : ex) "go to work"

• cannot be acted upon directly• must be broken down into more concrete compoonents

– Concrete goal : ex) "take a step"• can be acted upon directly

15 /24

Example : PEACTIDM (4)

• Intending– Intend function : implemented as a Soar operator– If the agent is currently one step away from the goal,

then it creates a goal achievement prediction. – Along with the prediction, the agent also generates an

Outcome Probability appraisal.

• Decode and motor– Soar’s standard method of communicating

16 /24

Example : Emotion (1)

• Over a long period of time in this task how do emotions affect each other over time?

• Emotion– Many theories : Hudlicka, 2004, Gratch & Marsella, 2004,

Damasio, 1994; Damasio, 2003, ... • Feelings = perception of our emotions• Emotion : short-lived• Mood : tend to longer

– Modeling• Feeling : intensity of appraisal frame• Emotion : feeling + feeling intensity• Mood : "moves" toward the emotion

each time step

17 /24

Example : Emotion (2)

18 /24

Example : The Influence of Emo-tion, mood and feeling upon behav-ior

• Feeling– Additional knowledge to the state representation

• Current = emotion, Past = mood– Guide control influence behavior

• [Forgas, 1999], [Gross & John, 2003]– Integration with action tendencies [Frijda et al., 1989]

included to demonstrate the possibility of feelings influ-encing behavior and focusing on one aspect of coping

• coping by giving up on goals• Giving up : a kind of Tasking

– Emotional feedback can detect is not making progress toward the goal– Subtask can give up if agents current feeling of Con-

duciveness is negative• Mood : motivation to go

19 /24

Evaluation

20 /24

Evaluation Result

21 /24

Related Work

• EMA [Gratch and Marsella, 2004]– Emotion and Adaptation– A computational model of a simple appraisal theory im-

plemented in Soar 7• MAMID [Hudlicka, 2004]

– Building emotions into a cognitive architecture• OCC/Em [Ortony et al, 1988]

– OCC model– OCC only briefly touches on mood, but leaves it unspeci-

fied• Kismet [Breazeal, 2003]

– social robot

22 /24

Summary

• (1) Appraisals are a functionally required part of cognitive processing; they cannot be replaced by some other emotion generation theory.

• (2) Appraisals provide a task-independent language for con-trol knowledge, although their values can be determined by task-dependent knowledge. Emotion and mood, by virtue of being derived from appraisals, abstract summaries of the current and past states, respectively. Feeling, then, aug-ments the current state representation with knowledge that combines the emotion and mood representations and can influence control.

• (3) The integration of appraisal and PEACTIDM implies a partial ordering of appraisal generation.

• (4) This partial ordering specifies a time course of appraisal generation, which leads to time courses for emotion, mood and feeling.

• (5) Emotion intensity is largely determined by expectations and consequences for the agent; thus, even seemingly mundane tasks can be emotional under the right circum-stances.

• (6) In general, appraisals may require an arbitrary amount of inference to be generated

23 /24

용어• CSP - Constraint Satisfaction ProblemEBG - Explana-tion-Based Generalisation

• EBL - Explanation-Based Learning• GOMS - Goals, Operators, Methods, and Selection rules

• HISoar - Highly Interactive Soar• ILP - Inductive Logic Programming• NNPSCM - New New Problem Space Computational Model

• NTD - NASA Test Director• PEACTIDM - Perceive, Encode, Attend, Comprehend, Task, Intend, Decode, Move

• SCA - Symbolic Concept Acquisition

24 /24