human-level ai for computer games

Human-Level AI for Computer Games

John E. Laird

University of Michigan

[email protected]://ai.eecs.umich.edu/people/laird

Why I’m Interested in Video Games• AI is going to be more important in games

• A great environment for developing human-level AI

• Academic AI has something to offer the game industry

Applications of AI to Games• Synthetic Characters

• Sports games, FPS, RTS, …

• Dynamic Game Control• Modify play calling based on experience (coaching)• Directing characters for plot and story• Providing play-by-play, camera control, …

• Development of Games• Automatic testing

Human-Level AI & Games

• FPS Enemies: Quake, Unreal, Half-Life, ...

• FPS Partners: Spec Ops, Rainbow Six

• Adventure Characters: Blade Runner

• RPG Characters: Baldur’s Gate

• Massively Multiplayer Games: Asheron’s Call, Everquest

Human Behavior Modeling

Time Scales

Seconds

104 1 hour Game

103 10 min Level

102 1 min Engagement

101 10 sec Unit task

100 1 sec Operations

10-1 100 msec Deliberative Act

10-2 10 msec Neural Circuit

10-3 1 msec Neuron

10-4 100 sec Organelle

Rational Band

Cognitive Band

Neural Band

Human Behavior As A Sequence Of Deliberative Actions

• Consensus:• Human task-level behavior (> 100 milliseconds) can be

model as a sequence of deliberate actions, each taking approximately 50 milliseconds.

• Basic Perception takes > 100 milliseconds.• Simplest action (key press) takes 70 milliseconds.

• “Press a key when the light goes on.”Perception Thinking Action Total100 msec. + 50 msec. + 70 msec. = 220

msec.We found reaction time (shooting or sidestepping) to enemy in Quake II to be .5-2.5 seconds

Processing is Pipelined• Perception/Cognition/Motor can run in parallel

Perception

Cognition

Motor

Dual Task: Tracking/Choice

Vertical separation between tracking and CRT tasks (degrees)

0.0 3.2 6.4 9.6 12.8 16.0 19.2 22.4 25.6 28.8 32.0 35.2

RM

S T

rack

ing

Err

or

0

5

10

15

20

25

30

35

40

45

50

Observed EPICLockout (novice)Lockout + prepare

Vertical separation between tracking and CRT tasks (degrees)

0.0 3.2 6.4 9.6 12.8 16.0 19.2 22.4 25.6 28.8 32.0 35.2

RM

S T

rack

ing

Err

or

0

5

10

15

20

25

30

35

40

45

50

Observed EPICLockout + Promotions + Interleaving (expert)Interleaving + Promotions + Retrained (expert)

Cognitive Architectures• ACT-R (John Anderson, Carnegie Mellon)

• Detailed models of behavior and learning• 10 milliseconds-10 seconds.

• Used by cognitive psychologists• Issues on performance and scalabiltiy

• EPIC (David Kieras, University of Michigan)• Detailed models of human-computer interaction (no learning)

• Emphasizes models of perception and action• 50 milliseconds-10 seconds

• Used by cognitive psychologist

• Soar (John Laird, University of Michigan)• Models of human behavior and learning

• 50 milliseconds-1 hour• Emphasis more on complex behavior and learning

• Used by AI researchers and cognitive psychologist

Conflicting Goals for Human-level AI in Games

Goal Driven

Reactive

Human

CharacteristicsKnowledge

Intensive

Low CPU & Memory Usage

Fast & Easy Development

• Build an AI Engine around the Soar AI architecture• Soar/Quake II• Soar/Descent 3• Soar/Half-life

• Build autonomous, unscripted AIs

Soar/Games Project

InterfaceDLL

Quake II

AI Engine(Soar)

KnowledgeFiles

Actions

Sensor Data

Soar• AI engine to support multi-method problem solving

• Applied to wide variety of tasks and methods

• Proposed unified theory for modeling human cognition

• Models a wide variety of human behavior: language, HCI, ...

• Combines reactive and goal-directed symbolic processing

• Supports very large bodies of knowledge (>100,000 rules)

• Optimized implementation in ANSI C

• In the public domain

Soar Quakebot• Goal: Human-like behavior

• Fun and challenging to play against• Not super-human• Currently plays as well as good player

• Completely implemented in Soar• > 750 rules• Spread across operators for exploring, wandering, chasing, …

• Sensing similar to human

• Runs on separate PC from game• Uses ~5-10% of 400 MHz Win-95/98/NT machine

Execution Flow of an AI Engine

Sense

Think

Act

G

A

M

E

Simulated PerceptionSelf: orientation, position, healthSenses: vision, hearing, smell, taste, touchAugmented Senses: radar, night-vision, …> 80 sensors in Quakebot

Simulated ActionsMovement, weapons’ use, communication~20 actions in Quakebot

Obstacle

Field of Vision

Execution Flow of an AI Engine

Sense

Think

Act

G

A

M

E

?

Finite-state machines

Subsumption

Neural nets

Rule-based systems

C code

Planning systems

Rule-based Systems• Program = set of if-then rules, not sequential code.

• Use whatever knowledge that is relevant to current situation

Working Memorysensing, elaborations, persistent memories, goals, actions …

Rule MemoryTactics, Movement Knowledge, ...

[ ][ ] [ ][ ]

[ ]

[ ][ ][ ]

Match Changes

[ ] [ ][ ]

[ ] [ ][ ]

Rules• Conditions contain variables and other first-order tests

• If sense a weapon and do not have that weapon, pickup the weapon.(p rule1 (sense weapon ?weapon1) (self weapon <> ?weapon1) (action pickup ?weapon1))

• A single rule can have multiple instantiations• A single condition can match multiple working memory

elements

• Compilers convert rules into discrimination networks• Only test changes to working memory.• Share tests across similar rules• Efficient matching

Which Rule Should Fire?

• Matching finds which rules can fire in current situation.

• Conflict resolution picks which rules actually fire.• Based only on syntactic features of rules and data.• Must engineer rules so correct rule is selected each time.

RuleMemory

WorkingMemory

[ ] [ ] [ ] [ ][ ] [ ] [ ] [ ]

[ ] [ ]

[ ] [ ]

[ ] [ ]

[ ] [ ]

Rule Instantiations

?Conflict resolution

Match

Weaknesses in Rules• Conflict resolution based on syntactic features

• Not knowledge of the task

• Rules combine three types of knowledge about actions• What is possible• What should be done (based on conflict resolution)• How to do itLeads to duplication if have multiplicity in one of the types

• No hierarchical organization of knowledge and goals

Soar’s Approach• Organize knowledge as operators:

• Primitive & abstract actions• Turn, move, climb, goto-door, get-item, wander, attack, chase• Provides trace of behavior comparable to human decisions

• Use rules to propose, select, and apply operators• Rules fire in parallel - act as an associative memory• Provides fine-grain, situation-dependent behavior• Conditional selection and execution determined dynamically

Inpu

t

Elaboration(propose operators)

Decide(select operator)

Elaboration(apply operator)

Out

put

Inpu

t

Dec

ide

Out

put

Inpu

t

Dec

ide

Elaboration(terminate operator & propose)

Execution Flow of Soar Engine

Elaboration

Propose Operators

Evaluate Proposed Operators

Select One Operator

Perform Operator Actions

Sense

Think

Act

G

A

M

E

Soar

Get item

Go through door

Get item in room

Detect item missing

Face item Move to item

Stop moving to item

Exit room

Go to door

Align with door

Face door

Move to door

Record at door

Shoot Slide to door

Stop move to door

Stop slide to door

Soar Quakebot Top OperatorsWanderAttack ExploreRetreat Record

EnemyRemove Enemy

Chase


EnemyRemove Enemy

If enemy visible and my health is > very-low-health-value (20%) and his weapon is not much better than mineTHEN propose attack

Chase

If enemy visible and my health is < very-low-health-value (20%) or his weapon is much better than mineTHEN propose retreat

If no enemy visible or recorded and explored levelTHEN propose wander

If no enemy visible or recorded and not explored levelTHEN propose explore


EnemyRemove Enemy

If enemy just became not visible and no enemy visible or recordedTHEN propose record-enemy

Chase

If record-enemy selectedTHEN record enemy’s last position and time + 20 seconds.

If hear enemy and no enemy visible or recorded THEN propose record-enemy

If time > recorded enemy’s saved timeTHEN propose remove-enemy

If remove-enemy is selectedTHEN remove recorded enemy structure

If there is a recorded enemy structureTHEN propose chase

Example Rules

IF attacking an enemy and there is an object I need and that object is closer than attack-get-item-rangeTHEN propose get-item for that object

IF I see a weapon that is much better than any I haveTHEN I need that weapon

IF my health is less than low-health-value (40%)THEN I need any health object

IF two get-item operators are proposedTHEN prefer selecting the one for the closer object

Attack

Get-item Face-enemy Side-step Approach Shoot

Get item

Go through door

Get item in room

Detect item missing

Face item Move to item

Stop moving to item

Exit room

Go to door

Align with door

Face door

Move to door

Record at door

Shoot Slide to door

Stop move to door

Stop slide to door

Example Tactics• Collect-Powerups

• Pick up best weapons from spawn locations • Remember when missing items will respawn• Use shortest paths to get objects• Get health/armor if low on health/armor• Pickup up other good weapons/ammo if close by to deny enemy

• Attack • Use circle-strafe• Move to best distance for current weapon

• Chase enemy based on sound of running

• Ambush in corner that can’t be seen by enemy

• Hunt at nearest spawn room after killing enemy

Mapping• Need information on location of walls, doors, etc.

• Many tactics require this information

• Built up through exploration of a level• Similar to a robot exploring with range sensors• Saved and reused when return to level

• Represented internally as a graph structure• Currently restricted to 2D rectangles

Anticipation?Dennis (Thresh) Fong:

“Say my opponent walks into a room. I'm visualizing him walking

in, picking up the weapon. On his way out, I'm waiting at the

doorway and I fire a rocket two seconds before he even rounds the

corner. A lot of people rely strictly on aim, but everybody has their

bad aim days. So even if I'm having a bad day, I can still pull out a

win. That's why I've never lost a tournament.”Newsweek, 11/21/99

Anticipation• Predicts enemy’s actions

• Creates internal model of enemy’s state

• Uses own knowledge to predict what enemy would do• Can be modulated by knowledge of enemy’s tactics

• Stops planning when it knows what to do or uncertainty in actions

• Uses prediction to select actions: • Ambush, Deny-Powerups

His Distance: 1

My Distance: 1

His Distance: 2

My Distance: 2

His Distance: 3

My Distance: 1 (but hall)

His Distance: 4

My Distance: 0 Ambush!

References• The Psychology of Human-Computer Interaction,

Card, Moran, and Newell, LEA, 1983.

• Unified Theories of Cognition, Newell, Harvard Press, 1990.

• Modeling Human and Organizational Behavior: Applications to Military Simulations, National Research Council, National Academy Press, 1998.

• Soar Homepage, http://ai.eecs.umich.edu/soar

human-level ai for computer games

Documents