3 states and a plan: the ai of f.e.a.r. jeff orkin monolith productions/ mit media lab
TRANSCRIPT
3 States and a Plan:The AI of F.E.A.R.
Jeff Orkin
Monolith Productions/
MIT Media Lab
Survey Says!
Survey Says!
FSM: 3 States
FSM: 3 States
Goto Animate
UseSmartObject
FSM: 3 States
Goto Animate
Animate
FSM: 2 States!
Goto Animate
FSM: Transition Logic
void StateAttack::Update(){
//...
if( Ammo == 0 ){
pState = Reload(bCrouch);return;
}
//...}
Shogo, 1998
No One Lives Forever, 2000
F.E.A.R., 2005
Halo 2, 2004Allowable behaviors for infantry, drivers, and passengers
FSM vs Planning
FSM Planning
- How - What
FSM vs Planning
FSM Planning
- How - What
- Procedural - Declarative
Motivation
Here’s the Plan:
• STRIPS Planning Overview
• Planning in F.E.A.R.
• Differences from STRIPS
• Squad Behaviors & Communication
• Beyond F.E.A.R.
What is Planning?
• Planning is a formalized process of searching for sequence of actions to satisfy a goal.
• Process is called “Plan Formulation.”
STRIPS Planning
…in a nutshell
STRIPS Planning
STRIPS =
STanford Research Institute
Problem Solver
STRIPS Planning
• STRIPS Goal: Desired state of the world to reach.
• STRIPS Actions:– Preconditions– Effects
States
States: FSM
Attack Search
States: PlanningRepresented as a logical sentence:
AtLocation(Home) ^ Wearing(Tie)
Represented as a vector:
(AtLocation, Wearing) = (Home, Tie)
States: PlanningExample: Lemonade Stand
( weather, #lemons, $$ )=
( , , ) or
( , , )
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
State: (phone#, recipe, hungry?)
Goal: ( -- , -- , NO )
( , -- ,YES) ( , -- ,NO )
( -- , ,YES) ( -- , ,NO )
STRIPS Planning Example
State: (phone#, recipe, hungry?)Goal: ( -- , -- , NO )
( , ,NO )
( , ,YES) ( , ,NO )
STRIPS Planning Example
STRIPS Planning Example
State: (phone#, recipe, hungry?)
ActionPreconditions: ( , -- , -- )
Effects:
Delete List: Hungry( YES )
Add List: Hungry( NO )
STRIPS Planning Example
State: (phone#, recipe, hungry?)
ActionPreconditions: ( , -- , -- )
Effects:
Hungry( NO )
Hungry( YES ) ^ Hungry( NO )
STRIPS Planning Example
Action: Buy( object )Preconditions: ( -- )
Effects:
Delete List: --
Add List: Own( object )
Own( ) ^ Own( ) ^ Own( )
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
STRIPS Planning Example
FPS Statistics
Average lifespan of FPS enemy AI:
FPS Statistics
Average lifespan of FPS enemy AI:
12.23 seconds
FPS Statistics
Average lifespan of FPS enemy AI:
12.23 seconds
Number of enemy AI who die in
FPS’s per day:
FPS Statistics
Average lifespan of FPS enemy AI:
12.23 seconds
Number of enemy AI who die in
FPS’s per day:
19,789,203
FPS Statistics
Sally Struthers says “Save the AI!”
Here’s the Plan:
• STRIPS Planning Overview
• Planning in F.E.A.R.
• Differences from STRIPS
• Squad Behaviors & Communication
• Beyond F.E.A.R.
Design Philosophy
Designer’s job is:
Create environments that allow AI to showcase their behaviors.
Designer’s job is NOT:
Script behavior of individual AI.
Planning Video #1
Planning Video #1
Planning Video #1
Planning Video #1
Planning Video #1
Planning Video #1
Planning Video #2
Planning Video #3
Planning Video #4
Planning Video #4Soldier Assassin Rat
Benefits of Planning
1. Decoupling Goals & Actions
2. Layering Behaviors
3. Dynamic Problem Solving
Decoupling Goals & Actions
Decoupling Goals & Actions
AttackFromCover
tt
t
t
t
Priority: 8 .0Sensors: V is ion, CoverNodes
Patrol
tt
t
t
t
Priority: 1 .0Sensors: PatrolNodes
Investigate
tt
t
t
t
Priority: 3 .0Sensors: HearDisturbance
Search
tt
t
t
t
Priority: 6 .0Sensors: V is ion, SearchNodes
Decoupling Goals & Actions
Decoupling Goals & Actions
+ =
Decoupling Goals & Actions
AttackFromCover
tt
t
t
t
Priority: 8 .0Sensors: V is ion, CoverNodes
Patrol
tt
t
t
t
Priority: 1 .0Sensors: PatrolNodes
Investigate
tt
t
t
t
Priority: 3 .0Sensors: HearDisturbance
Search
tt
t
t
t
Priority: 6 .0Sensors: V is ion, SearchNodes
Decoupling Goals & Actions
Decoupling Goals & Actions
Decoupling Goals & Actions
Work
tt
t
t
t
Priority: 8 .0
Death
tt
t
t
t
Priority: 100.0
Stunned
tt
t
t
t
Priority: 80.0
Decoupling Goals & ActionsWork
RecoilStand
Up
SitDown
Anim ate
Goto
Priority: 8 .0
Death
Priority: 100.0
Stunned
Priority: 80.0
Working Memory
Benefits of Planning
1. Decoupling Goals & Actions
2. Layering Behaviors
3. Dynamic Problem Solving
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Layering Behaviors
Kill Enemy
Attack Ranged
Layering Behaviors
Kill Enemy
Attack Ranged
Attack Melee
Layering Behaviors
Kill Enemy
Attack Ranged
Attack Melee
Dodge Shuffle
Dodge Roll
Dodge
Layering Behaviors
Kill Enemy
Attack Ranged
Attack Melee
Dodge Shuffle
Dodge Roll
Dodge
Cover
Goto
Dodge Covered
Layering Behaviors
Kill Enemy
Attack Ranged
Attack Melee
Dodge Shuffle
Dodge Roll
Dodge
Cover
Goto
Dodge Covered
Blind Fire
Layering Behaviors
Kill Enemy
Attack Ranged
Attack Melee
Dodge Shuffle
Dodge Roll
Dodge
Cover
Goto
Dodge Covered
Blind Fire
Ambush
Layering Behaviors
Kill Enemy
Attack Ranged
Attack Melee
Dodge Shuffle
Dodge Roll
Dodge
Cover
Goto
Dodge Covered
Blind Fire
Ambush
Layering Behaviors
Benefits of Planning
1. Decoupling Goals & Actions
2. Layering Behaviors
3. Dynamic Problem Solving
Dynamic Problem Solving
Dynamic Problem Solving
Differences From STRIPS
1. Cost Per Action
2. No Add/Delete Lists
3. Procedural Preconditions
4. Procedural Effects
Cost Per Action
Cost Per Action
2 8
Cost Per Action
2 8
*A
Cost Per Action
A
Cost Per Action
A* Navigation Planning
Nodes: NavMesh Polys World States
Edges: NavMesh Poly Actions
Edges
Goal: NavMesh Poly World State
Cost Per Action
A
Differences From STRIPS
1. Cost Per Action
2. No Add/Delete Lists
3. Procedural Preconditions
4. Procedural Effects
No Add/Delete Lists
State: (phone#, recipe, hungry?)
ActionPreconditions: ( , -- , -- )
Effects:
Delete List: Hungry( YES )
Add List: Hungry( NO )
No Add/Delete Lists
State: [phone#, recipe, hungry?]
ActionPreconditions: [ , -- , -- ]
Effects: [ -- , -- , NO ]
No Add/Delete Lists
AttackPreconditions: [ -- , -- , ]
ReloadEffects: [ -- , -- , ]
No Add/Delete Lists
[ -- , -- , -- ] 4-byte values
TargetDead [bool]WeaponLoaded [bool]OnVehicleType [enum]AtNode [HANDLE] - or -AtNode [variable*]
No Add/Delete Lists
Goal Cover: [ -- , CoverNode63, -- ]
Action GotoEffect: [ -- , variable*, -- ]
No Add/Delete Lists
[ -- , -- , ]
No Add/Delete Lists
[ -- , -- , ]
Differences From STRIPS
1. Cost Per Action
2. No Add/Delete Lists
3. Procedural Preconditions
4. Procedural Effects
Procedural Preconditions
class Action{
// [ -- , -- , -- ]WORLD_STATE m_Preconditions;WORLD_STATE m_Effects;
bool CheckPreconditions();
};
Procedural Preconditions
Differences From STRIPS
1. Cost Per Action
2. No Add/Delete Lists
3. Procedural Preconditions
4. Procedural Effects
Procedural Effects
class Action{
// [ -- , -- , -- ]WORLD_STATE m_Preconditions;WORLD_STATE m_Effects;
bool CheckPreconditions(); void ActivateAction();
};
Procedural Effects
void ActionFlee::ActivateAction()
{
// ...
}
Goto Animate
<x,y,z>
Squad Behaviors
Global Coordinator
Global Coordinator
Squad 1
Squad 2
Squad 3
Squad Behaviors
1. Simple
2. Complex
Squad Behaviors
1. Simple– Suppression fire– Goto positions– Follow leader
2. Complex– Flanking– Coordinated strikes– Retreats– Reinforcements
Simple Squad Behaviors
• Get-to-Cover
• Advance-Cover
• Orderly-Advance
• Search
Simple Squad Behaviors
1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Get-to-Cover
1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player
Get-to-Cover
1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player
Get-to-Cover
1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player
Get-to-Cover
1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player
Get-to-Cover
1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player
Get-to-Cover
1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player
Get-to-Cover
1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player
Get-to-Cover
1. Find participants.
2. Send orders.
3. Monitor progress.
4. Succeed or fail.
Player
Squad Behaviors
1. Simple– Suppression fire– Goto positions– Follow leader
2. Complex– Flanking– Coordinated strikes– Retreats– Reinforcements
Complex Squad Behaviors
• There are none!!
Complex Squad Behaviors
Player
Complex Squad Behaviors
Player
Complex Squad Behaviors
Player
Squad Behavior Implementation
• Key is separation between squad layer and individual behaviors.
• Possibly formalize with Hierarchical Task Network (HTN) planning.
Squad Communication
“I need reinforcements!”
Squad Communication
Dialogue, instead of announcements: – Announcement: “Ouch!”– Dialogue:
• AI1: “What’s your status?”• AI2: “I’m hit!” (or “I’m alright!”)
Squad Communication
Dialogue, instead of announcements: – Announcement: “Where did he go?”– Dialogue:
• AI1: “Can you see anything?”• AI2: “No sir!” (or “He’s behind
the boxes!”)
Squad Communication
Dialogue also explains lack of action: – Dialogue:
• AI1: “He’s aiming at you!
Get out of there!”• AI2: “I’ve got nowhere to go!”
Squad Communication
“…Not only do they give each other orders, but they actually DO what they’re told!”
Squad Communication
“…Not only do they give each other orders, but they actually DO what they’re told!”
Reality:• Squad behavior has bird’s eye view of the
situation.• Find appropriate dialogue sequence after
the fact.
Here’s the Plan:
• STRIPS Planning Overview
• Planning in F.E.A.R.
• Differences from STRIPS
• Squad Behaviors & Communication
• Beyond F.E.A.R.
“… just like the marines in Half-Life 1”
“… just like the marines in Half-Life 1”
1998 2005
MIT Media Lab:Cognitive Machines Group
http://www.media.mit.edu/cogmac
Dialogue Integration
“What’s your status?”
Unified Planning for Actions and Speech
“Look out! Grenade!”Precondition:
Effect:
Unified Planning for Actions and Speech
OpenDoor -or- “Open the door!”Precondition:
door is closed
Effect:
door is open
Plan Recognition
UnlockDoor
CreateKey
LightFire GotoForge
GetGold
Plan Recognition
Plan as Context Free Grammar (CFG):
UnlockDoor CreateKey
LightFire GotoForge
GetGold
CreateKey
GotoForge :
:
:
Plan Recognition
PLAN RECOGNITION VIDEO
Questions?Jeff Orkin
Monolith is hiring!•Engineers•Artists•Designers
http://www.lith.com/jobs