probabilistic smart terrain dr. john r. sullins youngstown state university

Probabilistic Smart Terrain

Dr. John R. SullinsYoungstown State University

John Sullins Youngstown State University

Probabilistic Smart Terrain ICTAI 2009

2

Outline

• What is Smart Terrain?

• Why do we need to add probabilities?

• Estimating expected distances to objects that meet character needs

• Plausibility benchmarks and experimental results

• Adding learned knowledge during exploration

• Hierarchical application to games



3

Smart Terrain

• Solves complex navigation problems in real time



4

Smart Terrain

• Characters have “needs”– Example: hunger

• Objects in world meet needs– Example: refrigerator with food inside

• Characters move towards objects that meet needs



5

Smart Terrain

• Objects meets needs transmits “signal”– Signal weakens with distance– Signal moves around objects



6

Smart Terrain

• Characters follow signal to objects– Move in direction of

increasing signal

– Only need to compute map once when level created

– No need for complex navigation!



7

Need for Probabilities

• Smart terrain can result in implausible actions

– Room character has never visited– Contains empty refrigerator

• Does not transmit signal• Character ignores it

– Not plausible behavior!



8


• Objects broadcast signal of form“I meet need n”

“I may meet need n with probability P ”

• Probability = uncertainty that object meets need

• Character might explore uncertain objects along path



9


• Theoretical goal:Move to closest object with highest probability

• Problem: Optimizing two separate criteria!

• Actual Goal: Plausible behavior for characters

Meets “hunger”

need with P = 0.7

At distance 8

Meets “hunger” need with P = 0.6

At distance 6



10

Expected Distances

• Expected number of tiles character must travel to reach object that fulfills need

• Use to determine which tile to move to next– Compute expected distance for four surrounding tiles– Move to surrounding tile with lowest value for

expected number of tiles to travel



11

Expected Distances

P(t): probability no objects within t tiles meet need

P(t) = (1 – pi ) (Equation 1)

where di < t

• Based on:– di: distances to each object i– pi: probabilities each object i meets need– Assumption of conditional independence



12

Expected Distances

• t < 6: P(t) = 1• 6 ≤ t < 8: P(t) = (1 – 0.6) = 0.4• t ≥ 8: P(t) = (1 – 0.6)(1 – 0.7) = 0.12

Distance: 6 Prob: 0.6

Distance: 8Prob: 0.7



13

Expected Distances

Expected distance from tile T to tile that meets need

E(T) = Σ P(t) (Equation 2) t

t < 6: P(t) = 16 ≤ t < 8: P(t) = 0.4

t ≥ 8: P(t) =0.12



14

Expected Distances

• Problem: Sum could be infinite• Solution: Limit t to some tmax tmax > di i

tmax

E(T) = Σ P(t) (Equation 3)

t



15

Expected Distances

• Compute expected distance E(T) for all tiles T• Character moves to adjacent tile with lowest E(T)



16

Plausibility Benchmarks

• Goal for games:Non-player characters should behave plausibly– Move in direction that “makes sense” to player

• Benchmarks for plausible behavior:– Objects similar in either distance or probability – Group of objects in same direction– Objects that meet need with complete certainty



17

Plausibility Benchmarks• Objects at same distance move to higher probability

• Objects with same probability move to closer one



18

Plausibility Benchmarks• Nearly same distance move to much higher probability

• Nearly same probability move to much closer object



19


Aggregate probabilities benchmark:• Multiple objects > single object with higher probability

– Assumption of conditional independence



20


Complete Certainty benchmark:• Single object with probability = 1 >

multiple objects with probability < 1



21

Learned Knowledge

• Probabilities changed when object reached– Object meets need probability becomes 1– Does not meet need probability becomes 0

• Should affect future actions

Refrigerator empty

Move towards another goal



22

Learned Knowledge• Changing global map affects all characters

– Will also appear to have learned this knowledge

New character enters room

Also ignores empty refrigerator



23

Learned Knowledge

• Each character stores own world model– Belief object meets needs– Initially based on probabilities– Modified when objects explored

Refrigerator R1 70%Refrigerator R2 80%

Object Belief object meets need

0%



24

Learned Knowledge

• Each object propagates raw data to tiles– Probability it meets need– Distance to that tile



25

Learned Knowledge• Character examines surrounding tiles

– Modify probabilities using world model

– Compute expected distances for each



26

Hierarchical Smart Terrain

• More realistic scenario:– Know whether objects meet needs– Don’t know if object is present in given area

• Go to entrance of most likely area• If object present, move to it.• Otherwise, move to another area



27


• “Area attractors” at entrances to rooms– Broadcast to entire level– Probability object that meets need is in room– Probability set to 0 when reached by character

• Objects in room– Signal range = size of room– Probability = 1 if present in room



28


• Compute expected distances from area attractors• Move to “best” room



29


• Object is present in area:– Now in range of object, probability meets need = 1– Character will move directly to object



30


• Object is not present in area:– Set probability of area attractor = 0– Character will move to next plausible attractor



31

Conclusions

• Probabilities added to Smart Terrain algorithm

• Characters move to adjacent tile with shortest expected distance to a tile that meets need

• Algorithm produces plausible behavior for benchmarks

• Probabilities overridden by learned knowledge

• Hierarchical algorithm for realistic play



32

Ongoing Work

• Algorithm modification to avoid local minima

• Characters with multiple needs at different levels– Low-probability object that meets critical need– High-probability object that meets less critical need– Which to move towards?

• Objects that change over time– Empty refrigerator now may be restocked in future



33

Local Minima

Caused when paths to low probability objects overlap

Overlap in paths to P=0.23 objects

Tiles nearer to object appear farther away



34

Local Minima

Solution: Weight estimated tiles by distance tmax

E(T) = Σ P(t) k t t

• Nearby objects “appear” even closer

• Any weight k > 0 seemsto work



35

Multiple Needs

• Characters can have multiple needs– Hunger, Fun

• Some needs more critical than others– Hunger = 10 Fun = 5

• Objects may only partially fulfill needs– Donuts: Hunger-7– Cookies: Hunger-3– TV: Fun-6

• Needs increase over time (each tile traversed)– Hunger += 0.5 per tile Fun += 0.2 per tile



36

Multiple Needs

• Goal: Minimize total “discontentment”

Σ (needj)2

j

• Problem: Balancing different factors



37

Multiple Needs• Terminology:

– n j = current level of need j

– di j = distance to object i

– pi j = probability object i meets need j

– ai j = amount that need j decreased by if it meets need)

– c j = increase in need j for each tile traversed

• Expected decrease in need j caused by all objects i = Σ pi

j ai

j within t tiles i



38

Multiple Needs

• Expected level of need j if move t tiles:

max (0, n j + tc j

- Σ pi j ai

j ) where di < t

i

• Expected discontentment if move t tiles:

Σ (max (0, n j + tc j

- Σ pi j ai

j ))2 where di < t j i



39

Multiple Needs

• Total expected discontentment at given tile:

tmax

Σ Σ (max (0, n j + tc j

- Σ pi j ai

j ))2 t j i

• Compute for surrounding tiles• Move to tile with lowest expected discontentment

probabilistic smart terrain dr. john r. sullins youngstown state university

Documents