cs510 ai and games

CS510 AI and Games

Final Report on Dec. 09 2008

Juncao Li

juncao@cs.pdx.edu

Computer Science, Portland State University 2

Agenda• About Advanced Protection• My Design and Work• Results and Evaluations• Conclusion and Discussion

juncao@cs.pdx.edu

Advanced Protection (AP)• Author: Soren Johnson• Turn-based strategy game

– Player makes strategy each turn– AI plays against player’s strategy

• Static AI V.S. adaptive AI – Player learns the strategy of static AIs– Adaptive AI adapts based on users performance

juncao@cs.pdx.edu

AP (Cont.)

juncao@cs.pdx.edu

AP (Cont.)Figure A-1: Terrain Types

Terrain Color Farming Bonus Defense Bonus Impassable?Water

B l u e 0 0 Y

W e t l a n d s L i g h t G r e e n 2 0 N

F o r e s t D a r k G r e e n 1 0 N

H i l l s B r o w n 0 1 N

M o u n t a i n s G r a y - 1 2 N

Figure A-2: Human Units

Unit Cost SalvageAttack

StrengthDefense Strength

Farming Rate

Special Ability

D r o n e $ 2 0 $ 2 0 0 1 1 -

M i n e $ 5 0 $ 2 5 0 0 0 ( 1 )

F a r m e r $ 1 0 0 $ 8 0 0 1 1 -

I n f a n t r y $ 1 0 0 $ 9 0 3 6 0 -

S e t t l e r $ 1 5 0 $ 1 2 0 2 4 3 -

J a m m e r $ 1 5 0 $ 9 0 0 2 0 ( 2 )

A r m o r $ 2 0 0 $ 1 5 0 8 1 0 0 -

A r t i l l e r y $ 4 0 0 $ 3 0 0 3 2 0 ( 3 )

juncao@cs.pdx.edu

AP (Cont.)Figure A-3: Chaos's Minions

Unit CostAttack

StrengthDefense Strength

Movement Rate

Hearing Radius

Special Ability

Scout$ 1 0 0 1 1 - ( 4 )

S c a v e n g e r $ 1 0 0 0 1 1 - ( 5 )

C o m m o n B a r b a r i a n $ 5 0 6 3 1 2 -

A m p h i b i o u s B a r b a r i a n $ 6 0 6 3 1 2 ( 6 )

M o b i l e B a r b a r i a n $ 7 0 5 3 2 3 -

J u m p B a r b a r i a n $ 7 0 6 2 1 2 ( 7 )

K a m i k a z e e B a r b a r i a n $ 7 0 0 1 0 2 2 ( 8 )

A r m o r e d B a r b a r i a n $ 2 0 0 1 5 1 0 1 3 ( 9 )

juncao@cs.pdx.edu

AP (Cont.)• Features

– Each minion has a brain (automaton) encoded by a 128-bit string

– Four behaviors depending on the input:• Move forward, turn right, turn left and do actions

– 250 brains for a minion to choose• 20 hardcoded, 230 from genetic algorithm• During the play, brains are rated based on their

performance against the player– Dynamically choose best-fit brains

• Each player has their own easy and hard minion brains • Players does not loose or win too much

juncao@cs.pdx.edu

Minion InputsFigure 3: Scout InputsInput Meaning000

N o i n p u t

0 0 1 F a c i n g i m p a s s a b l e t e r r a i n

0 1 0 S e n s e s j a m m i n g r a d i u s

0 1 1 S e n s e s a r t i l l e r y r a d i u s

1 0 0 F a c i n g d r o n e o r f a r m e r

1 0 1 F a c i n g i n f a n t r y , s e t t l e r , j a m m e r , o r a r t i l l e r y

1 1 0 F a c i n g a r m o r

1 1 1 F a c i n g m i n e

Figure 4: Scavenger InputsInput Meaning000

F a c i n g h u m a n

0 1 0 S e n s e g o l d

0 1 1 F a c i n g g o l d

1 0 0 N o f a r m i n g b o n u s

1 0 1 + 1 f a r m i n g b o n u s

1 1 0 + 2 f a r m i n g b o n u s

1 1 1 + 3 f a r m i n g b o n u s

Figure 5: Barbarian InputsInput Meaning000

N o i n p u t

0 1 0 S e n s e s s c o u t b r o a d c a s t

0 1 1 F a c i n g s c o u t b r o a d c a s t

1 0 0 F a c i n g d r o n e o r f a r m e r

1 0 1 F a c i n g i n f a n t r y o r s e t t l e r

1 1 0 F a c i n g a r m o r

1 1 1 F a c i n g j a m m e r o r a r t i l l e r y

juncao@cs.pdx.edu

Why Brain Matters

juncao@cs.pdx.edu

Why Brain Matters (Cont.)

juncao@cs.pdx.edu

Why Brain Matters (Cont.)

juncao@cs.pdx.edu

My Goal & Why• Design AIs that play as players

– Try to learn the player’s strategy– Try to perform best on all 250 brains

• Because Chaos selects brains based on player’s strategy

• Win (defined later) brains as much as it could• Design benchmark for the AIs on each side

– How well each AI performs• Learning about players helps game designs

– Player will not like a game that they can never win!

– Player will not like a game that they can easily win!

juncao@cs.pdx.edu

Finite State Machine (FSM)• Code the game strategy in a FSM• Inputs or what matters

– Treasury: Human and Chaos– Terrain and current human nodes on the map

• Output– Where to place the units– What units to place– Based on current state

• Allocate more farmers for earning money• Allocate more infantry for attacking Chaos

juncao@cs.pdx.edu

FSM (Cont.)

juncao@cs.pdx.edu

FSM (Cont.)

juncao@cs.pdx.edu

FSM (Cont.)

juncao@cs.pdx.edu

FSM (Cont.)• My FSM is static• Hard to make it dynamic and adaptive

– Too many possibilities (24*24 size map plus human ……)

• Works well as a benchmark– How well the Chaos AI performs

• Train Neural Network (NN)– Hopefully, teach the NN to recognize the map

juncao@cs.pdx.edu

Use Neural Network (NN)• Input nodes: MAP_SIZE+2

– Map size: 24*24 = 576– Human+Chaos treasury: 1+1

• Output nodes: MAP_SIZE– Human units placement

• One hidden layer: MAP_SIZE+2• Normalize the inputs/outputs range: [0.0,

1.0]• Use other’s NN code:

– AI Game Engine: well developed, but not efficient– Tim Jones book: simple and efficient NN

juncao@cs.pdx.edu

Train the NN• Firstly, use the static FSM to train the NN

– Random generated maps– Random initial Human/Chaos treasury – FSM generates outputs as the training case

• Secondly, improve the NN– Randomly generate strategy– If it performs better than current NN– Train NN with the strategy

juncao@cs.pdx.edu

Understand the NN Outputs• NN doesn’t always tell the exact answer• So we have to “guess”

Human_T MatchHuman(float type){float threashold = 0.2;type = type*HUMAN_TYPES;if(type<=threashold || type>HUMAN_TYPES+threashold)

return NO_HUMAN;

if(fabs(type-float(DRONE))<threashold)return DRONE;

else if(fabs(type-float(MINE))<threashold)return MINE;

else if(fabs(type-float(FARMER))<threashold)return FARMER;

else if(fabs(type-float(INFANTRY))<threashold)return INFANTRY;

else if(fabs(type-float(SETTLER))<threashold)return SETTLER;

else if(fabs(type-float(ARMOR))<threashold)return ARMOR;

else if(fabs(type-float(JAMMER))<threashold)return JAMMER;

else if(fabs(type-float(ARTILLERY))<threashold)return ARTILLERY;

return NO_HUMAN;}

CNNPlayer::DoTurns(){ ……//normalize the outputfloat low = actual[0];float high = actual[0];float avg = 0;for(i=0; i<OUTPUT_NEURONS; i++){

if(low>actual[i])low=actual[i];

if(high<actual[i])high=actual[i];

avg+=actual[i]/OUTPUT_NEURONS;}

for(i=0; i<OUTPUT_NEURONS; i++){if(actual[i]<=0)

actual[i] = 0;else{

actual[i] = (actual[i]-low)/(high-low);//actual[i] = actual[i]/avg;

}}……}

juncao@cs.pdx.edu

NN Outputs (4 hour & 1M inter.)

juncao@cs.pdx.edu

Recall the FSM

juncao@cs.pdx.edu

A Random NN Give You This!

juncao@cs.pdx.edu

Evaluations• Test based on each turn

– Only for each turn, the Chaos’ brain is certain– My AI performs well (win) if and only if:

• Given random initial treasury and map• It has advantage against most Chaos’ brains (money

earned) • Good performance on each turn leads to

final win

juncao@cs.pdx.edu

Statistics of My FSM

Win out of 250 brains

Initial Treasury

We want this!

Not well dealing with money between this range

juncao@cs.pdx.edu

Statistics of My NN AI

Win out of 250 brains

Initial Treasury

We want this!

juncao@cs.pdx.edu

Conclusion• Studied AP

– Adaptive turn-based strategy game • Static FSM

– Benchmark– Train NN

• NN– Developed a demo – Trained by FSM– Need more time to train it

juncao@cs.pdx.edu

Discussion• Static FSM

– Efficient to code– Pros: do exactly what you want– Cons: do exactly what you want

• NN– Like dealing with a child

• Patient• Be careful what you feed to it• Try to understand a child’s language• More trained, usually better performance

juncao@cs.pdx.edu

Discussion (Cont.)• Coding & debugging issues

– Error in coding NN is not easy to be detected• Wrong training data• Inconsistent between training inputs and using inputs

– Be careful about important parts• Training data• Training process

– Set breakpoint to check– Code review– Train it by the same example, see if it adapts as

expected

juncao@cs.pdx.edu

Questions ?

cs510 ai and games

portland state university4ap

portland state university5ap

portland state university6ap

portland state university15fsm

portland state university16fsm

portland state university17fsm

players strategystatic

game strategy

Documents

cs510 ai and games

introduzione ai microarray games

sports games ai

module d'approfondissement : ai for games and simulation

ai games spring12

ai for games

practical ai in games

cs510 concurrent systems class 2

csc3203: ai for games informed search (1)

human-level ai for computer games

ai innovation games on the semantic internet

cs510 concurrent systems

cs510 concurrent systems introduction to concurrency

tactical ai in real time strategy games

lecture 1 ai for games movement behaviors -...

cs510 concurrent systems jonathan walpole

ai in flash games

cs510 ai and games final report on dec. 09 2008 juncao li

adaptive ai engine for rts games

ai in games