Turn-Based Games

Héctor Muñoz-Avila

sources: •http://www.game-research.com/•www.gamespot.com•Wikipedia.org•Russell & Norvig AI Book; Chapter 5 (and slides)•My own

Turn-Based Strategy GamesEarly strategy games was dominated by turn-based games

•Derivate from board games•Chess•The Battle for Normandy (1982)•Nato Division Commanders (1985)

Turn-based strategy:•game flow is partitioned in turns or rounds. • Turns separate analysis by the player from actions•“harvest, build, destroy” in turns• Two classes:

•Simultaneous•Mini-turns

Turn-Based Games Continues to be A Popular Game Genre

• At least 3 sub-styles are very popular:– “Civilization”-style games

• Civilization IV came out last week

– Fantasy-style (RPG)• Heroes of Might and Magic series

– Poker games• Poker Academy

Some Historical Highlights

• 1952 Turing design a chess algorithm. Around the same time Claude Shannon also develop a chess program

• 1956 Maniac versus Human• 1970 Hamurabi. A game about building an economy for a

kingdom• The Battle for Normandy (1982)• 1987 Pirates!• 1990 Civilization• 1995 HoMM• 1996 Civilization II

• The best game ever?• …• 2005 Civilization IV• 2006 HoMM V

Side-tracking: Game Design: Contradicting Principles

• Principle: All actions can be done from a single screen.

• Classical example: Civ IV

• But: HoMM uses two interfaces: HoMM IV

Coming back: How to Construct Good AI?

• Idea: Lets just use A* and define a good heuristic for the game

Search space: a bipartite treeAfter all didn’t we use it with the 9-puzzle game?

• Problems with this idea: Adversarial: we need to consider possible moves of our opponent (s)

Time limit: (think Chess)

Types of AdversarialTBGs (from AI perspective)

Perfect information

Imperfect information

Deterministic Chance

Chess, Go, rock-paper-scissors

Battleships, Stratego

Backgammon, monopoly

Civilization, HoMMBridge, Poker

Game tree (2-player, deterministic, turns)

Concepts:•State: node in search space•Operator: valid move•Terminal test: game over•Utility function: value for outcome of the game•MAX: 1st player, maximizing its own utility•MIN: 2nd player, minimizing Max’s utility

Minimax

• Finding perfect play for deterministic games• Idea: choose move to position with highest minimax

value = best achievable payoff against best play

• E.g., 2-play game:

Minimax algorithm

Properties of minimax

• Complete? • Optimal? • Time complexity?

– b: branching factor– m: # moves in a game

Yes (if tree is finite)Yes (against an optimal opponent)

O(bm)

•For chess, b ≈ 35, m ≈100 for "reasonable" gamesTherefore, exact solution is infeasible

Minimax algorithm with Imperfect Decisions

evaluationFunction(state)

evaluationFunction(state)

Cutoff-test(state)

Cutoff-test(state)

Evaluation Function

• Evaluation Function– Is an estimate of the actual utility– Typically represented as a linear function:

EF(state) = w1f1(state) + w2f2(state) + … + wnfn(state) – Example:

Chess• weight: Piece Number

(w1) Pawn 1 (w2) Knight 3(w3) Bishop 3(w4) Rook 5 (w5) Queen 9

•Function; state Numberf1 = #(pawns,b) #(pawns,w)f2 = #(knight,b) #(knight,w)f3 = #(bishop,b) #(bishop,w)f4 = #(rook,b) #(rook,w)f5 = #(knight,b) #(knight,w)

Evaluation Function (2)

• Obviously, the quality of the AI player depends on the evaluation function

• Conditions for evaluation functions:If n is a terminal node, Computing EF should not take longEF should reflect chances of winning

EF(n) = Utility(n)

If EF(state) > 3 then is almost-certain that blacks win

Cutting Off Search

α-β pruning example

α-β pruning example

α-β pruning example

α-β pruning example

α-β pruning example

Properties of α-β

• Pruning does not affect final result

• Good move ordering improves effectiveness of pruning

• With "perfect ordering," time complexity = O(bm/2) doubles depth of search

• A simple example of the value of reasoning about which computations are relevant (a form of metareasoning)

•••

Why is it called α-β?

• α is the value of the best (i.e., highest-value) choice found so far at any choice point along the path for max

• If v is worse than α, max will avoid it prune that branch

• Define β similarly for min•–

••

The α-β algorithm

The α-β algorithm

Resource limits

Suppose we have 100 secs, explore 104 nodes/sec 106 nodes per move

Standard approach:• cutoff test:

e.g., depth limit (perhaps add quiescence search)• evaluation function

= estimated desirability of position––

•

Evaluation functions

• For chess, typically linear weighted sum of featuresEval(s) = w1 f1(s) + w2 f2(s) + … + wn fn(s)

• e.g., w1 = 9 with

f1(s) = (number of white queens) – (number of black queens), etc.

•

Cutting off search

MinimaxCutoff is identical to MinimaxValue except1. Terminal? is replaced by Cutoff?2. Utility is replaced by Eval

Does it work in practice?bm = 106, b=35 m=4

4-ply lookahead is a hopeless chess player!– 4-ply ≈ human novice– 8-ply ≈ typical PC, human master– 12-ply ≈ Deep Blue, Kasparov–

•

1.

Deterministic games in practice

• Checkers: Chinook ended 40-year-reign of human world champion Marion Tinsley in 1994. Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions.

»• Chess: Deep Blue defeated human world champion Garry Kasparov in a six-game match

in 1997. Deep Blue searches 200 million positions per second, uses very sophisticated evaluation, and undisclosed methods for extending some lines of search up to 40 ply.

• Othello: human champions refuse to compete against computers, who are too good.

• Go: human champions refuse to compete against computers, who are too bad. In go, b > 300, so most programs use pattern knowledge bases to suggest plausible moves.

•••

»

Summary

• Games are fun to work on!• They illustrate several important points about AI• perfection is unattainable must approximate• good idea to think about what to think about••