game and tree searching

10/19/2004 TCSS435A Isabelle Bichindaritz 1

Game and Tree Searching

10/19/2004 TCSS435A Isabelle Bichindaritz 2

Learning Objectives

• Introduction to games

• Optimal decisions in games

• Alpha-Beta pruning

• Real-time decisions

10/19/2004 TCSS435A Isabelle Bichindaritz 3

Introduction to Games• Games as Search Problems

– Frameworks: two-player, multi-player; zero-sum; perfect information

– Minimax algorithm• Perfect decisions• Imperfect decisions (based upon static evaluation function)

– Issues• Quiescence• Horizon effect

– Need for pruning (alpha-beta pruning)

10/19/2004 TCSS435A Isabelle Bichindaritz 4

Introduction to Games• Perfect Play

– General framework(s)– What could agent do with perfect info?

• Resource Limits– Search ply– Static evaluation: from heuristic search to heuristic game tree search

– Examples• Tic-tac-toe, connect four, checkers, connect-five • Chess, go

• Games with Uncertainty– Explicit: games of chance (e.g., backgammon, Monopoly, blackjack)– Implicit

10/19/2004 TCSS435A Isabelle Bichindaritz 5

Games versus Search Problems• Unpredictable Opponent

– Games are adversarial search problems– Solution is strategy, contingency plan– Time limits

• Unlikely to find goal• Must approximate

• Plan of Attack– Algorithm for perfect play (J. von Neumann, 1944)– Finite horizon, approximate evaluation (C. Zuse, 1945; C.

Shannon, 1950, A. Samuel, 1952-1957)– First chess program (Turing, 1951)– Pruning to allow deeper search (J. McCarthy, 1956)

10/19/2004 TCSS435A Isabelle Bichindaritz 6

Types of Games• Information: Can Know (Observe)

– … outcomes of actions / moves?– … moves committed by opponent?

• Uncertainty– Deterministic vs. nondeterministic outcomes– Thought exercise: sources of nondeterminism?

10/19/2004 TCSS435A Isabelle Bichindaritz 7

Minimax• Games with two players MIN and MAX are a search problem

with:– Initial state– Successor function– Terminal test / state– Utility function (objective / payoff)

• Win / loss / draw (chess)• +1 / -1 / 0 (chess)• -192 … +192 (backgammon)

10/19/2004 TCSS435A Isabelle Bichindaritz 8

Minimax

10/19/2004 TCSS435A Isabelle Bichindaritz 9

Minimax

• Perfect play for deterministic, perfect-information games

• Choose move to position with highest minimax value= best achievable payoff against best play

• Simple example: 2-ply game, a ply being a half-move

• Game ends after one move each by MAX and MIN (one move deep)

10/19/2004 TCSS435A Isabelle Bichindaritz 10

Minimax

10/19/2004 TCSS435A Isabelle Bichindaritz 11

Minimax Algorithm:Decision and Evaluation

what’s this? what’s this?

10/19/2004 TCSS435A Isabelle Bichindaritz 12

Properties of Minimax• Complete?

10/19/2004 TCSS435A Isabelle Bichindaritz 13

Properties of Minimax• Complete?

– … yes, provided following are finite:• Number of possible legal moves (generative breadth of tree)• “Length of game” (depth of tree) – more specifically?• A finite strategy can exist even in an infinite tree!

– Perfect vs. imperfect information?• Q: What search is perfect minimax analogous to?• A: Bottom-up breadth-first

10/19/2004 TCSS435A Isabelle Bichindaritz 14

Properties of Minimax• Complete?

– … yes, provided the tree is finite:• Number of possible legal moves (generative breadth of tree)• “Length of game” (depth of tree) – more specifically?

• Optimal?

10/19/2004 TCSS435A Isabelle Bichindaritz 15

Properties of Minimax• Complete?

– … yes, provided the tree is finite

• Optimal?– … yes, provided perfect info (evaluation function) and

opponent is optimal!– … otherwise, guaranteed if evaluation function is

correct

• Time Complexity?

10/19/2004 TCSS435A Isabelle Bichindaritz 16

Properties of Minimax• Complete?

– … yes, provided the tree is finite:• Optimal?

– … yes, provided perfect info (evaluation function) and opponent is optimal!

• Time Complexity?– Depth of tree: m– Legal moves at each point: b– O(bm) – NB, m 100, b 35 for chess!

• Space Complexity?

10/19/2004 TCSS435A Isabelle Bichindaritz 17

Properties of Minimax• Complete?

– … yes, provided the tree is finite:• Optimal?

– … yes, provided perfect info (evaluation function) and opponent is optimal!

• Time Complexity?– Depth of tree: m– Legal moves at each point: b– O(bm) – NB, m 100, b 35 for chess!

• Space Complexity? O(bm) – why?

10/19/2004 TCSS435A Isabelle Bichindaritz 18

Resource Limits

10/19/2004 TCSS435A Isabelle Bichindaritz 19

Static Evaluation Function Example: Chess

10/19/2004 TCSS435A Isabelle Bichindaritz 20

Do Exact Values Matter?

10/19/2004 TCSS435A Isabelle Bichindaritz 21

Cutting Off Search

10/19/2004 TCSS435A Isabelle Bichindaritz 22

Cutting Off Search• Issues

– Quiescence• Play has “settled down”• Evaluation function unlikely to exhibit wild swings in value in near

future– Horizon effect

• “Stalling for time”• Postpones inevitable win or damaging move by opponent• See: Figure 6.9 R&N

• Solutions?– Quiescence search: expand non-quiescent positions further– No general solution to horizon problem at present

10/19/2004 TCSS435A Isabelle Bichindaritz 23

Why Prune?

10/19/2004 TCSS435A Isabelle Bichindaritz 24

Game Tree Search• Perfect Play (TBD from Perfect Information)

– Common framework: two-player (or multi-player), zero-sum– Deterministic or known probability distribution function, totally

observable– Minimax algorithm given ability to search for arbitrary ply

• Assumptions about– … game?– … opponent?

• Specifications and Ramifications– Combinatorics: finite depth (m), breadth (b) of search– Resource limits: need for static evaluation function

• Practical Issues– Effects of depth cutoff– Need for pruning

10/19/2004 TCSS435A Isabelle Bichindaritz 25

- Pruning• Properties of - pruning

– Pruning does not affect final result

– Good move ordering improves effectiveness of pruning

– With “perfect ordering,” time complexity = O(bm/2)• Depth of tree: m• Legal moves at each point: b• Effective branching factor becomes b1/2

• Can easily reach depth 8 and play good chess– A simple example of the value of reasoning about which computations

are relevant ( a form of metareasoning)

10/19/2004 TCSS435A Isabelle Bichindaritz 26

- Pruning• is the value of the best (i.e. highest value) choice we have found so far at

any choice point along the path for MAX

is the value of the best (i.e. lowest value) choice we have found so far at any choice point along the path for MIN.

10/19/2004 TCSS435A Isabelle Bichindaritz 27

Alpha-Beta (-) Pruning:Definitions

10/19/2004 TCSS435A Isabelle Bichindaritz 28

Alpha-Beta (-) Pruning Example

≥ 3

3

3

MAX

MIN

MAX12 8

≤ 2

2 14 5

≤ 14

2

≤5 2

10/19/2004 TCSS435A Isabelle Bichindaritz 29

≥ 3

3

3

MAX

MIN

MAX12 8

≤ 2

2 14 5

≤ 14

2

≤5 2

Alpha-Beta (-) Pruning Example What are , , values here? here?

10/19/2004 TCSS435A Isabelle Bichindaritz 30

Alpha-Beta (-) Pruning:Modified Minimax Algorithm

10/19/2004 TCSS435A Isabelle Bichindaritz 31

Alpha-Beta (-) Pruning Properties• Perfect Play (TBD from Perfect Information)

– Common framework: two-player (or multi-player), zero-sum

– Deterministic or known probability distribution function, totally observable

– Minimax algorithm given ability to search to arbitrary ply• Assumptions about

– … game?

– … opponent?• Specifications and Ramifications

– Combinatorics: finite depth (m), breadth (b) of search

– Resource limits: need for static evaluation function• Practical Issues

– Effects of depth cutoff

– Need for pruning