1 chapter 6 section 1 – 4 adversarial search. 2 outline optimal decisions α-β pruning imperfect,...
TRANSCRIPT
![Page 1: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/1.jpg)
1
Chapter 6
Section 1 – 4
Adversarial SearchAdversarial Search
![Page 2: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/2.jpg)
2
OutlineOutline
• Optimal decisions
• α-β pruning
• Imperfect, real-time decisions
![Page 3: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/3.jpg)
3
Games vs. search problemsGames vs. search problems
• "Unpredictable" opponent specifying a move for every possible opponent reply
• For Chess, average branching 35; and search 50 moves by each player– Time limits (35^100) unlikely to find goal, must
approximate
![Page 4: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/4.jpg)
4
Two-Agent GamesTwo-Agent Games
• Two-agent, perfect information, zero-sum games
• Two agents move in turn until either one of them wins or the result is a draw.
• Each player has a complete model of the environment and of its own and the other’s possible actions and their effects.
![Page 5: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/5.jpg)
5
Minimax Procedure (1)Minimax Procedure (1)• Two player : MAX and MIN• Task : find a “best” move for MAX• Assume that MAX moves first, and that the two
players move alternately.• MAX node
– nodes at even-numbered depths correspond to positions in which it is MAX’s move next
• MIN node– nodes at odd-numbered depths correspond to positions in
which it is MIN’s move next
![Page 6: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/6.jpg)
66
Minimax Procedure (2)Minimax Procedure (2)• Estimate of the best-first move
– apply a static evaluation function to the leaf nodes– measure the “worth” of the leaf nodes.– The measurement is based on various features
thought to influence this worth.– Usually, analyze game trees to adopt the convention
• game positions favorable to MAX cause the evaluation function to have a positive value
• positions favorable to MIN cause the evaluation function to have negative value
• Values near zero correspond to game positions not particularly favorable to either MAX or MIN.
![Page 7: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/7.jpg)
7
Game tree (2-player, deterministic, Game tree (2-player, deterministic, turns)turns)
tic-tac-toe
![Page 8: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/8.jpg)
8
MinimaxMinimax• Perfect play for deterministic games
• Idea: choose move to position with highest minimax value = best achievable payoff against best play
• E.g., 2-ply game:
![Page 9: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/9.jpg)
9
Minimax algorithmMinimax algorithm
![Page 10: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/10.jpg)
10
Properties of minimaxProperties of minimax• Complete? Yes (if tree is finite)
• Optimal? Yes (against an optimal opponent)
• Time complexity? O(bm) (b-legal moves; m- max tree depth)
• Space complexity? O(bm) (depth-first exploration)
For chess, b ≈ 35, m ≈100 for "reasonable" games exact solution completely infeasible
![Page 11: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/11.jpg)
11
Example : Tic-Tac-Toe (1)Example : Tic-Tac-Toe (1)
• MAX marks crosses MIN marks circles• it is MAX’s turn to play first.
– With a depth bound of 2, conduct a breadth-first search
– evaluation function e(p) of a position p• If p is not a winning for either player,
e(p) = (no. of complete rows, columns, or diagonals that are still open for MAX) - (no. of complete rows, columns, or diagonals that are still open for MIN)
• If p is a win of MAX, e(p) = • If p is a win of MIN, e(p) = -
![Page 12: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/12.jpg)
12
– evaluation function e(p) of a position p• If p is not a winning for either player,
e(p) = (no. of complete rows, columns, or diagonals that are still open for MAX) - (no. of complete rows, columns, or diagonals that are still open for MIN)
e(p)=5-4=1 e(p)=6-4=2 e(p)=5-4=1 e(p)=6-4=2
![Page 13: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/13.jpg)
13
– evaluation function e(p) of a position p• If p is not a winning for either player,
e(p) = (no. of complete rows, columns, or diagonals that are still open for MAX) - (no. of complete rows, columns, or diagonals that are still open for MIN)
e(p)= e(p)= e(p)= e(p)=
![Page 14: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/14.jpg)
14
Example : Tic-Tac-Toe (2)Example : Tic-Tac-Toe (2)
• First move
![Page 15: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/15.jpg)
15
Example : Tic-Tac-Toe (3)Example : Tic-Tac-Toe (3)
![Page 16: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/16.jpg)
16
Example : Tic-Tac-Toe (4)Example : Tic-Tac-Toe (4)
![Page 17: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/17.jpg)
17
Question?
• How to improve search efficiency?
• It is possible to cut-off some unnecessary subtrees?
![Page 18: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/18.jpg)
18
α-β Pruning Exampleα-β Pruning Example
![Page 19: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/19.jpg)
19
α-β pruning exampleα-β pruning example
![Page 20: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/20.jpg)
20
α-β pruning exampleα-β pruning example
![Page 21: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/21.jpg)
21
α-β pruning exampleα-β pruning example
![Page 22: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/22.jpg)
22
α-β pruning exampleα-β pruning example
![Page 23: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/23.jpg)
23
Properties of α-βProperties of α-βPruning does not affect final result
Good move ordering improves effectiveness of pruning
With "perfect ordering," time complexity = O(bm/2) doubles depth of search
A simple example of the value of reasoning about which computations are relevant (a form of metareasoning)
![Page 24: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/24.jpg)
24
α is the value of the best (i.e., highest-value) choice found so far at any choice point along the path for max
If v is worse than α, max will avoid it prune that branch
Define β similarly for min
Why is it called α-β?
![Page 25: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/25.jpg)
25
The α-β algorithmThe α-β algorithm
![Page 26: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/26.jpg)
26
The α-β algorithmThe α-β algorithm
![Page 27: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/27.jpg)
27
An ExampleAn Example• (a) Compute the backed-up values calculated by the minimax algorithm. Show your
answer by writing values at the appropriate nodes in the above tree.
• (b) Which nodes will not be examined by the alpha-beta procedure?
5 3 4 6 5 3 6 4 7 5 2 4 5 3 8 2
![Page 28: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/28.jpg)
28
Resource limitsResource limits
Suppose we have 100 secs, explore 104 nodes/sec 106 nodes per move
Standard approach: cutoff test:
e.g., depth limit (perhaps add quiescence search)
evaluation function = estimated desirability of position
–
–
•
![Page 29: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/29.jpg)
29
Evaluation functionsEvaluation functions• For chess, typically linear weighted sum of features
Eval(s) = w1 f1(s) + w2 f2(s) + … + wn fn(s)
• e.g., w1 = 9 with
• f1(s) = (number of white queens) – (number of black queens), etc.
•
• Third, for nonterminal states, the evaluation function should be strongly correlated with the actual chances of winning.
• First, the evaluation function should order the terminal states in the same way as the true utility function;
• Second, the computation must not take too long!
![Page 30: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/30.jpg)
30
Cutting off searchCutting off search
MinimaxCutoff is identical to MinimaxValue except1. Terminal? is replaced by Cutoff?2. Utility is replaced by Eval
TERMINAL-TEST-->if CUTOFF-TEST(stated, depth) then return EVAL(state)
Does it work in practice?bm = 106, b=35 m=4
4-ply lookahead is a hopeless chess player!– 4-ply ≈ human novice– 8-ply ≈ typical PC, human master– 12-ply ≈ Deep Blue, Kasparov
![Page 31: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/31.jpg)
31
GameGame IncludeInclude an Element of Chancean Element of Chance
Backgammon
![Page 32: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/32.jpg)
32
GameGame IncludeInclude an Element of Chancean Element of Chance
![Page 33: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/33.jpg)
33
Deterministic games in practiceDeterministic games in practice• Checkers: ChinookChinook ended 40-year-reign of human world champion Marion
Tinsley in 1994. Used a precomputed endgame database defining perfect play for all positions involving 8 or fewer pieces on the board, a total of 444 billion positions.
• Chess: Deep Blue Deep Blue defeated human world champion Garry Kasparov in a six-game match in 1997. Deep Blue searches 200 million positions per second, uses very sophisticated evaluation, and undisclosed methods for extending some lines of search up to 40 ply.
• OthelloOthello: human champions refuse to compete against computers, who are too good.
• GoGo: human champions refuse to compete against computers, who are too bad. In go, b > 300, so most programs use pattern knowledge bases to suggest plausible moves.
![Page 34: 1 Chapter 6 Section 1 – 4 Adversarial Search. 2 Outline Optimal decisions α-β pruning Imperfect, real-time decisions](https://reader036.vdocuments.net/reader036/viewer/2022062308/56649ccf5503460f9499a634/html5/thumbnails/34.jpg)
34
SummarySummary• Games are fun to work on!
• They illustrate several important points about AI
• Perfection is unattainable must approximate
• Good idea to think about what to think about