12. classic strategy games

Upload: aporat

Post on 03-Apr-2018

213 views

Category:

Documents


0 download

TRANSCRIPT

  • 7/29/2019 12. Classic Strategy Games

    1/10

    25/11/12my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    1/10my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    Username: Avi Porat Book:AI Game Engine Programming. No part of any chapter or book may be reproduced or

    transmitted in any form by any means without the prior written permission for reprints and excerpts from the publisher of

    the book or chapter. Redistribution or other use that violates the fair use privilege under U.S. copyright laws (see 17

    USC107) or that otherwise violates these Terms of Service is strictly prohibited. Violators will be prosecuted to the full

    extent of U.S. Federal and Massachusetts laws.

    12. Classic Strategy Games

    In This Chapter

    Common AI Elements

    Useful AI Techniques

    Areas That Need Improvement

    Summary

    Game theorycan be roughly thought of as the study of human behavior whendealing with interactions in which the outcomes depend on the strategies of two ormore persons who have opposing or, at best, mixed motives. John von Neumannvirtually founded the field in 1928 by studying the concept of bluffing in poker anddiscovering that the analysis had significant ramifications for economics. He officiallyfathered the field in 1944 with the publishing of his classic Theory of Games andEconomic Behavior(written with Oskar Morgenstern). The book took his earlierresearched work on minimax theory (discussed later in the chapter) and extended itto include more complex games, like economics.

    In game theory, the concept of a game takes on special meaning. Instead of themore common entertainment-oriented definition of the word, game theory uses amore broad meaning; a game is an undertaking in which several agents strive tomaximize their payoff by taking actions, but the result relies on the actions of all the

    players. By discovering that this generalization exists across different types ofgames, game theory hopes to explain some kinds of human interactions acrossmany varying playfields, from business to war, and from the checkerboard tooverpopulation.

    Some of the classic games that have been studied under game theory includebarbarians at the gate, mutually assured destruction, the prisoners dilemma, andcaveat emptor. These are all mathematical constructs that attempt to define whatare called dominant strategies of the various human behaviors that each detail.

    In some of his earliest work, von Neumann made a very important discovery, withone very large requirement. The discovery was that for some games, rationality(meaning the best action to take) could be mathematically calculated, given thestrategies and payoffs inherent in the game. The requirement was that the game bewhat is called azero-sum game, which is a game in which one players winningactions directly result in anothers equivalent loss. In other words, these are games

    in which a number of players engage in a system of pure competition, in which thereis only one winner.

    This is not a trivial requirement. Many of the more socially important problems thatgame theory had hoped to tackle (such as economics, dealing with use of natural

    http://my.safaribooksonline.com/9781584505723/ch12lev1sec4#ch12lev1sec4http://my.safaribooksonline.com/9781584505723/ch12lev1sec3#ch12lev1sec3http://my.safaribooksonline.com/9781584505723/ch12lev1sec1#ch12lev1sec1http://my.safaribooksonline.com/9781584505723/ch12lev1sec4#ch12lev1sec4http://my.safaribooksonline.com/9781584505723/ch12lev1sec3#ch12lev1sec3http://my.safaribooksonline.com/9781584505723/ch12lev1sec2#ch12lev1sec2http://my.safaribooksonline.com/9781584505723/ch12lev1sec1#ch12lev1sec1
  • 7/29/2019 12. Classic Strategy Games

    2/10

    25/11/12my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    2/10my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    resources, and political systems) are not zero-sum games. Although game theorycan still give insights into these other kinds of games, it cannot help define game-specific rationality like it can in the limited world of zero-sum games.

    Von Neumanns work became a foundation for early AI researchers work, as theyset out to create programs that could accomplish complex tasks requiringrationality. How best to test their creations than by finding some abstract version ofworldly problems, that also manages to fit neatly into a clean mathematical model,

    so that rationality can be assured? Zero-sum games answered the call and are stillsome of the most studied of all AI problems.

    Classical strategy games such as chess, checkers, tic-tac-toe, and even poker are allexamples of zero-sum games. It also turns out that non-zero-sum games likeMonopoly(in which it might be possible that two people could form an alliance, andboth win money from the bank) can be converted to a zero-sum game byconsidering one of the players to be the board itself (or the bank, in Monopoly). Thisghost player is in essence losing the sum of the amount won by the players, andthus all the formal assumptions and proofs concerning zero-sum gaming can beemployed.

    Researchers began using computers to build an intelligent program capable ofplaying these games almost as soon as computers made their appearance. AlanTuring (of the Turing test fame) and Claude Shannon wrote some of the first chessprograms in 1950, barely five years after ENIAC came online. Both men put forththat a program that could competently play these games epitomized the definitionof something requiring (and exhibiting) intelligence.

    This brings up an interesting parable about AI problems in general. In the past, if atask was too difficult for a computer to accomplish, it was said that if someone could

    devise a program to do that task, then that program would be intelligent. But, afteryears of work, when someone finally does release a program that performs the task,the detractors declare it to be simple brute force search (or whatever computertechnique the program uses), and not real intelligence. Thus, AI never gets toactually solve any problems. In effect, the bar keeps moving.

    Researchers turned to games for a number of reasons. They are more complex andlend themselves more to real-world situations than so-called toyproblems do andrepresent a more uncertain and (somewhat) exciting world than massive searchventures like the traveling salesman problem (finding the optimal non-repeatingroute a salesman should take to connect a number of cities), or integrated circuit

    design.

    Classic strategy games also personify the optimal conditions for classic AI searchtechniques. They are games of perfect information (both players know everythingabout the game world), the moves are mostly global in effect (rather than withinsome small sphere of influence). The games are turn-based, which gives thecomputer time to think. Strategy games are also very complex (in terms of statespace), thus requiring intelligent methods for finding rational solutions.

    This is precisely the list of attributes that typically make a good computer AIsimulation. However, because these games also add the element of an opponent,they provide the problem with elements of uncertainty and, more specifically,directed uncertainty. Undirected uncertainty would be randomness introduced bydice or some similar means, and is thus unbiased and is merely part of the cost ofplaying. But directed uncertainty deals with things like bluffing, mixing strategies to

  • 7/29/2019 12. Classic Strategy Games

    3/10

    25/11/12my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    3/10my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    appear random, or using irrational moves to confuse your opponent.

    If you consider the previously mentioned optimal conditions for AI problem solving,it is easy to determine the parts of strategy games that will be weak for an AIsystem. Closed chess endgames (the term closed refers to a state with a numberof interlocked pawns across the middle of the board; see Figure 12.1) arenotoriously difficult for traditional AI systems. The reason? The moves are no longerglobal, in effect. Suddenly, we can cut up the chessboard into separate chunks and

    throw off the computer by making diversionary moves on the other side, to makethe AI system think something is going on. Tactics like this are one way that GaryKasparov beats many of the computer chess programs (and because hes one of thebest chess players in the history of the game, of course).

    Figure 12.1. A closed chess game position.

    What separates most academic studies from more traditional entertainment versionsof classical game playing programs is the notion of a time limit. Given theunreasonable request of an infinite amount of time, the best solution can almostalways be found. But given the limits of the real world, gameplaying programsalways have some form of time limitation, and we must make do with the amount oftime that we have allotted to us. Of course, as computation speeds increase, we aregetting closer and closer to the point when brute force methods will be possible,given even modest time constraints. But there will always be another, more complexgame that will force AI researchers to use alternate methods to find better solutions

    fast, without relying on total brute search.

    AI researchers have solved several of these games, meaning that the entire statespace has been mapped out and can be easily searched by todays computers toresult in optimal performance (that being a win for the first player to move, or adraw). Games that have been solved include tic-tac-toe, checkers, Connect Four,

  • 7/29/2019 12. Classic Strategy Games

    4/10

    25/11/12my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    4/10my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    Go-Moku, and Othello. Several others are in various states of being solved. Chess isgetting close. The highest-classed chess programs use a stored opening book(chains of moves that have been researched over the centuries by chess masters togive good play) for the opening moves. They use a smart search technique of somekind for the transitory middle game phase, and then have another stored databaseof good moves for the endgame phase. See Figure 12.2 for a listing of solved andpartially solved games. Bear in mind that while much was made of IBMs Deep Bluebeating Gary Kasparov in 1997, most chess programs were able to beat most human

    players long before that (the first real computer chess programs that came out inthe late 1950s could surely have beaten most human players).

    Figure 12.2. Classic games that have been solved, in whole or partially.

    Some games can have such huge state spaces (the game ofGo has a game tree size

    of around 10400, which is a number larger than the amount of atoms in theuniverse, give or take) that they are all but immune to brute force search methods

    and, thus, require either very clever directed search routines within recognizedportions of the state space, or intelligent algorithms to develop novel solutions giventhe game rules. Either way, these are some of the most classically-defined AIproblems there are.

    Listing 12.1 shows the search() and think() functions from the open source chess

    program, Faile, written by Adrien M. Regimbald. The entire source is on the CD-ROM, along with its corresponding Web links for more information. Faile is a verycompact (the entire source zip file is 42 K), yet full-featured, alpha-beta searchsystem, which gives this tiny little program expert-level AI play capability.

    Notice that the search function uses bounded optimality, in that it has a time limit,and will make decisions based on the best move it has seen given the time it hasleft, and will even make decisions on whether to continue searching or not based ontime. More detail will be given on this later in the chapter when alpha-beta search isdiscussed.

  • 7/29/2019 12. Classic Strategy Games

    5/10

    25/11/12my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    5/10my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    Listing 12.1. search() and think() from Faile. Distributed under the MIT

    license.

    long int search (int alpha, int beta, int depth, bool do_null) {

    /* search the current node using alpha-beta with negamax search */

    move_s moves[MOVE_BUFF], h_move;int num_moves, i, j, ep_temp, extensions = 0, h_type;long int score = -INF, move_ordering[MOVE_BUFF],

    null_score = -INF, i_alpha,h_score;bool no_moves, legal_move;d_long temp_hash;

    /* before we do anything, see if were out of timeor we have input: */

    if (i_depth > mindepth && !(nodes & 4095)) {if (rdifftime (rtime (), start_time) >= time_for_move) {

    /* see if our score has suddenly dropped, and if so,try to allocate some extra time: */

    if (allow_more_time && bad_root_score) {allow_more_time = FALSE;if (time_left > (5*time_for_move)) {

    time_for_move *= 2;}else {

    time_exit = TRUE;return 0;

    }

    }else {time_exit = TRUE;return 0;

    }}#ifndef ANSIif (xb_mode && bioskey ()) {

    time_exit = TRUE;return 0;

    }#endif

    }/* check for a draw by repetition before continuing: */if (is_draw ()) {

    return 0;}

    pv_length[ply] = ply;

    /* see what info we can get from our hash table: */h_score = chk_hash (alpha, beta, depth, &h_type, &h_move);if (h_type != no_info) {

    switch (h_type) {case exact:

    return (h_score);case u_bound:

    return (h_score);case l_bound:

  • 7/29/2019 12. Classic Strategy Games

    6/10

    return (h_score);case avoid_null:

    do_null = FALSE;break;

    default:break;}

    }

    temp_hash = cur_pos;ep_temp = ep_square;i_alpha = alpha;

    /* perform check extensions if we havent gone past maxdepth: */if (in_check ()) {

    if (ply < maxdepth+1) extensions++;}/* if not in check, look into null moves: */else {

    /* conditions for null move:

    - not in check- we didnt just make a null move- we dont have a risk of zugzwang by being in the endgame- depth is >= R + 1what we do after null move:- if score is close to

    -mated, were in danger, increase depth- if score is >= beta, we can get an early cutoff and exit */

    if (do_null && null_red && piece_count >= 5 &&depth >= null_red+1) {

    /* update the rep_history just so things dont get funky: */rep_history[game_ply++] = cur_pos;

    fifty++;

    xor (&cur_pos, color_h_values[0]);xor (&cur_pos, color_h_values[1]);xor (&cur_pos, ep_h_values[ep_square]);xor (&cur_pos, ep_h_values[0]);

    white_to_move ^= 1;ply++;ep_square = 0;null_score = -search (-beta, -beta+1,

    depth-null_red-1, FALSE);

    ep_square = ep_temp;ply;white_to_move ^= 1;

    game_ply;fifty;

    xor (&cur_pos, color_h_values[0]);xor (&cur_pos, color_h_values[1]);xor (&cur_pos, ep_h_values[ep_square]);

    xor (&cur_pos, ep_h_values[0]);assert (cur_pos.x1 == compute_hash ().x1 &&

    cur_pos.x2 == compute_hash ().x2);

    /* check to see if we ran out of time: */

  • 7/29/2019 12. Classic Strategy Games

    7/10

    25/11/12my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    7/10my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    if (time_exit)return 0;

    /* check to see if we can get a quickcutoff from our null move: */

    if (null_score >= beta)return beta;

    if (null_score < -INF+10*maxdepth)

    extensions++;}

    }/* try to find a stable position before passing

    the position to eval (): */if (!(depth+extensions)) {

    captures = TRUE;score = qsearch (alpha, beta, maxdepth);captures = FALSE;return score;

    }

    num_moves = 0;no_moves = TRUE;

    /* generate and order moves: */gen (&moves[0], &num_moves);order_moves (&moves[0], &move_ordering[0], num_moves, &h_move);

    /* loop through the moves at the current node: */while (remove_one (&i, &move_ordering[0], num_moves)) {

    make (&moves[0], i);

    assert (cur_pos.x1 == compute_hash ().x1 &&cur_pos.x2 == compute_hash ().x2);

    ply++;legal_move = FALSE;

    /* go deeper if its a legal move: */if (check_legal (&moves[0], i)) {

    nodes++;score = -search (-beta, -alpha, depth-1+extensions, TRUE);no_moves = FALSE;legal_move = TRUE;

    }

    ply;unmake (&moves[0], i);ep_square = ep_temp;cur_pos = temp_hash;

    /* return if weve run out of time: */if (time_exit) return 0;

    /* check our current score vs. alpha: */

    if (score > alpha && legal_move) {/* update the history heuristic since we have a cutoff: */history_h[moves[i].from][moves[i].target] += depth;

    /* try for an early cutoff: */

  • 7/29/2019 12. Classic Strategy Games

    8/10

    if (score >= beta) {u_killers (moves[i], score);store_hash (i_alpha, depth, score, l_bound, moves[i]);return beta;

    }alpha = score;

    /* update the pv: */pv[ply][ply] = moves[i];

    for (j = ply+1; j < pv_length[ply+1]; j++)pv[ply][j] = pv[ply+1][j];

    pv_length[ply] = pv_length[ply+1];}

    }

    /* check for mate / stalemate: */if (no_moves) {

    if (in_check ()) {alpha = -INF+ply;

    }else {alpha = 0;

    }}else {

    /* check the 50 move rule if no mate situationis on the board: */

    if (fifty > 100) {return 0;

    }}

    /* store our hash info: */if (alpha > i_alpha)

    store_hash (i_alpha, depth, alpha, exact, pv[ply][ply]);else

    store_hash (i_alpha, depth, alpha, u_bound, dummy);return alpha;

    }//--------------------move_s think (void) {

    /* Perform iterative deepening to go further in the search */

    move_s comp_move, temp_move;int ep_temp, i, j;long int elapsed;

    /* see if we can get a book move: */comp_move = book_move ();if (is_valid_comp (comp_move)) {

    /* print out a pv line indicating a book move: */printf (0 0 0 0 (Book move)\n);

    return (comp_move);}

    nodes = 0;qnodes = 0;

  • 7/29/2019 12. Classic Strategy Games

    9/10

    25/11/12my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    9/10my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    allow_more_time = TRUE;

    /* allocate our time for this move: */time_for_move = allocate_time ();

    /* clear the pv before a new search: */for (i = 0; i < PV_BUFF; i++)

    for (j = 0; j < PV_BUFF; j++)pv[i][j] = dummy;

    /* clear the history heuristic: */memset (history_h, 0, sizeof (history_h));

    /* clear the killer moves: */for (i = 0; i < PV_BUFF; i++) {

    killer_scores[i] = -INF;killer_scores2[i] = -INF;killer1[i] = dummy;killer2[i] = dummy;killer3[i] = dummy;

    }for (i_depth = 1; i_depth time_for_move*2.0/3.0 && i_depth > mindepth)

    break;

    ep_temp = ep_square;temp_move = search_root (-INF, INF, i_depth);

    ep_square = ep_temp;

    /* if we havent aborted our search on time,set the computers move

    and post our thinking: */if (!time_failure) {

    /* if our search score suddenly drops, andwe ran out of time on the

    search, just use previous results */comp_move = temp_move;last_root_score = cur_score;

    /* if our PV is really short, try to get someof it from hash info(dont modify this if it is a mate / draw though): */

    if (pv_length[1] 1 &&abs (cur_score) < (INF-100) &&

    result != stalemate && result != draw_by_fifty &&result != draw_by_rep)hash_to_pv (i_depth);

    if (post && i_depth >= mindepth)post_thinking (cur_score);

    }

    /* reset the killer scores (we can keep themoves for move ordering for now, but thescores may not be accurate at higher depths, so we need

    to reset them): */for (j = 0; j < PV_BUFF; j++) {

  • 7/29/2019 12. Classic Strategy Games

    10/10

    25/11/12my.safaribooksonline.com/print?xmlid=9781584505723%2Fch12

    killer_scores[j] = -INF;killer_scores2[j] = -INF;

    }

    }/* update our elapsed time_cushion: */if (moves_to_tc) {

    elapsed = rdifftime (rtime (), start_time);time_cushion += time_for_move-elapsed+inc;

    }

    return comp_move;

    }