ai search std

Chapter 2Heuristic Search TechniquesHeuristic Search Techniques

Ms Smita Selot

Contents A framework for describing search methods is

provided and several general purpose search techniques are discussed.

All are varieties of Heuristic Search: All are varieties of Heuristic Search: Generate and test Hill Climbing Best First Search (A*) Problem Reduction (AO*) Constraint Satisfaction Branch and bound Means-ends analysis

Heuristic Is a technique that improves the search

method Like our tour guides Improve the quality of search path Example general purpose heuristic- Example general purpose heuristic-

nearest neighbourhood Applying it to Travelling salesman problem

Select a city arbitrarily Select the next city closest to current city and not

visited yet Repeat step 2 till all cities have been visited

Travelling salesmen problemStart from A Next city it B as cost is minimum

From B select D From D select C Finally A-B-D-C This executes in time proportional to N*N

better than !N

AB

3

46 5

C D

4 2

1

5A

B C D

C D

C

3 46

5 2

1

Heuristic

Word heuristic come from greek word heuriskeinmeaning to discover

Heuristics is a technique that improves the efficiency of search process by sacrificing the claims of completeness

Good heuristic leads in promising directionGood heuristic leads in promising direction

Bad heuristic will yield inefficient search path

Heuristic function is a function that map from problem state descriptions to measure of desirability, represented as numbersThey describe goodness of a stateHeuristics are not always as general as TSP problem

h=f(x,y) where f can be squaring, identity, function

Heuristics Heuristics are important in real life

problem solving process- why? Rarely we look for optimum solutions

Good solutions do well People are satisfier rather than optimizers People are satisfier rather than optimizers Seek solution which satisfy requirements and quit

as they find one- example parking space Approximation produced may not be very

good in worst case. Worst case hardly occur in real time

Analysing heuristic- leads to better understanding of the problem

Heuristic function There are 2 ways in which domain specific , heuristic

knowledge can be incorporated in rule based search procedure In rule themselves, where rule depicts set of sensible moves

instead of all moves A heuristic function that evaluates individual problem states and

determine the degree of desirability

Problem state description

Measure of desirability ( number)Heuristic

function maps the two

Heuristic function Can be simple or complex At times

higher values -good position Lower values good position

Objective of heuristic function : is to guide search process in most profitable direction by suggesting path to follow when more than one profitable direction by suggesting path to follow when more than one are available

Good heuristic function less time in search Bad heuristic function will not improve the search Cost of computing heuristic overweighs the effort saved in search

process There is trade-off between cost of evaluating hf and the saving in

search time that function provides

Problem characteristics

Heuristics are applicable to variety of problems and effectiveness of heuristic depends on class of problem

Before applying the heuristic we must analyse the problem under following heads Is problem decomposable? Is problem decomposable? Can solution step be ignored? Is problem universal predictable? Is good solution to the problem is obvious? Is desired solution a state or path to state? Is knowledge required to solve problem or constraint the search

Problem characteristics Is problem decomposable?

Take two problems Integration Block domain problem

Can solution step be ignored? Ignorable in which solution step can be ignored

eg(Mathematical theorem proving)eg(Mathematical theorem proving) Recoverable in which solution step can be undone (8 puzzle) Irrecoverable in which solution step cannot be undone eg (

chess) Control strategy required may be

simple for ignorable requiring backtracking for recoverable Require great effort in taking decision in irrecoverable- planning

Problem characteristics Is the universe predictable?

8 puzzle can use planning to identify the exact solution Bridge exact planning not possible so navigate several plans

and then choose Difference is due to certain outcome and uncertain outcome Hardest type of problem is one with uncertain outcome and

irrecoverable playing bridge controlling robot arm lawyer helping to defend client from murder charge

Problem characteristics Is good solution to problem is obvious

Example problem using logic (FOPL) and TSP In logic one may ask question and interested in answer here optimum path

does not matter In TSP best path matters Difference is any path solution or best path solution Best path problem is computationally harder to achieve

Is solution a state or path Is solution a state or path Eg semantic analysis in NLP require a state as goal Water jug problem requires a path

What is the role of knowledge Even if unlimited computing power is available Knowledge about good moves , will always help search strategy to perform

better Example Chess or newspaper content to decide which party will win next

election

Problem classification When actual problems are analysed , we find

that there are several broad classes into which problem falls

So classes can be associated with generic So classes can be associated with generic control strategy for solving the problem

Example classification problem

Medical diagnosis ,fault diagnosis in vehicle Propose and refine

Design and planning process

Problem classification Various problem solving method No single way to solve all kind of problem Each problem may not require a complete

new strategynew strategy Instead

Analyse the problem Sort the various problem solving method by

kinds of problem which they solve We can solve the new problem

Problem and problem space1.Define the problem

Problem definition and state space representation of

problem

PRODUCTION SYSTEM

3.Create knowledge base2. Analyse the problem

problem

Problem characteristic

3.Create knowledge base

4. Find best control strategy

Production system

A good way to describe operations that can be performed in a search for solution to problem

A production system consist of

A set of rulesOne or more knowledge baseA control strategyA rule applier

Production system characteristics

Can production system be represented by set of characteristics

What relation exists between problem types and types of production system

Classes of production system Monotonic production system is a production system

in which application of a rule never prevents later application of another rule that could have been applied when at the time the first rule was selected .

Non monotonic system never allows this Non monotonic system never allows this Partially commutative production system if

application of particular sequence of rules transform state x to state y , then any permutation of those rules also transform state x to y

Commutative production system is one which is both partially commutative as well as monotonic

Categories of production systemMonotonic Non monotonic

Partially commutative

Theorem proving Robot navigation

Not partially commutative

Chemical synthesis

Bridge

Issues in design of search programs

Every search process is traversal in tree structure

It finds path connecting initial and final statestate

Tree could be constructed from rules that define allowable moves in state space

It is too large and most of it need never be explored


Hence instead of building an explicit tree and then searching it

Most search program represent tree implicitly in the rulesimplicitly in the rules

And generate those part that are explicitly explored by algorithm.


Direction in which to conduct the search Forward

Initial to final moving left to right in rules backward reasoning

Final to initial moving right to left Final to initial moving right to left

How to select applicable rules (matching) Production system spend most of the time looking

for rules to apply How to represent each node

Generate-and-Test Algorithm:

1. Generate a possible solution. For some problems, this means generating a particular point in the problem space. For others it means generating a path from a start statepath from a start state

2. Test to see if this is actually a solution by comparing the chosen point or the endpoint of the chosen path to the set of acceptable goal states.

3. If a solution has been found, quit, Otherwise return to step 1.

Generate-and-Test It is a depth first search procedure since complete

solutions must be generated before they can be tested. In its most systematic form, it is simply an exhaustive

search of the problem space. Operate by generating solutions randomly. Operate by generating solutions randomly. Also called as British Museum algorithm If a sufficient number of monkeys were placed in front of

a set of typewriters, and left alone long enough, then they would eventually produce all the works of Shakespeare.

Dendral: which infers the struture of organic compounds using NMR spectrogram. It uses plan-generate-test.

Generate-and-Test

Problem solution valid solution

Generator Tester

Rejected solution

Eg: Mapping pictures

Hill Climbing Is a variant of generate-and test in which feedback from the

test procedure is used to help the generator decide which direction to move in search space.

The test function is augmented with a heuristic function that provides an estimate of how close a given state is to the goal provides an estimate of how close a given state is to the goal state.

Computation of heuristic function can be done with negligible amount of computation.

Hill climbing is often used when a good heuristic function is available for evaluating states but when no other useful knowledge is available.

Simple Hill ClimbingAlgorithm:1. Evaluate the initial state. If it is also goal state, then return it

and quit. Otherwise continue with the initial state as the current state.

2. Loop until a solution is found or until there are no new operators left to be applied in the current state:

2. Loop until a solution is found or until there are no new operators left to be applied in the current state:

a. Select an operator that has not yet been applied to the current state and apply it to produce a new state

b. Evaluate the new statei. If it is the goal state, then return it and quit.ii. If it is not a goal state but it is better than the current state, then make it

the current state.iii. If it is not better than the current state, then continue in the loop.

Simple Hill Climbing The key difference between Simple Hill climbing and

Generate-and-test is the use of evaluation function as a way to inject task specific knowledge into the control process.

Is one state better than another ? For this algorithm to Is one state better than another ? For this algorithm to work, precise definition of better must be provided.

Steepest-Ascent Hill Climbing This is a variation of simple hill climbing which considers all

the moves from the current state and selects the best one as the next state.

Also known as Gradient search

Steepest-Ascent Hill ClimbingAlgorithm:1. Evaluate the initial state. If it is also a goal state, then return it and

quit. Otherwise, continue with the initial state as the current state.2. Loop until a solution is found or until a complete iteration

produces no change to current state:a. Let SUCC be a state such that any possible successor of the current state will a. Let SUCC be a state such that any possible successor of the current state will

be better than SUCCb. For each operator that applies to the current state do:

i. Apply the operator and generate a new stateii. Evaluate the new state. If is is a goal state, then return it and quit. If not,

compare it to SUCC. If it is better, then set SUCC to this state. If it is not better, leave SUCC alone.

c. If the SUCC is better than the current state, then set current state to SUCC,

: Hill-climbing This simple policy has three well-known drawbacks:

1. Local Maxima: a local maximumas opposed to global maximum.

2. Plateaus: An area of the searchspace where evaluation function is

(a)

(b)space where evaluation function isflat, thus requiring random walk.

3. Ridge: Where there are steepslopes and the search direction isnot towards the top but towards theside.

(b)

(c)

Figure 5.9 Local maxima, Plateaus andridge situation for Hill Climbing

Hill-climbing

In each of the previous cases (local maxima, plateaus & ridge), thealgorithm reaches a point at which no progress is being made.

A solution is to do a random-restart hill-climbing - where randominitial states are generated, running each until it halts or makes noinitial states are generated, running each until it halts or makes nodiscernible progress. The best result is then chosen.

Figure 5.10 Random-restart hill-climbing (6 initial values) for 5.9(a)

Solution to problems of Hill-climbing

Backtrack to some earlier node-this may give a different direction. A path which was equally promising can be chosen .It requires maintaining list of path followed .Good for handling local maxima .

Make a big jump-to reach a new section of search .Good for dealing with plateaus.

Apply two or more rules before doing the test-moving in several direction at once .Good for ridges

Best First Search Combines the advantages of both DFS and BFS into a single

method. DFS is good because it allows a solution to be found without

all competing branches having to be expanded. BFS is good because it does not get branches on dead end

paths. paths. One way of combining the two is to follow a single path at a

time, but switch paths whenever some competing path looks more promising than the current one does.

At each step of the BFS search process, we select the most promising of the nodes we have generated so far.

This is done by applying an appropriate heuristic function to each of them.

We then expand the chosen node by using the rules to generate its successors

Best First Search Similar to Steepest ascent hill climbing with two exceptions:

In hill climbing, one move is selected and all the others are rejected, never to be reconsidered. This produces the straight-line behavior that is characteristic of hill climbing.

In BFS, one move is selected, but the others are kept around so that they can be revisited later if the selected path becomes less promising.

The best available state is selected in the BFS, even if that state has a value that is lower than the value of the state that was just explored. This contrasts with hill climbing, which will stop if there are no successor states with better values than the current state.

OR-graph It is sometimes important to search graphs so that duplicate

paths will not be pursued.

An algorithm to do this will operate by searching a directed graph in which each node represents a point in problem space.

Each node will contain: Description of problem state it represents Indication of how promising it is Parent link that points back to the best node from which it came List of nodes that were generated from it

OR-Graph Parent link will make it possible to recover the path to

the goal once the goal is found.

The list of successors will make it possible, if a better path is found to an already existing node, to path is found to an already existing node, to propagate the improvement down to its successors.

This is called OR-graph, since each of its branches represents an alternative problem solving path

BFSStep 1 Step 2 Step 3

A A

B C D3 5 1

A

B C D3 5 1

Step 4 Step 5 E F4 6A

B C D3 5 1

E F4 6G H6 5

A

B C D3 5 1

E F4 6G H6 5

I J2 1

Implementation of OR graphs We need two lists of nodes:

OPEN LIST Nodes that have been generated and have had the heuristic function

applied to them but which have not yet been examined. OPEN is actually a priority queue in which the elements with the

highest priority are those with the most promising value of the highest priority are those with the most promising value of the heuristic function.

CLOSED LIST Nodes that have already been examined. We need to keep these

nodes in memory if we want to search a graph rather than a tree since whenever a new node is generated, we need to check whether

it has been generated before.

Close

open

BFSStep 1 Step 2 Step 3

A A

B C D3 5 1

A

B C D3 5 1

openStep 4 Step 5 E F4 6A

B C D3 5 1

E F4 6G H6 5

A

B C D3 5 1

E F4 6G H6 5

I J2 1

Algorithm: BFSGiven : s-starting state, n-any node, g- goal state, o-set of operators, c()-cost function1. Start with OPEN containing just the initial state

initialization open = {s} closed={ } c{s}=02. Until a goal is found or there are no nodes left on OPEN

do{ fail if open ={ }; terminate if n G then terminate with { fail if open ={ }; terminate if n G then terminate with

success }a. Pick the best node on OPEN

select : select n from openb Generate its successors

Expand Generate the successor of n using set of operators- o

b. For each successor do:i. If it has not been generated before, evaluate it, add it to OPEN,

and record its parent.V succ m , if m ~ {open U closed}set C(m)=C(n)+C(n,m)insert m in open

ii If it has been generated before, change the parent if this new path is better than the previous one. In that case, update the cost of getting to this node and to any successors that this node may getting to this node and to any successors that this node may already have.

if m {open U closed}set C(m) = min{ C(m),C(n)+C(n,m)}if C{m} has decreased

and m {closed} move it to openif m {open} change the cost

BFS : simple explanation It proceeds in steps, expanding one node at each step,

until it generates a node that corresponds to a goal state.

At each step, it picks the most promising of the nodes that have so far been generated but not expanded.

It generates the successors of the chosen node, applies the heuristic function to them, and adds them to the list of open nodes, after checking to see if any of them have been generated before.

By doing this check, we can guarantee that each node only appears once in the graph, although many nodes may point to it as a successor.

Examples (2): 8-puzzle f1(T) = the number correctly placed tiles on

the board: 11 33 2244776655

88f1f1 = 4= 4

f2(T) = number or incorrectly placed tiles on board:f2(T) = number or incorrectly placed tiles on board:

44

Most often, distance to goal heuristics are more useful !Most often, distance to goal heuristics are more useful !

f2f2 = 4= 411 33 22

44776655

88

f2(T) = number or incorrectly placed tiles on board:f2(T) = number or incorrectly placed tiles on board: gives (rough!) estimate of how far we are from goalgives (rough!) estimate of how far we are from goal

Examples (3):Manhattan distance

f3(T) = the sum of ( the horizontal + vertical distance that each tile is away from its final destination):

45

from its final destination): gives a better estimate of distance from the

goal nodef3f3 = 1 + 4 + 2 + 3 = 10= 1 + 4 + 2 + 3 = 10

11 55 2244776633

88

Heuristics: Example 8-puzzle: h(n) = tiles out of place

h(n) = 31 2 38 6

7 5 4

1 2 3

8 4

7 6 5

Goal state

Example - cont

h(n) = 31 2 38 6

7 5 4

h(n) = 2 h(n) = 41 2

8 6 3

7 5 4

1 2 3

8 6 4

7 5

1 2 3

8 6

7 5 4

h(n) = 3 h(n) = 2 h(n) = 4

h(n) = 31 2 38 6

7 5 4

1 2

8 6 3

7 5 4

1 2 3

8 6 4

7 5

1 2 3

8 6

7 5 4

h(n) = 3 h(n) = 2 h(n) = 4

h(n) = 1h(n) = 31 2 3

8 6

7 5 4

1 2 3

8 6 4

7 5

h(n) = 1h(n) = 3

A* Algorithm BFS is a simplification of A* Algorithm Presented by Hart et al Algorithm uses:

f: Heuristic function that estimates the merits of each node we generate. This is sum of two components, g and h and f represents an estimate of the cost of getting from the initial state to a goal state along with the path that generated the current represents an estimate of the cost of getting from the initial state to a goal state along with the path that generated the current node.

g : The function g is a measure of the cost of getting from initial state to the current node.

h : The function h is an estimate of the additional cost of getting from the current node to a goal state.

OPEN CLOSED

A* Algorithm1. Start with OPEN containing only initial node. Set that

nodes g value to 0, its h value to whatever it is, and its f value to h+0 or h. Set CLOSED to empty list.Initialize open = {s} closed = { } g(s)=0 f(s)=h(s)

2. Until a goal node is found, repeat the following 2. Until a goal node is found, repeat the following procedure:

(a)If there are no nodes on OPEN, report failure.fail if open={ } terminate with failure

.

A* Algorithm(b)Otherwise pick the node on OPEN with the lowest f

value. Call it BESTNODE. Remove it from OPEN. Place it in CLOSED. Select Select minimum cost state n from OPEN Save n in CLOSEin CLOSE

(c )See if the BESTNODE is a goal state. If so exit and report a solution.

terminate if n G terminate with success and f(n)

A* algorithmd)Otherwise, generate the successors of BESTNODE

For each of the SUCCESSOR, do the following:expand: SUCC m of n

if m ~ {open U close }set g(m) = g(n)+C(n,m)

set f(m)=g(m)+h(m)Insert m in openInsert m in openset the link to parent node

if m {open U close }set g(m)=min[g(m),g(n)+C(n,m)]set f(m)=g(m)+h(m)

if f(m) has decreased and m {close } move it to openif f(m) has decreased and m {open} update the cost and

change the link to parent node

Observations about A* Role of g function: This lets us choose which

node to expand next on the basis of not only of how good the node itself looks, but also on the basis of how good the path to the node was.

h, the distance of a node to the goal. If h is a perfect estimator of h, then A* will converge immediately to the goal with no search.

8-Puzzle Evaluation Function

f(n) = d(n) + W(n)

W(n) counts the number of displaced tiles in thatdatabase associated with the node nd(n) is the depth of the node in the search tree

Playing 8-Puzzle2 8 31 6 47 5

4

2 8 31 6 4

7 5

2 8 31 47 6 5

2 8 31 6 47 5

66 4

Playing 8-Puzzle2 8 31 47 6 5

4

2 8 31 4

7 6 5

2 31 8 47 6 5

2 8 31 47 6 5

65 5

8 32 1 47 6 5

2 8 37 1 4

6 5

2 31 8 47 6 5

2 31 8 47 6 5

6 7 5 7

Playing 8-Puzzle2 3

1 8 47 6 5

5

1 2 38 4

7 6 5

5

1 2 38 47 6 5

1 2 37 8 4

6 5

5 7

A* special casesf(n)=g(n)+h(n)

When h(n) = 0. => Uniform Cost Method Search only depends on g

When g(n) = 1, h(n) = 0 => Breadth First When g(n)=0 => Best First Search/A* When g(n)=0 => Best First Search/A* If non-admissible heuristic

g(n) = 0, h(n) = 1/depth => depth first One code, many algorithms

Behavior of A* search

Underestimation and overestimation

Underestimation:

If we can guarantee that h never over estimates actual value from current to goal, then A* algorithm is guaranteed to find an optimal path to a goal, if one exists.optimal path to a goal, if one exists.

Consider the following example.

Assume the cost of all arcs is 1. We see that f (E) = 5 = f (C) Suppose we resolve in favor of E, the path currently we are expanding. Clearly expansion of F( f = 6) is stopped and we will now expand C. Thus we see that by underestimating h(B), we have wasted some effort but eventually discovered that B was farther away than we thought. Then we go back and try another path, and will find optimal path.

Overestimation:

Now consider another situation. We expand B to E,E to F and F to G for a solution path of length 4.Butsuppose that there a direct path from D to a solutiongiving a path of length 2.

We will never find it because of overestimating We will never find it because of overestimatingh(D). We may find some other worse solutionwithout ever expanding D.

So by overestimating h, we can not be guaranteed tofind the cheaper path solution.

A Overestimated (1+3) B (1+4) C (1+5) D

(2+2) E(2+2) E

(3+1)F

(4+0) G

Properties of A* A* generates an optimal solution if h(n) is an admissible

heuristic and the search space is a tree: h(n) is admissible if it never overestimates the cost to reach

the destination node

A* generates an optimal solution if h(n) is a consistent heuristic and the search space is a graph:and the search space is a graph: h(n) is consistent if for every node n and for every

successor node n of n:h(n) c(n,n) + h(n)

n

n

dh(n)

c(n,n) h(n)

If h(n) is consistent then h(n) is admissibleFrequently when h(n) is admissible, it is also consistent

Admissibility of A* A search algorithm is admissible, if for any graph, italways terminates in an optimal path from initial state to goalstate, if path exists. If heuristic function h is underestimate of actual valuefrom current state to goal state, then the it is calledadmissible function. So, we can say that A* always terminates with the optimal So, we can say that A* always terminates with the optimalpath in case h(x) is an admissible heuristic function.Monotonicity: A heuristic function h is monotone if1. states Xi and Xj such that Xj is successor of Xi

h(Xi) h(Xj) cost (Xi, Xj) i.e., actual cost ofgoing from Xi to Xj

2. h (goal) = 0

Gracefull Decay of Admissibility If h rarely overestimates h by more than ,

then A* algorithm will rarely find a solution whose cost is more than greater than the cost of the optimal solution.

Under certain conditions, the A* algorithm can be shown to be optimal in that it generates the fewest nodes in the process of finding a solution to a problem.

AND-OR graphs AND-OR graph (or tree) is useful for representing the

solution of problems that can be solved by decomposing them into a set of smaller problems, all of which must then be solved.

One AND arc may point to any number of successor nodes, all of which must be solved in order for the arc to nodes, all of which must be solved in order for the arc to point to a solution.

Goal: Acquire TV Set

Goal: Steal a TV Set Goal: Earn some money Goal: Buy TV Set

AND-OR graph examples

A

B C D5 3 4

A

B C D389

B C D5 3 4 B C D

F GE H I J

5 10 3 4 15 10

17 9 27

Problem Reduction FUTILITY is chosen to correspond to a threshold such than any solution with a

cost above it is too expensive to be practical, even if it could ever be found.

Algorithm : Problem Reduction1. Initialize the graph to the starting node.

2. Loop until the starting node is labeled SOLVED or until its cost goes above FUTILITY:

a. Traverse the graph, a. Traverse the graph, starting at the initial node following the current best path, accumulate the set of nodes that are on that path and have not yet

been expanded or labeled as solved.b. Pick one of these nodes

expand it. If there are no successors, assign FUTILITY as the value of this node. Otherwise, add its successors to the graph for each of them compute f. if f of any node is 0, mark that node as SOLVED.

Problem reductionc. Change the f estimate

of the newly expanded node to reflect the new information provided by its successors.

Propagate this change backward through the graph.

Note:This propagation of revised cost estimates back up the Note:This propagation of revised cost estimates back up the tree was not necessary in the BFS algorithm because only unexpanded nodes were examined. But now expanded nodes must be reexamined so that the best current path can be selected.

AO* algorithm GRAPH consist of node representing initial state call

this state as INIT Let G={s}

Until INIT is labelled SOLVED or h > FUTILITY repeat the following the following select for expansion one of unexpanded node from marked sub

tree and call it NODE n Expand For every successor m of n

Add succ m to graph If succ m is SOLVED , mark it as SOLVED If not SOLVED compute h(m)

Back propagation of the newly discovered information Call cost revision(n)

AO* algorithm contd.. Cost Revision(n) :Let S be set of nodes that have been SOLVED or

whose value has been changed. Repeat following steps till S is empty Initialize S ={n} Select from S a node m (select a node possibly whose

descendent occurs in G).Remove m from S If s={ } return Compute the cost of each arc emerging from m

If m is AND node with succ r1, r2, r3 ........... rk If m is AND node with succ r1, r2, r3 ........... rk Set h(m)= h(ri)+C(m,ri) Mark each succ of m If every succ is SOLVED , mark m as SOLVED

If m is OR node with succ r1, r2 , r3 ............. rk Set h(m) = min( h(ri)+c(m,ri)) Mark the best succ If best succ is SOLVED mark m as SOLVED

If cost of m has changed then change must be propagated , hence insert all ancestor of m in S for which m is marked successor

Some observations Cost must be propagated to all path

backwards . Hence all ancestor must be added to S- why?

Problem may come if graph contain cycles Problem may come if graph contain cycles At time longer path may be better than

shorter path Disadvantage : No interaction amongst

subgoals

Constraint Satisfaction Constraint Satisfaction problems in AI have goal of

discovering some problem state that satisfies a given set of constraints.

Design tasks can be viewed as constraint satisfaction problems in which a design must be created within fixed limits on time, cost, and materials.

Constraint satisfaction is a search procedure that operates in a space of constraint sets. The initial state contains the constraints that are originally given in the

problem description. A goal state is any state that has been constrained enough where

enough must be defined for each problem. For example, in crypt arithmetic, enough means that each letter has

been assigned a unique numeric value.

Constraint Satisfaction Constraint Satisfaction is a two step process:

First constraints are discovered and propagated as far as possible throughout the system.

Then if there still not a solution, search begins. A guess about something is made and added as a new constraint Propagation can then occur with this new constraint Propagation can then occur with this new constraint

First step propagation occurs from fact that there are dependencies amongst constraint These dependencies occur because many constraints involve more than one object and many object participate in more than one constraint

Constraint propagation also occurs due to presence of inference rules that allow additional constraints to be added which are inferred form the given ones.

Algorithm: Constraint Satisfaction1. Propagate available constraints.

a. Set OPEN to set of all objects that must have values assigned to them in a complete solution.

b. Then do until an inconsistency is detected or until OPEN is empty:

i. Select an object OB from OPEN. Strengthen as much as possible the set of constraints that apply to OB.of constraints that apply to OB.

ii. If this set is different from the set that was assigned the last time OB was examined or if this is the first time OB has been examined, then add to OPEN all objects that share any constraints with OB.

iii. Remove OB from OPEN.

2. If the union of the constraints discovered above defines a solution, then quit and report the solution.

Algorithm: Constraint Satisfaction3. If the union of the constraints discovered above defines a

contradiction, then return the failure.

4. If neither of the above occurs, then make a guess at in order to proceed.

5. To do this loop until a solution is found or all possible solutions have been eliminated:

a. Select an object whose value is not yet determined b. select a way of strengthening the constraints on that object.c. Recursively invoke constraint satisfaction with the current set of

constraints augmented by strengthening constraint just selected.

Constraint Satisfaction: Example Cryptarithmetic Problem:SEND

+MORE-----------

MONEYMONEYInitial State: No two letters have the same value The sums of the digits must be as shown in the problemGoal State: All letters have been assigned a digit in such a way that

all the initial constraints are satisfied.

Cryptasithmetic Problem: Constraint Satisfaction

The solution process proceeds in cycles. At each cycle, two significant things are done:

1. Constraints are propagated by using rules that correspond to the properties of arithmetic.

2. A value is guessed for some letter whose value is not yet determined.

A few Heuristics can help to select the best guess to try first:

If there is a letter that has only two possible values and other with six possible values, there is a better chance of guessing right on the first than on the second.

Another useful Heuristic is that if there is a letter that participates in many constraints then it is a good idea to prefer it to a letter that participates in a few.

Solving a Cryptarithmetic ProblemInitial state

M=1

S= 8 or 9O = 0 or 1 -> O =0,S=9N= E or E+1 -> N= E+1

C2 = 1,C3=0N+R >8

C4 C3 C2 C1 S E N D

+ M O R EM O N E Y

N+R >8E 9

N=3R= 8 or 92+D = Y or 2+D = 10 +Y

2+D =Y

N+R = 10+ER =9S =8

2+D = 10 +YD = 8+YD = 8 or 9

Y= 0 ; Conflict Y =1 ; Conflict

E=2

C2= 0 C2= 1

D=8 D=9

E=5N=6;6+R=5 or 15R=8,9=> R=8

C2+N+R=14 E=4;Conflict

C2+N+R=15E=5

D+5=10+YD=7;Y=2

C2= 0 C2= 1

Game Playing Games as search problems Efforts in developing game playing algorithm

Charles Babbage and Bowden thought of extending analytical engine to play chess

Shannon wrote a paper describing program that can play chess Shannon wrote a paper describing program that can play chess Alan Turing described chess playing algo, which could not be

built Samuel succeeded in building first significant operational game

playing program checkers it could play and learn from mistake

Game Playing Two main reason that game appears to be a good

domain for machine intelligence: Provide a structured task in which it is easy to find

success or failure Did not require large amount of knowledge Did not require large amount of knowledge

Second reason is true only for small games (tic-tac-toe) Example Chess

Average branching factor = 35 On an average game one player can make 50 moves

To examine a complete tree we would have to examine 35100positions

Game Playing Search Algorithm like generate and test

Generate: generates all possible moves Tester: Evaluates each move

In Chess all 35 moves are genearte and In Chess all 35 moves are genearte and tester must evaluate each of them

A better solution is to have Improved generator which will produce only

good path Improved test procedure so that best moves

will be recognized first

Game Playing Use Plausible move generator in which small number of promising

move are generated With such a generator, test procedure can spend more time in

evaluating these nodes. Search procedure will find a path till goal is reached

Game Playing

Goal: winning / loosing state Hence, like heuristics , use static evaluation function to find the best

node Tic-tac-toe = number of winning moves for X- number of winning

moves for O Chess: add number of blacks(w), add number of whites(W) quotient Chess: add number of blacks(w), add number of whites(W) quotient

W/BTwo concern parameter in game search is depth and branching factor

Even with plausible generator, in games like chess, it is not possible to search till goal is found.

MinMax Algorithm Idealization and simplification:

Depth first depth limited search Two players Alternate moves Alternate moves

MAX player MIN player

Available information: Perfect: chess, chequers, tic-tac-toe (no chance,

same knowledge for the two players) Imperfect: poker, Stratego, bridge

Game representation Searching a game tree using OR graph

with two types of nodes Min Node: select minimum cost successor Max Node: select maximum cost successor Max Node: select maximum cost successor Terminal nodes: winning or loosing states Usually impossible to search upto terminal

nodes

Max

Min

Can propagate W or L from bottom up to root nodeDifficult is state space is largeCan be done for simple game like tic-tac-toe

Game playing-MinMax Games like chess ;complete state space cannot

be analyzed We expand the nodes to certain depth (ply) Apply heuristic function Apply heuristic function Knowledge of game is required to calculate the

cost Apply minmax: its like look ahead strategy Also called zero sum game as

if A gets more B will get less If A win B will loose

Search with an opponent

Search with an opponent Trivial approximation: generating the tree for

all moves Terminal moves are tagged with a utility value,

for example: +1 or -1 depending on if the winner is MAX or MIN.

The goal is to find a path to a winning state. Even if a depth-first search would minimize

memory space, in complex games this kind of search cannot be carried out.

Even a simple game like tic-tac-toe is too complex to draw the entire game tree.

Search with an opponent

Search with an opponent Heuristic approximation: defining an evaluation

function which indicates how close a state is from a winning (or losing) move

This function includes domain information. It does not represent a cost or a distance in steps. It does not represent a cost or a distance in steps. Conventionally:

A winning move is represented by the value+. A losing move is represented by the value -. The algorithm searches with limited depth.

Each new decision implies repeating part of the search.

Minimax

Minimax-value(n): utility for MAX of being in state n, assuming both players are playing optimally =

Utility(n), if n is a terminal state maxs Successors(n) Minimax-value(s), if n is a MAX state mins Successors(n) Minimax-value(s), if n is a MIN state

Example: tic-tac-toe e (evaluation function integer) =

number of available rows, columns, diagonals for MAX - number of available rows, columns, diagonals for MIN

MAX plays with X and desires maximizing e.

MIN plays with 0 and desires minimizing e.

Symmetries are taken into account. A depth limit is used (2, in the example).

Example: tic-tac-toe

The minimax algorithm It is depth first depth limited search

Uses plausible move generator to generate the promising successor(moves/nodes)

Uses Static evaluation function with large values indicating good values for us

99 99

values for us +10 -------------------------- -10Win for us win for opponent

Value generated by static evaluation function is backed up to parents node

Since static evaluation function is an estimate , it is better to carry search more than one ply in game tree look ahead

The minimax algorithmOur goal is to maximize our moveOpponent goal is to minimize our move Hence the name minmax algorithm

A

(-6)(-2) (-4)

(-2)

100 100

B C D

E F G H I J K

(9) (-6) (0) (0) (-2) (-4) (-3)

(-6)(-2) (-4)

Initial call to minmax will be minmax(current,0,p1)-if player 1 starts otherwise minmax(current,0,p2)

The minimax algorithmRequirement:

MOVEGEN (Position Player) generates list of nodes representing the moves that can be made by player in position

STATIC(Position , Player) returns a number representing goodness of position with respect to player

101 101

with respect to player

When to stop recursion ?Uses function DEEP ENOUGH which returnstrue if search can be stoppedFalse otherwise

Return value by the algorithmValue : backed up value Path itself

Hence a structure containing both the values is returned

Minmax (position, depth, player ){Step1:If DEEP-ENOUGH(position,player) then return structure

VALUE=STATIC(position , player)PATH=nil

Step2:Else{

SUCCESSORS=MOVE_GEN(position,player)Step3: if (SUCCESSOR= empty) return structure // same as DEEP_ENOUGH

Step4: if (! Empty(SUCCESSORSvalue )){ BEST_SCORE=min that STATIC can generate

102

{ BEST_SCORE=min that STATIC can generateFor every SUCC of SUCCESSOR

4(a) { RESULT_SUCC= MINMAX(SUCC, depth+1,OPPOSITE(player)

4(b) NEW-VALUE= - VALUE(RESULT_SUCC)4(c) if NEW-VALUE > BEST_SCORE

(i) { BSET_SCORE=NEW-VALUE(ii) BEST-PATH= SUCC + PATH(RESULT_SUCC)

path from CURRENT to SUCC .}// end if

}// end for

Step5: value=BEST_SCORE path=BEST_PATH return structure }// end if }// else }// minmax

Minmax(A,0,p1)1 false2 move-gen(A,p1)3 F 4 succ=(B,C,D)

for Bresult_suc=minmax(B,1,p2)1 F2 move-gen(b,p2)3 F4 succ=(E,F,G)4 succ=(E,F,G)

for Eresult_succ=minmax(E,2,p1)

1. Tval=static(E,p1)=9path=nilreturn (path=nil,val(9))

newval=-9best-score=-9best-path=Ereturn(BP,BS)

The minimax algorithm

105

The algorithm first recurses down to the tree bottom-left nodes and uses the Utility function on them to

discover that their values are 3, 12 and 8.105

The minimax algorithm

B

A

106

Then it takes the minimum of these values, 3, and returns it as the backed-up value of node B.

Similar process for the other nodes. 106

The minimax algorithm The minimax algorithm performs a

complete depth-first exploration of the game tree.

In minimax, at each point in the process,

107

In minimax, at each point in the process, only the nodes along a path of the tree are considered and kept in memory.

107

The minimax algorithm If the maximum depth of the tree is m, and

there are b legal moves at each point, then the time complexity is O(bm).

The space complexity is:

108

The space complexity is: O(bm) for an algorithm that generates all

successors at once O(m) if it generates successors one at a time.

108

The minimax algorithm: problems

For real games the time cost of minimax is totally impractical, but this algorithm serves as the basis: for the mathematical analysis of games and

109

for the mathematical analysis of games and for more practical algorithms

Problem with minimax search: The number of game states it has to examine

is exponential in the number of moves. Unfortunately, the exponent cant be

eliminated, but it can be cut in half. 109

Alpha-beta pruning It is possible to compute the correct

minimax decision without looking at every node in the game tree.

Alpha-beta pruning allows to eliminate

110

Alpha-beta pruning allows to eliminate large parts of the tree from consideration, without influencing the final decision.

110

BAlpha-beta pruning

The leaves below B have the values 3, 12 and 8. The value of B is exactly 3. It can be inferred that the value at the root is at least

3, because MAX has a choice worth 3.111 111

Alpha-beta pruning

B C

C, which is a MIN node, has a value of at most 2. But B is worth 3, so MAX would never choose C. Therefore, there is no point in looking at the other

successors of C.112 112

Alpha-beta pruning

B C D

D, which is a MIN node, is worth at most 14. This is still higher than MAXs best alternative (i.e.,

3), so Ds other successors are explored.113 113

Alpha-beta pruning

B C D

114

The second successor of D is worth 5, so the exploration continues.

114

Alpha-beta pruning

B C D

115

The third successor is worth 2, so now D is worth exactly 2.

MAXs decision at the root is to move to B, giving a value of 3

115

Alpha-beta pruning Alpha-beta pruning gets its name from two

parameters. They describe bounds on the values that

appear anywhere along the path under

116

appear anywhere along the path under consideration:

= the value of the best (i.e., highest value) choice found so far along the path for MAX

= the value of the best (i.e., lowest value) choice found so far along the path for MIN

116

Alpha-beta pruning Alpha-beta search updates the values of

and as it goes along. It prunes the remaining branches at a

node (i.e., terminates the recursive call)

117

node (i.e., terminates the recursive call) as soon as the value of the current node is

known to be worse than the current or value for MAX or MIN, respectively.

117

Alpha-beta bounds Alpha bound of j

Maximum current value of all max ancestor of j

Exploration of a min node ,j, is stopped when its value equals or falls below alphaits value equals or falls below alpha

In min node we update beta Beta bound of j

The minimum current of all MIN ancestor of j Exploration of a max node j is stopped when

its value equals or exceeds beta In a MAX node we update beta

MAX

Vi

{, }If Vi > , modify If Vi , pruning

Return

Alpha-beta pruning

{, }If Vi < , modify If Vi , pruning

Return

MIN

Vi

and bounds are transmitted from parent to child in the order of node visit. The effectiveness of pruning highly depends on the order in which successors are examined.

The I subtree can be pruned, because I is a

A

CB

ED

3

{-, +}

A

CB

ED

3 5

3{-, 3}

A

CB3

{3, +}

{3, +}

{alpha = -, beta = +}

{-, 3}{-, +}

{3, +}

pruned, because I is a min node and the value of v(K) = 0 is < = 3

CB

D

3

HF G

JI

LK

{3, +}{3, +}

{3, +}

0

ACB

D

3

HF

{3, +}

G

J

{3, +}

{3, +}

5

5

A

CB

D

3

HF

{3, +}

G

J

{3, 5}

5

5

NM 7A 4

{3, 5}

The G subtree can be pruned, because G is a max node and the value of v(M) = 7 is > = 5

A

CB

D

3

HF

4

J

4

5

5 4

Minmax _A_B(position, depth, player ,UT,PT){Step1:If DEEP-ENOUGH(position,player) then return structure

VALUE=STATIC(position , player)PATH=nil

Step2:Else{

SUCCESSORS=MOVE_GEN(position,player)Step3: if (SUCCESSOR= empty) return structure // same as DEEP_ENOUGH

Step4: if (! Empty(SUCCESSORSvalue )){ BEST_SCORE=min that STATIC can generate

For every SUCC of SUCCESSORFor every SUCC of SUCCESSOR4(a) { RESULT_SUCC= MINMAX_A_B(SUCC, depth+1,OPPOSITE(player),-PT,-UT)4(b) NEW-VALUE= - VALUE(RESULT_SUCC)4(c) if NEW-VALUE > PT

(i) { PT=NEW-VALUE(ii) BEST-PATH= SUCC + PATH(RESULT_SUCC)

path from CURRENT to SUCC .}// end if

(d) if PT >= UT then stop examining this branch and return value=PT path=BEST-PATH}// end for

Step5: value=PTpath=BEST_PATHreturn structure

}// end if}// else

Some observation in alpha beta Effectiveness of alpha beta depends upon the order in

which the nodes are considered If nodes are perfectly ordered then

If x= number of terminal nodes in alpha beta at depth d And y= number of terminal nodes without alpha beta at depth d/2 And y= number of terminal nodes without alpha beta at depth d/2 Then x=2*y Doubling the depth only at cost of double the number of terminal

nodes implies a significant gain This idea can be extended to cut off path that are only

slightly better than current path Exploration of subtree is terminated that offers little possibility of

improvement , hence called futility cut off

Additional refinements Waiting for Quiescence

At times great difference in value is observed as we move from one level to another

Our decision to change path should not occur on such short term measures

Secondary search If total expansion was upto n level we can expand one node an

extra level to reassure the path-singular extension Book moves maintaining the catalogue of all moves (list

of possible moves Incorporates knowledge and search together

Final comments about alpha-beta pruning

Pruning does not affect final results. Entire subtrees can be pruned, not just leaves. Good move ordering improves effectiveness of

pruning.

125

pruning. With perfect ordering, time complexity is

O(bm/2). Effective branching factor of sqrt(b) Consequence: alpha-beta pruning can look twice as

deep as minimax in the same amount of time.

125

SummaryFour steps to design AI Problem solving:1. Define the problem precisely. Specify the

problem space, the operators for moving within the space, and the starting and goal state.

2. Analyze the problem to determine where it falls 2. Analyze the problem to determine where it falls with respect to seven important issues.

3. Isolate and represent the task knowledge required

4. Choose problem solving technique and apply them to problem.

Summary What the states in search spaces represent. Sometimes the states

represent complete potential solutions. Sometimes they represent solutions that are partially specified.

How, at each stage of the search process, a state is selected for expansion.

How operators to be applied to that node are selected. Whether an optimal solution can be guaranteed. Whether an optimal solution can be guaranteed. Whether a given state may end up being considered more than

once. How many state descriptions must be maintained throughout the

search process. Under what circumstances should a particular search path be

abandoned.

ai search std

Documents