ITCS 3153Artificial Intelligence

Lecture 6Lecture 6

Informed SearchesInformed Searches

Lecture 6Lecture 6

Informed SearchesInformed Searches

Compare two heuristics

Compare these two heuristics

hh22 is always better than h is always better than h11

• for any node, n, hfor any node, n, h22(n) >= h(n) >= h11(n)(n)

• hh22 dominatesdominates h h11

• Recall all nodes with f(n) < C* will be expanded?Recall all nodes with f(n) < C* will be expanded?

– This means all nodes, h(n) < C* - g(n), will be expandedThis means all nodes, h(n) < C* - g(n), will be expanded

– All nodes hAll nodes h22 expands will also be expanded by h expands will also be expanded by h11 and and

because hbecause h11 is smaller, others will be expanded as well is smaller, others will be expanded as well

hh22 is always better than h is always better than h11

• for any node, n, hfor any node, n, h22(n) >= h(n) >= h11(n)(n)

• hh22 dominatesdominates h h11

• Recall all nodes with f(n) < C* will be expanded?Recall all nodes with f(n) < C* will be expanded?

– This means all nodes, h(n) < C* - g(n), will be expandedThis means all nodes, h(n) < C* - g(n), will be expanded

– All nodes hAll nodes h22 expands will also be expanded by h expands will also be expanded by h11 and and

because hbecause h11 is smaller, others will be expanded as well is smaller, others will be expanded as well

Inventing admissible heuristic funcs

How can you create h(n)?How can you create h(n)?

• Simplify problem by reducing restrictions on actionsSimplify problem by reducing restrictions on actions

– Allow 8-puzzle pieces to sit atop on anotherAllow 8-puzzle pieces to sit atop on another

– Call this a Call this a relaxed problemrelaxed problem

– The cost of optimal solution to relaxed problem is The cost of optimal solution to relaxed problem is admissible heuristic for original problemadmissible heuristic for original problem

It is at least as expensive for the original problemIt is at least as expensive for the original problem

How can you create h(n)?How can you create h(n)?

• Simplify problem by reducing restrictions on actionsSimplify problem by reducing restrictions on actions

– Allow 8-puzzle pieces to sit atop on anotherAllow 8-puzzle pieces to sit atop on another

– Call this a Call this a relaxed problemrelaxed problem

– The cost of optimal solution to relaxed problem is The cost of optimal solution to relaxed problem is admissible heuristic for original problemadmissible heuristic for original problem

It is at least as expensive for the original problemIt is at least as expensive for the original problem

Examples of relaxed problems

A tile can move from square A tile can move from square AA to square to square BB if if

AA is horizontally or vertically adjacent to is horizontally or vertically adjacent to BB

and and BB is blank is blank

• A tile can move from A to B if A is adjacent to B (overlap)A tile can move from A to B if A is adjacent to B (overlap)

• A tile can move from A to B if B is blank (teleport)A tile can move from A to B if B is blank (teleport)

• A tile can move from A to B (teleport and overlap)A tile can move from A to B (teleport and overlap)

Solutions to these relaxed problems can be computed Solutions to these relaxed problems can be computed without search and therefore heuristic is easy to computewithout search and therefore heuristic is easy to compute

A tile can move from square A tile can move from square AA to square to square BB if if

AA is horizontally or vertically adjacent to is horizontally or vertically adjacent to BB

and and BB is blank is blank

• A tile can move from A to B if A is adjacent to B (overlap)A tile can move from A to B if A is adjacent to B (overlap)

• A tile can move from A to B if B is blank (teleport)A tile can move from A to B if B is blank (teleport)

• A tile can move from A to B (teleport and overlap)A tile can move from A to B (teleport and overlap)

Solutions to these relaxed problems can be computed Solutions to these relaxed problems can be computed without search and therefore heuristic is easy to computewithout search and therefore heuristic is easy to compute

Multiple Heuristics

If multiple heuristics available:If multiple heuristics available:

• h(n) = max {hh(n) = max {h11(n), h(n), h22(n), …, h(n), …, hmm(n)}(n)}

If multiple heuristics available:If multiple heuristics available:

• h(n) = max {hh(n) = max {h11(n), h(n), h22(n), …, h(n), …, hmm(n)}(n)}

Use solution to subproblem as heuristic

What is optimal cost of solving some portion of What is optimal cost of solving some portion of original problem?original problem?

• subproblem solution is heuristic of original problemsubproblem solution is heuristic of original problem

What is optimal cost of solving some portion of What is optimal cost of solving some portion of original problem?original problem?

• subproblem solution is heuristic of original problemsubproblem solution is heuristic of original problem

Pattern Databases

Store optimal solutions to subproblems in databaseStore optimal solutions to subproblems in database

• We use an exhaustive search to solve every permutation of the We use an exhaustive search to solve every permutation of the 1,2,3,4 piece subproblem of the 8-puzzle1,2,3,4 piece subproblem of the 8-puzzle

• During solution of 8-puzzle, look up optimal cost to solve the During solution of 8-puzzle, look up optimal cost to solve the 1,2,3,4 piece subproblem and use as heuristic1,2,3,4 piece subproblem and use as heuristic

Store optimal solutions to subproblems in databaseStore optimal solutions to subproblems in database

• We use an exhaustive search to solve every permutation of the We use an exhaustive search to solve every permutation of the 1,2,3,4 piece subproblem of the 8-puzzle1,2,3,4 piece subproblem of the 8-puzzle

• During solution of 8-puzzle, look up optimal cost to solve the During solution of 8-puzzle, look up optimal cost to solve the 1,2,3,4 piece subproblem and use as heuristic1,2,3,4 piece subproblem and use as heuristic

Learning

Could also build pattern database while solving Could also build pattern database while solving cases of the 8-puzzlecases of the 8-puzzle

• Must keep track of intermediate states and true final cost of Must keep track of intermediate states and true final cost of solutionsolution

• Inductive learningInductive learning builds mapping of state -> cost builds mapping of state -> cost

• Because too many permutations of actual statesBecause too many permutations of actual states

– Construct important Construct important features features to reduce size of spaceto reduce size of space

Could also build pattern database while solving Could also build pattern database while solving cases of the 8-puzzlecases of the 8-puzzle

• Must keep track of intermediate states and true final cost of Must keep track of intermediate states and true final cost of solutionsolution

• Inductive learningInductive learning builds mapping of state -> cost builds mapping of state -> cost

• Because too many permutations of actual statesBecause too many permutations of actual states

– Construct important Construct important features features to reduce size of spaceto reduce size of space

Local Search Algorithms andOptimization Problems

Characterize Techniques

Uninformed SearchUninformed Search• Looking for a solution where solution is a Looking for a solution where solution is a pathpath from start to from start to

goalgoal

• At each intermediate point along a path, we have no At each intermediate point along a path, we have no prediction of value of pathprediction of value of path

Informed SearchInformed Search• Again, looking for a path from start to goalAgain, looking for a path from start to goal

• This time we have insight regarding the value of intermediate This time we have insight regarding the value of intermediate solutionssolutions

Uninformed SearchUninformed Search• Looking for a solution where solution is a Looking for a solution where solution is a pathpath from start to from start to

goalgoal

• At each intermediate point along a path, we have no At each intermediate point along a path, we have no prediction of value of pathprediction of value of path

Informed SearchInformed Search• Again, looking for a path from start to goalAgain, looking for a path from start to goal

• This time we have insight regarding the value of intermediate This time we have insight regarding the value of intermediate solutionssolutions

Now change things a bit

What if the path isn’t important, just the goal?What if the path isn’t important, just the goal?

• So the goal is unknownSo the goal is unknown

• The path to the goal need not be solvedThe path to the goal need not be solved

ExamplesExamples

• What quantities of quarters, nickels, and dimes add up to What quantities of quarters, nickels, and dimes add up to $17.45 and minimizes the total number of coins$17.45 and minimizes the total number of coins

• Is the price of Microsoft stock going up tomorrow?Is the price of Microsoft stock going up tomorrow?

What if the path isn’t important, just the goal?What if the path isn’t important, just the goal?

• So the goal is unknownSo the goal is unknown

• The path to the goal need not be solvedThe path to the goal need not be solved

ExamplesExamples

• What quantities of quarters, nickels, and dimes add up to What quantities of quarters, nickels, and dimes add up to $17.45 and minimizes the total number of coins$17.45 and minimizes the total number of coins

• Is the price of Microsoft stock going up tomorrow?Is the price of Microsoft stock going up tomorrow?

Local Search

Local search does not keep track of previous Local search does not keep track of previous solutionssolutions• Instead it keeps track of current solution (current state)Instead it keeps track of current solution (current state)

• Uses a method of generating alternative solution candidatesUses a method of generating alternative solution candidates

AdvantagesAdvantages• Use a small amount of memory (usually constant amount)Use a small amount of memory (usually constant amount)

• They can find reasonable (note we aren’t saying optimal) They can find reasonable (note we aren’t saying optimal) solutions in infinite search spaces solutions in infinite search spaces

Local search does not keep track of previous Local search does not keep track of previous solutionssolutions• Instead it keeps track of current solution (current state)Instead it keeps track of current solution (current state)

• Uses a method of generating alternative solution candidatesUses a method of generating alternative solution candidates

AdvantagesAdvantages• Use a small amount of memory (usually constant amount)Use a small amount of memory (usually constant amount)

• They can find reasonable (note we aren’t saying optimal) They can find reasonable (note we aren’t saying optimal) solutions in infinite search spaces solutions in infinite search spaces

Optimization Problems

Objective FunctionObjective Function• A function with vector inputs and scalar outputA function with vector inputs and scalar output

– goal is to search through candidate input vectors in order goal is to search through candidate input vectors in order to minimize or maximize objective functionto minimize or maximize objective function

ExampleExample• f (q, d, n) = 1,000,000 if q*0.25 + d*0.1 + n*0.05 != 17.45f (q, d, n) = 1,000,000 if q*0.25 + d*0.1 + n*0.05 != 17.45

= q + n + d otherwise = q + n + d otherwise

• minimize fminimize f

Objective FunctionObjective Function• A function with vector inputs and scalar outputA function with vector inputs and scalar output

– goal is to search through candidate input vectors in order goal is to search through candidate input vectors in order to minimize or maximize objective functionto minimize or maximize objective function

ExampleExample• f (q, d, n) = 1,000,000 if q*0.25 + d*0.1 + n*0.05 != 17.45f (q, d, n) = 1,000,000 if q*0.25 + d*0.1 + n*0.05 != 17.45

= q + n + d otherwise = q + n + d otherwise

• minimize fminimize f

Search Space

The realm of feasible input vectorsThe realm of feasible input vectors• Also called state space landscapeAlso called state space landscape

• Usually described byUsually described by

– number of dimensions (3 for our change example)number of dimensions (3 for our change example)

– domain of dimensions (q is discrete from 0 to 69…)domain of dimensions (q is discrete from 0 to 69…)

– nature of relationship between input vector and objective function nature of relationship between input vector and objective function outputoutput

no relationshipno relationship

smoothly varyingsmoothly varying

discontinuitiesdiscontinuities

The realm of feasible input vectorsThe realm of feasible input vectors• Also called state space landscapeAlso called state space landscape

• Usually described byUsually described by

– number of dimensions (3 for our change example)number of dimensions (3 for our change example)

– domain of dimensions (q is discrete from 0 to 69…)domain of dimensions (q is discrete from 0 to 69…)

– nature of relationship between input vector and objective function nature of relationship between input vector and objective function outputoutput

no relationshipno relationship

smoothly varyingsmoothly varying

discontinuitiesdiscontinuities

Search Space

Looking for global maximum (or minimum)Looking for global maximum (or minimum)Looking for global maximum (or minimum)Looking for global maximum (or minimum)

Hill Climbing

Also called Greedy SearchAlso called Greedy Search• Select a starting point and set Select a starting point and set currentcurrent

• evaluate (evaluate (current)current)

• loop doloop do

– neighborneighbor = highest value successor of = highest value successor of currentcurrent

– if evaluate (if evaluate (neighborneighbor) <= evaluate () <= evaluate (currentcurrent))

return return currentcurrent

– else else currentcurrent = = neighborneighbor

Also called Greedy SearchAlso called Greedy Search• Select a starting point and set Select a starting point and set currentcurrent

• evaluate (evaluate (current)current)

• loop doloop do

– neighborneighbor = highest value successor of = highest value successor of currentcurrent

– if evaluate (if evaluate (neighborneighbor) <= evaluate () <= evaluate (currentcurrent))

return return currentcurrent

– else else currentcurrent = = neighborneighbor

Hill climbing gets stuck

Hiking metaphor (you are wearing glasses that Hiking metaphor (you are wearing glasses that limit your vision to 10 feet)limit your vision to 10 feet)Hiking metaphor (you are wearing glasses that Hiking metaphor (you are wearing glasses that limit your vision to 10 feet)limit your vision to 10 feet)

• Local maximaLocal maxima

– RidgesRidges

• PlateauPlateau

– why is this a problem?why is this a problem?

• Local maximaLocal maxima

– RidgesRidges

• PlateauPlateau

– why is this a problem?why is this a problem?

Hill Climbing Gadgets

Variants on hill climbing play special rolesVariants on hill climbing play special roles

• stochastic hill climbingstochastic hill climbing

– don’t always choose the best successordon’t always choose the best successor

• first-choice hill climbingfirst-choice hill climbing

– pick the first good successor you findpick the first good successor you find

useful if number of successors is largeuseful if number of successors is large

• random restartrandom restart

– follow steepest ascent from multiple starting statesfollow steepest ascent from multiple starting states

– probability of finding global max increases with number of startsprobability of finding global max increases with number of starts

Variants on hill climbing play special rolesVariants on hill climbing play special roles

• stochastic hill climbingstochastic hill climbing

– don’t always choose the best successordon’t always choose the best successor

• first-choice hill climbingfirst-choice hill climbing

– pick the first good successor you findpick the first good successor you find

useful if number of successors is largeuseful if number of successors is large

• random restartrandom restart

– follow steepest ascent from multiple starting statesfollow steepest ascent from multiple starting states

– probability of finding global max increases with number of startsprobability of finding global max increases with number of starts

Hill Climbing Usefulness

It DependsIt Depends

• Shape of state space greatly influences hill climbingShape of state space greatly influences hill climbing

• local maxima are the Achilles heellocal maxima are the Achilles heel

• what is cost of evaluation?what is cost of evaluation?

• what is cost of finding a random starting location?what is cost of finding a random starting location?

It DependsIt Depends

• Shape of state space greatly influences hill climbingShape of state space greatly influences hill climbing

• local maxima are the Achilles heellocal maxima are the Achilles heel

• what is cost of evaluation?what is cost of evaluation?

• what is cost of finding a random starting location?what is cost of finding a random starting location?

Simulated Annealing

A term borrowed from metalworkingA term borrowed from metalworking• We want metal molecules to find a stable location relative to We want metal molecules to find a stable location relative to

neighborsneighbors

• heating causes metal molecules to move around and to take on heating causes metal molecules to move around and to take on undesirable locationsundesirable locations

• during cooling, molecules reduce their movement and settle during cooling, molecules reduce their movement and settle into a more stable position into a more stable position

• annealing is process of heating metal and letting it cool slowly annealing is process of heating metal and letting it cool slowly to lock in the stable locations of the moleculesto lock in the stable locations of the molecules

A term borrowed from metalworkingA term borrowed from metalworking• We want metal molecules to find a stable location relative to We want metal molecules to find a stable location relative to

neighborsneighbors

• heating causes metal molecules to move around and to take on heating causes metal molecules to move around and to take on undesirable locationsundesirable locations

• during cooling, molecules reduce their movement and settle during cooling, molecules reduce their movement and settle into a more stable position into a more stable position

• annealing is process of heating metal and letting it cool slowly annealing is process of heating metal and letting it cool slowly to lock in the stable locations of the moleculesto lock in the stable locations of the molecules

Simulated Annealing

Be the BallBe the Ball

• You have a wrinkled sheet of metalYou have a wrinkled sheet of metal

• Place a BB on the sheet and what happens?Place a BB on the sheet and what happens?

– BB rolls downhillBB rolls downhill

– BB stops at bottom of hill (local or global min?)BB stops at bottom of hill (local or global min?)

– BB momentum carries it out of hill into another (local or global)BB momentum carries it out of hill into another (local or global)

• By shaking metal sheet, your are adding energy (heat)By shaking metal sheet, your are adding energy (heat)

• How hard do you shake?How hard do you shake?

Be the BallBe the Ball

• You have a wrinkled sheet of metalYou have a wrinkled sheet of metal

• Place a BB on the sheet and what happens?Place a BB on the sheet and what happens?

– BB rolls downhillBB rolls downhill

– BB stops at bottom of hill (local or global min?)BB stops at bottom of hill (local or global min?)

– BB momentum carries it out of hill into another (local or global)BB momentum carries it out of hill into another (local or global)

• By shaking metal sheet, your are adding energy (heat)By shaking metal sheet, your are adding energy (heat)

• How hard do you shake?How hard do you shake?

Our Simulated Annealing Algorithm

You’re not being the ball, Danny You’re not being the ball, Danny (Caddy Shack)(Caddy Shack)

• Gravity is great because it tells the ball which way is downhill Gravity is great because it tells the ball which way is downhill at all timesat all times

• We don’t have gravity, so how do we decide a successor We don’t have gravity, so how do we decide a successor state?state?

– RandomnessRandomness

aka aka Monte CarloMonte Carlo

aka aka StochasticStochastic

You’re not being the ball, Danny You’re not being the ball, Danny (Caddy Shack)(Caddy Shack)

• Gravity is great because it tells the ball which way is downhill Gravity is great because it tells the ball which way is downhill at all timesat all times

• We don’t have gravity, so how do we decide a successor We don’t have gravity, so how do we decide a successor state?state?

– RandomnessRandomness

aka aka Monte CarloMonte Carlo

aka aka StochasticStochastic

Algorithm OutlineSelect some initial guess of evaluation function parameters: xSelect some initial guess of evaluation function parameters: x00

Evaluate evaluation function, E(xEvaluate evaluation function, E(x00)=v)=v

Compute a random displacement, x’Compute a random displacement, x’00

• The Monte Carlo eventThe Monte Carlo event

Evaluate E(x’Evaluate E(x’00) = v’) = v’

• If v’ < v; set new state, xIf v’ < v; set new state, x11 = x’ = x’00

• Else set xElse set x11 = x’ = x’00 with Prob(E,T) with Prob(E,T)

– This is the Metropolis stepThis is the Metropolis step

Repeat with updated state and tempRepeat with updated state and temp

Select some initial guess of evaluation function parameters: xSelect some initial guess of evaluation function parameters: x00

Evaluate evaluation function, E(xEvaluate evaluation function, E(x00)=v)=v

Compute a random displacement, x’Compute a random displacement, x’00

• The Monte Carlo eventThe Monte Carlo event

Evaluate E(x’Evaluate E(x’00) = v’) = v’

• If v’ < v; set new state, xIf v’ < v; set new state, x11 = x’ = x’00

• Else set xElse set x11 = x’ = x’00 with Prob(E,T) with Prob(E,T)

– This is the Metropolis stepThis is the Metropolis step

Repeat with updated state and tempRepeat with updated state and temp

Metropolis StepWe approximate nature’s alignment of molecules by allowing We approximate nature’s alignment of molecules by allowing uphill transitions with probabilityuphill transitions with probability• Prob (in energy state E) ~ exp (-E/kT) Prob (in energy state E) ~ exp (-E/kT)

– Boltzmann Probabilty DistributionBoltzmann Probabilty Distribution

– Even when T is small, there is still a chance in high energy stateEven when T is small, there is still a chance in high energy state

• Prob (transferring from EProb (transferring from E11 to E to E22) = exp (-(E) = exp (-(E22-E-E11) / kT)) / kT)

– Metropolis StepMetropolis Step

– if Eif E22 < E < E11, prob () is greater than 1, prob () is greater than 1

– if Eif E22 > E > E11, we may transfer to higher energy state, we may transfer to higher energy state

The rate at which T is decreased and the amount The rate at which T is decreased and the amount it is decreased is prescribed by anit is decreased is prescribed by an annealing schedule annealing schedule

We approximate nature’s alignment of molecules by allowing We approximate nature’s alignment of molecules by allowing uphill transitions with probabilityuphill transitions with probability• Prob (in energy state E) ~ exp (-E/kT) Prob (in energy state E) ~ exp (-E/kT)

– Boltzmann Probabilty DistributionBoltzmann Probabilty Distribution

– Even when T is small, there is still a chance in high energy stateEven when T is small, there is still a chance in high energy state

• Prob (transferring from EProb (transferring from E11 to E to E22) = exp (-(E) = exp (-(E22-E-E11) / kT)) / kT)

– Metropolis StepMetropolis Step

– if Eif E22 < E < E11, prob () is greater than 1, prob () is greater than 1

– if Eif E22 > E > E11, we may transfer to higher energy state, we may transfer to higher energy state

The rate at which T is decreased and the amount The rate at which T is decreased and the amount it is decreased is prescribed by anit is decreased is prescribed by an annealing schedule annealing schedule

What have we got?

Always move downhill if possibleAlways move downhill if possible

Sometimes go uphillSometimes go uphill

• More likely at start when T is highMore likely at start when T is high

Optimality guaranteed with slow annealing scheduleOptimality guaranteed with slow annealing schedule

No need for smooth search spaceNo need for smooth search space

• We do not need to know what nearby successor isWe do not need to know what nearby successor is

Can be discrete search spaceCan be discrete search space

• Traveling salesman problemTraveling salesman problem

Always move downhill if possibleAlways move downhill if possible

Sometimes go uphillSometimes go uphill

• More likely at start when T is highMore likely at start when T is high

Optimality guaranteed with slow annealing scheduleOptimality guaranteed with slow annealing schedule

No need for smooth search spaceNo need for smooth search space

• We do not need to know what nearby successor isWe do not need to know what nearby successor is

Can be discrete search spaceCan be discrete search space

• Traveling salesman problemTraveling salesman problem

Local Beam Search

Keep more previous states in memoryKeep more previous states in memory

• Simulated Annealing just kept one previous state in memorySimulated Annealing just kept one previous state in memory

• This search keeps k states in memoryThis search keeps k states in memory

Generate k initial statesGenerate k initial states

if any state is a goal, terminateif any state is a goal, terminate

else, generate all successors and select best kelse, generate all successors and select best k

repeatrepeat

Keep more previous states in memoryKeep more previous states in memory

• Simulated Annealing just kept one previous state in memorySimulated Annealing just kept one previous state in memory

• This search keeps k states in memoryThis search keeps k states in memory

Generate k initial statesGenerate k initial states

if any state is a goal, terminateif any state is a goal, terminate

else, generate all successors and select best kelse, generate all successors and select best k

repeatrepeat

Isn’t this steepest ascent in parallel?

Information is shared between k search pointsInformation is shared between k search points

• Each k state generates successorsEach k state generates successors

• Best k successors are selectedBest k successors are selected

• Some search points may contribute none to best successorsSome search points may contribute none to best successors

• One search point may contribute all k successorsOne search point may contribute all k successors

• If executed in parallel, no search points would be terminated like thisIf executed in parallel, no search points would be terminated like this

• Draw itDraw it

Information is shared between k search pointsInformation is shared between k search points

• Each k state generates successorsEach k state generates successors

• Best k successors are selectedBest k successors are selected

• Some search points may contribute none to best successorsSome search points may contribute none to best successors

• One search point may contribute all k successorsOne search point may contribute all k successors

• If executed in parallel, no search points would be terminated like thisIf executed in parallel, no search points would be terminated like this

• Draw itDraw it

Beam Search

Premature termination of search paths?Premature termination of search paths?

• Stochastic beam searchStochastic beam search

• Choose k successors from total list at randomChoose k successors from total list at random

Premature termination of search paths?Premature termination of search paths?

• Stochastic beam searchStochastic beam search

• Choose k successors from total list at randomChoose k successors from total list at random