cs 416 artificial intelligence lecture 6 informed searches lecture 6 informed searches
TRANSCRIPT
CS 416Artificial Intelligence
Lecture 6Lecture 6
Informed SearchesInformed Searches
Lecture 6Lecture 6
Informed SearchesInformed Searches
A sad day
No more Western Union telegramsNo more Western Union telegrams
• 1844 - First telegram (Morse) “What hath God wrought”1844 - First telegram (Morse) “What hath God wrought”
• 1858 - First transatlantic telegram from Queen Victoria to 1858 - First transatlantic telegram from Queen Victoria to President BuchananPresident Buchanan
• Break, break, break… 1866 working againBreak, break, break… 1866 working again
– A few words per minuteA few words per minute
– Punctuation cost extra. “stop” was cheaper.Punctuation cost extra. “stop” was cheaper.
No more Western Union telegramsNo more Western Union telegrams
• 1844 - First telegram (Morse) “What hath God wrought”1844 - First telegram (Morse) “What hath God wrought”
• 1858 - First transatlantic telegram from Queen Victoria to 1858 - First transatlantic telegram from Queen Victoria to President BuchananPresident Buchanan
• Break, break, break… 1866 working againBreak, break, break… 1866 working again
– A few words per minuteA few words per minute
– Punctuation cost extra. “stop” was cheaper.Punctuation cost extra. “stop” was cheaper.
Assignment 1
Getting Visual StudioGetting Visual Studio
Signing up for Thursday (3-5) Friday (2-3:30)Signing up for Thursday (3-5) Friday (2-3:30)
Explanation of IsNormal ( )Explanation of IsNormal ( )
Getting Visual StudioGetting Visual Studio
Signing up for Thursday (3-5) Friday (2-3:30)Signing up for Thursday (3-5) Friday (2-3:30)
Explanation of IsNormal ( )Explanation of IsNormal ( )
A* without admissibility
9A
4B
200C
0D
5
4 5
5
Never explored!
A* with admissibility
9A
4B
1C
0D
5
4 5
5
Another A* without admissibility
9A
1B
6C
0D
5
4 5
5
Never explored!
Admissible w/o Consistency
3A
2B
1C
1D
1
1 1
0
02
Meta-foo
What does “meta” mean in AI?What does “meta” mean in AI?
• Frequently it means step back a level from fooFrequently it means step back a level from foo
• Metareasoning = reasoning about reasoningMetareasoning = reasoning about reasoning
• These informed search algorithms have pros and cons These informed search algorithms have pros and cons regarding how they choose to explore new levelsregarding how they choose to explore new levels
– a metalevel learning algorithm may learn how to combine a metalevel learning algorithm may learn how to combine search techniques to suit application domainsearch techniques to suit application domain
What does “meta” mean in AI?What does “meta” mean in AI?
• Frequently it means step back a level from fooFrequently it means step back a level from foo
• Metareasoning = reasoning about reasoningMetareasoning = reasoning about reasoning
• These informed search algorithms have pros and cons These informed search algorithms have pros and cons regarding how they choose to explore new levelsregarding how they choose to explore new levels
– a metalevel learning algorithm may learn how to combine a metalevel learning algorithm may learn how to combine search techniques to suit application domainsearch techniques to suit application domain
Heuristic Functions
8-puzzle problem8-puzzle problem8-puzzle problem8-puzzle problem
Avg Depth=22
Branching = approx 3
322 states
170,000 repeated
Heuristics
The number of misplaced tilesThe number of misplaced tiles
• Admissible because at least n moves required to solve n Admissible because at least n moves required to solve n misplaced tilesmisplaced tiles
The distance from each tile to its goal positionThe distance from each tile to its goal position
• No diagonals, so use No diagonals, so use Manhattan DistanceManhattan Distance
– As if walking around rectilinear city blocksAs if walking around rectilinear city blocks
• also admissiblealso admissible
The number of misplaced tilesThe number of misplaced tiles
• Admissible because at least n moves required to solve n Admissible because at least n moves required to solve n misplaced tilesmisplaced tiles
The distance from each tile to its goal positionThe distance from each tile to its goal position
• No diagonals, so use No diagonals, so use Manhattan DistanceManhattan Distance
– As if walking around rectilinear city blocksAs if walking around rectilinear city blocks
• also admissiblealso admissible
Compare these two heuristics
Effective Branching Factor, b*Effective Branching Factor, b*
• If A* generates If A* generates NN nodes to find the goal at depth nodes to find the goal at depth dd
– b* = branching factor such that a uniform tree of depth d b* = branching factor such that a uniform tree of depth d contains N+1 nodes (we add one for the root node that contains N+1 nodes (we add one for the root node that wasn’t included in N)wasn’t included in N)
N+1 = 1 + b* + (b*)N+1 = 1 + b* + (b*)22 + … + (b*) + … + (b*)dd
Effective Branching Factor, b*Effective Branching Factor, b*
• If A* generates If A* generates NN nodes to find the goal at depth nodes to find the goal at depth dd
– b* = branching factor such that a uniform tree of depth d b* = branching factor such that a uniform tree of depth d contains N+1 nodes (we add one for the root node that contains N+1 nodes (we add one for the root node that wasn’t included in N)wasn’t included in N)
N+1 = 1 + b* + (b*)N+1 = 1 + b* + (b*)22 + … + (b*) + … + (b*)dd
Compare these two heuristics
Effective Branching Factor, b*Effective Branching Factor, b*
• b* close to 1 is ideal b* close to 1 is ideal
– because this means the heuristic guided the A* search because this means the heuristic guided the A* search linearlylinearly
– If b* were 100, on average, the heuristic had to consider If b* were 100, on average, the heuristic had to consider 100 children for each node100 children for each node
– Compare heuristics based on their b*Compare heuristics based on their b*
Effective Branching Factor, b*Effective Branching Factor, b*
• b* close to 1 is ideal b* close to 1 is ideal
– because this means the heuristic guided the A* search because this means the heuristic guided the A* search linearlylinearly
– If b* were 100, on average, the heuristic had to consider If b* were 100, on average, the heuristic had to consider 100 children for each node100 children for each node
– Compare heuristics based on their b*Compare heuristics based on their b*
Compare these two heuristics
Compare these two heuristics
hh22 is always better than h is always better than h11
• for any node, n, for any node, n, hh22(n) >= h(n) >= h11(n)(n)
• hh22 dominatesdominates h h11
• Recall all nodes with f(n) < C* will be expanded?Recall all nodes with f(n) < C* will be expanded?
– This means all nodes, h(n) + g(n) < C*, will be expandedThis means all nodes, h(n) + g(n) < C*, will be expanded
All nodes where h(n) < C* - g(n) will be expandedAll nodes where h(n) < C* - g(n) will be expanded
– All nodes hAll nodes h22 expands will also be expanded by h expands will also be expanded by h11 and and
because hbecause h11 is smaller, others will be expanded as well is smaller, others will be expanded as well
hh22 is always better than h is always better than h11
• for any node, n, for any node, n, hh22(n) >= h(n) >= h11(n)(n)
• hh22 dominatesdominates h h11
• Recall all nodes with f(n) < C* will be expanded?Recall all nodes with f(n) < C* will be expanded?
– This means all nodes, h(n) + g(n) < C*, will be expandedThis means all nodes, h(n) + g(n) < C*, will be expanded
All nodes where h(n) < C* - g(n) will be expandedAll nodes where h(n) < C* - g(n) will be expanded
– All nodes hAll nodes h22 expands will also be expanded by h expands will also be expanded by h11 and and
because hbecause h11 is smaller, others will be expanded as well is smaller, others will be expanded as well
Inventing admissible heuristic funcs
How can you create h(n)?How can you create h(n)?
• Simplify problem by reducing restrictions on actionsSimplify problem by reducing restrictions on actions
– Allow 8-puzzle pieces to sit atop on anotherAllow 8-puzzle pieces to sit atop on another
– Call this a Call this a relaxed problemrelaxed problem
– The cost of optimal solution to relaxed problem is The cost of optimal solution to relaxed problem is admissible heuristic for original problemadmissible heuristic for original problem
It is at least as expensive for the original problemIt is at least as expensive for the original problem
How can you create h(n)?How can you create h(n)?
• Simplify problem by reducing restrictions on actionsSimplify problem by reducing restrictions on actions
– Allow 8-puzzle pieces to sit atop on anotherAllow 8-puzzle pieces to sit atop on another
– Call this a Call this a relaxed problemrelaxed problem
– The cost of optimal solution to relaxed problem is The cost of optimal solution to relaxed problem is admissible heuristic for original problemadmissible heuristic for original problem
It is at least as expensive for the original problemIt is at least as expensive for the original problem
Examples of relaxed problems
A tile can move from square A tile can move from square AA to square to square BB if if
AA is horizontally or vertically adjacent to is horizontally or vertically adjacent to BB
and and BB is blank is blank
• A tile can move from A to B if A is adjacent to B (overlap)A tile can move from A to B if A is adjacent to B (overlap)
• A tile can move from A to B if B is blank (teleport)A tile can move from A to B if B is blank (teleport)
• A tile can move from A to B (teleport and overlap)A tile can move from A to B (teleport and overlap)
Solutions to these relaxed problems can be computed Solutions to these relaxed problems can be computed without search and therefore heuristic is easy to computewithout search and therefore heuristic is easy to compute
A tile can move from square A tile can move from square AA to square to square BB if if
AA is horizontally or vertically adjacent to is horizontally or vertically adjacent to BB
and and BB is blank is blank
• A tile can move from A to B if A is adjacent to B (overlap)A tile can move from A to B if A is adjacent to B (overlap)
• A tile can move from A to B if B is blank (teleport)A tile can move from A to B if B is blank (teleport)
• A tile can move from A to B (teleport and overlap)A tile can move from A to B (teleport and overlap)
Solutions to these relaxed problems can be computed Solutions to these relaxed problems can be computed without search and therefore heuristic is easy to computewithout search and therefore heuristic is easy to compute
Multiple Heuristics
If multiple heuristics available:If multiple heuristics available:
• h(n) = max {hh(n) = max {h11(n), h(n), h22(n), …, h(n), …, hmm(n)}(n)}
If multiple heuristics available:If multiple heuristics available:
• h(n) = max {hh(n) = max {h11(n), h(n), h22(n), …, h(n), …, hmm(n)}(n)}
Use solution to subproblem as heuristic
What is optimal cost of solving some portion of What is optimal cost of solving some portion of original problem?original problem?
• subproblem solution is heuristic of original problemsubproblem solution is heuristic of original problem
What is optimal cost of solving some portion of What is optimal cost of solving some portion of original problem?original problem?
• subproblem solution is heuristic of original problemsubproblem solution is heuristic of original problem
Pattern Databases
Store optimal solutions to subproblems in databaseStore optimal solutions to subproblems in database
• We use an exhaustive search to solve every permutation of the We use an exhaustive search to solve every permutation of the 1,2,3,4-piece subproblem of the 8-puzzle1,2,3,4-piece subproblem of the 8-puzzle
• During solution of 8-puzzle, look up optimal cost to solve the During solution of 8-puzzle, look up optimal cost to solve the 1,2,3,4-piece subproblem and use as heuristic1,2,3,4-piece subproblem and use as heuristic
Store optimal solutions to subproblems in databaseStore optimal solutions to subproblems in database
• We use an exhaustive search to solve every permutation of the We use an exhaustive search to solve every permutation of the 1,2,3,4-piece subproblem of the 8-puzzle1,2,3,4-piece subproblem of the 8-puzzle
• During solution of 8-puzzle, look up optimal cost to solve the During solution of 8-puzzle, look up optimal cost to solve the 1,2,3,4-piece subproblem and use as heuristic1,2,3,4-piece subproblem and use as heuristic
Learning
Could also build pattern database while solving Could also build pattern database while solving cases of the 8-puzzlecases of the 8-puzzle
• Must keep track of intermediate states and true final cost of Must keep track of intermediate states and true final cost of solutionsolution
• Inductive learningInductive learning builds mapping of state -> cost builds mapping of state -> cost
• Because too many permutations of actual statesBecause too many permutations of actual states
– Construct important Construct important features features to reduce size of spaceto reduce size of space
Could also build pattern database while solving Could also build pattern database while solving cases of the 8-puzzlecases of the 8-puzzle
• Must keep track of intermediate states and true final cost of Must keep track of intermediate states and true final cost of solutionsolution
• Inductive learningInductive learning builds mapping of state -> cost builds mapping of state -> cost
• Because too many permutations of actual statesBecause too many permutations of actual states
– Construct important Construct important features features to reduce size of spaceto reduce size of space
Local Search Algorithms andOptimization Problems
Characterize Techniques
Uninformed SearchUninformed Search• Looking for a solution where solution is a Looking for a solution where solution is a pathpath from start to from start to
goalgoal
• At each intermediate point along a path, we have no At each intermediate point along a path, we have no prediction of the future value of the pathprediction of the future value of the path
Informed SearchInformed Search• Again, looking for a path from start to goalAgain, looking for a path from start to goal
• This time we have more insight regarding the value of This time we have more insight regarding the value of intermediate solutionsintermediate solutions
Uninformed SearchUninformed Search• Looking for a solution where solution is a Looking for a solution where solution is a pathpath from start to from start to
goalgoal
• At each intermediate point along a path, we have no At each intermediate point along a path, we have no prediction of the future value of the pathprediction of the future value of the path
Informed SearchInformed Search• Again, looking for a path from start to goalAgain, looking for a path from start to goal
• This time we have more insight regarding the value of This time we have more insight regarding the value of intermediate solutionsintermediate solutions
Now change things a bit
What if the path isn’t important, just the goal?What if the path isn’t important, just the goal?
• So the goal is unknownSo the goal is unknown
• The path to the goal need not be solvedThe path to the goal need not be solved
ExamplesExamples
• What quantities of quarters, nickels, and dimes add up to What quantities of quarters, nickels, and dimes add up to $17.45 and minimizes the total number of coins$17.45 and minimizes the total number of coins
• Is the price of Microsoft stock going up tomorrow?Is the price of Microsoft stock going up tomorrow?
What if the path isn’t important, just the goal?What if the path isn’t important, just the goal?
• So the goal is unknownSo the goal is unknown
• The path to the goal need not be solvedThe path to the goal need not be solved
ExamplesExamples
• What quantities of quarters, nickels, and dimes add up to What quantities of quarters, nickels, and dimes add up to $17.45 and minimizes the total number of coins$17.45 and minimizes the total number of coins
• Is the price of Microsoft stock going up tomorrow?Is the price of Microsoft stock going up tomorrow?
Local Search
Local search does not keep track of previous Local search does not keep track of previous solutionssolutions• Instead it keeps track of current solution (current state)Instead it keeps track of current solution (current state)
• Uses a method of generating alternative solution candidatesUses a method of generating alternative solution candidates
AdvantagesAdvantages• Use a small amount of memory (usually constant amount)Use a small amount of memory (usually constant amount)
• They can find reasonable (note we aren’t saying optimal) They can find reasonable (note we aren’t saying optimal) solutions in infinite search spaces solutions in infinite search spaces
Local search does not keep track of previous Local search does not keep track of previous solutionssolutions• Instead it keeps track of current solution (current state)Instead it keeps track of current solution (current state)
• Uses a method of generating alternative solution candidatesUses a method of generating alternative solution candidates
AdvantagesAdvantages• Use a small amount of memory (usually constant amount)Use a small amount of memory (usually constant amount)
• They can find reasonable (note we aren’t saying optimal) They can find reasonable (note we aren’t saying optimal) solutions in infinite search spaces solutions in infinite search spaces
Optimization Problems
Objective FunctionObjective Function• A function with vector inputs and scalar outputA function with vector inputs and scalar output
– goal is to search through candidate input vectors in order goal is to search through candidate input vectors in order to minimize or maximize objective functionto minimize or maximize objective function
ExampleExample• f (q, d, n) = 1,000,000 if q*0.25 + d*0.1 + n*0.05 != 17.45f (q, d, n) = 1,000,000 if q*0.25 + d*0.1 + n*0.05 != 17.45
= q + n + d otherwise = q + n + d otherwise
• minimize fminimize f
Objective FunctionObjective Function• A function with vector inputs and scalar outputA function with vector inputs and scalar output
– goal is to search through candidate input vectors in order goal is to search through candidate input vectors in order to minimize or maximize objective functionto minimize or maximize objective function
ExampleExample• f (q, d, n) = 1,000,000 if q*0.25 + d*0.1 + n*0.05 != 17.45f (q, d, n) = 1,000,000 if q*0.25 + d*0.1 + n*0.05 != 17.45
= q + n + d otherwise = q + n + d otherwise
• minimize fminimize f
Search SpaceThe realm of feasible input vectorsThe realm of feasible input vectors• Also called state-space landscapeAlso called state-space landscape
• Usually described byUsually described by
– number of number of dimensionsdimensions (3 for our change example) (3 for our change example)
– domaindomain of each dimension (# of each dimension (#quartersquarters is discrete from 0 to 69…) is discrete from 0 to 69…)
– functional relationshipfunctional relationship between input vector and objective function between input vector and objective function outputoutput
no relationship (chaos or seemingly random)no relationship (chaos or seemingly random)
smoothly varyingsmoothly varying
discontinuitiesdiscontinuities
The realm of feasible input vectorsThe realm of feasible input vectors• Also called state-space landscapeAlso called state-space landscape
• Usually described byUsually described by
– number of number of dimensionsdimensions (3 for our change example) (3 for our change example)
– domaindomain of each dimension (# of each dimension (#quartersquarters is discrete from 0 to 69…) is discrete from 0 to 69…)
– functional relationshipfunctional relationship between input vector and objective function between input vector and objective function outputoutput
no relationship (chaos or seemingly random)no relationship (chaos or seemingly random)
smoothly varyingsmoothly varying
discontinuitiesdiscontinuities
Search Space
Looking for global maximum (or minimum)Looking for global maximum (or minimum)Looking for global maximum (or minimum)Looking for global maximum (or minimum)
Hill Climbing
Also called Greedy SearchAlso called Greedy Search
• Select a starting point and set Select a starting point and set currentcurrent
• evaluate (evaluate (current)current)
• loop doloop do
– neighborneighbor = highest value successor of = highest value successor of currentcurrent
– if evaluate (if evaluate (neighborneighbor) <= evaluate () <= evaluate (currentcurrent))
return return currentcurrent
– else else currentcurrent = = neighborneighbor
Also called Greedy SearchAlso called Greedy Search
• Select a starting point and set Select a starting point and set currentcurrent
• evaluate (evaluate (current)current)
• loop doloop do
– neighborneighbor = highest value successor of = highest value successor of currentcurrent
– if evaluate (if evaluate (neighborneighbor) <= evaluate () <= evaluate (currentcurrent))
return return currentcurrent
– else else currentcurrent = = neighborneighbor
Hill climbing gets stuck
Hiking metaphor (you are wearing glasses that Hiking metaphor (you are wearing glasses that limit your vision to 10 feet)limit your vision to 10 feet)Hiking metaphor (you are wearing glasses that Hiking metaphor (you are wearing glasses that limit your vision to 10 feet)limit your vision to 10 feet)
• Local maximaLocal maxima
– Ridges (in cases when you can’t walk along the ridge)Ridges (in cases when you can’t walk along the ridge)
• PlateauPlateau
– why is this a problem?why is this a problem?
• Local maximaLocal maxima
– Ridges (in cases when you can’t walk along the ridge)Ridges (in cases when you can’t walk along the ridge)
• PlateauPlateau
– why is this a problem?why is this a problem?
Hill Climbing GadgetsVariants on hill climbing play special rolesVariants on hill climbing play special roles
• stochastic hill climbingstochastic hill climbing
– don’t always choose the best successordon’t always choose the best successor
• first-choice hill climbingfirst-choice hill climbing
– pick the first good successor you findpick the first good successor you find
useful if number of successors is largeuseful if number of successors is large
• random restartrandom restart
– follow steepest ascent from multiple starting statesfollow steepest ascent from multiple starting states
– probability of finding global max increases with number of startsprobability of finding global max increases with number of starts
Variants on hill climbing play special rolesVariants on hill climbing play special roles
• stochastic hill climbingstochastic hill climbing
– don’t always choose the best successordon’t always choose the best successor
• first-choice hill climbingfirst-choice hill climbing
– pick the first good successor you findpick the first good successor you find
useful if number of successors is largeuseful if number of successors is large
• random restartrandom restart
– follow steepest ascent from multiple starting statesfollow steepest ascent from multiple starting states
– probability of finding global max increases with number of startsprobability of finding global max increases with number of starts
Hill Climbing Usefulness
It DependsIt Depends
• Shape of state space greatly influences hill climbingShape of state space greatly influences hill climbing
• local maxima are the Achilles heellocal maxima are the Achilles heel
• what is cost of evaluation?what is cost of evaluation?
• what is cost of finding a random starting location?what is cost of finding a random starting location?
It DependsIt Depends
• Shape of state space greatly influences hill climbingShape of state space greatly influences hill climbing
• local maxima are the Achilles heellocal maxima are the Achilles heel
• what is cost of evaluation?what is cost of evaluation?
• what is cost of finding a random starting location?what is cost of finding a random starting location?
Simulated Annealing
A term borrowed from metalworkingA term borrowed from metalworking• We want metal molecules to find a stable location relative to We want metal molecules to find a stable location relative to
neighborsneighbors
• heating causes metal molecules to jump around and to take on heating causes metal molecules to jump around and to take on undesirable (high energy) locationsundesirable (high energy) locations
• during cooling, molecules reduce their movement and settle during cooling, molecules reduce their movement and settle into a more stable (low energy) position into a more stable (low energy) position
• annealing is process of heating metal and letting it cool slowly annealing is process of heating metal and letting it cool slowly to lock in the stable locations of the moleculesto lock in the stable locations of the molecules
A term borrowed from metalworkingA term borrowed from metalworking• We want metal molecules to find a stable location relative to We want metal molecules to find a stable location relative to
neighborsneighbors
• heating causes metal molecules to jump around and to take on heating causes metal molecules to jump around and to take on undesirable (high energy) locationsundesirable (high energy) locations
• during cooling, molecules reduce their movement and settle during cooling, molecules reduce their movement and settle into a more stable (low energy) position into a more stable (low energy) position
• annealing is process of heating metal and letting it cool slowly annealing is process of heating metal and letting it cool slowly to lock in the stable locations of the moleculesto lock in the stable locations of the molecules
Simulated Annealing
““Be the Ball”Be the Ball”
• You have a wrinkled sheet of metalYou have a wrinkled sheet of metal
• Place a BB on the sheet and what happens?Place a BB on the sheet and what happens?
– BB rolls downhillBB rolls downhill
– BB stops at bottom of hill (local or global min?)BB stops at bottom of hill (local or global min?)
– BB momentum may carry it out of hill into another (local or global)BB momentum may carry it out of hill into another (local or global)
• By shaking metal sheet, your are adding energy (heat)By shaking metal sheet, your are adding energy (heat)
• How hard do you shake?How hard do you shake?
““Be the Ball”Be the Ball”
• You have a wrinkled sheet of metalYou have a wrinkled sheet of metal
• Place a BB on the sheet and what happens?Place a BB on the sheet and what happens?
– BB rolls downhillBB rolls downhill
– BB stops at bottom of hill (local or global min?)BB stops at bottom of hill (local or global min?)
– BB momentum may carry it out of hill into another (local or global)BB momentum may carry it out of hill into another (local or global)
• By shaking metal sheet, your are adding energy (heat)By shaking metal sheet, your are adding energy (heat)
• How hard do you shake?How hard do you shake?
Our Simulated Annealing Algorithm
““You’re not being the ball, Danny” You’re not being the ball, Danny” (Caddy Shack)(Caddy Shack)
• Gravity is great because it tells the ball which way is downhill Gravity is great because it tells the ball which way is downhill at all timesat all times
• We don’t have gravity, so how do we find a successor state?We don’t have gravity, so how do we find a successor state?
– RandomnessRandomness
AKA AKA Monte CarloMonte Carlo
AKA AKA StochasticStochastic
““You’re not being the ball, Danny” You’re not being the ball, Danny” (Caddy Shack)(Caddy Shack)
• Gravity is great because it tells the ball which way is downhill Gravity is great because it tells the ball which way is downhill at all timesat all times
• We don’t have gravity, so how do we find a successor state?We don’t have gravity, so how do we find a successor state?
– RandomnessRandomness
AKA AKA Monte CarloMonte Carlo
AKA AKA StochasticStochastic
Algorithm OutlineSelect some initial guess of evaluation function parameters: Select some initial guess of evaluation function parameters:
Evaluate evaluation function, Evaluate evaluation function,
Compute a random displacement,Compute a random displacement,
• The Monte Carlo eventThe Monte Carlo event
EvaluateEvaluate
• If If v’ < vv’ < v; set new state,; set new state,
• Else set with Prob(E,T)Else set with Prob(E,T)
– This is the Metropolis stepThis is the Metropolis step
Repeat with updated state and tempRepeat with updated state and temp
Select some initial guess of evaluation function parameters: Select some initial guess of evaluation function parameters:
Evaluate evaluation function, Evaluate evaluation function,
Compute a random displacement,Compute a random displacement,
• The Monte Carlo eventThe Monte Carlo event
EvaluateEvaluate
• If If v’ < vv’ < v; set new state,; set new state,
• Else set with Prob(E,T)Else set with Prob(E,T)
– This is the Metropolis stepThis is the Metropolis step
Repeat with updated state and tempRepeat with updated state and temp
Metropolis Step
We approximate nature’s alignment of molecules by We approximate nature’s alignment of molecules by allowing uphill transitions with some probabilityallowing uphill transitions with some probability•
Prob (in energy state E) ~ Prob (in energy state E) ~
– Boltzmann Probability DistributionBoltzmann Probability Distribution
– Even when T is small, there is still a chance in high energy stateEven when T is small, there is still a chance in high energy state
• Prob (transferring from EProb (transferring from E11 to E to E22) =) =
– Metropolis StepMetropolis Step
– if Eif E22 < E < E11, prob () is greater than 1, prob () is greater than 1
– if Eif E22 > E > E11, we may transfer to higher energy state, we may transfer to higher energy state
The rate at which T is decreased and the amount The rate at which T is decreased and the amount it is decreased is prescribed by anit is decreased is prescribed by an annealing schedule annealing schedule
We approximate nature’s alignment of molecules by We approximate nature’s alignment of molecules by allowing uphill transitions with some probabilityallowing uphill transitions with some probability•
Prob (in energy state E) ~ Prob (in energy state E) ~
– Boltzmann Probability DistributionBoltzmann Probability Distribution
– Even when T is small, there is still a chance in high energy stateEven when T is small, there is still a chance in high energy state
• Prob (transferring from EProb (transferring from E11 to E to E22) =) =
– Metropolis StepMetropolis Step
– if Eif E22 < E < E11, prob () is greater than 1, prob () is greater than 1
– if Eif E22 > E > E11, we may transfer to higher energy state, we may transfer to higher energy state
The rate at which T is decreased and the amount The rate at which T is decreased and the amount it is decreased is prescribed by anit is decreased is prescribed by an annealing schedule annealing schedule
What have we got?Always move downhill if possibleAlways move downhill if possible
Sometimes go uphillSometimes go uphill
• More likely at start when T is highMore likely at start when T is high
Optimality guaranteed with slow annealing scheduleOptimality guaranteed with slow annealing schedule
No need for smooth search spaceNo need for smooth search space
• We do not need to know what nearby successor isWe do not need to know what nearby successor is
Can be discrete search spaceCan be discrete search space
• Traveling salesman problemTraveling salesman problem
Always move downhill if possibleAlways move downhill if possible
Sometimes go uphillSometimes go uphill
• More likely at start when T is highMore likely at start when T is high
Optimality guaranteed with slow annealing scheduleOptimality guaranteed with slow annealing schedule
No need for smooth search spaceNo need for smooth search space
• We do not need to know what nearby successor isWe do not need to know what nearby successor is
Can be discrete search spaceCan be discrete search space
• Traveling salesman problemTraveling salesman problem
More info: Numerical Recipes in C (online) Chapter 10.9
Local Beam Search
Keep more previous states in memoryKeep more previous states in memory• Simulated Annealing just kept one previous state in memorySimulated Annealing just kept one previous state in memory
• This search keeps k states in memoryThis search keeps k states in memory
Generate k initial statesGenerate k initial states
if any state is a goal, terminateif any state is a goal, terminate
else, generate all successors and select best kelse, generate all successors and select best k
repeatrepeat
Keep more previous states in memoryKeep more previous states in memory• Simulated Annealing just kept one previous state in memorySimulated Annealing just kept one previous state in memory
• This search keeps k states in memoryThis search keeps k states in memory
Generate k initial statesGenerate k initial states
if any state is a goal, terminateif any state is a goal, terminate
else, generate all successors and select best kelse, generate all successors and select best k
repeatrepeat
Isn’t this steepest ascent in parallel?
Information is shared between k search pointsInformation is shared between k search points• Each k state generates successorsEach k state generates successors
• Best k successors are selectedBest k successors are selected
• Some search points may contribute none to best successorsSome search points may contribute none to best successors
• One search point may contribute all k successorsOne search point may contribute all k successors
– ““Come over here, the grass is greener” Come over here, the grass is greener” (Russell and Norvig)(Russell and Norvig)
• If executed in parallel, no search points would be terminated If executed in parallel, no search points would be terminated like thislike this
Information is shared between k search pointsInformation is shared between k search points• Each k state generates successorsEach k state generates successors
• Best k successors are selectedBest k successors are selected
• Some search points may contribute none to best successorsSome search points may contribute none to best successors
• One search point may contribute all k successorsOne search point may contribute all k successors
– ““Come over here, the grass is greener” Come over here, the grass is greener” (Russell and Norvig)(Russell and Norvig)
• If executed in parallel, no search points would be terminated If executed in parallel, no search points would be terminated like thislike this
Beam Search
Premature termination of search paths?Premature termination of search paths?
• Stochastic beam searchStochastic beam search
– Instead of choosing best K successorsInstead of choosing best K successors
– Choose k successors at randomChoose k successors at random
Premature termination of search paths?Premature termination of search paths?
• Stochastic beam searchStochastic beam search
– Instead of choosing best K successorsInstead of choosing best K successors
– Choose k successors at randomChoose k successors at random