grasp: a sampling meta-heuristic
DESCRIPTION
GRASP: A Sampling Meta-Heuristic. Topics What is GRASP The Procedure Applications Merit. What is GRASP. GRASP : Greedy Randomized Adaptive Search Procedure Random Construction : TSP: randomly select next city to add High Solution Variance Low Solution Quality - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/1.jpg)
GRASP: A Sampling Meta-Heuristic
Topics• What is GRASP • The Procedure• Applications • Merit
![Page 2: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/2.jpg)
What is GRASP
GRASP : Greedy Randomized Adaptive Search Procedure
Random Construction: TSP: randomly select next city to addHigh Solution VarianceLow Solution QualityTSP: randomly select next city to add
Greedy Construction: TSP: select nearest city to add High Solution Quality Low Solution Variance
GRASP: Tries to Combine the Advantages of Random and Greedy Solution Construction Together.
![Page 3: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/3.jpg)
The Knapsack Example
• Knapsack problem– Backpack: 8 units of space, 4 items to pick – Item Value in terms of dollars: 2,5,7,9– Item Cost in terms of space units:
1,3,5,7
• Construction Heuristic – Pick the Most Valuable Item– Pick the Most Valuable Per Unit
![Page 4: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/4.jpg)
Solution Quality
• Solution Quality– For Heuristic 1: (1,4) , Value 11– For Heuristic 2: (1,4), Value 11.– Optimal Solution: (2,3), Value 12
• None of them gives the Optimal solution• This is true for any heuristic• Theoretically, for a NP-Hard problem,
there is no polynomial algorithm
![Page 5: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/5.jpg)
Semi-Greedy Heuristics
• Add at each step, not necessarily the highest rated solution components
• Do the following – Put high (not only the highest) solution
components into a restricted candidate list (RCL)
– Choose one element of the RCL randomly and add it to the partial solution
– Adaptive element: The greedy function depends on the partial solution constructed so far.
• Until a full solution is constructed.
![Page 6: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/6.jpg)
Mechanism of RCL
• Size of the Restricted Candidate List– 1) If we set size of the RCL to be really big,
then the semi-greedy heuristic turns into a pure random heuristic
– 2) If we set the size of RCL to be 1, the sem-greedy heuristic turns into the pure greedy heuristic
• Typically, this size is set between 3~5.
![Page 7: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/7.jpg)
GRASP
• Do the following– Phase I: Construct the current solution
according to a greedy myopic measure of goodness (GMMOG) with random selection from a restricted candidate list
– Phase II: Using a local search improvement heuristic to get better solutions
• While the stopping criteria unsatisfied
![Page 8: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/8.jpg)
GRASP• GRASP is a combination of semi-greedy
heuristic with a local search procedure• Local search from a Random Construction:
– Best solution often better than greedy, if not too large prob.
– Average solution quality worse than greedy heuristic
– High variance• Local Search from Greedy Construction:
– Average solution quality better than random– Low (No Variance)
![Page 9: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/9.jpg)
The Knapsack Example
• Knapsack problem– Backpack: 8 units of space, 4 items to pick – Item Value in terms of dollars: 2,5,7,9– Item Cost in terms of space units:
1,3,5,7
• Two Greedy Functions– Pick the Most Valuable Item– Pick the Most Valuable Per Unit
![Page 10: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/10.jpg)
GRASP
• The Most Valuable Item with RCL=2– Items 4 and 3 with values 9,7 are in the RCL – Flip a coin, we select ….
• The Most Valuable Per Unit with RCL = 2– Items 1 and 2 are selected with values 2/1
=2 and 5/3 = 1.7, – Flip a coin, we select ….
![Page 11: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/11.jpg)
GRASP extensions
• Merits– Fast– High Quality Solution– Time Critical Decision– Few Parameters to tune
• Extension– Reactive GRASP – The RCL Size– The use of Elite Solutions found – Long term memory, Path relinking
![Page 12: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/12.jpg)
Literature• T.A.Feo and M.G.C. Resende, “A probabilistic Heuristic
for a computational Difficult Set covering Problem,” Operations Research Letters, 8:67-71, 1989
• P. Festa and M.G.C. Resende, “GRASP: An annotated Biblograph” in P. Hansen and C.C. Ribeiro, editors, “Essays and Surveys on Metaheuristics, Kluwer Academic Publishers, 2001
• M.G.C.Resende and C.C.Ribeiro, “Greedy Randomized Adaptive Search Procedure”, in Handbook of Metaheuristics, F. Glover and G. Kochenberger, eds, Kluwer Academic Publishers, 219-249, 2002
![Page 13: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/13.jpg)
Neighbourhood• For each solution S S, N(S) S
is a neighbourhood
• In some sense each T N(S) is in some sense “close” to S
• Defined in terms of some operation• Very like the “action” in search
![Page 14: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/14.jpg)
Neighbourhood
Exchange neighbourhood:Exchange k things in a sequence or partition
Examples:
• Knapsack problem: exchange k1 things inside the bag with k2 not in. (for ki, k2 = {0, 1, 2, 3})
• Matching problem: exchange one marriage for another
![Page 15: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/15.jpg)
2-opt Exchange
![Page 16: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/16.jpg)
2-opt Exchange
![Page 17: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/17.jpg)
2-opt Exchange
![Page 18: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/18.jpg)
2-opt Exchange
![Page 19: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/19.jpg)
2-opt Exchange
![Page 20: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/20.jpg)
2-opt Exchange
![Page 21: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/21.jpg)
3-opt exchange
• Select three arcs• Replace with three others• 2 orientations possible
![Page 22: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/22.jpg)
3-opt exchange
![Page 23: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/23.jpg)
3-opt exchange
![Page 24: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/24.jpg)
3-opt exchange
![Page 25: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/25.jpg)
3-opt exchange
![Page 26: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/26.jpg)
3-opt exchange
![Page 27: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/27.jpg)
3-opt exchange
![Page 28: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/28.jpg)
3-opt exchange
![Page 29: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/29.jpg)
3-opt exchange
![Page 30: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/30.jpg)
3-opt exchange
![Page 31: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/31.jpg)
3-opt exchange
![Page 32: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/32.jpg)
3-opt exchange
![Page 33: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/33.jpg)
Neighbourhood
Strongly connected:• Any solution can be reached from
any other(e.g. 2-opt)
Weakly optimally connected• The optimum can be reached from
any starting solution
![Page 34: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/34.jpg)
Neighbourhood• Hard constraints create solution
impenetrable mountain ranges• Soft constraints allow passes through the
mountains
• E.g. Map Colouring (k-colouring)– Colour a map (graph) so that no two adjacent
countries (nodes) are the same colour– Use at most k colours– Minimize number of colours
![Page 35: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/35.jpg)
Map Colouring
Starting sol Two optimal solutionsDefine neighbourhood as:
Change the colour of at most one vertex
?Make k-colour constraint soft…
![Page 36: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/36.jpg)
Variable Neighbourhood Search
• Large Neighbourhoods are expensive
• Small neighbourhoods are less effective
Only search larger neighbourhood when smaller is exhausted
![Page 37: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/37.jpg)
Variable Neighbourhood Search
• m Neighbourhoods Ni
• |N1| < |N2| < |N3| < … < |Nm|
1. Find initial sol S ; best = z (S)2. k = 1; 3. Search Nk(S) to find best sol T4. If z(T) < z(S)
S = Tk = 1
elsek = k+1
![Page 38: GRASP: A Sampling Meta-Heuristic](https://reader033.vdocuments.net/reader033/viewer/2022061605/56813065550346895d963ca3/html5/thumbnails/38.jpg)
• VNS does not follow a trajectory– Like SA, tabu search