Transcript
Page 1: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

The Traveling Salesman Problem

Page 2: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Problem Statement• Motivation

– How to find the shortest tour through a set of cities. Applications in VLSI, transportation.

• Why should we care? – One of the most well-studied optimization problems.

Shows what we can do in the face of NP-Complete problems.

• Mathematically– Given a complete graph G(V,E) with some distance

function d(E), find the minimum cost Hamiltonian cycle.

Page 3: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Problem Difficulty

• A naïve approach tries all possible tours O(n!)

• Held and Karp (Berkeley) improved this to O(2nn2) in 1962, which is the best known still.

• TSP is NP-Hard, but in practice what we can do is pretty amazing.

Page 4: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Problem Difficulty Continued

Much/most of this progress is due to improved algorithms, not hardware. Moore's Law says computer speed has increased exponentially. Problem difficulty increases exponentially with size. → Therefore, hardware only improvements would imply only a linear increase in the size of solved problems (graph on the right). But historically we have improved faster.

Page 5: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Problem Difficulty Continued

• 2006: VLSI 85,900 locations.

Page 6: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Problem Classification

• Metric– Satisfies triangle inequality: d(e

ij) d(e≦

ik) + d(e

kj)

• Symmetric– d(e

ij) = d(e

ji)

• Euclidean TSP– Distance satisfies the euclidean norm. Implies both

metric and symmetric.

Page 7: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Method of Attack

• Lower Bound– A solution to an easier and relaxed problem.– In practice: Linear Programming with Branch and Cut

• Upper Bound– A feasible solution to the current problem.– In practice: Simulated Annealing, Local Search,

Genetic Algorithms, Christofides Algorithm.

• Optimality when the two bounds match.

Page 8: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Approximation Difficulty• Non-metric

– Any approximation is NP-Hard by reduction to Hamiltonian Cycle.

• Symmetric and Metric– Christofides: 3/2 approximation ratio by using a MST and

perfect matching. > 20 years old– Many believe 4/3 is possible.– 220/119 is NP-Complete (Papadimitriou)

• s,t TSP– Very recently shown to have a 1.618 approximation ratio.

Page 9: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Held-Karp Relaxation

• Integrality gap is in the range of 3/2 to 4/3.

• But how do we implement this LP?

Page 10: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Held-Karp Relaxation

• S is a subset of V, so we have 2n constraints.

• But, we only need to find and use the violated ones which we can find efficiently with network flows.

Page 11: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Branch and Cut

Branch and Cut is a very powerful technique to solve integer programs.

• Branch– Solve the LP for the optimal x*– For a non-integer result, branch into two subproblems

• x*i 0, x*≦

i 1≧

• Cut– Use a cutting plane to remove the LP optimum but not remove

any integer solutions.

Page 12: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Branch and Cut Continued

• Cutting Planes– Gomory Cuts, elegant and very generalized cutting planes.– Lift and Project Cuts– Problem Specific Cuts

• Comb, Clique Tree, Path, Wheelbarrow, Bicycle, Ladder, and Crown inequalities.

• Pool data structure– Too many cuts makes the LP slower to solve, so we

eventually drop old cuts and keep the newer ones.

Page 13: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Polyhedral Combinatorics

• Branching is naïve and computationally expensive.

• Cutting however only adds constraints to an LP.

• If we had the tightest cuts possible, facet-defining cuts, it would be easy to find integer solutions.

• How do we find the proper cuts?

Page 14: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Polyhedral Combinatorics Cont.

• Gomory and Lift and Project Cuts:– Augment any constraint matrix A with a cut.– Problem: Can take a huge number of cuts,

and not numerically stable.

• Problem Specific Cuts– Example: Knapsack – Inequalities

• At least for knapsack, guaranteed to result in the proper solution eventually

• Easy to 'separate'– What about TSP Cuts?

Page 15: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Polyhedral Combinatorics Cont.

• Comb Inequalities– Good news: They are facet defining– Bad news: There's no known polynomial

separation algorithm.– However: Comb separation hasn't been

shown to be NP-Complete

Contrast to:

• Subtour Elimination Cuts– Bad news: Exponentially many– Good news: Polynomial separation

Page 16: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Heuristics: k-Opt

• A problem is called k-Optimal if we cannot improve the tour by switching k edges.

• k-Optimality takes O(nk) time.

• Is k-Optimality enough for full optimality?– Yes if k = n (clearly). But can we do better?

→ No. Papadimitriou (Berkeley) has shown a TSP can have exponentially many local optima that are arbitrarily far from the global optima and require k = O(na) and (0 < a < 1).

Page 17: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Heuristics: k-Opt Continued

• Despite the grim picture, k-Opt does work well.

• Furthermore, O(nk) implies we look at all k groups. However, we expect a peturbation to work if it is local.

• Why don't we restrict ourselves to the p nearest neighbors to get O(pk)?

Page 18: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Heuristics: k-Opt Continued

• P Nearest Neighbors– At optimality most nodes connect to their nearest neighbors.

– However, some nodes connect to very far neighbors. i.e. the distribution has a long tail.

– Another metric of nearest neighbor is needed.

Page 19: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Heuristics: k-Opt Continued• Optimizations:

– Use alpha nearness, or distance on a minimum 1-tree.– Restrict ourselves to 2,3 - Opt moves but allow sequences of

moves.– Escape local minima with 'kicks' where we purposefully cross

edges and hope the problem relaxes lower.– First implemented by Lin and Kernighan and 'perfected' with

LKH (Keld Helsgaun).

• Results:– LKH has found every optimum and holds every record for

unknown problems (including world TSP)

Page 20: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

State of the Art

• Strong heuristics speed up the relaxation, and vice versa.

• Strongest Heuristics (Upper Bounds):– Initialize a tour with Christofides Algorithm– Run LKH and possibly use genetic algorithms.

• TSP is especially suited to genetic algorithms.

– Stop when we reach lower bound.

• Strongest Relaxations (Lower Bounds):– Held-Karp LP with many problem specific cutting planes.– Use best feasible solutions to prune the search tree.

Page 21: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Conclusion

• TSP has motivated many advances in integer programming and combinatorial optimization.

• It shows how good we can do on even NP-Hard problems in practice.

• It is a simple problem with a very deep and elegant mathematical framework.

Page 22: The Traveling Salesman Problem - Stanford Universityweb.stanford.edu/class/cme334/docs/2012-11-14-Firouz_TSP.pdf · The Traveling Salesman Problem. The Traveling Salesman - Omede

The Traveling Salesman - Omede Firouz

Conclusion

• 2011– World TSP 1,900,000 locations– 7,515,778,188 using LKH vs 7,512,218,268 from Concorde + CPLEX– .0474% Optimality Gap


Top Related