Applications of Dynamic Programming and Heuristics to the Traveling Salesman ProblemERIC SALMON & JOSEPH SEWELL

Traveling Salesman Problem

Distances between n cities are stores in a distance matrix D with elements dij where i, j = 1 … n and the diagonal elements dii are zero. A tour can be represented by a cyclic permutation π of { 1, 2, …, n} where π(i) represents the city that follows city i on the tour. The traveling salesman problem is then the optimization problem to find a permutation π that minimizes the length of the tour denoted by:

Hasler & Hornik (Journal of Statistical Software, 2007 V. 23 I. 22)

Heuristic Algorithms for TSP

A heuristic is a technique for solving a problem quicker when other methods are too slow or for finding an approximate solution to a problem.

Random Search

Generate random permutation for a tour

Genetic Algorithm

Mimic evolution to arrive at a tolerable tour

Simulated Annealing

Find a solution by moving slowly towards a global optimum without being trapped in local optimums

Genetic Algorithm

A genetic algorithm is a search heuristic that mimics the process of natural selection.

Simulated Annealing

Name inspired from metal work

Heating and cooling an object to alter its properties

While the algorithm is ‘hot’, it is allowed to jump out of its local optimums

As the algorithm ‘cools’ it begins to hone on the global optimum

Simulated Annealing (cont.)

Dynamic Programming

A method for solving complex problems by breaking them down into simpler sub-problems

Exploits sub-problem overlap

Example: Finding Fibonacci numbers.

F(n) = F(n-2) + F(n-1)

To find F(n) you must also compute F(n-2) and F(n-1)

These values will be recomputed for each F(n) you want to find

Using Dynamic Programming, every computed value would be stored which would then be looked up before computation.

Computing Fibonacci (naïve)

fib(n)

if n <= 2 : f = 1

else : f = fib(n-1) + fib(n-2)

return f

Dynamic Programing: Fibonacci

array = {}

fib(n):

if n in array: return array[n]

if n <= 2 : f = 1

else: f = fib(n-1) + fib(n-2)

array[n] = f

return f

Branch & Bound

An algorithm design for optimization problems.

Enumerations are possible solutions

Candidate partial solutions are child nodes from the root

Before enumerating child node, this branch is checked against upper/lower bounds compared to optimal solution

In the case of TSP this would be total distance up to that node

If this value is greater than the bound, discard entire branch

No added distance would ever decrease total distance

Continue enumeration through tree until solution found

Branch & Bound

is an algorithm design paradigm for discrete and combinatorial optimization problems. A branch-and-bound algorithm consists of a systematic enumeration of candidate solutions by means of state space search: the set of candidate solutions is thought of as forming a rooted tree with the full set at the root. The algorithm explores branches of this tree, which represent subsets of the solution set. Before enumerating the candidate solutions of a branch, the branch is checked against upper and lower estimated bounds on the optimal solution, and is discarded if it cannot produce a better solution than the best one found so far by the algorithm.

http://en.wikipedia.org/wiki/Branch_and_bound

Branch & Bound

A branch-and-bound procedure requires two tools. The first one is a splitting procedure that, given a set S of candidates, returns two or more smaller sets S1, S2, … whose union covers S. Note that the minimum of f(x) over S is min{v1, v2, …}, where each vi is the minimum of f(x) within Si. This step is called branching, since its recursive application defines a search tree whose nodes are the subsets of S.

http://en.wikipedia.org/wiki/Branch_and_bound

Branch & Bound

The second tool is a procedure that computes upper and lower bounds for the minimum value of f(x) within a given subset of S. This step is called bounding.

The key idea of the BB algorithm is: if the lower bound for some tree node (set of candidates) A is greater than the upper bound for some other node B, then A may be safely discarded from the search. This step is called pruning, and is usually implemented by maintaining a global variable m (shared among all nodes of the tree) that records the minimum upper bound seen among all sub-regions examined so far. Any node whose lower bound is greater than m can be discarded.

http://en.wikipedia.org/wiki/Branch_and_bound

Branch & Bound

The recursion stops when the current candidate set S is reduced to a single element, or when the upper bound for set S matches the lower bound. Either way, any element of S will be a minimum of the function within S.

http://en.wikipedia.org/wiki/Branch_and_bound

Branch & Bound on TSP

Given:

Branch & Bound on TSP