exponential time algorithms algorithms and networks

Exponential time algorithms

Algorithms and networks

Exponential time algorithms2

Today

• Exponential time algorithms: introduction

• Techniques

• 3-coloring

• 4-coloring

• Coloring

• Maximum Independent Set

• TSP


What to do if a problem is NP-complete?

• Solve on special cases

• Heuristics and approximations

• Algorithms that are fast on average

• Good exponential time algorithms

• …


Good exponential time algorithms

• Algorithms with a running time of cn.p(n)– c a constant– p() a polynomial– Notation: O*(cn)

• Smaller c helps a lot!


Important techniques

• Dynamic programming

• Branch and reduce– Measure and conquer (and design)

• Divide and conquer

• Clever enumeration

• Local search

• Inclusion-exclusion


Running example

• Graph coloring– Several applications: scheduling, frequency

assignment (usually more complex variants)– Different algorithms for small fixed number of

colors (3-coloring, 4-coloring, …) and arbitrary number of colors

– 2-coloring is easy in O(n+m) time


3-coloring

• O*(3n) is trivial

• Can we do this faster?


3-coloring in O*(2n) time

• G is 3-colorable, if and only if there is a set of vertices S with– S is independent– G[V-S] is 2-colorable

• Algorithm: enumerate all sets, and test these properties (2n tests of O(n+m) time each)


3-coloring

• Lawler, 1976:– We may assume S is a maximal independent set

– Enumerating all maximal independent sets in O*(3n/3) = O*(1.4423n) time

• There are O*(3n/3) maximal independent sets (will be proved later.)

– Thus O*(1.4423n) time algorithm for 3-coloring

• Schiermeyer, 1994; O*(1.398n) time• Beigel, Eppstein, 1995: O*(1.3446n) time


4-coloring in O*(2n) time

• Lawler, 1976

• G is 4-colorable, if and only if we can partition the vertices in two sets X and Y such that G[X] and G[Y] are both 2-colorable

• Enumerate all partitions– For each, check both halves in O(n+m) time


4-coloring

• Using 3-coloring– Enumerate all maximal independent sets S– For each, check 3-colorability of G[V-S]– 1.4423n * 1.3446n = 1.939n

• Better: there is always a color with at least n/4 vertices – Enumerate all m.i.s. S with at least n/4 vertices– For each, check 3-colorability of G[V-S]– 1.4423n * 1.34463n/4 = 1.8009n

• Byskov, 2004: O*(1.7504n) time


Coloring

• Arbitrary number of colors• With polynomial memory: Björklund, Husfeld,

2005• Divide and conquer approach• Set some constant r, 0 < r < 1• There is a color with at least rn vertices, or we can

divide the colors in two groups, such that for each group, there are between (1- r)/2 n and (1+r)/2 n vertices with a color in the group


Algorithm

• Enumerate all sets S– If S is a m.i.s. with at least rn vertices, compute

chromatic number of G[V-S] recursively; add 1– If (1-r)/2 n |S| (1+r)/2 n then recursively

compute chromatic number of G[S] and recursively compute chromatic number of G[V-S], and add these numbers

• Output the best over all |S|.


Setting r

• Using Maple or Mathematica for finding the value of r that gives the best solution

• r=0.19903 gives an algorithm using O*(5.283n) time– Analysis of number of m.i.s. of certain sizes,

etc.

• Other technique (2005): O*(2.4423n)


Coloring with dynamic programming

• Lawler, 1976: using DP for solving graph coloring.

(G) = minS is m.i.s. in G 1+X(G[V-S])• Tabulate chromatic number of G[W] over

all subsets W– In increasing size– Using formula above– 2n * 1.4423n = 2.8868n


Coloring

• Lawler 1976: 2.4423n (improved analysis)

• Eppstein, 2003: 2.4151n

• Byskov, 2004: 2.4023n – All using O*(2n) memory– Improvements on DP method

• Björklund, Husfeld, 2005: 2.3236n

• 2006 ….


Inclusion-exclusion

• Björklund and Husfeld, 2006, and independently Koivisto, 2006

• O*(2n) time algorithm for coloring

• Expression: number of ways to cover all vertices with k independent sets


First formula

• Let ck(G) be the number of ways we can cover all vertices in G with k independent sets, where the stable sets may be overlapping, or even the same– Sequences (V1,…,Vk) with the union of the Vi’s

= V, and each Vi independent

• Lemma: G is k-colorable, if and only if ck(G) > 0


Counting independent sets

• Let s(X) be the number of independent sets that do not intersect X, i.e., the number of independent sets in G(V-X).

• We can compute all values s(X) in O*(2n) time. – s(X) = s(X {v}) + s(X {v} N(v)) for v X

• Count IS’s with v and IS’s without v

– Now use DP and store all values

– Polynomial space slower algorithm also possible, by computing s(X) each time again


Expressing ck in s

• s(X)k counts the number of ways to pick k independent sets from V-X

• If a pick covers all vertices, it is counted in s()• If a pick does not cover all vertices, suppose it covers all

vertices in V-Y, then it is counted in all X that are a subset in Y– With a +1 if X is even, and a -1 if X is odd– Y has equally many even as odd subsets: total contribution is 0

VX

kXk XsGc )()1()( ||


Explanations

• Consider the number of k-tuples (W(1), … , W(k)) with each W(i) an independent set in G

• If we count all these k-tuples, we count all colourings, but also some wrong k-tuples: those which avoid some vertices

• So, subtract from this number all k-tuples of independent sets that avoid a vertex v, for all v

• However, we now subtract too many, as k-tuples that avoid two or more vertices are subtracted twice

• So, add for all pairs {v,w}, the number of k-tuples that avoid both v and w

• But, then what happens to k-tuples that avoid 3 vertices???• Continue, and note that the parity tells if we add or subtract…• This gives the formula of the previous slide


The algorithm

• Tabulate all s(X)

• Compute values ck(G) with the formula

• Take the smallest k for which ck(G) > 0


Maximum independent set

• Branch and reduce algorithm (folklore)

• Uses:– Branching rule– Reduction rules


Two simple reduction rules

• Reduction rule 1: if v has degree 0, put v in the solution set and recurse on G-v

• Reduction rule 2: if v has degree 1, then put v in the solution set. Suppose v has neighbor w. Recurse on G – {v,w}.– If v has degree 1, then there is always a

maximum independent set containing v


Idea for branching rule

• Consider some vertex v with neighbors w1, w2, … , wd. Suppose S is a maximum independent set. One of the following cases must hold:

1. v S. Then w1, w2, … , wd are not in S.

2. For some i, 1 id, v, w1, w2, … , wi-1

are not in S and wi S. Also, no neighbor of wi is in S.


Branching rule

• Take a vertex v of minimum degree.• Suppose v has neighbors w1, w2, …, wd.• Set best to 1 + what we get when we recurse on G – {v, w1,

w2, …, wd}. (Here we put v in the solution set.)• For i = 1 to d do

– Recurse on G – {v, w1, w2, …, wi} – N(wi). Say, it gives a solution of value x. (N(wi) is set of neighbors of wi. Here we put wi in S.)

– Set best = max (best, x+1).• Return best

Using some bookkeeping gives the corresponding set S

Using some bookkeeping gives the corresponding set S


Analysis

• Say T(n) is the number of leaves in the search tree when we have a graph with n vertices.

• If v has degree d, then we have T(n) (d+1) T(n-d-1).– Note that each wi has degree d as v had minimum

degree, so we always recurse on a graph with at least d-1 fewer vertices.

• d > 1 (because of reduction rules).• With induction: T(n) 3n/3.• Total time is O*(3n/3) = O*(1.4423n).


Number of maximal independent sets

• Suppose M(n) is maximum number of m.i.s.’s in graph with n vertices

• Choose v of minimum degree d.

• If v has degree 0: number of m.i.s.’s is at most 1* M(n)

• If v has degree 1: number of m.i.s.’s is at most 2* M(n-2)

• If v has degree d>1: number of m.i.s’s is at most (d+1)* M(n – d – 1)

• M(n) 3n/3 with induction


Some remarks

• Can be done without reduction step

• Bound on number of m.i.s.’s sharp: consider a collection of triangles


A faster algorithm

• Reduction rule 3: if all vertices of G have degree at most two, solve problem directly. (Easy in O(n+m) time.)

• New branching rule:– Take vertex v of maximum degree– Take best of two recursive steps:

• v not in solution: recurse of G – {v}

• v in solution: recurse on G – {v} – N(v); add 1.


Analysis

• Time on graph with n vertices T(n).• We have T(n) T(n – 1) + T(n – 4) +

O(n+m)– As v has degree at least 3, we loose in the

second case at least 4 vertices

• Induction: T(n) = O*(1.3803n)– Solve (with e.g., Maple or Mathematica)

• x4 = x3 + 1


Maximum Independent SetFinal remarks

• More detailed analysis gives better bounds• Current best known: O(1.1844n) (Robson, 2001)

– Extensive, computer generated case analysis!

– Includes memorization (DP)

• 2005: Fomin, Grandoni, Kratsch: the measure and conquer technique for better analysis of branch and reduce algorithms – Much simpler and only slightly slower compared to

Robson


Analysis

• More complicated measure (measure and conquer)

• Very simplified example here:• Suppose algorithm costs p(n)* aq * br time if

we have a graph with q vertices of degree three and r vertices with degree at least four

• Suppose a b• Case analysis: …


If largest degree is five or more

• If we do not select, r decreases by 1

• If we select, r decreases by at least 6

• For T(q,r) p(n)* aq * br to hold as induction hypothesis, we want:– aq * br aq * br-1 + aq * br-6 or

– b6 b5 + 1


If largest degree is four (1)

• If we do not select the vertex, we loose one vertex of degree four, and for each of its neighbors, we get a vertex that gets a degree one smaller– If the degree was 2, it has degree one, and is removed,

and then its neighbor gets a degree one smaller

– So, 4 vertices of degree 3 or 4 get a degree one smaller

– If all neighbors have degree 4, then we get r decreases by 5, and q increases by 3

– Other cases similar


If largest degree is four (2)

• If we select the vertex, then it, and its neighbors will be removed. The possible case is that these form a clique, and no other vertices are affected.– Here you see a possibility to improve the algorithm…

– If all neighbors have degree four, then we get that we want that aq * br aq+3 * br-4 + aq * br-4 which gives b4 a3 +1

– Other cases …


If largest degree is 3

• Exercise set 8, and similar to previous cases:

• We get in both cases that q drops by four, and r stays the same (namely, 0).

• As condition, we get that a4 2


Analysis

• Now, solve the set of equations

• Example: with Excel

• We get here as optimum 1.285n


Polynomial space algorithm for coloring

• Using the current best polynomial space algorithm to compute s(X) for each X, we can get an algorithm that uses O*(2.2461n) time and polynomial space for coloring


Remarks on measure and conquer

• More complicated manner to count number of subproblems

• For each degree d, we have a cost cd , e.g., vertices with degree 2 cost 0.7 (c2 = 0.7), etc.

• To an instance, associate the sum over all vertices of their cost (depending on their degree)

• From the rules, get a recurrence on these values, and bound it

• To select the costs that give the best bounds, a complicated problem has to be solved (quadratic programming)

• 2006, van Rooij: design by measure and conquer


Dominating set

• Now: exact algorithm for Dominating Set– Given: Graph G=(V,E)

– Question: find a minimum size set S V, such that for all v V: v S or v has a neighbor in S

• Translate to Set Cover:– Given: Collection Z of subsets of V

– Question: find a minimum subcollection such that each v V is in at least one set of the subcollection

• Note: Z is set of sets


Reduction rules for set cover: 1

• If S R, S and R both in Z, then there is an optimal solution without S– Replace S by R and we cover V with equally

many sets

• Rule 1: If S R, S and R both in Z, remove S from Z


Reduction rules for set cover: 2

• If, for v V, there is only one set in the collection Z that contains v, say S, then any set cover must include S

• Rule 2: If, for v V, there is only one set in Z that contains v, say S, then – Make a new instance, where we set all R Z to

R-S ={w R | w S}– Solve new instance recursively, and add 1


All sets of size 2

• If all sets have size at most 2, then the problem can be solved in polynomial time– After rules 1 and 2 cannot be applied, all sets

have size 2– The problem can be solved by maximum

matching• A maximum matching gives the largest number of

sets that cover two vertices each• All vertices not in the matching cost 1 entire set


Algorithm

• while rule 1 or rule 2 can be applied do– Apply rule 1 or 2

• if all sets have size at most 2– Solve problem exactly by matching

• else– Take a set S from collection of maximum size– Recurse: put S in the solution (remove S from Z, set

each R Z to R-S, add 1 to result)– Recurse: remove S from Z (S not in solution)– Output the best of these two


Simple analysis (1)

• Take as size: |V| + |Z|• Size starts at 2n• Rules 1 and 2, and all sets of degree 2 cost

polynomial time• Branching: two subproblems with sizes

decreased by 1 and by at least 4• Let T(n) be the time when size is n• T(n) T(n – 1) + T(n – 4)


Solving T(n) = T(n – 1) + T(n – 4)

• cn = cn-1 + cn-4

• Divide by cn-4

• c4 = c3 + 1 has a unique positive solution: namely c = 1.3082…

• Use e.g., Mathematica, Maple or Excel• So T(n) 1.3083n

• Algorithm for Dominating set uses at most O*(1.30832n) O*(1.7116n) time


More complicated cost measure (1)

• Idea: smaller sets are nicer than larger sets– Closer to reduction rules

– Similarly: elements that appear in few sets are better than elements that appear in many sets

• Frequency of element v: number of sets R Z with v R

• Write ni = number of sets in Z with size i

• Write mi = number of elements with frequency i


More complicated cost measure (2)

• For some values vi and wj, that will be chosen in a clever way later, we take as cost measure

• The values will be between 0 and 1, and increasing (larger is worse)

• Elements of frequency 1 and sets of size 1 are all removed by rules 1 and 2

• Set v1 = w1 = 0

22 j

jji

ii mvnw


Analysis• If the largest set S has size k, then

– Branch 1: choosing S in the solution, reduces nk by 1and k vertices are removed

• We gain wk for removing S• We gain k times a value vj (different js depending on frequency of

vertices in S, but each j>1)• Each v in S with frequency j means that j sets will have frequency one

smaller

– Branch 2: not choosing S in the solution, reduces nk by 1, and k vertices have frequency 1 smaller

• We gain wk for removing S• If one of these vertices had frequency 2, then it gets now frequency 1

and will be removed


Details, details

• Different cases for frequencies and sizes give a large collection of recurrences

• To simplify, all values for 6 and larger are set to 1 (appears not to contribute anyhow to better analysis)

• For a choice of vis and wjs we get a collection of recurrences

• A program in Mathematica or Maple (or for simpler instances Excell) can give the values where these recurrences give the largest value


(Near) Optimal values

• w2 = 0.3774

• w3 = 0.7548

• w4 = 0.9095

• w5 = 0.9764

• wk = 1 if k > 5

• v2 = 0.3996

• v3 = 0.7677

• v4 = 0.9300

• v5 = 0.9856

• vk = 1 if k > 5


And then

• From the recurrences, we see that a cost n instance solves at most 1.236n subproblems

• Initial cost is at most 2n (n elements, and n sets, each of cost at most 1)

• So, total time of this algorithm for Dominating Set is bounded by O*(1.2362n ) = O*(1.5263n)


Improvement

• van Rooij, 2006• Design by measure and conquer method:

– Analyze which of the recurrences is sharp– Look at that case, and try to build a new rule

that reduces that case

• For DS: one addition rule gives O*(1.5134n) and, using DP and storing some subproblems O*(1.5086n)


Held-Karp algorithm for TSP

• O*(2n) algorithm for TSP• Uses Dynamic programming• Take some starting vertex s• For set of vertices R (s R), vertex w R, let

– B(R,w) = minimum length of a path, that• Starts in s

• Visits all vertices in R (and no other vertices)

• Ends in w


TSP: Recursive formulation

• B({s},s) = 0

• If |S| > 1, then– B(S,w) = minv S – {w}B(S-{w}, v}

• If we have all B(V,v) then we can solve TSP.

• Gives requested algorithm using DP-techniques.


Final remarks

• Techniques for designing exponential time algorithms

• Other techniques, e.g., local search

• Combination of techniques

• Several interesting open problems

exponential time algorithms algorithms and networks

Documents

n time slide

n time algorithm

n time lawler

n time g

n time beigel

exponential time algorithms7

exponential time algorithms2

exponential time algorithms11