approximation algorithms - tunielomaa/teach/appral-15-1.pdf · 1. an introduction to approximation...

3/19/2015

1

Approximation Algorithms

Prof. Tapio Elomaa

[email protected]

Course Basics

• A new 4 credit unit course• Part of Theoretical Computer Science

courses at the Department of Mathematics• There will be 4 hours of lectures per week• Weekly exercises start next week• We will assume familiarity with

– Necessary mathematics– Basic programming

19-Mar-15MAT-72606 ApprAl, Spring 2015 2

3/19/2015

2

Organization & Timetable

• Lectures: Prof. Tapio Elomaa– Tue & Thu 12–14 in TB219 & TB223– Mar. 17 – May 12, 2015– Easter break: Apr. 2–8

• Exercises: M.Sc. Juho LauriMon 10–12 TC315

• Exam: Mon. May 25, 2015


Course Grading


• Exam: Maximum of 30 points• Weekly exercises yield extra points

• 40% of questions answered: 1 point• 80% answered: 6 points• In between: linear scale (so that

decimals are possible)

3/19/2015

3

Material

• The textbook of the course is– David P. Williamson & David B. Shmoys:

The Design of Approximation Algorithms,Cambridge University Press, 2011

• There is no prepared material, the slidesappear in the web as the lectures proceed– http://www.cs.tut.fi/~elomaa/teach/72606.html

• The exam is based on the lectures (i.e., noton the slides only)


Content (Plan)

1. An introduction to approximation algorithms2. Greedy algorithms and local search3. Rounding data and dynamic programming4. Deterministic rounding of linear programs5. Random sampling and randomized rounding of

linear programs6. Randomized rounding of semidefinite programs7. The primal-dual method8. Cuts and metrics


3/19/2015

4

1. Introduction

The whats and whysThe set cover problem

A deterministic rounding algorithmRounding a dual solutionThe primal-dual method

A greedy algorithmA randomized rounding algorithm

2. Greedy algorithms andlocal search

Scheduling jobs with deadlinesThe -center problem

Scheduling jobs on parallel machinesThe traveling salesman problem

Maximizing float in bank accountsMinimum-degree spanning trees

Edge coloring

3/19/2015

5

3. Rounding data anddynamic programming

The knapsack problemScheduling jobs on identical parallel

machinesThe bin-packing problem

4. Deterministic roundingof linear programs

Minimizing the sum of completion timesMinimizing the weighted sum of completion

timesSolving large linear programs in polynomial

time via the ellipsoid methodThe prize-collecting Steiner tree problem

The uncapacitated facility location problemThe bin-packing problem

3/19/2015

6

5. Random sampling andrandomized rounding of

linear programsSimple algorithms for MAX SAT and MAX CUT

DerandomizationFlipping biased coins

Randomized roundingChoosing the better of two solutions

Non-linear randomized roundingThe prize-collecting Steiner tree problem

The uncapacitated facility location problemScheduling a single machine with release dates

Chernoff boundsInteger multicommodity flows

Random sampling and coloring dense 3-colorable graphs

6. Randomized rounding ofsemidefinite programs

A brief introduction to semidefiniteprogramming

Finding large cutsApproximating quadratic programs

Finding a correlation clusteringColoring 3-colorable graphs

3/19/2015

7

7. The primal-dual method

The set cover problem: a reviewChoosing variables to increaseCleaning up the primal solution

Increasing multiple variables at onceStrengthening inequalities

The uncapacitated facility location problemLagrangean relaxation and the -median

problem

8. Cuts and metrics

The multiway cut problem and a minimum-cut-basedalgorithm

The multiway cut problem and an LP rounding algorithmThe multicut problem

Balanced cutsProbabilistic approximation of metrics by tree metrics

An application of tree metrics: Buy-at-bulk network designSpreading metrics, tree metrics, and linear arrangement

3/19/2015

8

1.1 The whats and whys

• Many interesting discrete optimization problemsare NP-hard

• If P NP, we can’t have algorithms that1) find optimal solutions2) in polynomial time3) for any instance

• At least one of these requirements must berelaxed in any approach to dealing with an NP-hard optimization problem


Definition 1.1: An -approximation algorithm foran optimization problem is a polynomial-timealgorithm that for all instances of the problemproduces a solution whose value is within a factorof of the value of an optimal solution.

• We call the performance guarantee of thealgorithm

• It is also often called the approximation ratio orapproximation factor of the algorithm

• We will follow the convention that > 1 forminimization problems, while < 1 formaximization problems


3/19/2015

9

Definition 1.2: A polynomial-time approximationscheme (PTAS) is a family of algorithms },where there is an algorithm for each > 0, suchthat is a (1 + )-approximation algorithm (forminimization problems) or a (1 )-approximationalgorithm (for maximization problems).• E.g., the knapsack problem and the Euclidean

traveling salesman problem, both have a PTAS• A class of problems that is not so easy is MAX

SNP• It contains problems, such as the maximum

satisfiability problem and the maximum cutproblem


Theorem 1.3: For any MAX SNP-hard problem, theredoes not exist a PTAS, unless P = NP.• E.g., maximum clique problem is very hard:

– Input: an undirected graph = ( )– Goal: find a maximum-size clique; i.e., find

that maximizes | so that for each pair , itmust be the case that

• Almost any nontrivial approximation guarantee ismost likely unattainable:

Theorem 1.4: Let be the number of vertices in aninput graph, and consider any constant > 0. Thenthere does not exist an )-approximationalgorithm for the max clique problem, unless P = NP


3/19/2015

10

• Observe that it is very easy to get an -approximation algorithm for the problem:– just output a single vertex

• This gives a clique of size 1, whereas the size ofthe largest clique can be at most , the numberof vertices in the input

• The theorem states that finding something onlyslightly better than this completely trivialapproximation algorithm implies that P = NP


1.2 Linear programs:The set cover problem

• Given a ground set of elements = { , … , },some subsets of those elements , … , whereeach , and a nonnegative weight 0for each subset

• Find a minimum-weight collection of subsets thatcovers all of ; i.e., we wish to find an{1, … , } that minimizes subject to

• If = 1 for each , the problem is unweighted


3/19/2015

11

• The SC was used in the development of anantivirus product, which detects computerviruses

• It was desired to find salient features that occurin viruses designed for the boot sector of acomputer, such that the features do not occur intypical computer applications

• These features were then incorporated into aneural network for detecting these boot sectorviruses

• The elements of the SC were the known bootsector viruses (about 150 at the time)


• Each set corresponded to some three-bytesequence occurring in these viruses but not intypical computer programs; there were about21,000 such sequences

• Each set contained all the viruses that had thecorresponding sequence somewhere in it

• The goal was to find a small number of suchsequences ( 150) that would be useful for theneural network

• By using an approximation algorithm to solve theproblem, a small set of sequences was found,and the neural network was able to detect manypreviously unanalyzed boot sector viruses


3/19/2015

12

• The set cover problem also generalizes thevertex cover problem

• In the vertex cover problem, we are given anundirected graph = ( ) and a nonnegativeweight 0 for each vertex

• The goal is to find a minimum-weight subset ofvertices such that for each edge ,either or

• Again, if = 1 for each vertex , the problem isunweighted


• To see that the vertex cover problem is a specialcase of the set cover problem:

• for any instance of the vertex cover problem,create an instance of the set cover problem inwhich– the ground set is the set of edges, and– a subset of weight is created for each

vertex containing the edges incident to• It is not difficult to see that for any vertex cover

, there is a set cover of the same weight,and vice versa


3/19/2015

13

MAT-72606 ApprAl, Spring 2015 19-Mar-15

12

342-3

1-3

1-2

2-4S2

S4

S3S1


• Linear programming plays a central role in thedesign and analysis of approximation algorithms

• Each linear program (LP) or integer program (IP)is formulated in terms of some number ofdecision variables that represent some sort ofdecision that needs to be made

• The variables are constrained by a number oflinear inequalities and equalities calledconstraints

• Any assignment of real numbers to the variablessuch that all of the constraints are satisfied iscalled a feasible solution

3/19/2015

14

• In the set cover problem, we need to decidewhich subsets to use in the solution

• A decision variable to represent this choice• We would like to be 1 if the set is included

in the solution, and 0 otherwise• Thus, we introduce constraints 1 and 0

for all subsets• This does not guarantee that 0,1 , so we

formulate the problem as an IP to excludefractional solutions

• Requiring to be integer along with theconstraints suffices to guarantee that 0,1


• We also want to make sure that any feasiblesolution corresponds to a set cover, so weintroduce additional constraints

• In order to ensure that every element iscovered, it must be the case that at least one ofthe subsets containing is selected

• This will be the case if

1 ,

for each , = 1, … ,


3/19/2015

15

• In addition to the constraints, LPs and IPs aredefined by a linear function of the decisionvariables called the objective function

• The LP or IP seeks to find a feasible solutionthat either maximizes or minimizes this objectivefunction

• Such a solution is called an optimal solution• The value of the objective function for a

particular feasible solution is called the value ofthat solution

• The value of the objective function for an optimalsolution is called the value of the LP or IP


• We say we solve the LP if we find an optimalsolution

• In the case of the set cover problem, we want tofind a set cover of minimum weight

• Given the decision variables and constraintsdescribed above, the weight of a set cover giventhe variables is

• Thus, the objective function of the IP is, and we wish to minimize this function


3/19/2015

16

• IPs and LPs are usually written in a compactform stating first the objective function and thenthe constraints

• The problem of finding a minimum-weight setcover is equivalent to the following IP:

minimize

subject to 1 , = 1, … ,

0,1 , = 1, … ,


• Let denote the optimum value of this IP for agiven instance of the set cover problem

• Since the IP exactly models the problem, wehave that = OPT, where OPT is the value ofan optimum solution to the set cover problem

• In general, IPs cannot be solved in polynomialtime

• This is clear because the set cover problem isNP-hard, so solving the IP above for any setcover input in polynomial time would imply thatP = NP


3/19/2015

17

• However, LPs are polynomial-time solvable• We cannot require the variables to be integers• Even in cases such as the set cover problem,

we are still able to derive useful information fromLPs

• If we replace the constraints 0,1 with theconstraints 0, we obtain the following LP,which can be solved in polynomial time

minimizesubject to 1 = 1, … ,

0, = 1, … ,


• We could also add the constraints 1, foreach = 1, … , , but they would be redundant:– in any optimal solution to the problem, we can

reduce any > 1 to = 1 without affectingthe feasibility of the solution and withoutincreasing its cost

• The LP is a relaxation of the original IP:1. every feasible solution for the original IP is

feasible for this LP; and2. the value of any feasible solution for the IP

has the same value in the LP


3/19/2015

18

• To see that the LP is a relaxation, note that anysolution for the IP such that– 0,1 for each = 1, … , and– 1 for each = 1, … ,

will certainly satisfy all the constraints of the LP• Furthermore, the objective functions of both the

IP and LP are the same, so that any feasiblesolution for the IP has the same value for the LP

• Let denote the optimum value of this LP• Any optimal solution to the IP is feasible for the

LP and has value


• Thus, any opt. solution to the LP will have value= OPT, since this minimization LP

finds a feasible solution of lowest possible value• Using a poly-time solvable relaxation in order to

obtain a lower bound (in min) or an upper bound(in max) on the optimum value of the problem isa common concept

• We will see that a fractional solution to the LPcan be rounded to a solution to the IP ofobjective function value that is within a certainfactor of the value of the LP

• Thus, the integer solution will cost OPT


3/19/2015

19

1.3 A deterministic rounding algorithm

• Suppose that we solve the LP relaxation of theset cover problem

• Let denote an optimal solution to the LP• How then can we recover a solution to the set

cover problem?• Given the LP solution , we include subset in

our solution iff , where is themaximum number of sets in which any elementappears


• More formally, let = be the numberof sets in which element appears, = 1, … , ;then = max

,…,

• Let denote the indices of the subsets in thissolution

• In effect, we round the fractional solution toan integer solution by setting = 1 if

, and = 0 otherwise• It is straightforward to prove that is a feasible

solution to the IP, and indeed indexes a setcover


3/19/2015

20

Lemma 1.5: The collection of subsets , , is aset cover.Proof: Consider the solution specified by thelemma. An element is covered if this solutioncontains some subset containing .Show that each element is covered. Becausethe optimal solution is a feasible solution to theLP, we know that 1 for element . Bythe definition of and , there are terms inthe sum, so at least one term must be .Thus, for some such that , .Therefore, , and element is covered.


Theorem 1.6: The rounding algorithm is an-approx. algorithm for the set cover problem.

Proof: It is clear that the algorithm runs inpolynomial time. By our construction,for each . From this, and the fact that eachterm is nonnegative for = 1, … , , we seethat

OPT

where the final inequality follows from the fact thatOPT.


3/19/2015

21

• In the vertex cover, = 2 for each , sinceeach edge is incident to exactly two vertices

• Thus, the rounding algorithm gives a 2-approx.algorithm for the vertex cover problem

• This particular algorithm allows us to have an afortiori guarantee for each input

• While we know that for any input, the solutionproduced has cost at most a factor of morethan the cost of an optimal solution, we can forany input compare the value of the solution wefind with the value of the linear programmingrelaxation


• If the algorithm finds a set cover , let

=

• From the proof above, we know that• However, for any given input, it could be the

case that is significantly smaller than ; in thiscase we know that

OPT,

and the solution is within a factor of of optimal• The algorithm can easily compute , given that it

computes and solves the LP relaxation19-Mar-15MAT-72606 ApprAl, Spring 2015 42

3/19/2015

22

1.4 Rounding a dual solution

• Suppose that each element is charged somenonnegative price 0 for its coverage by aset cover

• Intuitively, it might be the case that someelements can be covered with low-weightsubsets, while other elements might requirehigh-weight subsets to cover them– we would like to be able to capture this distinction

by charging low prices to the former and highprices to the latter


• In order for the prices to be reasonable, it cannotbe the case that the sum of the prices ofelements in a subset is more than the weightof the set, since we are able to cover all of thoseelements by paying weight

• Thus, for each subset we have the followinglimit on the prices:

• We can find the highest total price that theelements can be charged by the following LP:


3/19/2015

23

maximize

subject to = 1, … , ,

0, = 1, … ,• This is the dual LP of the set cover LP relaxation• If we derive a dual for a given LP, the given

program is called the primal LP• This dual has a variable for each constraint of

the primal LP (i.e., for 1), and has aconstraint for each variable of the primal

• This is true of dual linear programs in general19-Mar-15MAT-72606 ApprAl, Spring 2015 45

• Dual LPs have a number of very interesting anduseful properties

• E.g., let be any feasible solution to the setcover LP relaxation, and let be any feasible setof prices (any feasible solution to the dual LP)

• Then consider the value of the dual solution :

,

• since for any , 1 by the feasibilityof


3/19/2015

24

• Rewriting the RHS of this inequality, we have

=

• Finally, noticing that since is a feasible solutionto the dual LP, we know that forany , so that


• So we have shown that

i.e., any feasible solution to the dual LP has avalue any feasible solution to the primal LP

• In particular, any feasible solution to the dual LPhas a value the optimal solution to the primalLP, so for any feasible ,

• This is called the weak duality property of LPs• Since we previously argued that OPT, we

have that for any feasible , OPT


3/19/2015

25

• Additionally, there is a quite amazing strongduality property of LPs– as long as there exist feasible solutions to

both the primal and dual LPs, their optimalvalues are equal

• Thus, if is an optimal solution to the set coverLP relaxation, and is an optimal solution tothe dual LP, then

=


approximation algorithms - tunielomaa/teach/appral-15-1.pdf · 1. an introduction to approximation...

Documents