lecture10 - p and np

8/14/2019 Lecture10 - P and NP

1/55

COMP2230 Introduction toAlgorithmics

Lecture 10

A/Prof Ljiljana Brankovic


2/55


3/55

Lecture Overview

Motivating Examples

The Complexity of Computations,Algorithms and Problems

Complexity Classes: P, NP and NP-Complete

NP-completeness proofs


4/55

Motivating Examples

Suppose that you were given the following 3problems to solve:

1. Given a map containing cities and roadsbetween cities, design a route that startsat a given city, traverses each road

exactly once and returns to the startingcity (if there is such a route).


5/55

Motivating Examples

2. Given a map containing cities, roadsbetween cities, and the length of each

road, design a minimum length route thatstarts at a given city, visits each cityexactly once and returns to the startingcity.


6/55

Motivating Examples

3. Design an algorithm that takes as input aprogram and an input to it and determineswhether or not the program will halt onthat input.


7/55

Motivating Examples1. The first problem is known as finding an Eulerian

cycle in a graph and can be solved in polynomialtime.

2. The second problem is known as Travelling

Salesperson Problem (TSP) and there is no knowpolynomial time algorithm for solving it.

3. The third problem is the so-called Halting

problem; this problem is undecidible, that is,there is no algorithm that can solve this problem.


8/55

Motivating ExamplesProblems that cannot be solved in polynomial time are

called intractable. Intractable problems areeither so difficult that a solution cannot befound in polynomial time, or the solution itself isso extensive that it cannot be described in

polynomial time (e.g., list all possible routes atravelling salesman can take the number ofroutes is exponential)

We are primarily concerned with problems for whichneither there are known polynomial timealgorithms, nor it has been proved that suchalgorithms do not exist.


9/55

Decision Problems

We focus our attention on so-called decision problems.These are problems which only require yes or noanswer, e.g., Given a graph G, is there an Euleriancycle in G?

All instances of a decision problem can be divided intoyes instances (those for which the answer is yes)and no instances (those for which the answer is no).


10/55

Decision Problems

It is known that a graph has an Eulerian cycle if and onlyif it is connected and all its vertices have evendegrees.

1

Thus in the Eulerian cycle problem, all connected graphsin which every vertex has an even degree correspondto yes instances of the problem, and all the othergraphs to no instances.


11/55

Turing machines

In what follows we shall use a Turing machine as a generalmodel of computation.

Recall that a Turing machine has an infinite tape divided

into cells, and a read-write head that can read thecontent of a cell, overwrite the content of a cell, ormove left or right one step at the time.


12/55

Turing machines

A Turing machine has a finite number of states and basedon the current state and the content of the currentcell it makes decision on what action to take (overwritethe cell and move one cell to the left or right) andwhat new state to go to.

Any problem that can be solved on any Random AccessMachine (RAM) in polynomial time, can also be solvedon a Turing machine in polynomial time.


13/55

The Complexity ofComputations

The time complexityof a computation is theamount of time required to perform thecomputation.

The space complexityof a computation is thespace required to perform the computation.


14/55

The Complexity of Computations

The time complexity of a Turing machine computationis the number of single transitions executed duringthe computation.

The space complexity of a Turing machinecomputation is the number of tape cells required bythe computation.

If the time complexity of a Turing machinecomputation is n, then the space complexity of thecomputation is no more than n+1.


15/55

The Complexity of Algorithms

A Turing machine can be seen as the implementation of asingle algorithm.

In general, the time complexity of a Turing machinecomputation is a function of the length of the input.

best-case, worst-case and average-case performance average-case :

pi

ci

where piis the probability of computation with

complexity ci


16/55

The Complexity of Problems

The complexity of a problem is not the same as thecomplexity of a particular algorithm.

We may say that the complexity of a problem is thecomplexity of its simplest solution.


17/55


The complexity of a problem is (f) if it can besolved by an algorithm whose complexity is (f)and if any better solution also have the complexity

(f).


18/55


Definition:

Let t: NN be a function. We define the time

complexity class, TIME(t(n)), to be:

TIME(t(n)) = { L | L is a problem decided by a O(t(n))time Turing machine }


19/55

The Complexity of ProblemsExample:

Consider the language A = {0n1n | n 0}, and the

following algorithm for deciding whether or not a string

is in the language A:

M1 = On input string : Scan across the tape and reject if a 0 is found to the right of a 1.

Repeat the following if both 0s and 1s remain on the tape:

Scan across the tape, crossing off a single 0 and a single 1.

If 0s still remain after all the 1s have been crossed off, or if 1sstill remain after all the 0s have been crossed off, reject;otherwise, accept.


20/55


Example (contd):M1decides A in time O(n2), and thus A TIME(n2).

M2decides A in time O(n logn), and thus A TIME(n logn).

M2 = On input string : Scan across the tape and reject if a 0 is found to the right of

a 1.

Repeat the following if both 0s and 1s remain on the tape:

Scan across the tape, checking if total number of 0s and 1s

remaining is even; if not, reject. Scan again across the tape, crossing off every other 0 starting

with the first 0, and then crossing off every other 1 startingwith the first 1.

If no 0s and no 1s remain on the tape, accept; otherwise,

reject.


21/55


Example (contd):M3is a two-tape TM that decides A in time O(n).

M3 = On input string : Scan across the tape and reject if a 0 is found to the right of

a 1. Scan across the 0s on T1until the first 1. At the same time,

copy the 0s onto T2.

Scan across the 1s on T1until the end of input. For each 1 on

T1, cross off a 0 on T2. If all the 0s are crossed off before all1s are read, reject.

If all the 0s have been crossed off, accept; otherwise,reject.


22/55

The Complexity of ProblemsTheorem:

Let t(n) be a function such that t(n) n. Then every t(n)

time multiple-tape Turing machine has an equivalent

O(t2(n)) time single-tape Turing machine.

Definition: Let N be a nondeterministic Turing decider. The

running time of M is the function f:NN, where f(n) is

the maximum number of steps that M uses on a shortestaccepting branch of its computation, over all inputs of

length n.


23/55


Theorem:

Let t(n) be a function such that t(n) n. Then every t(n)time nondeterministic single-tape Turing machine has an

equivalent 2O(t(n))time deterministic single-tape Turing

machine.


24/55

The Class PDefinition:The class P is the class of decision problems that are

decidable in polynomial time on a deterministic single-

tape Turing machine.

P is invariant for all computational models that are

polynomially equivalent to the deterministic single-tape

Turing machine.

P roughly corresponds to the class of problems that are

realistically solvable on a computer.

k

knTIMEP )(


25/55

The Class P

Example:Let G be a directed graph and let s and t be two vertices of

G (s,t V(G)). The PATH problem is to determine

whether there exists a directed path from s to t.

PATH = { | G is a directed graph that has a directed

path from s to t }

Theorem:

PATH P.


26/55

The Class NP

Definition:A nondeterministic algorithm can be thought of as a twostage algorithm; in the first stage, algorithmguesses asolution to the problem; in the second stage the

algorithm verifies the solution.A verifierfor a problem A is an algorithm V which takes as

input an instance and a candidate solution c (guess or

witness), and verifies it.

Apolynomial time verifier runs in polynomial time in lengthof .


27/55

The Class NP

Definition:The class NP is the class of decision problems (languages)

that have polynomial time verifiers.

Theorem:A language is in NP if and only if it is decided by some

nondeterministic polynomial time Turing machine.


28/55

The Class NP

Definition:NTIME(t(n)) = { L | L is a problem decided by a

O(t(n)) time nondeterministic Turing machine}

Corollary:

kk

nNTIMENP )(


29/55

The Class NP

Example:A clique in an undirected graph G is a

subgraph of G such that every two

vertices in the clique are connected by anedge. A k-clique is a clique that contains k

vertices.

CLIQUE = { | G is an undirected graph with

a k-clique}


30/55

The Class NP

Theorem:CLIQUE NP

Proof:A polynomial time verifier:

V = On input: Test whether c is a set of k vertices in G.

Test whether every two vertices in c are connected by an

edge in G.

If both answers are yes, accept; otherwise, reject.


31/55

The Class NP

Alternative Proof:A polynomial time NTM:

N = On input


32/55

The Class NP

Example:SUBSETSUM = { | S = {x1,x2,,xk} and for

some {y1,y2,,ys} {x1,x2,,xk} we have yi=t}

Theorem:

SUBSETSUM NP.


33/55

The Class NP

Proof:A polynomial time verifier:

V = On input:

Test whether c is a collection of numbers that sum to t. Test whether c S.

If both answers are yes, accept; otherwise, reject.


34/55

P versus NP

P is the class of decision problems that can bedecided in polynomial time.

NP is the class of decision problems that can be

verified in polynomial time. The most outstanding unsolved question in

computer science:

NPP

?


35/55

P versus NP

NP

P

P = NPOR ?


36/55

NP-Completeness

NP-completeproblems are the problems in NP with

the following property: If a polynomial time algorithmexists for any of these problems, then all problems inNP would have a polynomial time algorithm.


37/55

NP-Completeness

Boolean variables are variables that can take valueTRUE (1) or FALSE (0).

Boolean operations: AND, OR and NOT.

A Boolean formula is an expression that involves Booleanvariables and operations (evaluates to TRUE or FALSE).

A Boolean formula is satisfiable if there exists anassignment of 0s and 1s to the variables such that theformula evaluates to 1 (TRUE).


38/55

NP-CompletenessThesatisfiability problem is to test whether a Boolean

formula is satisfiable.

A literalis a Boolean variable or a negated Boolean variable.

A clauseis several literals connected with ORs.

A Boolean formula in conjunctive normal formcomprisesseveral clauses connected with AND.

Example:)()()(

523653321 xxxxxxxxx


39/55

NP-Completeness

Cook-Levin theorem:

SAT P iff P = NP


40/55

NP-Completeness

Definition:

A function f: **is a polynomial time computablefunction if there exists a polynomial time Turingmachine M that halts with f() on its tape when startedon input .


41/55

NP-Completeness

Definition:A problem A is polynomial time reducible to problem B, written ApB, if there exists a polynomial time computable function f: ** such that for every ,

A f() B.

The function f is called the polynomial time reduction from A to B.

In other words, the function f is a mapping from instances of problem

A to the instances of problem B, such that every yes instance ofproblem A is reduced to a yes instance of problem B, and everyno instance of problem A is reduced to a no instance of problemB.


42/55

NP-Completeness

Theorem:

If A pB and B P, then A P.


43/55

NP-Completeness

Proof:Let M be the polynomial time algorithm deciding B and let

f be the polynomial time reduction from A to B. Then Nis a polynomial time algorithm that decides A.

N = On input :

Compute f().

Run M on input f() and output what M outputs.


44/55

NP-Completeness

3SAT is SAT in 3 conjunctive normal form (all clauseshave exactly 3 literals).

Theorem:

3SAT is polynomial time reducible to Independent set(proof is in the text).


45/55

NP-Completeness

Definition:

A language B is NP-complete if it satisfies the followingtwo conditions:

B is in NP

every A in NP is polynomial time reducible to B.


46/55

NP-Completeness

Theorem:

If B is NP-complete and B is in P, then P = NP.

Theorem:

If B is NP-complete and B p C and C is in NP, then C isNP-complete.


47/55

NP-Completeness

Theorem (Levin-Cook):SAT is NP-complete.

Theorem:3SAT is NP-complete.Proof:We use reduction from SAT. We first need to

prove that 3SAT is in NP.

Indeed, if we have a truth assignment for all the variablesit is easy to check whether this assignment satisfies allthe clauses; we can do this in polynomial time, bychecking clause by clause (O(m), where m is the numberof clauses).


48/55

NP-Completeness

Proof contd:

We next need to show that there exists a polynomial timereduction from SAT to 3SAT, that takes an arbitraryinstance of SAT and maps it into an instance of 3SAT.

Then we need to prove that the reduction can beperformed in polynomial time and that the instance ofSAT is a yes instance if and only if the correspondinginstance of 3SAT is a yes instance.


49/55

NP-Completeness

Proof contd:Reduction: Take an arbitrary instance of SAT.

Consider a clause by clause. We distinguishbetween 4 cases:

1. the clause has one literal

2. The clause has 2 literals

3. The clause has 3 literals

4. The clause has more than 3 literals


50/55

NP-Completeness

Proof contd:1. The clause has one literal:

Let the original clause be (x); we replace it with the

following set of clauses each of which has exactly 3literals:(x, y1, y2), (x, y1, y2), (x, y1, y2), (x, y1, y2)

Here y1

and y2

are variables that appear in these fourclauses and nowhere else. Note that for every truthassignment to y1and y2there will always be one clausethat is not satisfied by y1and y2, so x has to be truein order to satisfy all four clauses.


51/55

NP-Completeness

Proof contd:2. The clause has two literals:

Let the original clause be (x1, x2); we replace it with

the following set of clauses each of which has exactly3 literals:(x1, x2, y), (x1, x2, y)

Again, y is a variable that appears in these twoclauses and nowhere else. Note that for every truthassignment to y there will always be one clause that isnot satisfied by y, so x1or x2has to be true in orderto satisfy all four clauses.


52/55

NP-Completeness

Proof contd:3. The clause has three literals:

Let the original clause be (x1, x

2, x

3); we keep

this clause as it is.


53/55

NP-Completeness

Proof contd:4. The clause has more than three literals:

Let the original clause be (x1, . . ., xk); we replace itwith the following set of clauses each of which has

exactly 3 literals:(x1, x2, y1), (y1, x3, y2), (y2, x4, y3), . . . , (yk-3, xk-1, xk)

Here y1, . . ., yk-3are variables that appear in these k-2 clauses and nowhere else. Note that for every truth

assignment to y1, . . ., yk-3there will always be oneclause that is not satisfied by y1, . . ., yk-3, so one ofthe x1, . . . , xkhas to be true in order to satisfy allthe clauses.


54/55

NP-CompletenessProof contd:The final thing we need to show is that this reduction can

be performed in polynomial time. Each clause can beconstructed in constant time, since each clause hasexactly 3 literals. Therefore we only need to show

that the total number of clauses is polynomial interms of the number n of variables and number m ofclauses of the instance of the original SAT problem.

For each original clause with 1 literal we construct 4clauses; for each clause with 2 literals, 2 clauses; for

each clause with 3 literals, 1 clause, and for eachclause with k literals, k 4, k-2 clauses; since k2n, wehave no more than 2mn clauses, therefore thereduction can be performed in polynomial time.


55/55

NP-Completeness

Theorem:

Independent set is NP-complete.

Proof:

We already proved that Independent set is in NP, andthat 3SAT is polynomial time reducible to Independent

set.

lecture10 - p and np

Documents