Theory of Computing

Lecture 17MAS 714

Hartmut Klauck

Nondeterministic Turing Machine

• A nondeterministic Turing machine is a 7-tuple(Q, ¡, b, §, q0, A, ±)

• Q: set of states of the machine• ¡: tape alphabet• b2¡: blank symbol• §µ¡-{b}: input alphabet• q02 Q: initial state• A µ Q: accepting states• ±: Q- A £ ¡

Subset of {Set of all {Q£¡£{left,stay,right}}: transition

Notes

• NTIME(f(n)) is the class of all languages that have nondeterministic time complexity at most O(f(n))

• NP is the class of languages L such that the time complexity of L can be upper bounded by a polynomial in n

NP

• Guess and check definition:• A language L is in NP, if there is a language R2P such that for all x: [There is a poly(|x|) length string y s.t. x#y2R] , x2 L

• y is called proof or witness

Relation to other classes

• Pµ NP by definition• NPµ EXP– enumerate all witnesses and check each

• Better: NPµ PSPACE– PSPACE: languages decidable by TM using

polynomial space– enumerate all witnesses (re-use the same space)

Completeness

• We will identify a class of problems that capture the hardness of NP

• P=NP if and only if one of these problems is in P– None are known to be in P

• These are the problems L in NP, such that any other problem in NP can be solved by a deterministic polynomial time TM given a free subroutine that solves L

Completeness

• Informally, L reduces to S, if, given an `oracle’ that decides S for free we can compute L in polynomial time (deterministically)

• NP-complete: L is in NP and all S in NP reduce to L– L is hardest in NP

Cook-Reductions

• Named after Stephen Cook• L reduces to S by a Cook-reduction, if there is

an algorithm that solves L, using calls to a subroutine to S, and runs in polynomial time

• Formal definition uses a model of oracle Turing machine

Karp-Reductions

• We will use a simpler definition of reductions• Also called ``many-one reductions’’• Definition:

– Let Lµ¡* and Sµ§*– A Karp-reduction from L to S is a polynomial time computable

function f that sends x2¡* to f(x)2§* such that x2 L , f(x)2 S

• Clearly: a polynomial time algorithm for S implies a polynomial time algorithm for L

• Notation: L·P S• Vice versa: if L is not in P, then S is not in P

Definition: NP-complete

• Definition: a language L is NP-hard, if every problem S 2 NP reduces to L ( S·P L )

• Definition: a language L is NP-complete, if L is NP-hard and L 2 NP

• We use Karp reductions. Cook reductions may result in a larger set of NP-complete problems (this is an open problem)

NP-completeness

• Theorem: P=NP iff there is an NP-complete problem in P

• Counterpositive: If P NP then no NP-complete problem is in P

Proof

• Assume an NP-complete problem L is in P:– For any (other) problem S in NP we can perform

the reduction and then decide L– This is a polytime algorithm for all S in NP

• P=NP: implies that all problems in NP are in P

How to identify a hard problem

• Given a language L, is it NP-complete?• Find a known NP-complete problem S• Reduce S to L

• Then L is hard under the assumption that S is hard

• How do we get our first NP-complete problem?

Finding an NP-complete Problem

• We will define the problem CircuitSAT and show how every problem in NP reduces to it

• Then we can show further NP-hardness results by reducing from CircuitSAT, or any other NP-hard problem we already know

• Important property of reductions:• Lemma: If L·p S and S·p T then L·p T [Transitivity]• Proof: exercise

Reducing from any L2 NP

• Given L2 NP we need to find a reduction to CircuitSAT

• L2 NP) there is a language S2 P such that x2 L iff 9y: (x#y)2S– y is short and S has a polytime TM M– the length of y is known given x

• We will model the computation of M using Boolean circuits

Circuits

• A Boolean circuit is a directed acyclic graph where each inner node has in-degree 1 or 2. The sources are labeled by variables from {x1,…,xn}. Inner nodes are labeled with the gate functions AND, OR, NOT

• Notation of AND: ÆOR: ÇNOT: :

• The sink nodes are the outputs of the circuit

Example

Computation of a circuit

• Inputs x1,…,xn are instantiated to some Boolean values

• Value of the source nodes are the values of the corresponding variables

• Value of the inner nodes: – AND(x,y)=1 iff x=1 and y=1– OR(x,y)=1 iff x=1 or y=1– NOT(x)=1 iff x=0

• Process nodes in topological order

Circuits

• A circuit computes a function f:{0,1}n{0,1}k if the output gates are correct for every input

• A family of circuits is an infinite sequence C1,C2, C3,… of circuits, such that Cn has n inputs

• A family computes a function f:{0,1}* {0,1} if each Ci computes f restricted to {0,1}n

Uniformity

• A family of circuits is uniform, if each circuit Cn is easy to compute, given n– This means a TM can output a description of Cn in time poly(n)

• Otherwise the circuit model is too powerful

• The size of a circuit family is the function that maps the input length n to the number of nodes of Cn

• Note: P=class of Boolean functions computable by polynomial size uniform circuit families.

CircuitSAT

• Input: a Boolean circuit C with n inputs and 1 output and size m– Encoded suitably

• Output: accept if and only if there is an input x that is accepted by C

• Circuit Satisfiablity– circuit is satisfiable if there is an input that will

make it accept/compute 1

Circuit simulating Turing Machine

• Theorem:– If M is a polytime Turing machine, and n is the

fixed input length we consider, then there is a (uniform) circuit Cn, that decides the same language as M on inputs of length n. The size of Cn is polynomial in n

Proof Sketch

1. For simplicity we consider only TM with alphabet {0,1} (original symbols and blanks are encoded as strings )

2. Change the Turing machine so it moves obliviously (left to right all the way and back etc.)

3. Construct a circuit for the transition function4. `Glue’ copies of the transition circuit together

into a circuit that simulates the TM

Pieces of the proof:

• A Turing machine is sweeping, if it moves (on every input) from the left end of the space used to the right end and back until it stops on the left end

• Lemma: Making M sweeping increases the running time from T(n) to O(T2(n))

• Proof: Leave a marker for the head position on the tape. For each time step of M sweep the whole tape region used from left to right and back twice. Move the marker according to the computation of M.

Pieces of the proof:

• Every Boolean function {0,1}n {0,1} can be computed by a Boolean circuit [Shannon]

• Then: the transition function can be computed by circuits

• Proof: exercise

• Note: most functions need exponential size circuits. Transition function is of constant size.

• Hence the circuits are of constant size.

Pieces of the proof:

• Gluing the circuits together:– We use T(n)2 copies of the transition function circuit.

Arrange them as a grid.– Copy i,j performs the computation of the transition

function when the TM is on cell j of the tape during the i-th traversal of the tape

– Inputs of copy i,j: tape symbol from copy i-1, state from the copy i, j-1 or i, j+1

– Outputs: tape symbol goes to copy i+1,j ; state goes to copy i,j+1 or i,j-1

– Special connections at j being left and right end of the tape

Conclusion:

• The size of the circuit is O(T(n)2)• The circuit is easy to construct from the TM,

i.e., if we do this construction for all n we get a uniform polynomial size family of circuits

NP-hardness

• Theorem: CircuitSAT is NP-hard• Take any L2 NP• Use the TM M for S2P from the guess and check definition of NP• x2L iff x#y2 S• For x2 {0,1}m find the circuit Cn that simulates M on length n

inputs (n=length of x and y combined)• Fix the source nodes of Cn reading xi variables to the actual

values of these variables• Now: x2 L iff Cn accepts some y (with x fixed)

iff Cn (with fixed x) is in CircuitSAT• Karp reduction to CircuitSAT !

NP-complete

• Theorem: CircuitSAT is NP-complete

• Left to show: CircuitSAT2 NP• This is easy to see: Guess an input x, simulate

the circuit C on a TM to check that C(x)=1 – exercise (circuit given as adjacency list)

Simplifying CircuitSAT

• A literal is a variable xi or its negation

• A clause is a term of the form (l1Ç l2Ç… lk)• A formula in conjunctive normal form (CNF) is a

AND of m clauses (of any length) on n variables• Special case of circuits• CNF-SAT: set of all CNF formulae that are

satisfiable (i.e., there is a setting of the variables that makes the formula accept)

CNF-SAT

• Theorem:– CNF-SAT is NP-complete

• Proof: Easy to see that CNF-SAT is in NP• Hardness: reduction from CircuitSAT• Let C be a circuit. We need to find a CNF

formula F that is satisfiable iff C is satisfiable– Also F cannot be too long

The reduction

• Assume that C has s gates and n variables x1,...,xn

• Assume that all negations are at the sources of C (why can we do this? De Morgan’s laws)

• Then F will have n+s variables x1,…,xn and g1,…,gs

• The CNF F is a conjunction (AND) of clauses• There are 3 clauses for every gate of C• Idea: gi guesses the value of gate i

The clauses

• Gate i is the AND of gates j and k:• We want to express that gi = gjÆgk

• Clauses: (gi Ç :gj Ç :gk) AND (:giÇ gj) AND (:gi Ç gj)

• Gate i is the OR of gates j and k:• We want that g i=gjÇgk

• Clauses: (giÇ:gj) AND (gi Ç:gk) AND (:gi Ç gjÇ gk)

• Last clause: (gO), the output gate

Correctness

• We need to show that C is satisfiable iff F is• Assume C is satisfied by x• Then the correct values of the gates make C accept• Then these values (for the g-variables) together with x

satisfy F• Conversely, given the values of x and g variables that

satisfy F, x must satisfy C

• Hence we have a Karp reduction (clearly F is of linear size in C)

Note

• Since in CircuitSAT all gates have in-degree 2 all clauses concern only 3 gates

• We have shown NP-completeness of 3-SAT:– a k-CNF is a CNF in which all clauses have 3 literals (or less)– k-SAT={set of all k-CNF that are satisfiable}

• Notes: Not hard to give a reduction from CNF-SAT to 3-SAT

• 2-SAT is in P (and hence not NP-complete unless P=NP)• 3-SAT is the basis of our reductions from now on

Reductions

• We will show reductions from 3-SAT to a few problems

• Due to transitivity we can use reductions from any of the problems for which we already have a reduction from 3-SAT– Often this is easier than directly reducing from 3-

SAT