Complexity Theory

Lecture 5

Lecturer: Moni Naor

Recap

Last week: Probabilistic Space and Time Complexity• Undirected Connectivity is in randomized logspace

This week:• Probabilistic Complexity

1. Schwatz-Zippel2. Approximating Counting Problems

Plus• Alternation

Famous Markov Chain: PageRank algorithm [Brin and Page 98]

• Good authorities should be pointed by good authorities

• Random walk on the web graph– pick a page at random– with probability α follow a random outgoing link– with probability 1- α jump to a random page

• Rank according to the stationary distribution

nqF

qPRpPR

pq

11

)(

)()(

Hot off the press news• Omer Reingold, Undirected ST-Connectivity in

Log-Space,Available: Electronic Colloquium on Computational

Complexity, Report TR04-094

Important Web Resources on Complexity:• ECCC: http://www.eccc.uni-trier.de• Lance Fortnow’s Computational Complexity Web

Log: http://fortnow.com/lance/complog/

Probabilistic Variants of P

RP: one-sided Error• For all x 2 L we have Pr[M stops with `yes’]>1/2• For all x L we have Pr[M stops with `no’]=1

BPP: two-sided Error• For all x 2 L we have Pr[M stops with `yes’]>2/3• For all x L we have Pr[M stops with `no’]>2/3

ZPP: No Error but stopping time is only expected polynomial time

Allows Amplication (error reduction)

Allows Amplification as long as difference is at

least 1/p(n)

The Schwartz-Zippel Algorithm/Theorem

Theorem: Let Q(x1, x2, …, xn) 2 F(x1, x2, …, xn) be a non-zero multivariate polynomial of total degree d. Fix any finite set S µ F and choose r1, r2, …, rn 2R S. Then

Pr[Q(r1, r2, …, rn)=0] · d/|S|

Proof: by induction on n

Useful when Q is not given explicitly and want to test equality to 0Determinant of a matrix

Matching in Bipartite Graphs

• Let G=(V,U,E) be a bipartite graph, |V|=|U|=n Define A to be the n £ n matrix with |E| variables

Aij= xij if (i,j) 2 E and 0 otherwise

Theorem (Edmonds): G has a perfect matching iff det(A)≠0

Proof:det(A)= 2 Sn

sgn() A1,(1) A1,(2)… A1,(n) •If there is no matching always zero

•If there is a matching cannot be canceled by another term

Algorithm for deciding whether a matching exists in bipartite graphs

• Fix a prime P larger than 2n• Choose {rij}(i,j) 2 E 2R GF[P]• Compute det(A({rij}(i,j)))• If non-zero declare matching. Ow, no matching

• In general computing det(A) more expensive than running combinatorial algorithm for matching

• Not true in parallel computation

Branching ProgramsA Branching Program (BP) for on n Boolean variables x1,

x2, … xn function is a DAG with outdegree 0 or 2. – One source– Each sink is labeled in {0,1}– Each internal node is labeled with a variable xi and the two

outgoing edge are labeled with 0 and 1Given an assignment: computation proceeds by traversing the

resulting path to a sinkExample: decision trees are all BPs

Question: Which functions have polynomial sized BPs?

Observation: If a function is in LogSpace, then it has a polynomial sized BP

Types of Branching Programs

A read-once BP (ROBP) is one where on every path from source to sink at no variable appears more than once

• Equivalence problem for BPs:Given two branching programs, do they compute the

same function?Homework: for general BPs the equivalence problem

is Co-NP CompleteTheorem: for read once BPs the equivalence problem

is in Co-RP

Arithmetization of BPs

• For a given BP B define a multivariable polynomial PB over the rationals– Each Boolean variable xi – variable Xi

– For each path from source to sink labeled 1 add a monomial with variables on the path

• For occurrence of xi put Xi and of :xi put (1-Xi)

Claim: for any assignment a 2 {0,1}n we have that B(a) = PB(a)Since each assignment makes only one path non-zero

Claim: for any rational assignment q to X1 ,X2 ,…, Xn

possible to evaluate PB(q) in polynomial time

Computing PB

To evaluate Pb at point q=(q1, q2, …, qn):• Assign source value 1• If a node labeled with xi has already been assigned

with value v, assign outgoing edges – Edge labeled with 1 gets assigned v qi

– Edge labeled with 0 gets assigned v (1-qi)

• If all the incoming edges of a node have been assigned: node is assigned their sum

An algorithm for the equivalence of read-once BPs

• Input B1 and B2. Output are they equivalent• Pick a subset of 2n values• Choose random assignment q to variables• Check whether PB1

(q) = PB2(q)

Key point: for ROBP PB1(X) ≡ PB2

(X) iff B1 and B2 are equivalentProof:

• Since PB1(X) and PB2

(X) are multilinear polynomials the Schwartz Zippel Theorem says they will agree with probability at most 1/2

Methods for separating LogSpace from P (and beyond)

Prove lower bounds on branching programs• If for any problem in P you prove a super

polynomial lower bound of the smallest BP for it, then LogSpace ( P

• If for any problem in P you prove that a short BP implies a lower bound on the size – a time-space tradeoff on almost any imaginable machine

•BP are the non-uniform variant of log-space

•There are lower bounds separating read-once from read-twice BP

Probabilistic Approximation

• Natural usage of randomization: approximating number of solutions:– Pick a random assignment and see what happens

• Can define class PP:PP: unbounded two-sided Error• For all x 2 L we have Pr[M stops with `yes’]>1/2• For all x L we have Pr[M stops with `no’]>1/2Prove: NP [ Co-NP ½ PP

Counting Problems

• A counting problem: given an instance x of a decision problem how many witness does it have?

• Examples– given a formula how many satisfying assignment does it

have• CNF• DNF

– Given a graph:• How many spanning trees does it have• How many perfect matchings • How many Hamiltonian Cycles

#P• A function f is in #P if there exists a NTM M

– running in polynomial time – M(x) has f(x) accepting paths for all x2 {0,1}n

• All above problems are in #P#PClaim: NP-Hard to compute all functions in #P#P

Claim: for all f 2 FP we have f 2 #P#P

Claim: if FP=#PFP=#P then P=NP

Approximation Schemes for #P

Polynomial Randomized approximation scheme (PRAS) for a function f in #P:

A probabilistic machine A that get x and and • in time polynomial time in |x| returns an answer• Pr[(1- )f(x) · A(x, ) · (1+ ) f(x)] ¸ 2/3

Fully Polynomial Randomized approximation scheme (FPRAS):• in time polynomial time in |x| and 1/

Homework: if a function f in has an fpras then there is an algorithm that in addition to x and get and – returns a correct approximation with probability at least 1- – in time polynomial time in |x|, 1/ and log 1/

Approximating satisfying assignments to DNFs

• Given a DNF formula F=D1Ç D2 Ç …, Dm count how many satisfying assignments

• First attempt: For ℓ times– generate an assignment at random and test whether satisfies F – Let Yi =1 if satisfies and 0 otherwise

Output Z = 2n 1/ ℓ i=1ℓ Yi

• Claim E[Z]=#F• Claim: if = #F/2n and ℓ ¸ 4/(2 ) log 2/ then

approximation is good with probability 1-

Not good enough when is small

Approximating DNFs• Good news: for each Di easy to compute # of sat

assignments• Second attempt:

– Sum up #Di

• What to do about overlapping assignments?– Idea: assign each satisfying assignment to the lexicographically

first clause it satisfies– Lost the ability to compute # of sat assignments exactly

• Instead: approximate for each Di number of lex first satisfying assignments by– generating random satisfying assignment and testing for lex firstEffectively: obtained a problem with ¸ 1/m

CircuitsLet B be a collection of Booleans functionsA Circuit for on n Boolean variables x1, x2, … xn over Basis B is a DAG with

– Each source is labeled with a literal– Unique sink– Each internal node is labeled with a function from B and has the appropriate indegree

• A circuit compute a function in the natural way• Which (families of) functions have circuits whose size is bounded by polynomial in the

input size?– B ={:, Ç, Æ}

Claim: all f 2 P have polynomial size circuitsProof: via the `usual’ tableau. Size of circuit T(n)S(n).

Using Oblivious Turing Machines T(n) log T(n)

Claim: most Boolean function f on n variables do not have polynomial sized circuitsFor any polynomial p(n) most functions need more p(n) gates

Need (2n/n) gates to compute all functions on n variables, which is tight

Can determine the inputs to a given gate by a log space machine

Polynomial Sized Circuits for BPP• Theorem: any f 2 BPP has a polynomial size circuit

– There exists a sequence of circuits C1, C2, … and a polynomial p(n) such that:

• circuit Cn computes f on inputs of size n • circuit Cn is of size at most p(n)

Several proofs:• Construction of hitting set (for f 2 RP)

– Claim that a sequence r1, r2, … rn exists where for each x 2 L at least one ri says `yes’

• Simulating large sample spaces by small ones• Amplification

– Reduce the error so that a single assignment will be good for all x

Polynomial Sized Circuits for BPPTheorem: any f 2 BPP has a polynomial size

circuitSimulating large sample spaces• Want to find a small collection of strings on

which the PTM behaves as on the large collection – If the PTM errs with probability at most , then

should err on at most + of the small collection• Choose m random strings• For input x event Ax is more than (+) of

the m strings fail the PTM

Pr[Ax] · e-22m < 2-2n

Pr[[x Ax] · x Pr[Ax] < 2n 2-2n=1

Good 1-

Bad

Collection that should resemble probability of success on ALL inputs

Chernoff

Alternation

• Non-determinism: a configuration leads to acceptance if there is an accepting leaf in the computation – Similar to Or

• Can also consider leads to acceptance if all leaves are accepting– Similar to And

• What if we alternate between the modes

Alternating Turing Machines

An Alternating Turing Machines (ATM) is a non-deterministic TM whose state S ={SAND [ SOR} are partitioned into two sets

• For input x consider the tree of computations where each node is a configuration

• The ATM is accepting if the root leads to an accepting configuration:– A leaf is accepting or not– A node in state s 2 SAND leads to acceptance if both its children

leads to acceptance– A node in state s 2 SOR leads to acceptance if one of its children

leads to acceptance

Alternating Time ClassesATIME(f(n))= class of language decided by an ATM where:• All computations halt after at most f(|x|) stepsASPACE(f(n))= class of language decided by an ATM where:• All computation halt • Never use more than f(|x|) cells

AL AL = ASPACE(log n)

Theorem: AL = PAL = PTheorem: by simulating circuitImportant point:Important point: if f 2 PP then the circuit is constructable in log space• Simulate the circuit from the output towards the inputs