cs151 complexity theory lecture 5 april 13, 2015
TRANSCRIPT
![Page 1: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/1.jpg)
CS151Complexity Theory
Lecture 5
April 13, 2015
![Page 2: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/2.jpg)
April 13, 2015
Introduction
Power from an unexpected source?
• we know P ≠ EXP, which implies no poly-time algorithm for Succinct CVAL
• poly-size Boolean circuits for Succinct CVAL ??
Does NP have linear-size, log-depth Boolean circuits ??
2
![Page 3: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/3.jpg)
April 13, 2015
Outline
• Boolean circuits and formulas
• uniformity and advice
• the NC hierarchy and parallel computation
• the quest for circuit lower bounds
• a lower bound for formulas
3
![Page 4: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/4.jpg)
April 13, 2015
Boolean circuits
• C computes function f:{0,1}n {0,1} in natural way – identify C with function f it computes
• circuit C– directed acyclic graph
– nodes: AND (); OR (); NOT (); variables xi
x1 x2
x3 … xn
4
![Page 5: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/5.jpg)
April 13, 2015
Boolean circuits
• size = # gates
• depth = longest path from input to output
• formula (or expression): graph is a tree
• every function f:{0,1}n {0,1} computable by a circuit of size at most O(n2n)
– AND of n literals for each x such that f(x) = 1– OR of up to 2n such terms
5
![Page 6: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/6.jpg)
April 13, 2015
Circuit families
• circuit family : a circuit for each input length C1, C2, C3, … = “{Cn}”
• “{Cn} computes f” iff for all x
C|x|(x) = f(x)
• Definition: circuit family {Cn} is logspace uniform iff TM M outputs Cn on input 1n and runs in O(log n) space
6
![Page 7: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/7.jpg)
April 13, 2015
TMs that take advice
• family {Cn} without uniformity constraint is called “non-uniform”
• regard “non-uniformity” as a limited resource just like time, space, as follows:– add read-only “advice” tape to TM M– M “decides L with advice A(n)” iff
M(x, A(|x|)) accepts x L– note: A(n) depends only on |x|
7
![Page 8: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/8.jpg)
April 13, 2015
TMs that take advice
• Definition: TIME(t(n))/f(n) = the set of those languages L for which:
– there exists A(n) s.t. |A(n)| ≤ f(n)
– TM M decides L with advice A(n) in time t(n)
• most important such class:
P/poly = k TIME(nk)/nk
8
![Page 9: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/9.jpg)
April 13, 2015
Uniformity
Theorem: P = languages decidable by logspace uniform, polynomial-size circuit families {Cn}.
• Proof:– already saw ()
– () on input x, generate C|x|, evaluate it and accept iff output = 1
9
![Page 10: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/10.jpg)
April 13, 2015
TMs that take advice
Theorem: L P/poly iff L decided by family of (non-uniform) polynomial size circuits.
• Proof:– () Cn from CVAL construction; hardwire
advice A(n)
– () define A(n) = description of Cn; on input x, TM simulates C|x|(x)
10
![Page 11: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/11.jpg)
April 13, 2015
Approach to P/NP
• Believe NP P– equivalent: “NP does not have uniform,
polynomial-size circuits”
• Even believe NP P/poly– equivalent: “NP (or, e.g. SAT) does not have
polynomial-size circuits”– implies P ≠ NP– many believe: best hope for P ≠ NP
11
![Page 12: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/12.jpg)
April 13, 2015
Parallelism
• uniform circuits allow refinement of polynomial time:
circuit C
depth parallel time
size parallel work
12
![Page 13: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/13.jpg)
April 13, 2015
Parallelism
• the NC (“Nick’s Class”) Hierarchy (of logspace uniform circuits):
NCk = O(logk n) depth, poly(n) size
NC = k NCk
• captures “efficiently parallelizable problems”
• not realistic? overly generous
• OK for proving non-parallelizable13
![Page 14: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/14.jpg)
April 13, 2015
Matrix Multiplication
• what is the parallel complexity of this problem?– work = poly(n) – time = logk(n)? (which k?)
n x n matrix A
n x n matrix B =
n x n matrix AB
14
![Page 15: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/15.jpg)
April 13, 2015
Matrix Multiplication
• two details– arithmetic matrix multiplication…
A = (ai, k) B = (bk, j) (AB)i,j = Σk (ai,k x bk, j)
… vs. Boolean matrix multiplication:
A = (ai, k) B = (bk, j) (AB)i,j = k (ai,k bk, j)
– single output bit: to make matrix multiplication a language: on input A, B, (i, j) output (AB)i,j
15
![Page 16: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/16.jpg)
April 13, 2015
Matrix Multiplication
• Boolean Matrix Multiplication is in NC1
– level 1: compute n ANDS: ai,k bk, j
– next log n levels: tree of ORS
– n2 subtrees for all pairs (i, j)– select correct one and output
16
![Page 17: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/17.jpg)
April 13, 2015
Boolean formulas and NC1
• Previous circuit is actually a formula. This is no accident:
Theorem: L NC1 iff decidable by polynomial-size uniform family of Boolean formulas.
Note: we measure formula size by leaf-size.17
![Page 18: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/18.jpg)
April 13, 2015
Boolean formulas and NC1
• Proof: – () convert NC1 circuit into formula
• recursively:
• note: logspace transformation (stack depth log n, stack record 1 bit – “left” or “right”)
18
![Page 19: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/19.jpg)
April 13, 2015
Boolean formulas and NC1
– () convert formula of size n into formula of depth O(log n) • note: size ≤ 2depth, so new formula has
poly(n) size
D
C C1
1
C0
0
D
D
key transformation
19
![Page 20: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/20.jpg)
April 13, 2015
Boolean formulas and NC1
– D any minimal subtree with size at least n/3 • implies size(D) ≤ 2n/3
– define T(n) = maximum depth required for any size n formula
– C1, C0, D all size ≤ 2n/3
T(n) ≤ T(2n/3) + 3
implies T(n) ≤ O(log n)
20
![Page 21: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/21.jpg)
April 13, 2015
Relation to other classes
• Clearly NCµ P– recall P uniform poly-size circuits
• NC1 µ L
– on input x, compose logspace algorithms for:• generating C|x|
• converting to formula• FVAL
21
![Page 22: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/22.jpg)
April 13, 2015
Relation to other classes
• NL µ NC2: S-T-CONN NC2
– given G = (V, E), vertices s, t– A = adjacency matrix (with self-loops)– (A2)i, j = 1 iff path of length ≤ 2 from node i to
node j– (An)i, j = 1 iff path of length ≤ n from node i to
node j– compute with depth log n tree of Boolean
matrix multiplications, output entry s, t – log2 n depth total
22
![Page 23: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/23.jpg)
April 13, 2015
NC vs. P
• can every efficient algorithm be efficiently parallelized?
NC = P• P-complete problems least-likely to be
parallelizable– if P-complete problem is in NC, then P = NC – Why?
we use logspace reductions to show problem P-complete; L in NC
?
23
![Page 24: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/24.jpg)
April 13, 2015
NC vs. P
• can every uniform, poly-size Boolean circuit family be converted into a uniform, poly-size Boolean formula family?
NC1 = P?
24
![Page 25: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/25.jpg)
April 13, 2015
NC Hierarchy Collapse
NC1 µ NC2 µ NC3 µ NC4 µ … µ NC
Exercise
if NCi = NCi+1, then NC = NCi
(prove for non-uniform versions of classes)
25
![Page 26: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/26.jpg)
April 13, 2015
Lower bounds
• Recall: “NP does not have polynomial-size circuits” (NP P/poly) implies P ≠ NP
• major goal: prove lower bounds on (non-uniform) circuit size for problems in NP– believe exponential – super-polynomial enough for P ≠ NP – best bound known: 4.5n– don’t even have super-polynomial bounds for
problems in NEXP26
![Page 27: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/27.jpg)
April 13, 2015
Lower bounds
• lots of work on lower bounds for restricted classes of circuits
– we’ll see two such lower bounds: • formulas • monotone circuits
27
![Page 28: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/28.jpg)
April 13, 2015
Shannon’s counting argument
• frustrating fact: almost all functions require huge circuits
Theorem (Shannon): With probability at least 1 – o(1), a random function
f:{0,1}n {0,1}
requires a circuit of size Ω(2n/n).
28
![Page 29: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/29.jpg)
April 13, 2015
Shannon’s counting argument
• Proof (counting):– B(n) = 22n
= # functions f:{0,1}n {0,1} – # circuits with n inputs + size s, is at most
C(n, s) ≤ ((n+3)s2)s
s gates
n+3 gate types 2 inputs per gate
29
![Page 30: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/30.jpg)
April 13, 2015
Shannon’s counting argument
– C(n, c2n/n) < ((2n)c222n/n2)(c2n/n)
< o(1)22c2n
< o(1)22n (if c ≤ ½)
– probability a random function has a circuit of size s = (½)2n/n is at most
C(n, s)/B(n) < o(1)
C(n, s) ≤ ((n+3)s2)s
30
![Page 31: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/31.jpg)
April 13, 2015
Shannon’s counting argument
• frustrating fact: almost all functions require huge formulas
Theorem (Shannon): With probability at least 1 – o(1), a random function
f:{0,1}n {0,1}
requires a formula of size Ω(2n/log n).
31
![Page 32: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/32.jpg)
April 13, 2015
Shannon’s counting argument
• Proof (counting):– B(n) = 22n
= # functions f:{0,1}n {0,1} – # formulas with n inputs + size s, is at most
F(n, s) ≤ 4s2s(2n)s
4s binary trees with s internal nodes 2 gate choices
per internal node
2n choices per leaf
32
![Page 33: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/33.jpg)
April 13, 2015
Shannon’s counting argument
– F(n, c2n/log n) < (16n)(c2n/log n)
< 16(c2n/log n)2(c2n) = (1 + o(1))2(c2n)
< o(1)22n (if c ≤ ½)
– probability a random function has a formula of size s = (½)2n/log n is at most F(n, s)/B(n) < o(1)
F(n, s) ≤ 4s2s(2n)s
33
![Page 34: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/34.jpg)
April 13, 2015
Andreev function
• best formula lower bound for language in NP:
Theorem (Andreev, Hastad ‘93): the Andreev function requires (,,)-formulas of size at least
Ω(n3-o(1)).
34
![Page 35: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/35.jpg)
April 13, 2015
Andreev function
selector
yi
n-bit string yXOR XOR
. . .
log n copies; n/log n bits each
the Andreev function A(x,y) A:{0,1}2n
{0,1}35
![Page 36: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/36.jpg)
April 13, 2015
Random restrictions
• key idea: given function f:{0,1}n {0,1}
restrict by ρ to get fρ
– ρ sets some variables to 0/1, others remain free
• R(n, єn) = set of restrictions that leave єn variables free
• Definition: L(f) = smallest (,,) formula computing f (measured as leaf-size)
36
![Page 37: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/37.jpg)
April 13, 2015
Random restrictions
• observation:
EρR(n, єn)[L(fρ)] ≤ єL(f)– each leaf survives with probability є
• may shrink more…– propogate constants
Lemma (Hastad 93): for all f
EρR(n, єn)[L(fρ)] ≤ O(є2-o(1)L(f))
37
![Page 38: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/38.jpg)
April 13, 2015
Hastad’s shrinkage result
• Proof of theorem:– Recall: there exists a function
h:{0,1}log n {0,1} for which L(h) > n/2loglog n.
– hardwire truth table of that function into y to get A*(x)
– apply random restriction from R(n, m = 2(log n)(ln log n))
to A*(x).
38
![Page 39: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/39.jpg)
April 13, 2015
The lower bound
• Proof of theorem (continued):– probability given XOR is killed by restriction is
probability that we “miss it” m times:
(1 – (n/log n)/n)m ≤ (1 – 1/log n)m
≤ (1/e)2ln log n ≤ 1/log2n– probability even one of XORs is killed by
restriction is at most:
log n(1/log2n) = 1/log n < ½.
39
![Page 40: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/40.jpg)
April 13, 2015
The lower bound
– (1): probability even one of XORs is killed by restriction is at most:
log n(1/log2n) = 1/log n < ½.– (2): by Markov:
Pr[ L(A*ρ) > 2 EρR(n, m)[L(A*
ρ)] ] < ½.
– Conclude: for some restriction ρ • all XORs survive, and
• L(A*ρ) ≤ 2 EρR(n, m) [L(A*
ρ)]
40
![Page 41: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/41.jpg)
April 13, 2015
The lower bound
• Proof of theorem (continued):– if all XORs survive, can restrict formula further
to compute hard function h • may need to add ’s
L(h) = n/2loglogn ≤ L(A*ρ)
≤ 2EρR(n, m)[L(A*ρ)] ≤ O((m/n)2-o(1)L(A*))
≤ O( ((log n)(ln log n)/n)2-o(1) L(A*) )
– implies Ω(n3-o(1)) ≤ L(A*) ≤ L(A).
41
![Page 42: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/42.jpg)
April 13, 2015 42
Clique
CLIQUE = { (G, k) | G is a graph with a clique of size ≥ k }
(clique = set of vertices every pair of which are connected by an edge)
• CLIQUE is NP-complete.
![Page 43: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/43.jpg)
April 13, 2015 43
Circuit lower bounds
• We think that NP requires exponential-size circuits.
• Where should we look for a problem to attempt to prove this?
• Intuition: “hardest problems” – i.e., NP-complete problems
![Page 44: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/44.jpg)
April 13, 2015 44
Circuit lower bounds
• Formally: – if any problem in NP requires super-
polynomial size circuits– then every NP-complete problem requires
super-polynomial size circuits
– Proof idea: poly-time reductions can be performed by poly-size circuits using a variant of CVAL construction
![Page 45: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/45.jpg)
April 13, 2015 45
Monotone problems
• Definition: monotone language = language
L {0,1}*
such that x L implies x’ L for all x ¹ x’.
– flipping a bit of the input from 0 to 1 can only change the output from “no” to “yes” (or not at all)
![Page 46: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/46.jpg)
April 13, 2015 46
Monotone problems
• some NP-complete languages are monotone– e.g. CLIQUE (given as adjacency matrix):
– others: HAMILTON CYCLE, SET COVER…– but not SAT, KNAPSACK…
![Page 47: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/47.jpg)
April 13, 2015 47
Monotone circuits
A restricted class of circuits:
• Definition: monotone circuit = circuit whose gates are ANDs (), ORs (), but no NOTs
• can compute exactly the monotone fns.– monotone functions closed under AND, OR
![Page 48: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/48.jpg)
April 13, 2015 48
Monotone circuits
• A question:
Do all
poly-time computable monotone functions
have
poly-size monotone circuits?
– recall: true in non-monotone case
![Page 49: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/49.jpg)
April 13, 2015 49
Monotone circuits
A monotone circuit for CLIQUEn,k
• Input: graph G = (V,E) as adj. matrix, |V|=n– variable xi,j for each possible edge (i,j)
• ISCLIQUE(S) = monotone circuit that = 1
iff S V is a clique: i,j S xi,j
• CLIQUEn,k computed by monotone circuit:
S V, |S| = k ISCLIQUE(S)
![Page 50: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/50.jpg)
April 13, 2015 50
Monotone circuits
• Size of this monotone circuit for CLIQUEn,k:
• when k = n1/4, size is approximately:
n k
k 2
1/ 41/ 4
n 1/ 4n
4
2
1/
nn
2n
n
![Page 51: CS151 Complexity Theory Lecture 5 April 13, 2015](https://reader030.vdocuments.net/reader030/viewer/2022032800/56649d355503460f94a0bac1/html5/thumbnails/51.jpg)
April 13, 2015 51
Monotone circuits
• Theorem (Razborov 85): monotone circuits for CLIQUEn,k with k = n1/4 must have size at least
2Ω(n1/8).
• Proof: – next lecture