average-case complexity luca trevisan uc berkeley

Average-case Complexity

Luca TrevisanUC Berkeley

Distributional Problem

<P,D>

P computational problem– e.g. SAT

D distribution over inputs– e.g. n vars 10n clauses

Positive Results:• Algorithm that solves P efficiently on most

inputs– Interesting when P useful problem, D

distribution arising “in practice”

Negative Results:• If <assumption>, then no such algorithm– P useful, D natural• guide algorithm design

– Manufactured P,D, • still interesting for crypto, derandomization

Holy Grail

If there is algorithm A that solves P efficiently on most inputs from D

Then there is an efficient worst-case algorithm for [the complexity class] P [belongs to]

Part (1)

In which the Holy Grail proves elusive

The Permanent

Perm (M) := Ss Pi M(i,s(i))

Perm() is #P-complete

Lipton (1990): If there is algorithm that solves Perm()

efficiently on most random matrices, Then there is an algorithm that solves it

efficiently on all matrices (and BPP=#P)

Lipton’s Reduction

Suppose operations are over finite field of size >n

A is good-on-average algorithm (wrong on < 1/(10(n+1)) fraction of matrices)

Given M, pick random X, compute A(M+X), A(M+2X),…,A(M+(n+1)X)

Whp the same as Perm(M+X),Perm(M+2X),…,Perm(M+(n+1)X)

Lipton’s Reduction

Given Perm(M+X),Perm(M+2X),…,Perm(M+

(n+1)X)

Find univariate degree-n polynomial p such thatp(t) = Perm(M+tX) for all t

Output p(0)

Improvements / Generalizations

• Can handle constant fraction of errors[Gemmel-Sudan]

• Works for PSPACE-complete, EXP-complete,…[Feigenbaum-Fortnow, Babai-Fortnow-Nisan-Wigderson]Encode the problem as a polynomial

Strong Average-Case Hardness

• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that– Size-t circuit correct on ½ + 1/t inputs implies– Size poly(t) circuit correct on all inputs

Motivation:[Nisan-Wigderson]

P=BPP if there is problem in E of exponential average-case complexity


• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that– Size-t circuit correct on ½ + 1/t inputs

implies– Size poly(t) circuit correct on all inputs

Motivation:[Impagliazzo-Wigderson]

P=BPP if there is problem in E of exponential average worst-case complexity

Open Question 1

• Suppose there are worst-case intractable problems in NP

• Are there average-case intractable problems?


• [Impagliazzo, Impagliazzo-Wigderson] Manufacture problems in E, EXP, such that– Size-t circuit correct on ½ + 1/t inputs implies– Size poly(t) circuit correct on all inputs

• [Sudan-T-Vadhan]– IW result can be seen as coding-theoretic

– Simpler proof by explicitly coding-theoretic ideas

Encoding Approach

• Viola proves that an error-correcting code cannot be computed in AC0

• The exponential-size error-correcting code computation not possible in PH

Problem-specific Approaches?

[Ajtai]

• Proves that there is a lattice problem such that:

– If there is efficient average-case algorithm

– There is efficient worst-case approximation algorithm

Ajtai’s Reduction

• Lattice Problem– If there is efficient average-case algorithm– There is efficient worst-case approximation

algorithm

The approximation problem is in NPIcoNPNot NP-hard

Holy Grail

• Distributional Problem:– If there is efficient average-case algorithm– P=NP

(or NP in BPP, or NP has poly-size circuits,…)

Already seen: no “encoding” approach works

Can extensions of Ajtai’s approach work?

A Class of Approaches

• L problem in NP, D distribution of inputs• R reduction of SAT to <L,D>:

• Given instance f of SAT,– R produces instances x1,…,xk of L, each distributed

according to D– Given L(x1),…,L(x1), R is able to decide f

If there is good-on-average algorithn for <L,D>, we solve SAT in polynomial time

[cf. Lipton’s work on Permanent]


• L,W problems in NP, D (samplable) distribution of inputs

• R reduction of W to <L,D>

• Given instance w of W,– R produces instances x1,…,xk of L, each distributed

according to D– Given L(x1),…,L(x1), R is able to decide w

If there is good-on-average algorithm for <L,D>, we solve W in polynomial time;

Can W be NP-complete?


• Given instance w of W,– R produces instances x1,…,xk of L, each

distributed according to D– Given L(x1),…,L(x1), R is able to decide w

Given good-on-average algorithm for <L,D>, we solve W in polynomial time;

If we have such reduction, and W is NP-complete, we have Holy Grail!

Feigenbaum-Fortnow: W is in “coNP”

Feigenbaum-Fortnow

• Given instance w of W,– R produces instances x1,…,xk of L, each

distributed according to D– Given L(x1),…,L(x1), R is able to decide w

• Using R, Feigenbaum-Fortnow design a 2-round interactive proof with advice for coW

• Given w, Prover convinces Verifier that R rejects w after seeing L(x1),…,L(x1)

Feigenbaum-Fortnow

• Given instance w of W,– R produces instances x of L distributed as in D– w in L iff x in L

Suppose we know PrD[ x in L]= ½

VP

w

R(w) = x1

R(w) = x2

. . .R(w) = xm

x1, x2,. . . , xm

(Yes,w1),No,. . . , (Yes, wm)

Accept iff all simulations of R rejectand m/2 +/- sqrt(m) answers are certified Yes

Feigenbaum-Fortnow

• Given instance w of W, p:= Pr[ xi in L]

– R produces instances x1,…,xk of L, each distrib. according to D

– Given L(x1),…,L(xk), R is able to decide wV

wR(w) -> x1

1,…,xk

1 . . .

R(w) -> x1m,

…,xkm

P

x11,…,xk

m

(Yes,w11),…,NO

Accept iff -pkm +/- sqrt(pkm) YES with certificates-R rejects in each case

Generalizations

• Bogdanov-Trevisan: arbitrary non-adaptive reductions

• Main Open Question:What happens with adaptive reductions?

Open Question 1

Prove the following:

Suppose: W,L are in NP, D is samplable distribution,

R is poly-time reduction such that– If A solves <L,D> on 1-1/poly(n) frac of

inputs– Then R with oracle A solves W on all inputs

Then W is in “coNP”

By the Way

• Probably impossible by current techniques:If NP not contained in BPPThere is a samplable distribution D and an NP

problem L Such that <L,D> is hard on average

By the Way

• Probably impossible by current techniques:If NP not contained in BPPThere is a samplable distribution D and an NP

problem L Such that for every efficient A A makes many mistakes solving L on D

By the Way

• Probably impossible by current techniques:If NP not contained in BPPThere is a samplable distribution D and an NP problem

L Such that for every efficient A A makes many mistakes solving L on D

• [Guttfreund-Shaltiel-TaShma] Prove:If NP not contained in BPPFor every efficient A There is a samplable distribution D Such that A makes many mistakes solving SAT on D

Part (2)

In which we amplify average-case complexity and we discuss a short paper

Revised Goal

• Proving“If NP contains worst-case intractable problems, then NP contains average-case intractable problems”

Might be impossible

• Average-case intractability comes in different quantitative degrees

• Equivalence?

Average-Case Hardness

What does it mean for <L,D> to be hard-on-average?

Suppose A is efficient algorithm Sample x ~ DThen A(x) is noticeably likely to be wrong

How noticeably?

Average-Case Hardness Amplification

Ideally:

• If there is <L,Uniform>, L in NP, such that every poly-time algorithm (poly-size circuit) makes > 1/poly(n) mistakes

• Then there is <L’,Uniform>, L’ in NP, such that every poly-time algorithm (poly-size circuit) makes > ½ - 1/poly(n) mistakes

Amplification

“Classical” approach: Yao’s XOR Lemma

Suppose: for every efficient APrD [ A(x) = L(x) ] < 1- d

Then: for every efficient A’ PrD [ A’(x1,…,xk) = L(x1) xor … xor L(xk) ]

< ½ + (1 - 2d)k + negligible

Yao’s XOR Lemma


Then: for every efficient A’ PrD [ A’(x1,…,xk) = L(x1) xor … xor L(xk) ]

< ½ + (1 - 2d)k + negligible

Note: computing L(x1) xor … xor L(xk) need not be in NP, even if L is in NP

O’Donnell Approach


Then: for every efficient A’ PrD [ A’(x1,…,xk) = g(L(x1), …, L(xk)) ]

< ½ + small(k, d)

For carefully chosen monotone function g

Now computing g(L(x1),…, L(xk)) is in NP, if L is in NP

Amplification (Circuits)

Ideally:• If there is <L,Uniform>, L in NP, such that every poly-time

algorithm (poly-size circuit) makes > 1/poly(n) mistakes• Then there is <L’,Uniform>, L’ in NP, such that every poly-

time algorithm (poly-size circuit) makes > ½ - 1/poly(n) mistakes

Achieved by [O’Donnell, Healy-Vadhan-Viola] for poly-size circuits

Amplification (Algorithms)

• If there is <L,Uniform>, L in NP, such that every poly-time algorithm makes > 1/poly(n) mistakes

• Then there is <L’,Uniform>, L’ in NP, such that every poly-time algorithm makes > ½ - 1/polylog(n) mistakes

[T]

[Impagliazzo-Jaiswal-Kabanets-Wigderson] ½ - 1/poly(n) but for PNP||

Open Question 2

Prove:

• If there is <L,Uniform>, L in NP, such that every poly-time algorithm makes > 1/poly(n) mistakes

• Then there is <L’,Uniform>, L’ in NP, such that every poly-time algorithm makes > ½ - 1/poly(n) mistakes

Completeness

• Suppose we believe there is L in NP, D distribution, such that <L,D> is hard

• Can we point to a specific problem C such that <C,Uniform> is also hard?

Completeness

• Suppose we believe there is L in NP, D distribution, such that <L,D> is hard


Must put restriction on D, otherwise assumption is the same as P != NP

Side Note

Let K be distribution such that x has probability proportional to 2-K(x)

Suppose A solves <L,K> on 1-1/poly(n) fraction of inputs of length n

Then A solves L on all but finitely many inputs

Exercise: prove it

Completeness

• Suppose we believe there is L in NP, D samplable distribution, such that <L,D> is hard


Completeness

• Suppose we believe there is L in NP, D samplable distribution, such that <L,D> is hard


Yes we can! [Levin, Impagliazzo-Levin]

Levin’s Completeness Result

• There is an NP problem C, such that

• If there is L in NP, D computable distribution, such that <L,D> is hard

• Then <C,Uniform> is also hard

Reduction

Need to define reduction that preserves efficiency on average

(Note: we haven’t yet defined efficiency on average)

R is a (Karp) average-case reduction from <A,DA> to <B,DB> if

1. x in A iff R(x) in B2. R(DA) is “dominated” by DB:

Pr[ R(DA)=y] < poly(n) * Pr [DB = y]

Reduction

R is an average-case reduction from <A, DA> to <B, DB> if

• x in A iff R(x) in B• R(DA) is “dominated” by DB:

Pr[ R(DA)=y] < poly(n) * Pr [DB = y]

Suppose we have good algorithm for <B, DB>

Then algorithm also good for <B,R(DA)>

Solving <A, DA> reduces to solving <B,R(DA)>

Reduction

If Pr[ Y=y] < poly(n) * Pr [DB = y]

and we have good algorithm for <B, DB >

Then algorithm also good for <B,Y>

Reduction works for any notion of average-case tractability for which above is true.


Follow presentation of [Goldreich]

• If <BH,Uniform> is easy on average

• Then for every L in NP, every D computable distribution, <L,D> is easy on average

BH is non-deterministic Bounded Halting: given <M,x,1t>,does M(x) accept with t steps?


BH, non-deterministic Bounded Halting: given <M,x,1t>,does M(x) accept with t steps?

Suppose we have good-on-average alg A

Want to solve <L,D>, where L solvable by NDTM M

First try: x -> <M,x, 1poly(n)>


First try: x -> <M,x, 1poly(n)>

Doesn’t work: x may have arbitrary distribution, we need target string to be nearly uniform (high entropy)

Second try: x -> <M’,C(x), 1poly(n)>Where C() is near-optimal compression alg,

M’ recover x from C(x), then runs M


Second try: x -> <M’,C(x), 1poly(n)>Where C() is near-optimal compression alg,

M’ recover x from C(x), then runs M

Works! Provided C(x) has length at mostO(log n) + log 1/PrD[x]

Possible if cumulative distribution function of D is computable.

Impagliazzo-Levin

Do the same but for all samplable distribution

Samplable distribution not necessarily efficiently compressible in coding theory sense. (E.g. output of PRG)

Hashing provides “non-constructive” compression

Complete Problems

BH with Uniform distribution

Tiling problem with Uniform distribution [Levin]

Generalized edge-coloring [Venkatesan-Levin] Matrix representability [Venkatesan-

Rajagopalan]Matrix transformation [Gurevich]. . .

Open Question 3

L in NP, M NDTM for L is specified by k bits

Levin’s reduction incurs 2k bits in fraction of “problematic” inputs(comparable to having 2k slowdown)

Limited to problems having non-deterministic algorithm of 5 bytes

Inherent?

More Reductions?

Still relatively few complete problems

Similar to study of inapproximability before Papadimitriou-Yannakakis and PCP

Would be good, as in Papadimitriou-Yannakakis, to find reductions between problems that are not known to be complete but are plausibly hard

Open Question 4

(Heard from Russell Impagliazzo)

Prove that

If 3SAT is hard on instances with n variables and 10n clauses,

Then it is also hard on instances with 12n clauses

See

• http://www.cs.berkeley.edu/~luca/average[slides, references, addendum to Bogdanov-T, coming soon]

• http://www.cs.uml.edu/~wang/acc-forum/ [average-case complexity forum]

• Impagliazzo A personal view of average-case complexityStructures’95

• Goldreich Notes on Levin’s theory of average-case complexityECCC TR-97-56

• Bogdanov-T. Average case complexityF&TTCS 2(1): (2006)

http://www.cs.berkeley.edu/~luca/average

http://www.cs.uml.edu/~wang/acc-forum/

average-case complexity luca trevisan uc berkeley

Documents