1 paul beame university of washington satisfiability and unsatisfiability: proof complexity and...
Post on 20-Dec-2015
216 views
TRANSCRIPT
1
Paul Beame University of Washington
Satisfiability and Unsatisfiability:Proof Complexity and Algorithms
2
Outline
Intro to proof complexityProof complexity and complete SAT algs Review Resolution/DPLL connection Application to random k-SAT Proof complexity and DPLL extensions
Clause learningCaching
Proof system survey
3
Proof Complexity
Study of the number of symbols required for proofs of unsatisfiability (or tautology) in propositional logic
Does not address algorithmic issue How would you find short proofs if they
existed?
Existence of short proofs for every unsatisfiable formula is equivalent to NP = co-NP (and is implied by P=NP) Generally believed that such proofs don’t exist
4
Propositional proof systems
A propositional proof system is a polynomial time algorithm V s.t. for all formulas F
F is unsatisfiable there exists a string P s.t. V accepts input (P,F)Note:
direction is usually called soundness direction is usually called completeness
Proof complexity is size of P as a function of the size of F
5
Sample propositional proof systems
Truth tables proof is a fully filled out truth table
easy to verify that it is filled out correctly and all truth assignments yield T
Frege systems inference rules: e.g. modus ponens
A, (A B) | B axioms: e.g. excluded middle | (A A) axioms & inference rules are schemas
can make consistent substitution of arbitrary formulas for variables in schema
e.g. excluded middle yields ((xy) (xy))
6
The graph of a proof
Axioms/inputs are sources
F7F1
F11
F3
F4
F8
F9 F12
F2
F10
F5 F6
F13
Inference ruleassociated with each node
Sink labelled by tautology (or for refutation)
7
Resolution
Start with clauses of CNF formula F
Resolution rule Given (A x), (B x) derive
(A B)
The empty clause is derivable F is unsatisfiable
Proof size = # of clauses used
8
Restricted forms of Resolution
Resolution In general, graph of inferences is a directed
acyclic graph
Tree Resolution Graph of inferences forms a binary tree
i.e., if you want to use a clause more than once you need to re-derive it
Regular Resolution Once a variable is resolved out of a clause
you can’t add it back later More general than Tree Resolution
9
Satisfiability Algorithms
Incomplete Algorithms Local search
GSAT [Selman,Levesque,Mitchell 92] Walksat [Kautz,Selman 96]
Belief PropagationSP [Braunstein, Mezard, Zecchina 02]
Complete Algorithms Backtracking search
DPLL [Davis,Putnam 60]
[Davis,Logeman,Loveland 62]DPLL + clause learning GRASP, SATO, zchaff, berkmin
etc
10
Proof Complexity and SAT Algorithms
Proof Complexity Unsatisfiable formulas Nondeterministic algorithms
SAT Algorithms Satisfiable formulas Deterministic or randomized
algorithms
11
Proof complexity and complete satisfiability
algorithmsIn general The transcript of the run of a
complete satisfiability algorithm on an unsatisfiable formula is a proof of its unsatisfiability
For backtracking style algorithms Even runs on satisfiable formulas
yield proofs of unsatisfiability of related formulas
12
Backtracking search/DPLL
DPLL(F) while F contains a 1-clause x
F F|x=1
if F is empty return 1 if F has an empty clause return 0
else select a literal x to branch on return DPLL(F|x=1) DPLL(F|x=0)
Residual formula
13
Resolution and DPLL
DPLL tree with any variable selection rule on an unsatisfiable formula F generates a Resolution refutation of F # of clauses running time
Note: Regular resolution corresponds to a similar DAG (read-once branching program) rather than a tree but general resolution has no such analogue
14
DPLL Refutation
Clauses1. a b c2. ab3. c4. a d5. d b
a
b b
c d3 3
21 4 5
a a
b
d
bb
d
b
a b c ac
b b
a d d b
c c
15
DPLL Refutation= Tree-like Resolution Proof
Clauses1. a b c2. ac3. b4. a d5. a b
a:
b: a b: a
d:a b3 3
21 4 5
a b c ac
b b
a d d b
c: a b
16
Random k-CNF formulas
Randomly choose m clauses over n variables independently each of size k Each size k clause is equally likely
Key parameter r=m/n, the ratio of clauses to variables:
Threshold value rk* of rBelow, almost certainly satisfiableAbove, almost certainly unsatisfiableHardest problems near threshold
17
DPLL on random 3-CNF*
0
1
probability
satisfiable
4.267
ratio of clauses to variables
# of DPLLbacktracks
* n = 50 variables
Proof complexity
shows 2(n/r) time is required for unsatisfiable
formulas for r r3*[B,Karp,Saks,Pitassi 98]
[Ben-Sasson 02]
What about satisfiableformulas below threshold?
r[Mitchell,Selman,Levesque 92]
18
Some simple select choices for DPLL algorithms
UC: Unit Clause/Ordered DLL Choose variables in a fixed order Always set True first
UCwm: Unit Clause with majority Choose variables in a fixed order Apply a majority vote among 3-clauses for
assigning each value
GUC: Generalized Unit Clause Choose a variable v in a shortest clause C Set v to satisfy C
19
Parameters of residual formulas follow trajectories
1
2/3
[Chao,Franco 88]
[Frieze,Suen 95]
[Achlioptas 00]
[Achlioptas,Sorkin 00]UCGUC
3.523-clause ratio
2-c
laus
e r
atio
8/3
[Kaporis et al 03]
0
20
Satisfiability for mixed random formulas: proven
properties
1
4.501
SAT
UNSAT
3.522/3
?
??
?
?
?
?
?
?
?
?
?
2.28
3-clause ratio
2-c
lau
se r
ati
o
[Achlioptas et al 96]
[Kaporis et al 03]
[Dubois 01]
21
Resolution proof complexity of mixed random formulas
Theorem: [Achlioptas, B, Molloy 01] For any constants r21 and r3 0 random formulas with r2n 2-clauses and r3n 3-clauses need exponential-size resolution proofs almost surely
Extends [Chvatal-Szemeredi 88]
22
Long-running DPLL Executions and Proof Complexity
Residual formula at is unsatisfiable
Algorithm’sproof of unsatisfiability is exponentially long
Every resolution2n
Residual formula at each node is a mix of2- and 3-clauses
23
Trajectory on 3-CNF
1UC Algorithm Trajectory
2-c
lau
se r
ati
o
4.51
Provably UNSAT& Hard
3.52 4.267
ProvablySAT & Easy
3-clause ratio
3.81
0
Exponential lower bounds for 3-CNF formulas below ratio 4.267
Corollary For almost all 3-CNF formulas, above ratio 3.81 UC takes exponential time 3.83 UCwm takes exponential time 4.01 GUC takes exponential time
25
Exponential lower bound below the proven k-CNF threshold
[Achlioptas, Peres 02] rk* 2k ln 2 – (k+4)/2
[Achlioptas, B, Molloy 04] For k 4, for almost all k-CNF formulas UC takes exponential time above ratio c 2k/k
Note These formulas have huge numbers of satisfying assignments (more than 2 (1-) n out of a possible 2n) but still are hard
26
Open Problem
Closing gap for unsatisfiability of mixed formulas would yield sharp threshold behavior for each algorithm A Below rA algorithm runs in linear time
Above rA algorithm requires exponential time
Conjecture: no polynomial time selection heuristic will work up to the threshold rA r3* for all algorithms A
Backtracking algorithms for other random problems with phase transitions? e.g. k-colorability on random graphs G(n,r/n)
Unsatisfiable phase exp(cn/rk) [B, Culberson, Mitchell, Moore 03]
27
Outline
Intro to proof complexityProof complexity and complete SAT algs Review Resolution/DPLL connection Application to random k-SAT Proof Complexity and DPLL
extensionsClause learningCaching
Proof system survey
28
Clause Learning
At each backtrack point in DPLL search add new clauses to the input formula that correspond to known causes of failure of the search Critical to the good behavior of the best
current algorithms
What is the power of clause learning? Intuitively: it makes DPLL trees DAG-like regular resolution? general
resolution?
29
Conflict Graph
Decision scheme(p q b)
1-UIP scheme(t)
p
q
b
a
x1
x2
x3
y
yfalset
Known Clauses(p q a)
( a b t)(t x1)(t x2)(t x3)
(x1 x2 x3 y)(x2 y)
Current decisionsp falseq falseb true
30
Proof complexity and clause learning
Clause learning is no stronger than Resolution Implication graph yields resolution
derivation of each learned clause
There are examples where clause learning works in polynomial time but any Regular Resolution proof is exponential [Sabharwal, B, Kautz 04] Clause learning can sometimes be much
stronger than Regular Resolution
31
Proof complexity and clause learning
Separation is true for any proper subsystem R of Resolution for which refuting F|x is no harder than F Clause learning does not have this property
unless it is as powerful as resolution
Idea: Take formula F easy for Resolution but hard for R and create new formula F’ that is F plus clauses for (CpC) for each C in Resolution proof of F where each pC is a new propositional variable Setting each pC to true yields original F but
branching on sequence of pC allows sequence of clauses C to be learned
32
Still Open
Can clause learning do everything regular resolution can?*
Is clause learning as powerful as general resolution?
* Note: Alan Van Gelder has pointed out that if one returns learned clauses up the tree and resolves them while backtracking then clause learning does efficiently simulate regular resolution
33
Proof Complexity and Formula Caching
Formula Caching: Memoization
Cache residual formulas that have been already found to be unsatisfiable
Do not make recursive call if the current residual formula is already cached
34
DPLL(F)if F is empty report satisfiable and haltif F contains the empty clause
backtrackelse select a literal x
DPLL(F|x)
DPLL(F|x)
DPLL
35
FC(F,L)if F is empty report satisfiable and haltif F contains the empty clause or FL
returnelse choose a literal x
FC(F|x,L)
FC(F|x,L)
add F to L
Basic Formula Caching
Start with FC(F,)
36
Proof Complexity of Formula Caching [B, Impagliazzo, Pitassi, Segerlind
03]
Basic Formula Caching can exponentially improve DPLL
Basic Formula Caching is not as powerful as regular resolution but some of its natural extensions are
These natural extensions can be exponentially more powerful than general resolution or even Res(k) for any k ! Res(k) is like resolution but uses k-DNFs
instead of clauses
37
Extensions of Formula Caching
Don’t just check membership in the cache F is trivially unsatisfiable given G
iff every clause of G contains some clause of F
Record the reasons why residual formulas are known to be unsatisfiable and check the reasons rather than the formulas
38
FC*(F,L)if F is empty report satisfiable and haltif F is trivially unsatisfiable given L
add F to Lreturn
else choose a literal xFC*(F|x,L)
FC*(F|x,L)
add F to L
More General Formula Caching
39
FCreason(F,L)if F is empty report satisfiable and haltif F is trivially unsatisfiable based on GL
add F to Lreturn(G)
else choose a literal xGFCreason(F|x,L)
HFCreason(F|x,L)J (xG) (xH) add F,J to Lreturn(J)
Adding Reasons
40
Combining Clause Learning and Formula Caching
Cachet system used to count # of satisfying assignments
[Sang, Bacchus, B., Kautz, Pitassi 04]
Built on zchaff’s clause learning Uses decomposition of residual
formula into components Caches components rather than the
whole formula Subtle interactions
41
Outline
Intro to proof complexityProof complexity and complete SAT algs Resolution/DPLL review Application to random k-SAT DPLL extensions
Clause learningCaching
Proof system survey
42
Proof Systems Hierarchy
Truth Tables
DPLL Nullstellensatz
Polynomial CalculusResolution
Cutting Planes
Frege
AC0-Frege
ZFC
Ext-Frege
PCR
LS
Res(k)
43
AC0-Frege
Constant-depth Frege (AC0-Frege) Like Frege but formulas in proof only
have constant # of alternations between and
Resolution is depth-1 Frege
44
Some Known Hard Problems for Resolution
Exact/Modular Counting Pigeonhole principle: n+1 pigeons can’t fit
into n pigeonholes without getting cosy with each other
Parity Principle: e.g. can’t pair up an odd number of people
Tseitin Tautologies These problems are also hard for AC0-Frege
Random Unsatisfiable k-CNF Graph k-colorability Existence of k-cliques in graphs
45
Cutting Planes/PseudoBoolean
Introduced to relate integer and linear programming [Gomory 59, Chvatal 73]: Objects are linear integer inequalities Clause (x1 x2 x3) becomes
inequality x1+(1-x2)+x3 1
Add inequalities xi 0 and 1-xi 0
Goal: derive 0 1
46
Cutting Planes rules
addition:
multiplication by positive integer:
Division by positive integer:
a1x1 + ... + anxn A
b1x1 + ... + bnxn B
(a1+b1)x1+...+(an+bn)xn A+B
a1x1 + ... + anxn A
ca1x1 + ... + canxn cA
ca1x + ... + canxn B
a1x1 + ... + anxn B/c
47
Limitations of Cutting Planes
At least as good as resolution
Can also handle pigeonhole principle parity principle
But can’t handle counting when it is a derived property Clique-Coloring
A graph can’t have a k-clique and be k-1-colorable
Tseitin tautologies still open
48
Proof Systems Hierarchy
Truth Tables
DPLL Nullstellensatz
Polynomial CalculusResolution
Cutting Planes
Frege
AC0-Frege
ZFC
Ext-Frege
PCR
LS
Res(k)
49
Hilbert’s Nullstellensatz
System of polynomials Q1(x1,…,xn)=0,…,Qm(x1,…,xn)=0
over field K has no solution in any extension field of K there exist polynomials P1(x1,…,xn),…,Pm(x1,…,xn) in K[x1,…,xn] s.t.
m
i ii=1
1QP
50
Nullstellensatz proof system
Clause (x1 x2 x3) becomes equation (1-x1)x2(1-x3)=0
Add equations xi2-xi =0 for each
variable Guarantees only 0-1 solutions
A proof is polynomials P1,…, Pm+n
proving unsatisfiability: i.e. such that j
m n2
j m+iCj=1 i=1
+ (x - x) 1QP P
C
QC
51
Polynomial CalculusSimilar to Nullstellensatz except: Begin with Q1,…,Qm+n as before Given polynomials R and S can infer
aR + bS for any a, b in KxiR
Derive constant polynomial 1 Degree = maximum degree of polynomial
appearing in the proof Can find proof of degree d in time nO(d) using
Groebner basis-like algorithm (linear algebra)
52
PCR = PC + ResolutionTwo variables x and x’ for each proposition x x’ stands for x include equations x+x’-1=0, x2-x=0, and
(x’)2-x’=0
Translate (x1 x2 x3) as x’1 x2 x’3=0
Same proof rules and proof search as polynomial calculus Efficiently simulates resolution: clausemonomial Degree-size relationship for PCR is same as width-
size relationship of resolution
53
Lovasz-Schrijver (LS) Proofs
Linear inequalitiesSteps: Lift to get degree 2 inequalities
E.g. multiply inequalities Project
Combine degree 2 inequalities plus x2=x to cancel out degree 2 terms
Captures properties of semi-definite programming
54
Proof Systems Hierarchy
Truth Tables
DPLL Nullstellensatz
Polynomial CalculusResolution
Cutting Planes
Frege
AC0-Frege
ZFC
Ext-Frege
PCR
LS
Res(k)
55
Conclusions
Exchange of ideas between proof complexity and SAT algorithms can be fruitful Better understanding of existing algorithms Source of ideas for new algorithms and
proof systems
Potential to branch out beyond DPLL derivatives Wide range of proof systems to explore