probability in the analysis of cnf satisfiability
TRANSCRIPT
Probability in the Analysis
of CNF Satisfiability
Algorithms and Properties
John FrancoComputer Science, University of Cincinnati
Why is it Worthwhile?
The Questions:
• Why are some problems so difficult?
• Are there algorithms that will make them easier?
Why Probability?
• Results and process tend to draw out intuition
− Identify properties that may be exploited by a fastalgorithm and properties that may prevent exploitation.
− Identify reasons for the hardness of various problems -why lots of instances are hard.
• Can explain the good or bad behavior of an algorithm
• Afford comparison of incomparable classes of formulas
• De-randomization may yield fast algorithms
TermsA variable looks like this: v9, takes a value from {0, 1}
A positive literal: v9, a negative literal: ¬v9
A clause looks like this: (¬v1 ∨ ¬v2 ∨ v5 ∨ v9)
An instance of SAT looks like this:(v1 ∨ ¬v2 ∨ v7) ∧ (¬v2 ∨ v6) ∧ (¬v2 ∨ ¬v4 ∨ ¬v5) ∧ (v10)...
Clause width: # literals; k-SAT: fixed width k
An assignment of values satisfying F is a model for F
An important splitting operation in solvers:
HHHHHj
������
tdddcc
tdddcctdddcc
(¬v1 ∨ ¬v2 ∨ vi) ∧ (¬vi ∨ v3) ∧ (¬v2 ∨ v4)
vi1 0
(v3) ∧ (¬v2 ∨ v4) (¬v1 ∨ ¬v2) ∧ (¬v2 ∨ v4)
Some problems
• Must assume an input distribution,often not reflecting reality(but sometimes we do not need to)
• Analysis can be difficult or impossible -algorithmic steps may significantly change distribution(known tools are limited)
• Can yield misleading results
A Misleading Result
Example: A probabilistic model for random formulas
Given: set L = {v1,¬v1, ..., vn,¬vn} of literals and 0 < p < 1
Construct clause c: l ∈ L independently in c with probability p
Construct formula: m independently constructed clauses
Justification: All formulas are equally likely if p = 1/3.
Applied to splitting (DPLL):
HHHHHj
������
tdddcc
tdddcctdddcc
(¬v1 ∨ ¬v2 ∨ vi) ∧ (¬vi ∨ v3) ∧ (¬v2 ∨ v4)
vi1 0
(v3) ∧ (¬v2 ∨ v4) (¬v1 ∨ ¬v2) ∧ (¬v2 ∨ v4)
A Misleading Result
Analysis sketch:
Given m clauses, the average number of clauses removedwhen a value is assigned is pm
Let Ti be the average number of clauses remainingon the ith iteration
Then T0 = m and Ti = (1 − p)Ti−1
For what i does Ti = 1? When 1 = m(1 − p)i orlg(m) = −i lg(1 − p)
i = lg(m)/ − lg(1 − p)
So, size of search space is 2i ≈ 2lg(m)/p = mc if p = 1/3
Conclusion: DPLL is fantastic, on the average!
A Misleading Result
Problems with the analysis:
• The input model is funky!The probability that a random assignment satisfiesa random formula is
(1 − (1 − p)2n)m ≈ e−me−2pn
which tends to 1 if ln(m)/pn = o(1) and means theaverage width of a clause can be at least ln(m).With p = 1/3, this holds if n > ln(m).
• If average clause width is constant (p = c/n)then the average search space size is
2− lg(m)/ lg(1−p) ≈ 2lg(m)/(c/n) = 2n lg(m)/c = mn/c
Exponential complexity!
Probabilistic Toolbox
Linearity of expectation:
E{∑
i
Xi} =∑
i
E{Xi}, Xi real valued r.v.s
First Moment Method: show Pr(P ) → 0 as n → ∞.
Let X be a count of some entitySuppose some property P holds if and only if X ≥ 1.
ThenPr(X > 1) ≤ E{X}
So, if E{X} → 0 then Pr(P ) → 0, as n → ∞.
First Moment Method
An Example:
Assume F is a random k-SAT formula, m clauses, n variables
Let P be the property that F has a model
Let X be the number of models for F
Let Xi =
{
1 if the ith assignment is a model for F0 otherwise
Pr(Xi = 1) = (1 − 2−k)m
E{X} =2n∑
i=1
Pr(Xi = 1) = 2n(1 − 2−k)m
Pr(F has a model) → 0 ifm
n>
1
lg(1 − 2−k)≈ 2k
For k = 3 F has no model w.h.p. ifm
n> 5.19
Probabilistic Toolbox
Flow Analysis:
Consider straight-line (non-backtracking) variants of DPLL
Let F be a CNF Boolean expressionSet M = ∅Repeat the following:
Choose an unassigned literal l in FIf l is a positive literal, set M = M ∪ {l}Set F = {c \ {¬l} : c ∈ F , l /∈ c}If F is satisfied then return MIf some clause in F is falsified return "give up"
When does this not give up with probability tending to 1?
Answer depends on the way literals are chosen
Flow Analysis
Ck(j) -
?
zk(j)
wk−1(j)
Ck−1(j) -
?
zk−1(j)
wk−2(j)
. . .w2(j)?
C2(j) -
?
z2(j)
w1(j)
C1(j) -
?
z1(j)
w0(j)
E{m4(j)}
j/n-0 1/4 1/2 3/4 1
m
0
E{m3(j)}
j/n-0 1/4 1/2 3/4 1
m
0
E{m2(j)}
j/n-0 1/4 1/2 3/4 1
m
0
Flow Analysis
Example:
Unit clause heuristic: When there is a clause with oneunassigned variable remaining, set the value of such avariable so as to satisfy its clause.
Intuitively: If the clause flow w1(j) < 1 then anyaccumulation of unit clauses can be prevented andno clauses will ever be eliminated
Analysis:
Write difference equations describing flows:
mi(j + 1) = mi(j) + wi(j) − wi−1(j) − zi(j), ∀ 0 ≤ i ≤ k, 1 < j < n
Take expectations and rearrange:
E{mi(j + 1) − mi(j)} = E{wi(j)} − E{wi−1(j)} − E{zi(j)}
Flow AnalysisCompute the expectations:
E{zi(j)} = E{E{zi(j)|mi(j)}} = E
{
i · mi(j)
2(n − j)
}
=i · E{mi(j)}
2(n − j)
E{wi(j)} = E{E{...}} = E
{
(i + 1)mi+1(j)
2(n − j)
}
=(i + 1)E{mi+1(j)}
2(n − j)
Substitute into difference equations:
E{mi(j + 1) − mi(j)} =(i + 1)E{mi+1(j)}
2(n − j)− i · E{mi(j)}
n − j, i < k
E{mk(j + 1) − mk(j)} = −k · E{mk(j)}n − j
Switch to differential equations (use x for j/n):
dm̄i(x)
dx=
(i + 1)m̄i+1(x)
2n(1 − x)− i · m̄i(x)
n − j; m̄k(0) = m/n, m̄i(0) = 0 for i < k
Solve: Translation:
m̄i(x) =1
2k−i
(m
n
)
(
k
i
)
(1 − x)ixk−i E{mi(j)} =m
2k−i
(
k
i
)(
1 − j
n
)i( j
n
)k−i
Flow AnalysisThe important flow:
E{w1(j)} =E{m2(j)}(n − j)
=1
2k−2
(
k
2
)(
1 − j
n
)2( j
n
)k−2
m
Find location of maximum (set derivative = 0):
j∗ =
(
k − 2
k − 1
)
n
So,
E{w1(j∗)} < 1 when
m
n<
2k−1
k
(
k − 1
k − 2
)k−1
for k = 3 this ism
n<
8
3
Conclusion: unit clause heuristic succeeds with probability
bounded by a constant whenm
n<
8
3.
Flow Analysis
What makes this work?
The clausal distribution is the same, up to parameters m and n,at each level (algorithm is myopic - no information is revealed)
There is pretty much never a sudden spurious flowPr(||Ci(j + 1)| − |Ci(j)|| > n0.2) = o(n−3)
The average flow change is pretty smoothE{|Ci(j + 1)| − |Ci(j)|} = fi(j/n, |C0(j)|/n, ..., |Ck(j)|/n) + o(1),
fi is continuous and
|fi(u1, ..., uk+2) − fi(v1, ..., vk+2)| ≤ L∑
1≤i≤j
|ui − vi|
Flow Analysis
good when m/n <
literal selection k-SAT 3-SAT
Choose from unit clause, otherwise randomly 2k/k 2.66
Choose var with maximum difference betweenoccurrences of positive and negative lits. Setvalue to maximize satisfied clauses
c · 2k/k 3.003
Choose randomly from a clause with fewestnon-falsified literals
1.125 · 2k/k 3.09
Best possible myopic algorithm c · 2k/k 3.26
literal selection, non-myopic algorithmsGreedy algorithm: maximize number of clausessatisfied, eliminate unit clauses when they exist
c · 2k/k? 3.52
Maximize the expected number of models ofthe reduced instance
c · 2k/k? > 3.6?
Flow Analysis
Pr(success)
1
0
0.46352k
k + 1
(
k − 1
k − 2
)k−2
0.77252k
k + 1
(
k − 1
k − 2
)k−2
m
n⇒
Typical probability curve of these algorithms
Is2k
kthe crossover to unsatisfiability or is it 2k?
Probabilistic Toolbox
Second Moment Method: show Pr(P ) → 1 as n → ∞Let P be some property of a formula
A witness for P : a structure whose presence in F implies P
Let W = {w : w is a witness for P in F}
Let Xi =
{
1 if structure si ∈ W
0 otherwise
Let X =∑
i
Xi and E{Xi} = q, then E{X} ∆= µ = q|W |
Suppose var(X) ∆= σ2 = o(µ2). Then, since
Pr(X = 0) ≤ σ2
µ2
we get
Pr(P ) → 1 as n → ∞
Second Moment MethodTo show σ2 = o(µ2):
Start with one witness w chosen arbitrarily
Let Aw be all witnesses sharing at least one clause with w
Let Dw be all witnesses sharing no clause with w. Then
σ2 = µ(1 − q) + µ
∑
z∈Aw
(Pr(z|w) − q) +∑
z∈Dw
(Pr(z|w) − q)
Need |Aw| ≪ |Dw| or “little” overlap among witnesses since
Pr(z|w) = Pr(z) = q if z ∈ Dw so∑
z∈Dw(Pr(z|w) − q) = 0.
If µ → ∞ and∑
z∈Aw
Pr(z|W ) → o(µ) as n → ∞ then
Pr(P ) → 1 as n → ∞
Where is the Crossover?
Cannot apply the second moment method directly to k-SAT- variance of the number of models is too highthe reason: the asymmetry of k-SAT
Let z and w be assignments agreeing in αn variables
Pr(z satisfies F | w satisfies F) =
(
1 − 1 − αk
2k − 1
)m
Variance gets too big, maximum occurs at α 6= 1/2
∑
z∈Aw
Pr(z|w) =∑
0<α≤1
( n
αn
)
(
1 − 1 − αk
2k − 1
)m
Where is the Crossover?
Analyze a different problem: Not All Equal k-SATA NAE model: one for which every clause hasat least one true and at least one false literal
Pr(∃ a model for F) > Pr(∃ a NAE-model for F)
Let X = number of models, XNAE = number of NAE models
Pr(XNAE = 0) > Pr(X = 0)
Pr(z NAE-satisfies F | w NAE-satisfies F) =
(
1 − 1 − αk − (1 − α)k
2k−1 − 1
)m
Variance is low, maximum term occurs at α = 1/2.
∑
0<α≤1
( n
αn
)
(
1 − 1 − αk − (1 − α)k
2k−1 − 1
)m
≈ 2n(1 − 21−k)m√n
≈ o(µ)
Pr(XNAE = 0) <1
µNAE
≈ 1
2n(1 − 2k−1)m→ 0 if
m
n>
1
lg(1 − 2k−1)≈ 2k−1
What About Easy Classes?
Examples:Horn, Hidden Horn, 2-SAT, Extended Horn, q-Horn,CC-balanced, SLUR, Matched, Linear Autarkies
Horn:every clause has at most one positive literalsolved in linear time by unit clause algorithm
Probabilistic analysis of polytime solvable classes can reveal:
• What critically distinguishes an easy classfrom more difficult classes
• Whether one class is much larger than anotherincomparable class in a probabilistic sense
Polynomial Time Solvable Classes
Example: q-Hornvariables
Horn
clauses
non-positive
-1 0 1 0 0 0 0 0 0 0 0
1 -1 -1 0 1 0 0 0 0 0 0
0 0 -1 -1 0 0 0 0 0 0 0
0 1 -1 0 0 0 0 0 0 0 0
0 -1 -1 0 0 0 0 1 0 1 0
-1 0 0 -1 0 1 0 0 -1 0 0
0 0 0 -1 0 0 0 0 0 -1 1
0 0 -1 -1 0 0 -1 0 0 0 0
Zero
2-SAT
If the satisfiability index of a given formula is no greaterthan 1, then the formula is q-Horn.
The class of formulas with a satisfiability index greater than1 + n−ǫ, for any ǫ, is NP-complete.
Polynomial Time Solvable Classes
Vulnerability of q-Horn to particular cycles
. . . (u2 ∨ ¬u1 ∨ ...) (u1 ∨ ¬v0 ∨ ¬u3p ∨ ...) (u3p ∨ ¬u3p−1 ∨ ...) . . .
. . . (¬up−1 ∨ up ∨ ...) (¬up ∨ ¬v0 ∨ up+1 ∨ ...) (¬up+1 ∨ up+2 ∨ ...) . . .
u1
up
v0
up+1
u3p
Pr(random formula is of specified class)
class as n → ∞ k-SAT 3-SAT
Horn 0 m > ǫ m > ǫ
Hidden Horn 0 m/n > 1/(k − lg(k + 1)) m/n > 1
q-Horn 0 m/n > 4/(k2 − k) m/n > 0.66
SLUR 0 m/n > 4/(k2 − k) m/n > 0.66
Matched 1 m/n < ck, ck → 1 m/n < 0.64
No Cycles 1 m/n < 1.36/(k2 − k) m/n < 0.226
Algorithms for Unsatisfiability
Resolution performs badly on random unsatisfiable formulas
Pr(random k-SAT formula is unsatisfiable) → 1 if mn > 2k
Pr(resolution does well) → 1 only if mn >
(
nlg(n)
)k−2
Are there better alternatives?Must avoid getting stuck on “sparse” nature of formulas
One possibility: Hitting Set
Focus attention on clauses which are all positive or negative
Let n+ = min number of 1 valued variables to satisfy positives
Let n− = min number of 0 valued variables to satisfy negatives
If n+ + n− > n then some variable must be set to 1 AND 0
That is impossible, conclude the formula is unsatisfiable
Hitting Set
Construct graphs G+, G−
Vertices are labeled as pairs of variables
Edge 〈a, b〉 ⇔ some clause contains all variables labeling a, b
(v1 ∨ v2 ∨ v3 ∨ v4) ∧ (v2 ∨ v3 ∨ v4 ∨ v5) ∧ (v1 ∨ v3 ∨ v4 ∨ v5)
v1, v2
yiihhg
v3, v4
yiihhg
AAAAAA
��
��
��
v2, v5
yiihhg
v1, v5
yiihhg
v2, v4
yiihhg
v1, v3
yiihhg
AAAAAA
��
��
��
v4, v5
yiihhg
v2, v3
yiihhg
AAAAAA
��
��
��
v1, v4
yiihhg
v3, v5
yiihhg
AAAAAA
Hitting Set
Construct graphs G+, G−
Vertices are labeled as pairs of variables
Edge 〈a, b〉 ⇔ some clause contains all variables labeling a, b
(v1 ∨ v2 ∨ v3 ∨ v4) ∧ (v2 ∨ v3 ∨ v4 ∨ v5) ∧ (v1 ∨ v3 ∨ v4 ∨ v5)
v1, v2
yiihhg
v3, v4
yiihhg
AAAAAA
��
��
��
v2, v5
yiihhg
v1, v5
yiihhg
v2, v4
yiihhg
v1, v3
yiihhg
AAAAAA
��
��
��
v4, v5
yiihhg
v2, v3
yiihhg
AAAAAA
��
��
��
v1, v4
yiihhg
v3, v5
yiihhg
AAAAAA
For k = 4, if a model exists, one graph has |IS| >n2
8
Hitting Set
Construct matrices M+,M−
Columns and rows are indexed on vertices
Mi,j =
−1−pp if there is an edge between vertices i and j
1 otherwise
where p represents a probability that can be adjusted
Exploit relationship between maximum eigenvalue and maximum IS
α(G+) < λ1(M+) and α(G−) < λ1(M
−)
For purposes of proving a bound
If p =ln7(n′)
n′then max
i{|λi(M)|} =
2n′
ln3.5(n′)(1 + o(1)) w.h.p.
This leads to Pr(HS does well) → 1 ifm
n> n(k−2)/2
Recall Pr(resolution does well) → 1 only ifm
n> nk−2.
A De-randomized Algorithm for MAXSAT
MAX k-SAT
Given: A CNF formula F with k literals per clause
Find: An assignment to the variables of F thatsatisfies a maximum number of its clauses.
Suppose F has variables v1, v2, . . . , vn. Define indicators
Ait1,...,tj
=
{
1 if clause i satisfied given v1 = t1, ..., vj = tj0 otherwise
What is the probability that Ait1,...,,tj
= 1 for random tj+1, ..., tn?
for example: Pr((¬v1 ∨ v3 ∨ ¬v5) = 1 | t1 = 1, t2 = 0) = Pr(Ai1,0) = 3/4
A De-randomized Algorithm for MAXSAT
Let N be the number of satisfied clauses in F. Then
E{N} =n∑
i=1
Pr(Ai) = (1 − 2−k)m
and
Pr(Ait1,...,tj−1
) = (Pr(Ait1,...,tj−1,1) + Pr(Ai
t1,...,tj−1,0))/2
≤ max {Pr(Ait1,...,tj
)}
so keep choosing a value tj, for j = 1, 2, ... that maximizesPr(Ai
t1,...,tj) to get
E{N} = (1 − 2−k)m =
n∑
i=1
Pr(Ai) ≤n∑
i=1
max {Pr(Ait1
)}...
≤n∑
i=1
max {Pr(Ait1,...,tn)} = # clauses satisfied given t1, ..., tn
A De-randomized Algorithm for MAXSAT
MAX k-SAT approximation algorithm uses this:
bool chooseValue (Variable *var) {double sum pos=0.0, sum neg=0.0, prob=0.5;for (int sz=1 ; sz <= k ; sz++) {
sum pos += var->no clauses as pos lit[sz]*prob;sum neg += var->no clauses as neg lit[sz]*prob;prob *= 0.5;
}return (sum pos >= sum neg) ? true : false;
}
and will always find an assignment that satisfies at least(1 − 2−k)m clauses - for any input formula!