1 exploiting random walk strategies in reasoning wei
TRANSCRIPT
![Page 1: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/1.jpg)
1
Exploiting Random Walk Strategies in
Reasoning
Wei Wei
![Page 2: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/2.jpg)
2
Universal inference engine One of the oldest AI dreams
General Problem Solver (Newell & Simon, 1961)Use logic reasoning (“Program with Common Sense”, McCart
hy 1968) Largely unsuccessful due to the tradeoff between co
mputational complexity and representation power. => many successful systems are domain-specific (e.g. Dendral, 1968-85, organic chemistry)
Recent improvement in computer’s abilities to perform large-scale search => Revisit the original idea
![Page 3: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/3.jpg)
3
Computational Power - SAT Hardware power Average-case
complexity New algorithmic tools:
randomization, learning
Increasing demand: verification
Research input: annual conference, competition
year #variable
1962 50
1992 300
1996 1,000
2002 1,000,000
![Page 4: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/4.jpg)
4
Boolean Satisfiability Problem Boolean Satisfiability Problem (SAT) asks if a Boolean
expression can be made true by assigning Boolean values to its variables.
The problem is well-studied in AI community with direct application in reasoning, planning, CSP, VLSI microprocessor verification etc.
Does statement s hold in world A (represented by a set of clauses)?
A s (¬s) ^ A unsatisfiable
![Page 5: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/5.jpg)
5
SAT SAT (even 3SAT) is NP-complete. Best theoreti
cal bound so far is (2-2/k)N randomized (Schoening 1999) or
(2-2/(k+1))N deterministic (Dantsin et al 2002) for k-SAT
In practice, there are two different kinds of solvers DPLL (Davis, Logemann and Loveland 1962) Local Search (Selman et al 1992)
![Page 6: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/6.jpg)
6
DPLL (x1 x2 x3) (x1 x2 x3) (x1 x2) DPLL was first proposed as a basic depth-
first tree search.
x1
x2
FT
T
null
F
solution
x2
Potential problem: early commitment
![Page 7: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/7.jpg)
7
DPLL Recently (since late 90’s), many
improvements: Randomization restarts out-of-order backtracking clause learning
![Page 8: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/8.jpg)
8
Local Search(x1 x2 x3) (x1 x2 x3) (x1 x2 ) The idea: Start with a random assignment.
And make local changes to the assignment until a solution is reached (010 011 001)
Pro: often efficient in practice. Sometimes the only feasible way for some problems
Con: Cannot prove nonexistence of solutions. Difficult to analyze theoretically.
Example GSAT (Selman et al. 1992)
![Page 9: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/9.jpg)
9
Local Search Schemes local search schemes used:
Simulated annealing Tabu search Genetic algorithms Random Walk and its variants the most successf
ul so far
![Page 10: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/10.jpg)
10
Unbiased (Pure) Random Walk for SATProcedure Random-Walk (RW)Start with a random truth assignmentRepeat
c:= an unsatisfied clause chosen at randomx:= a variable in c chosen at randomflip the truth value of x
Until a satisfying assignment is found
![Page 11: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/11.jpg)
11
Unbiased RW on any satisfiable 2SAT Formula
Given a satisfiable 2SAT formula with n variables, a satisfying assignment will be reached by Unbiased RW in O(n2) steps with high probability. (Papadimitriou, 1991)
![Page 12: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/12.jpg)
12
We have an unbiased random walk with a reflecting barrier at distance N from T (max Hamming distance) and an absorbing barrier (satisfying assignment) at distance 0.
We start at a Hamming distance of approx. ½ N.
Property of unbiased random walks: after N^2 flips, with high probability, we will hit the origin (the satisfying assignment). (Drunkards walk)
So, O(N^2) randomized algorithm (worst-case!) for 2-SAT.
T A0T A0
reflecting
![Page 13: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/13.jpg)
13
Unbiased RW on 3SAT Formulas
Random walk takes exponential number of steps to reach 0.
T A0
reflecting
![Page 14: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/14.jpg)
14
Comments on RW
1) Random Walk is highly “myopic” does not take into account any gradient of the objective function (= number of unsatisfied clauses)!
Purely “local” fixes.
2) Can we make RW practical for SAT? Yes --- inject greedy bias into walk biased Random Walk.
![Page 15: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/15.jpg)
15
Biased Random Walk
Procedure WalkSatRepeat
c:= an unsatisfied clause chosen at random
if there exist a variable x in c with break value = 0 // greedy biasflip the value of x (“freebie move”)
else with probability p // pure walk
x:= a variable in c chosen at random flip the value of x
with probability (1-p)x:= a variable in c with smallest break value // more greedy biasflip the value of x
Until a satisfying assignment is found
Note: tune parameter p.
![Page 16: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/16.jpg)
16
Random Walk algorithms Random walk algorithm (e.g. Walksat) offer sig
nificant improvement on performance over hill-climbing algorithms.
ID Vars clauses GSAT+w WalksatSsa7552-038 1501 3575 129 2.3
Ssa7552-158 1363 3034 90 2
Ssa7552-159 1363 3032 14 0.8
Ssa7552-160 1391 3126 18 1.5
![Page 17: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/17.jpg)
17
First, bringing out the worst in random walks…
(2-SAT)
X1 X2
X2 X3
X3 X4
Xn X4
Xn X1
Note: Only 2 satisfying assignments, all False and all True.
![Page 18: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/18.jpg)
18
Binary Chains
Walk is exactly balanced.
![Page 19: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/19.jpg)
19
Results: Speeding up Random Walks on Binary Chains
*: empirical results**: proof available
Pure binary chain
Chain withredundancies
RW (n2)** (n2)**
RWF (n2)** (n1.2)*
WalkSat (n2)** (n1.1)*
Becomesalmost likeunit prop.
![Page 20: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/20.jpg)
20
Ternary Chains
In general, even a small bias in the wrong direction leads to exponential time to reach 0.
reflecting
![Page 21: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/21.jpg)
21
What about 3-SAT? Again, consider “chain” formulas.
X1 & X2 X3
X2 & X3 X4
X_(n-2) & X_(n-1) X_n
X1 & X2
X1 & X2
X_floor(n/2) & X_(n-1) X_n
![Page 22: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/22.jpg)
22
Theoretical Results on 3-SAT Chains
Function low(i) Expected run time of pure RW
i-2 (highly local) ~ Fib(n) (i.e., exp.)
i/2 (interm. reach) ~ nlog n (i.e., quasi-poly)
log i (interm. reach) ~ n2 . (log n)2 (i.e., poly)
1 (full back reach) ~ n2
low(i) captures how far back the clauses reach.
![Page 23: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/23.jpg)
23
Recurrence RelationsOur formula structure gives us:
E(f(zi)) = (E(f(zlow(i)) + E(f(zi) + 1) * 1/3
+ (E(f(zi-1) + E(f(zi) + 1) * 1/3
+ 1 * 1/3
E(f(zi)) = E(f(zlow(i)) + E(f(zi-1) + 3
![Page 24: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/24.jpg)
24
Decompose: multiple “0”s
110101010101
110001010001
100001
111001110001
101001111001
111111111101
110101010101
110001100001
111001110001
110101
010001
101001111001111101111111
110111010111
110111100111
111111110111
111101
111001
101001111001111101111111
110111 111111
111101 111111
Start
Sat assign.
![Page 25: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/25.jpg)
25
Recap Chain Formula Results Adding implied constraints capturing long-range
dependencies speeds random walk on 2-Chain to near linear time.
Certain long-range dependencies in 3-SAT lead to poly-time convergence of random walks.
Can we take advantage of these results on practical problem instances?
Yes! (next)Our methodology: Identify, analyze, and “exploit” special tractable structure in large practical reasoning problems.
![Page 26: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/26.jpg)
26
Empirical Results
SSS-SAT-1.0 instances (Velev 1999). 100 total. level of redundancy added (20% near optimal)
Formulas(redun. level)
<40 sec <400 sec
<4000 sec
= 0.0 15 26 42
= 0.2 85 98 100
= 1.0 13 33 64
![Page 27: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/27.jpg)
28
Probabilistic Reasoning Previously, we asked “does statement s hold
in world A (represented by a set of clauses)?”
A s (¬s) ^ A unsatisfiable Now, what is the probability that statement s
holds in world A?
Pr = #( s ^ A ) / #A
![Page 28: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/28.jpg)
29
Probabilistic Reasoning Close connection between counting and samp
ling (Jerrum et al, 1986). Bayesian Net Queries can be encoded as #SAT.
(Kautz, 2004) How to Sample:
By repeated counting using DPLL algorithms Monte Carlo Markov Chain Method Use state-of-the-art local search methods Note: Random Walk not uniform
![Page 29: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/29.jpg)
30
Characteristics of Solution space: Solution Clustering Visualization with multi-dimensional scaling (MDS)
Solutions to specific 75 variable, 325 clause 3-SAT instances
75 dimensional solution projected to two dimensions Distance between points approximates hamming distance
![Page 30: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/30.jpg)
31-10 -5 0 5 10 15 20 25-25
-20
-15
-10
-5
0
5
10
15
20SA on SAT-75 - Multi-dimensional Scaling Using Hamming Distance Between Solutions
25 Most Often Found Solutions25 Next Most Often Found SolutionsMiddle Solutions25 Next Least Often Found 25 Least Often Found Solutions
Empirically determined each solution’s probability (uf75-01 - 75 variable, 325 clause 3-SAT instance)
WalkSat finds every solution, but with very large range of probabilities (1:104)
Probability Clusters
Solution Probability Using WalkSat Algorithm
![Page 31: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/31.jpg)
32
Probability Ranges in Different Domains
Instance Runs Hits Rarest
Hits Common
Common-to -Rare
Ratio
Random 50*106 53 9*105 1.7*104
Logistics 1* 106 84 4*103 50
![Page 32: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/32.jpg)
33
Improving the Uniformity of Sampling - mixing sampling strategy
To reduce the range of probabilities, we propose a hybrid local search algorithm: With probability p, the algorithm makes a biased
random walk move With probability 1-p, the algorithm makes a SA (si
mulated annealing) move In our experiments, we used
50% WalkSat + 50% SA at a fixed temperature
![Page 33: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/33.jpg)
34
Results of the Hybrid Approach
Our key figure.
![Page 34: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/34.jpg)
35
Solution Clusters
Results on a random 3-SAT instance (70 vars, 301 clauses, 2531 solutions).
![Page 35: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/35.jpg)
36
Summary
1) WalkSAT does sample all solutions.2) But, sampling can be highly biased.3) Using a new hybrid strategy, we can obtain effective near-uniform sampling.
Lesson: Hybrid of SA and biased walk, is a promising alternative to MCMC methods for sampling.
Idea: Use SAT solvers to sample solutions from a combinatorial space.Findings:
![Page 36: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/36.jpg)
37
Research Directions
Can we exploit techniques from DPLL solvers, such as learned clauses (inferred structure) in local search?
Can we identify other classes of structure in real-world problem that can further accelerate random walk style SAT solvers?
Can we design better SAT encodings based on our insights about structure?
![Page 37: 1 Exploiting Random Walk Strategies in Reasoning Wei](https://reader036.vdocuments.net/reader036/viewer/2022062321/56649e0f5503460f94af9f62/html5/thumbnails/37.jpg)
38
Research Directions, cont. Solution sampling
Main Challenge: moving between clusters evaluation tools.
Randomize DPLL to sample solution space Early indicator of solution counts.
Compare with Bayesian/probabilistic state-of-the-art inference methods. Can we outperform them?