Download - Experiments with STAGE
![Page 1: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/1.jpg)
1
Experiments with STAGEExperiments with STAGE
Wei Wei
![Page 2: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/2.jpg)
2
Introduction Introduction
STAGE- Developed by Boyan
Use value function approximation to automatically analyze sample trajectories.
Speed up many local search methods
![Page 3: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/3.jpg)
3
Diagram of STAGEDiagram of STAGE
Run to Optimize Obj Hillclimb to
Optimize V
Produces new training data
Produces good start states
![Page 4: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/4.jpg)
4
Apply it to SATApply it to SAT
The base algorithm is WalkSAT (modified)
Got results better than pure WalkSAT
![Page 5: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/5.jpg)
5
Overview Overview
We need to deal with four aspects of the problem: WalkSAT, STAGE, features, and to make the algorithm Markovian.
Hard to tune; not every combination works.
Marko-vianize
stageWalkSAT
features
![Page 6: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/6.jpg)
6
Features Features
%clauses unsatisfied (-)%clauses satisfied by 1 variable (+)%clauses satisfied by 2 variables (-)%critical variables (-)%variables set to naïve setting (~)
![Page 7: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/7.jpg)
7
MarkovianizeMarkovianize
S/W1 : patience based, not MarkovianS/W2 : best-so-farS/W3 : epsilon cutoff
![Page 8: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/8.jpg)
8
Parameter tuningParameter tuning
Noise 0.25 seems goodPatience 10,000Cutoff 1,000,000Epsilon .0001
![Page 9: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/9.jpg)
9
Function approximator V-bar-piFunction approximator V-bar-pi
Quadratic regressionLinear regression
Linear functions perform 25% better, and faster.
Linear functions are coarse approximators.
![Page 10: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/10.jpg)
10
resultsresults
algorithm Mean(obj) Time Accept%
WalkSAT 15.2 63min 100
S/W1 5.2 130min 60
S/W2 6.2 112min 58
S/W3 4.5 122min 97
![Page 11: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/11.jpg)
11
Results – Hemming Distance Results – Hemming Distance traveled by the V steptraveled by the V stepalgorithm Min Max Average TBN
S/W1 27 5028 2047 90%
S/W2 54 6982 2135 89%
S/W3 1 625 176 99%
![Page 12: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/12.jpg)
12
resultsresults
algorithm Linear Quadratic difference
S/HC 21.6 28.3 31%
S/W1 5.2 5.4 4%
S/W2 6.2 5.0 -19%
S/W3 4.4 5.6 27%
![Page 13: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/13.jpg)
13
Feature 1 and 2 only Feature 1 and 2 only
algorithm Mean(obj) Time Accept%
WalkSAT 15.2 63min 100
S/W1 8.2 98min 83
S/W2 8.5 96min 85
S/W3 7.3 102min 97
![Page 14: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/14.jpg)
14
Added feature: %variables set Added feature: %variables set to true to true
algorithm Mean(obj) Time Accept%
WalkSAT 15.2 63min 100
S/W1 5.4 143min 58
S/W2 5.9 118min 56
S/W3 4.6 135min 95
![Page 15: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/15.jpg)
15
Discussion(1)Discussion(1)
Linear regression is very bad approximation is this case, yet it gives better results than quadratic regression. Why?
Hit bottom very oftenLead to long more WalkSAT moves
![Page 16: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/16.jpg)
16
Discussion(2)Discussion(2)
Features – coefficients vary a lot among instances. But relatively stable within one instance.
The signs are relatively stable
![Page 17: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/17.jpg)
17
Discussion(3)Discussion(3)
Time vs evaluationWhen # of evaluation is fixed, STAGE
performs 3 times better, but time spent is doubled
When time is fixed, the result is 40% better than WalkSAT
![Page 18: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/18.jpg)
18
Discussion(4)Discussion(4)
Can it hit the finish line?It does vaguely(?) learn some concepts,
which hopefully can direct WalkSAT to a good place.
Par-? Is a good set of problems to solve?
![Page 19: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/19.jpg)
19
One featureOne feature
5 features 1 feature
WalkSAT (15.2)
S/w1 5.2 17.4
S/W2 6.2 18.3
S/W3 4.4 20.9
No improvement over WalkSAT.
![Page 20: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/20.jpg)
20
Random restartRandom restart
176 Random flips – Worse than S/W3, still better than WalkSAT
1000 Random flips – Worse than one-run WalkSAT
Complete new start points – similar to the case above.
Parameters: cutoff – 10,000. Restart – 100.
![Page 21: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/21.jpg)
21
HanoiHanoi
Parameters not yet carefully tunedIt would be interesting to see whether
Hanoi4 can be solved by carefully tuned S/W3. I ran WalkSAT for 50,000,000 flips, but failed to solve it.
![Page 22: Experiments with STAGE](https://reader036.vdocuments.net/reader036/viewer/2022081519/5681392b550346895da0d49e/html5/thumbnails/22.jpg)
22
Hanoi problemsHanoi problems
WalkSAT
GSAT S/W3
Hanoi5 8 18 5
Hanoi4 2 8 1