1
BLACKBOX: A New Paradigm for Planning
BLACKBOX: A New Paradigm for Planning
Bart SelmanCornell University
2
Search as Inference: DirectSearch as Inference: Direct
Abstract problem specification
General inference (NP complete)
Solution
Model in propositional logic
3
State-space PlanningState-space Planning
Find a sequence of operators that transform an initial state to a goal state
State = complete truth assignment to a set of variables (fluents)
Goal = partial truth assignment (set of states)
Operator = a partial function State State
• specified by three sets of variables:precondition, add list, delete
list
4
Some Applications of PlanningSome Applications of Planning
Autonomous systems
• NASA Deep Space One Remote Agent
Softbots - software robots
• Internet agents, program assistants
• Bots, characters in games
Program verification
• Jackson (1998) - finding bugs in protocols
- is there a sequence of actions that reaches an error state?
5
SATPLAN(Kautz & Selman 1996)SATPLAN
(Kautz & Selman 1996)
STRIPSModel in propositional logic
Walksat SAT engine Solution
6
Lessons from SATPLANLessons from SATPLAN
A general propositional theorem prover outperformed traditional AI planning systems (UCPOP, Nonlin, Prodigy, ...)
• Power of propositional logic– much better scaling than attempts in 1970’s using
first-order theorem proving
• Fast SAT engines– stochastic search - walksat
– large SAT/CSP community sharing ideas and code
– older planning systems can be viewed as adhoc, incomplete, poorly understood theorem provers!
• Importance of modeling– different axiomatizations can have vastly different
computational properties
7
Graphplan(Blum & Furst 1996)Graphplan(Blum & Furst 1996)
Planning as graph searchLike SATPLAN...
• Two phases: instantiation of propositional structure, followed by search
• Plan graph is very close to CNF
Unlike SATPLAN…
• Takes STRIPS operators directly as input
• Interleaves instantiation and pruning of plan graph
– results in much smaller structure
• Employs specialized search engine
• Graphplan - better instantiation
• SATPLAN - better search
Goal: Combine best features of both systems
8
Where Graphplan Gets its PowerWhere Graphplan Gets its Power
During instantiation, Graphplan computes mutex relationships between incompatible actions
• used for pruning, and later speeding search
• mutex algorithm is actually a form of limited resolution on binary negative clauses!
• polytime preprocessing O(n2)
Issue:
• research on graphplan failed to discover any useful extensions to mutex algorithm
• Can general polytime limited inference algorithms discover other kinds of useful local information?
9
Multistep Problem ReformulationMultistep Problem Reformulation
Domain specific model
Polytime domain specific inference
Combinatorial core - general language
Full general inference(NP complete)
SolutionPolytime general inference
Abstract problem specification
10
BlackboxBlackbox
STRIPSPlan Graph
Mutex computation
CNF Translation
Stochastic / Systematic SAT engines
SolutionLimited resolution - failed literal rule
11
IntuitionIntuition
Many real-world problems not tractable, but are nearly so
• domain specific polytime inference takes advance of special kinds of structure
• small number of practical methods for combinatorial core– can be highly optimized
– limited inference: variations of constraint propagation
– full inference: local search, smart backtracking, *randomized backtracking
12
Translation to CNFTranslation to CNF
Fact Act1 Act2
Act1 Pre1 Pre2
¬Act1 ¬Act2
Act1
Act2
Fact
Pre1
Pre2
Alternating layers of facts and actions• fully factored (nodes are propositions, not states!)
Not all atoms in a layer can hold simultaneously• solution = subgraph containing all goals, all supports, no mutexes
13
General Limited InferenceGeneral Limited Inference
Generated wff can be further simplified by consistency propagation techniques
Compact (Crawford & Auton 1996)
• unit propagation: is Wff inconsistant by resolution against unit clauses?
O(n)
• failed literal rule: is Wff + { P } inconsistant by unit propagation?
O(n2)
• binary failed literal rule: is Wff + { P V Q } inconsistant by unit propagation?
O(n3)
Complements domain specific limited inference
Discovers hidden local structure!
14
General Limited InferenceGeneral Limited Inference
Percent vars set byProblem Varsunitprop
failedlit
binaryfailed
bw.a 2452 10% 100% 100%bw.b 6358 5% 43% 99%bw.c 19158 2% 33% 99%log.a 2709 2% 36% 45%log.b 3287 2% 24% 30%log.c 4197 2% 23% 27%log.d 6151 1% 25% 33%
15
Randomized Sytematic SolversRandomized Sytematic Solvers
Stochastic local search solvers (walksat)
• when they work, scale well
• cannot show unsat
• fail on some domains
Systematic solvers (Davis Putnam)
• complete
• seem to scale badly
Can we combine best features of each approach?
16
Heavy TailsHeavy Tails
Bad scaling of systematic solvers can be caused by heavy tailed distributions
Deterministic algorithms get stuck on particular instances
• but that same instance might be easy for a different deterministic algorithm!
Expected (mean) solution time increases without limit over large distributions
17
Heavy Tailed Cost DistributionHeavy Tailed Cost Distribution
0.1
1
1 10 100 1000 10000 100000
log( Backtracks )
log
( 1
- F
(x)
)
18
Randomized RestartsRandomized Restarts
Solution: randomize the systematic solver
• Add noise to the heuristic branching (variable choice) function
• Cutoff and restart search after a fixed number of backtracks
Eliminates heavy tails
In practice: rapid restarts with low cutoff can dramatically improve performance
19
Rapid Restart SpeedupRapid Restart Speedup
1000
10000
100000
1000000
1 10 100 1000 10000 100000 1000000
log( cutoff )
log
( b
ackt
rack
s )
20
Blackbox as Experimental TestbedBlackbox as Experimental Testbed
All components of blackbox are parameterized
Can experiment with different schedules for instantiating, simplifying, and solving problems
blackbox -solver -maxsec 20 graphplan
-then compact -l
-then satz -cutoff 20 -restart 100
-then walksat -cutoff 1000000 -restart 10
21
blackbox version 9Bcommand line: blackbox -o logistics.pddl -f logistics_prob_d_len.pddl -solver compact -l -then satz -cutoff 25 -restart 10
----------------------------------------------------Converting graph to wff6151 variables243652 clausesInvoking simplifier compactVariables undetermined: 4633Non-unary clauses output: 139866----------------------------------------------------Invoking solver satz version satz-rand-2.1Wff loaded[1] begin restart[1] reached cutoff 25 --- back to root[2] begin restart[2] reached cutoff 25 --- back to root [3] begin restart[3] reached cutoff 25 --- back to root [4] begin restart[4] reached cutoff 25 --- back to root [5] begin restart**** the instance is satisfiable ********* verification of solution is OK ****
total elapsed seconds = 25.930000----------------------------------------------------Begin plan1 drive-truck_ny-truck_ny-central_ny-po_ny
22
Begin plan1 drive-truck_ny-truck_ny-central_ny-po_ny1 drive-truck_sf-truck_sf-airport_sf-po_sf1 load-truck_package5_bos-truck_bos-po1 drive-truck_pgh-truck_pgh-airport_pgh-central_pgh1 fly-airplane_airplane2_pgh-airport_sf-airport1 load-truck_package6_bos-truck_bos-po2 load-truck_package2_pgh-truck_pgh-central2 load-truck_package4_ny-truck_ny-po2 load-truck_package7_ny-truck_ny-po2 load-truck_package3_pgh-truck_pgh-central2 drive-truck_bos-truck_bos-po_bos-airport_bos2 load-airplane_package8_airplane2_sf-airport2 fly-airplane_airplane1_pgh-airport_sf-airport2 drive-truck_la-truck_la-po_la-airport_la3 fly-airplane_airplane2_sf-airport_bos-airport3 unload-truck_package6_bos-truck_bos-airport3 drive-truck_pgh-truck_pgh-central_pgh-airport_pgh3 fly-airplane_airplane1_sf-airport_pgh-airport3 unload-truck_package5_bos-truck_bos-airport3 drive-truck_ny-truck_ny-po_ny-airport_ny3 drive-truck_sf-truck_sf-po_sf-airport_sf4 unload-truck_package3_pgh-truck_pgh-airport4 unload-truck_package2_pgh-truck_pgh-airport4 unload-truck_package4_ny-truck_ny-airport4 load-airplane_package6_airplane2_bos-airport4 load-airplane_package5_airplane2_bos-airport4 drive-truck_la-truck_la-airport_la-po_la4 drive-truck_bos-truck_bos-airport_bos-central_bos4 unload-truck_package7_ny-truck_ny-airport5 drive-truck_ny-truck_ny-airport_ny-po_ny5 drive-truck_bos-truck_bos-central_bos-po_bos5 load-airplane_package2_airplane1_pgh-airport5 drive-truck_la-truck_la-po_la-central_la5 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh5 load-airplane_package3_airplane1_pgh-airport5 fly-airplane_airplane2_bos-airport_ny-airport6 drive-truck_sf-truck_sf-airport_sf-central_sf6 unload-airplane_package6_airplane2_ny-airport6 load-airplane_package4_airplane2_ny-airport6 drive-truck_la-truck_la-central_la-po_la6 drive-truck_bos-truck_bos-po_bos-airport_bos6 load-airplane_package7_airplane2_ny-airport6 drive-truck_ny-truck_ny-po_ny-airport_ny6 unload-airplane_package8_airplane2_ny-airport6 fly-airplane_airplane1_pgh-airport_sf-airport6 load-truck_package1_pgh-truck_pgh-po7 fly-airplane_airplane2_ny-airport_la-airport7 fly-airplane_airplane1_sf-airport_bos-airport7 load-truck_package9_sf-truck_sf-central7 load-truck_package6_ny-truck_ny-airport
7 drive-truck_bos-truck_bos-airport_bos-central_bos7 drive-truck_pgh-truck_pgh-po_pgh-airport_pgh7 load-truck_package8_ny-truck_ny-airport8 drive-truck_sf-truck_sf-central_sf-po_sf8 fly-airplane_airplane2_la-airport_pgh-airport8 unload-truck_package1_pgh-truck_pgh-airport8 drive-truck_bos-truck_bos-central_bos-po_bos8 drive-truck_ny-truck_ny-airport_ny-central_ny8 fly-airplane_airplane1_bos-airport_la-airport8 drive-truck_la-truck_la-po_la-airport_la9 unload-airplane_package7_airplane2_pgh-airport9 unload-truck_package8_ny-truck_ny-central9 unload-airplane_package5_airplane2_pgh-airport9 unload-truck_package9_sf-truck_sf-po9 unload-airplane_package3_airplane1_la-airport9 unload-truck_package6_ny-truck_ny-central9 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh9 load-airplane_package1_airplane2_pgh-airport10 drive-truck_ny-truck_ny-central_ny-po_ny10 fly-airplane_airplane2_pgh-airport_bos-airport10 load-truck_package3_la-truck_la-airport10 fly-airplane_airplane1_la-airport_ny-airport10 drive-truck_pgh-truck_pgh-po_pgh-airport_pgh11 drive-truck_bos-truck_bos-po_bos-airport_bos11 drive-truck_ny-truck_ny-po_ny-airport_ny11 unload-airplane_package2_airplane1_ny-airport11 drive-truck_la-truck_la-airport_la-central_la11 drive-truck_sf-truck_sf-po_sf-airport_sf11 unload-airplane_package1_airplane2_bos-airport11 load-truck_package7_pgh-truck_pgh-airport11 load-truck_package5_pgh-truck_pgh-airport12 drive-truck_sf-truck_sf-airport_sf-po_sf12 load-truck_package1_bos-truck_bos-airport12 fly-airplane_airplane2_bos-airport_la-airport12 load-truck_package2_ny-truck_ny-airport12 fly-airplane_airplane1_ny-airport_pgh-airport12 drive-truck_pgh-truck_pgh-airport_pgh-po_pgh12 unload-truck_package3_la-truck_la-central13 drive-truck_ny-truck_ny-airport_ny-po_ny13 load-truck_package3_la-truck_la-central13 load-truck_package9_sf-truck_sf-po13 drive-truck_bos-truck_bos-airport_bos-po_bos13 unload-truck_package5_pgh-truck_pgh-po13 unload-airplane_package4_airplane2_la-airport14 unload-truck_package9_sf-truck_sf-po14 unload-truck_package1_bos-truck_bos-po14 unload-truck_package7_pgh-truck_pgh-po14 unload-truck_package2_ny-truck_ny-po14 unload-truck_package3_la-truck_la-centralEnd plan
23
Blackbox ResultsBlackbox Results
0.01
0.1
1
10
100
1000
10000
rocket.a rocket.b log.a log.b log.c log.d
Graphplan
BB-walksat
BB-rand-sys
Handcoded-walksat
1016 states6,000 variables125,000 clauses
24
AI Planning Systems CompetitionCMU, 1998
AI Planning Systems CompetitionCMU, 1998
Team Number of Average FastestShortest
problems solution onsolutions
solved time (msec) for
Blackbox 10 3171 3 6
(AT&T Labs)
HSP 9 25875 1 5
(Venezuela)
IPP 8 (11) 11036 1(3) 6(8)
(Germany)
STAN 7 20947 5 4
(UK)
25
NotesNotes
All finalists based on SATPLAN, Graphplan, or A* !
• Traditional non-linear planning no longer competitive
• Knowledge-intensive approaches require too much human effort
Other new techniques• Type-theoretic analysis of operators: can infer
state invariants (package only in one vehicle, etc.)– powerful, generally applicable pre-processor
• Compilation of more expressive languages (conditional effects) to STRIPS
• Recent extensions to MDP’s of A* (Geffner), Graphplan (Blum), SATPLAN (Littman)
26
SummarySummary
Blackbox combines best features of Graphplan, SATPLAN, and new randomized systematic search engines
Automatic generation of wffs from standard STRIPS input
• No performance penalty over hand-encodings!
Testbed for bridging different planning paradigms
27
Current Research IssuesCurrent Research Issues
Incorporating explicit domain knowledge (Kautz & Selman, 1998)
• state invariants
• optimality conditions
• declarative constraints - independent of search engine
More expressive planning languages: optimizing resources
• can view bounded integer linear programming as generalization of SAT
• ILPPLAN - adapts SATPLAN framework to ILP, solve with WSAT(OIP) (local search for ILP)
• Initial results - can find better quality solutions (counting action costs) than previously known for benchmark logistics & scheduling problems(Kautz & Walser 1999)