satisfiability and state- transition systems: an ai perspective henry kautz university of washington

Satisfiability and State-Transition Systems: An AI

Perspective

Satisfiability and State-Transition Systems: An AI

Perspective

Henry Kautz

University of Washington

IntroductionIntroduction

Both the AI and CADE/CAV communities have long been concerned with reasoning about state-transition systems

• AI – Planning

• CADE/CAV – Hardware and software verification

Recently propositional satisfiability testing has turned out to be surprisingly powerful tool

• Planning – SATPLAN (Kautz & Selman)

• Verification – Bounded model checking (Clarke), Debugging relational specifications (Jackson)

Shift in KR&R Shift in KR&R

Traditional approach: specialized languages / specialized reasoning algorithms

New direction: • Compile combinatorial reasoning problems into a

common propositional form (SAT)

• Apply new, highly efficient general search engines

Combinatorial Task

SAT Encoding SAT Solver

Decoder

AdvantagesAdvantages

Rapid evolution of fast solvers• 1990: 100 variable hard SAT problems

• 2000: 100,000 variables

Sharing of algorithms and implementations from different fields of computer science

AI, theory, CAD, OR, CADE, CAV, …

Competitions - Germany 91 / China 96 / DIMACS-93/97/98

JAR Special Issues – SAT 2000

RISC vs CISC

Can compile control knowledge into encodings

OUTLINEOUTLINE

1. Planning Model Checking

2. Planning as Satisfiability

3. SAT + Petri Nets + Randomization = Blackbox

4. State of the Art

5. Using Domain-Specific Control Knowledge

6. Learning Domain-Specific Control Knowledge

GOAL: Overview of recent advances in planning that may (or may not!) be relevant to the CADE community!

1. Planning Model Checking1. Planning Model Checking

The AI Planning ProblemThe AI Planning Problem

Given a world description, set of primitive actions, and goal description (utility function), synthesize a control program to achieve those goals (maximize utility)

most general case covers huge area of computer science, OR, economics

program synthesis, control theory, decision theory, optimization …

STRIPS Style PlanningSTRIPS Style Planning

“Classic” work in AI has concentrated on STRIPS style planning (“state space”)

• Open loop – no sensing• Deterministic actions• Sequential (straight line) plans• SHAKEY THE ROBOT (Fikes & Nilsson 1971)

Terminology• Fluent – a time varying proposition, e.g. “on(A,B)”• State – complete truth assignment to a set of fluents• Goal – partial truth assignment (set of states)• Action – a partial function State State

specified by Operator schemas

Operator SchemasOperator Schemas

Each yields set of primitive actions, when instantiated over a given finite set of objects (constants)

Pickup(x, y)• precondition: on(x,y), clear(x), handempty

• delete: on(x,y), clear(x), handempty

• add: holding(x), clear(y)

Plan: A (shortest) sequence of actions that transforms the initial state into a goal state

• E.g.: Pickup(A,B); Putdown(A,C)

ParallelismParallelism

Useful extension: parallel composition of primitive actions

• Only allowed when all orderings are well defined and equivalent – no shared pre / effects

(act1 || act2)(s) = act2(act1(s)) = act1(act2(s))

• Can dramatically reduce size of search space

• Easy to serialize

• Distinguish:– number of actions in a plan – “sequential length”

– number of sequentially composition operators in a plan – “parallel length”, “horizon”

(a1 || a2); (a3 || a4 || a5) ; a6

- sequential length 6, parallel length 3

Some Applications of STRIPS-Style Planning

Some Applications of STRIPS-Style Planning

Autonomous systems• Deep Space One Remote Agent (Williams & Nayak 1997)

Natural language understanding• TRAINS (Allen 1998)

Internet agents• Rodney (Etzioni 1994)

Manufacturing• Supply chain management (Crawford 1998)

Abdundance of Negative Complexity Results

Abdundance of Negative Complexity Results

Unbounded STRIPS planning: PSPACE-complete• Exponentially long solutions

(Bylander 1991; Backstrom 1993)

Bounded STRIPS planning: NP-complete• Is there a solution of (sequential/parallel) length N?

(Chenoweth 1991; Gupta and Nau 1992)

Domain-specific planning: may depend on whether solutions must be the shortest such plan

• Blocks world –– Shortest plan – NP-hard– Approximately shortest plan – NP-hard

(Selman 1994)

– Plan of length 2 x number blocks – Linear time

Approaches to AI PlanningApproaches to AI Planning

Three main paradigms:• Forward-chaining heuristic search over state space

– original STRIPS system

– recent resurgence – TLPlan, FF, …

• “Causal link” Planning– search in “plan space”

– Much work in 1990’s (UCPOP, NONLIN, …), little now

• Constraint based planning– view planning as solving a large set of constraints

– constraints specify relationships between actions and their preconditions / effects

– SATPLAN (Kautz & Selman), Graphplan (Blum & Furst)

Relationship to Model CheckingRelationship to Model Checking

Model checking – determine whether a formula in temporal logic evaluates to “true” in a Kripke structure described by a finite state machine

• FSM may be represented explicitly or symbolically

STRIPS planning – special case where• Finite state matchine (transition relation) specified

by STRIPS operators– Very compact

– Expressive – can translate many other representations of FSM’s into STRIPS with little or no blowup

Relationship, continuedRelationship, continued

• Formula to be checked is of the form“exists path . eventually . GOAL”

– Reachability

– Distinctions between linear / branching temporal logics not important

Difference:• Concentration on finding shortest plans

• Emphasis on efficiently finding single witness (plan) as opposed to verifying a property holds in all states

– NP vs co-NP

Why Not Use OBDD’s?Why Not Use OBDD’s?

Size of OBDD explodes for typical AI benchmark domains

• Overkill – need not / cannot check all states, even if they are represented symbolically!

O(2n2) states

(But see recent work by M. Velosa on using OBDD’s for non-deterministic variant of STRIPS)

Verification using SATVerification using SAT

Similar phenomena occur in some verification domains

• Hardware multipliers

Has led to interest in using SAT techniques for verification and bug finding

• Bounded – fixed horizon

• Under certain conditions can prove that only considering a fixed horizon is adequate

– Empirically, most bugs found with small bounds

• E. Clarke – Bounded Model Checking– LTL specifications, FSM in SMV language

• D. Jackson – Nitpick– Debugging relational specifications in Z

2. Planning as Satisfiability2. Planning as Satisfiability

Planning as SatisfiabilityPlanning as Satisfiability

SAT encodings are designed so that plans correspond to satisfying assignments

Use recent efficient satisfiability procedures (systematic and stochastic) to solve

Evaluation performance on benchmark instances

SATPLANSATPLAN

axiomschemas instantiated

propositionalclauses

satisfyingmodelplan

length

problemdescription

SATengine(s)

instantiate

interpret

SAT EncodingsSAT Encodings

Target: Propositional conjunctive normal form

Sets of clauses specified by axiom schemas1. Create model by hand

2. Compile STRIPS operators

Discrete time, modeled by integers• upper bound on number of time steps

• predicates indexed by time at which fluent holds / action begins– each action takes 1 time step

– many actions may occur at the same step

fly(Plane, City1, City2, i) at(Plane, City2, i +1)

Solution to a Planning ProblemSolution to a Planning Problem

A solution is specified by any model (satisfying truth assignment) of the conjunction of the axioms describing the initial state, goal state, and operators

Easy to convert back to a STRIPS-style plan

Complete SAT AlgorithmsComplete SAT Algorithms

Davis-Putnam-Loveland-Logeman (DPLL)• Depth-first backtrack search on partial truth assignments

• Basis of nearly all practical complete SAT algorithms– Exception: “Stahlmark’s method”

• Key to efficiency: good variable choice at branch points

– 1961 – unit propagation, pure literal rule

– 1993 - explosion of improved heuristics and implementations

+ MOM’s heuristic

+ satz (Chu Min Li) – lookhead to maximize rate of creation of binary clauses

• Dependency directed backtracking – derive new clauses during search – rel_sat (Bayardo), GRASP (di Silva)

– See SATLIB 1998 / Hoos & Stutzle

Incomplete SAT AlgorithmsIncomplete SAT Algorithms

GSAT and Walksat (Kautz, Selman & Cohen 1993)

• Randomized local search over space of complete truth assignments

• Heuristic function: flip variables to minimize number of unsatisfied clauses

• Noisy “random walk” moves to escape local minima

• Provably solves 2CNF, empirically successful on a broad class of problems

– random CNF, graph coloring, circuit synthesis encodings (DIMACS 1993, 1997)

Planning Benchmark Test SetPlanning Benchmark Test Set

Extension of Graphplan benchmark set

logistics - transportation domain, ranging up to• 14 time slots, unlimited parallelism

• 2,165 possible actions per time slot

• optimal solutions containing 74 primitive actions

• 22000 legal states (60,000 Boolean variables)

Problems of this size not previously handled by any domain-independent planning system

Initial SATPLAN ResultsInitial SATPLAN Results

problem horizon / actions

Graphplan naïve SAT encoding

hand SAT encoding

rocket-b 7 / 30 9 min 16 min 41 sec

log-a 11 / 47 13 min 58 min 1.2 min

log-b 13 / 54 32 min * 1.3 min

log-c 13 / 63 * * 1.7 min

log-d 14 / 74 * * 3.5 min

SAT solver: Walksat (local search)

* indicates no solution found after 24 hours

How SATPLAN Spent its TimeHow SATPLAN Spent its Time

problem instantiation walksat DPLL satz

rocket-b 41 sec 0.04 sec 1.8 sec 0.3 sec

log-a 1.2 min 2.2 sec * 1.7 min

log-b 1.3 min 3.4 sec * 0.6 sec

log-c 1.7 min 2.1 sec * 4.3 sec

log-d 3.5 min 7.2 sec * 1.8 hours

Hand created SAT encodings


Automating EncodingsAutomating Encodings

While SATPLAN proved the feasibility of planning using satisfiability, modeling the transition function was problematic

• Direct naïve encoding of STRIPS operators as axiom schemas gave poor performance

• Handcrafted encodings gave good performance, but were labor intensive to create

– similar issues arise in work in verification – division of labor between user and model checker!

GOAL: fully automatic generation and solution of planning problems from STRIPS specifications

GraphplanGraphplan

Graphplan (Blum & Furst 1995)

Set new paradigm for planning

Like SATPLAN...• Two phases: instantiation of propositional structure,

followed by search

Unlike SATPLAN...• Efficient instantiation algorithm based on Petri-net

type reachability analysis

• Employs specialized search engine

Neither approach best for all domains• Can we combine advantages of both?

BlackboxBlackbox

STRIPSPlan Graph

Petri Net Analysis

CNF

GeneralSAT engines

Solution

SimplifierTranslator

CNF

Component 1: Petri-Net AnalysisComponent 1: Petri-Net Analysis

Graphplan instantiates a “plan graph” in a forward direction, pruning (some) unreachable nodes• plan graph unfolded Petri net (McMillian 1992)

Polynomial-time propagation of mutual-exclusion relationships between nodes• Incomplete – must be followed by search to

determine if all goals can be simultaneously reached

Growing the Plan GraphGrowing the Plan Graph

P0

facts factsactions actions


P0 A1

B1

P2

R2


Q2


P0 A1

B1

P2

R2

C3


Q2


P0 A1

B1

P2

R2


Q2

Component 2: TranslationComponent 2: Translation

P0 A1

B1

P2

R2


Q2

Action implies preconditions: A1 P0 , B1 P0

Mutual exclusion: A1 B1 , P2 Q2

Initial facts hold at time 0

Goals holds at time n

Component 3: SimplificationComponent 3: Simplification

Generated wff can be further simplified by more general consistency propagation techniques

• unit propagation: is Wff inconsistant by resolution against unit clauses?

O(n)

• failed literal rule: is Wff + { P } inconsistant by unit propagation?

O(n2)

• binary failed literal rule: is Wff + { P V Q } inconsistant by unit propagation?

O(n3)

General simplification techniques complement Petri net analysis

Effective of SimplificationEffective of Simplification

Percent vars set byProblem Varsunitprop

failedlit

binaryfailed

bw.a 2452 10% 100% 100%bw.b 6358 5% 43% 99%bw.c 19158 2% 33% 99%log.a 2709 2% 36% 45%log.b 3287 2% 24% 30%log.c 4197 2% 23% 27%log.d 6151 1% 25% 33%

Component 3: Randomized Systematic Solvers

Component 3: Randomized Systematic Solvers

BackgroundBackground

Combinatorial search methods often exhibit

a remarkable variability in performance. It is

common to observe significant differences

between:• different heuristics

• same heuristic on different instances

• different runs of same heuristic with different random seeds

How SATPLAN Spent its TimeHow SATPLAN Spent its Time

problem instantiation walksat DPLL satz

rocket-b 41 sec 0.04 sec 1.8 sec 0.3 sec

log-a 1.2 min 2.2 sec * 1.7 min

log-b 1.3 min 3.4 sec * 0.6 sec

log-c 1.7 min 2.1 sec * 4.3 sec

log-d 3.5 min 7.2 sec * 1.8 hours



Preview of StrategyPreview of Strategy

We’ll put variability / unpredictability to our advantage via randomization / averaging.

Cost DistributionsCost Distributions

Consider distribution of running times of backtrack search on a large set of “equivalent” problem instances

• renumber variables

• change random seed used to break ties

Observation (Gomes 1996): distributions often have heavy tails

• infinite variance

• mean increases without limit

• probability of long runs decays by power law (Pareto-Levy), rather than exponentially (Normal)

Heavy TailsHeavy Tails

Bad scaling of systematic solvers can be caused by heavy tailed distributions

Deterministic algorithms get stuck on particular instances

• but that same instance might be easy for a different deterministic algorithm!

• Expected (mean) solution time increases without limit over large distributions

• Log-log plot of distribution of running times approximately linear

Heavy-Tailed DistributionsHeavy-Tailed Distributions

… … infinite variance … infinite meaninfinite variance … infinite mean

Introduced by Pareto in the 1920’s “probabilistic curiosity”

Mandelbrot established the use of heavy-tailed distributions to model real-world fractal phenomena

• stock-market, Internet traffic delays, weather

New discovery: good model for backtrack search algorithms

• formal statement of “folk wisdom” of theorem proving community

Randomized RestartsRandomized Restarts

Solution: randomize the systematic solver• Add noise to the heuristic branching (variable choice)

function

• Cutoff and restart search after a fixed number of backtracks

Provably Eliminates heavy tails

In practice: rapid restarts with low cutoff can dramatically improve performance

(Gomes, Kautz, and Selman 1997, 1998)• Related analysis: Luby & Zuckerman 1993; Alt & Karp 1996

Rapid Restart on LOG.DRapid Restart on LOG.D

1000

10000

100000

1000000

1 10 100 1000 10000 100000 1000000

log( cutoff )

log

( b

ackt

rack

s )

Note Log Scale: Exponential speedup!

Overall insight:Overall insight:

Randomized tie-breaking with

rapid restarts can boost

systematic search algorithms

• Speed-up demonstrated in many versions of Davis-Putnam

– basic DPLL, satz, rel_sat, …

• Related analysis: Luby & Zuckerman 1993; Alt & Karp 1996

Blackbox ResultsBlackbox Results

problem naïve SAT encoding

hand SAT encoding

blackbox

walksat

blackbox

satz-rand

rocket-b 16 min 41 sec 2.5 sec 4.9 sec

log-a 58 min 1.2 min 7.4 sec 5.2 sec

log-b * 1.3 min 1.7 min 7.1 sec

log-c * 1.7 min 15 min 9.3 sec

log-d * 3.5 min * 52 sec

Naïve/Hand SAT solver: Walksat (local search)


4. State of the Art4. State of the Art

Which Strategies Work Best?Which Strategies Work Best?

Causal-link planning• <5 primitive actions in solutions

• Works best if few interactions between goals

Constraint-based planning• Graphplan, SATPLAN, + descendents

• 100+ primitive actions in solutions

• Moderate time horizon <30 time steps

• Handles interacting goals well

1995 – 1999 Constraint-based approaches dominate

• AIPS 1996, AIPS 1998

Graph Search vs. SATGraph Search vs. SAT

SATPLAN

Graphplan

Problem size / complexity

Tim

e

Caveat: on some domains SAT approach can exhaust memory even though direct graph search is easy

Blackbox withsolver schedule

Resurgence of A* SearchResurgence of A* Search

In most of 1980 – 1990’s forward chaining A* search was considered a non-starter for planning

Voices in the wilderness:

• TLPlan (Bacchus) – hand-tuned heuristic function could make approach feasible

• LRTA (Geffner) – can automatically derive good heuristic functions

Surprise – AIPS-2000 planning competition dominated by A* planners!

• What happened?

Solution Length vs HardnessSolution Length vs Hardness

Key issue: relationship between solution length and problem hardness

• RECALL: In many domains, finding solutions that minimize the number of time steps is NP-hard, while finding an arbitrary solution is in P

– Put all the blocks on the table first

– Deliver packages one at a time

• Long solutions minimize goal interactions, so little or no backtracking required by forward-chaining search

• AIPS-2000 Planning Competition did not consider plan length criteria!

Non-Optimal PlanningNon-Optimal Planning

0.01

0.1

1

10

100

1000

10000

100000

easy rocket-a rocket-b

blackbox

hsp

ff

Optimal-Length PlanningOptimal-Length Planning

0.01

0.1

1

10

100

1000

10000

100000

easy rocket-a rocket-b

blackbox

hsp

ff

Which Works Best, ContinuedWhich Works Best, Continued

Constraint-based planning• Short parallel solutions desired• Many interactions between goals• SAT translation a win for larger problems where

time is dominated by search (as opposed to instantiation and Petri net analysis)

Forward-chaining search• Long sequential solutions okay• Few interactions between goals

Much recent progress in domain-independent planning…

but further scaling to large real-world problems requires domain-dependent techniques!

Kinds of Domain-Specific Knowledge

Kinds of Domain-Specific Knowledge

Invariants true in every state• A truck is only in one location

Implicit constraints on optimal plans• Do not remove a package from its destination location

Simplifying assumptions• Do not unload a package from an airplane, if the

airplane is not at the package’s destination city– eliminates connecting flights

Expressing KnowledgeExpressing Knowledge

Such information is traditionally incorporated in the planning algorithm itself

Instead: use additional declarative axioms(Bacchus 1995; Kautz 1998; Huang, Kautz, & Selman 1999)

• Problem instance: operator axioms + initial and goal axioms + control axioms

• Control knowledge constraints on search and solution spaces

• Independent of any search engine strategy

Axiomatic FormAxiomatic Form

State Invariant:

at(truck,loc1,i) & loc1 loc2 at(truck,loc2,i)

Optimality:

at(pkg,loc,i) & at(pkg,loc,i+1) & i<j at(pkg,loc,j)

Simplifying Assumption

incity(airport,city) & at(pkg,loc,goal) & incity(airport,city)

unload(pkg,plane,airport)

Adding Control KnowledgeAdding Control Knowledge

ProblemSpecification

Axioms

Domain-specific Control Axioms

Instantiated Clauses

SAT Simplifier

SAT Engine

SAT “Core”

As control knowledge increases, Core shrinks!

Effect of Domain KnowledgeEffect of Domain Knowledge

problem walksat walksat +

Kx

DPLL DPLL +

Kx

rocket-b 0.04 sec 0.04 sec 1.8 sec 0.13 sec

log-a 2.2 sec 0.11 sec * 1.8 min

log-b 3.4 sec 0.08 sec * 11 sec

log-c 2.1 sec 0.12 sec * 7.8 min

log-d 7.2 sec 1.1 sec * *



Learning Control RulesLearning Control Rules

Axiomatizing domain-specific control knowledge by hand is a time consuming art…

• Certain kinds of knowledge can be efficiently deduced

– simple classes of invariants (Fox & Long; Gerevini & Schubert)

• Can more powerful control knowledge be automatically learned, by watching planner solve small instances?

Form of RulesForm of Rules

We will learn two kinds of control rules, specified as temporal logic programs

– (Huang, Selman, & Kautz 2000)

• Select rule: conditions under which an action must be performed at the current time instance

• Reject rule: conditions under which an action must not be performed at the current time instance

incity(airport,city) & GOAL(at(pkg,loc)) &incity(airport,city)

unload(pkg,plane,airport)

Training ExamplesTraining Examples

Blackbox initially solves a few small problem instances

Each instance yields• POSITIVE training examples – states at which

actions occur in the solution

• NEGATIVE training examples – states at which an action does NOT occur, even though its preconditions hold in that state

Note that this data is very noisy!

Rule InductionRule Induction

Rules are induced using a version of Quinlan’s FOIL inductive logic programming algorithm

• Generates rules one literal at time

• Select rules: maximize coverage of positive examples, but do not cover negative examples

• Reject rules: maximize coverage of negative examples, but do not cover positive examples

• Prune rules that are inconsistent with any of the problem instances

– For details, see “Learning Declarative Control Rules for Constraint-Based Planning”, Huang, Selman, & Kautz, ICML 2000

Logical Status of Induced RulesLogical Status of Induced Rules

Some of the learned rules could in principle be deduced from the domain operators together with a bound on the length on the plan

• Reject rules for unnecessary actions

But in general: rules are not deductive consequences

• Could rule out some feasible solutions

• In worst case: could rule out all solutions to some instances

– not a problem in practice: such rules are usually quickly pruned in the training phase

Effect of LearningEffect of Learning

problem horizon blackbox learning

blackbox

grid-a 13 21 4.8

grid-b 18 74 16.6

gripper-3 15 >7200 7.2

gripper-4 19 >7200 260

log-d 14 15.8 5.7

log-e 15 3522 291

mystery-10

8 >7200 47.2

mystery-13

8 161 12.2

AIPS-98 competition benchmarks

SummarySummary

• Close connections between much work in AI Planning and CADE/CAV work on model checking

• Remarkable recent success of general satisfiability testing programs on hard benchmark problems

• Success of Blackbox and Graphplan in combining ideas from planning and verification suggest many more synergies exist

• Techniques for learning and applying domain specific control knowledge dramatically boost performance for planning – could ideas also be applied to verification?

satisfiability and state- transition systems: an ai perspective henry kautz university of washington

Documents