sat solving

8/11/2019 SAT Solving

1/95

Satisability:Algorithms, Applications and Extensions

Javier Larrosa1 Ines Lynce 2

Joao Marques-Silva3

1 Universitat Politecnica de Catalunya, Spain

2 Technical University of Lisbon, Portugal

3 University College Dublin, Ireland

SAC 2010
http://find/


2/95

SAT: A Simple Example

Boolean Satisability (SAT) in a short sentence:

SAT is the problem of deciding (requires a yes/no answer) if there is an assignment to the variables of a Boolean formulasuch that the formula is satised

Consider the formula (a b ) (a c ) The assignment b = True and c = False satises the formula!
http://find/


3/95

SAT: A Practical Example

Consider the following constraints: John can only meet either on Monday, Wednesday or Thursday Catherine cannot meet on Wednesday Anne cannot meet on Friday Peter cannot meet neither on Tuesday nor on Thursday Question: When can the meeting take place?

Encode then into the following Boolean formula:(Mon Wed Thu ) (Wed ) (Fri ) (Tue Thu )

The meeting must take place on Monday
http://find/


4/95

Outline

Motivation

What is Boolean Satisability?

SAT Algorithms

Incomplete AlgorithmsLocal Search

Complete AlgorithmsBasic RulesResolutionStalmarcks MethodRecursive LearningBacktrack Search (DPLL)Conict-Driven Clause Learning (CDCL)
http://find/http://goback/


5/95

Outline
http://find/


6/95

Outline

Motivation


SAT Algorithms


http://find/


7/95

Motivation - Why SAT?

Boolean Satisability (SAT) has seen signicantimprovements in recent years

At the beginning is was simply the rst known NP-completeproblem [Stephen Cook, 1971]

After that mostly theoretical contributions followed In the 90s practical algorithms were developed and made

available Currently, SAT founds many practical applications

SAT extensions found even more applications
http://find/


8/95

Motivation - Some lessons from SAT I

Time is everything Good ideas are not enough, you have to be fast!

One thing is the algorithm, another thing is the implementation Make your source code available Otherwise people will have to wait for years before realising

what you have done At least provide an executable!
http://find/


9/95

Motivation - Some lessons from SAT II

Competitions are essential

To check the state-of-the-art of SAT solvers To keep the community alive (for almost a decade now) To get students involved

Part of the credibility of a community comes from thecorrectness and robustness of the tools made available
http://find/


10/95

Motivation - Some lessons from SAT III

There is no perfect solver! Do not expect your solver to beat all the other solvers on all

problem instances What makes a good solver? Correctness and robustness for sure... Being most often the best for its category: industrial,

handmade or random

Being able to solve instances from different problems
http://find/


11/95

www.satcompetition.org

Get all the info from the SAT competition web page

Organizers, judges, benchmarks, executables, source code Winners Industrial, Handmade and Random benchmarks SAT+UNSAT, SAT and UNSAT categories Gold, Silver and Bronze medals
http://find/


12/95

Outline

Motivation


SAT Algorithms


http://find/


13/95

Boolean Formulas

Boolean formula is dened over a set of propositionalvariables x 1 , . . . , x n , using the standard propositionalconnectives , , , , , and parenthesis

The domain of propositional variables is {0, 1}

Example: (x 1 , . . . , x 3) = (( x 1 x 2 ) x 3) (x 2 x 3 ) A formula in conjunctive normal form (CNF) is a

conjunction of disjunctions (clauses) of literals, where a literalis a variable or its complement

Example: (x 1 , . . . , x 3) = ( x 1 x 3 ) (x 2 x 3 ) (x 2 x 3)

Can encode any Boolean formula into CNF (more later)
http://find/


14/95

Boolean Satisability (SAT)

The Boolean satisability (SAT) problem: Find an assignment to the variables x 1 , . . . , x n such that(x 1 , . . . , x n ) = 1, or prove that no such assignment exists

SAT is an NP-complete decision problem [Cook71] SAT was the rst problem to be shown NP-complete There are no known polynomial time algorithms for SAT 39-year old conjecture:

Any algorithm that solves SAT is exponential in the number of variables, in the worst-case


15/95

Denitions

Propositional variables can be assigned value 0 or 1

In some contexts variables may be unassigned A clause is satised if at least one of its literals is assigned

value 1(x 1 x 2 x 3 )

A clause is unsatised if all of its literals are assigned value 0(x 1 x 2 x 3 )

A clause is unit if it contains one single unassigned literal andall other literals are assigned value 0

(x 1 x 2 x 3 ) A formula is satised if all of its clauses are satised A formula is unsatised if at least one of its clauses is

unsatised
http://find/


16/95

Outline

Motivation


SAT Algorithms


http://find/


17/95

Algorithms for SAT

Incomplete algorithms (i.e. can only prove (un)satisability):

Local search / hill-climbing Genetic algorithms Simulated annealing ...

Complete algorithms (i.e. can prove both satisability and

unsatisability): Proof system(s)

Natural deduction Resolution Stalmarcks method Recursive learning ...

Binary Decision Diagrams (BDDs) Backtrack search / DPLL

Conict-Driven Clause Learning (CDCL) ...

l
http://find/


18/95

Outline

Motivation


SAT Algorithms



O li
http://find/


19/95

Outline

Motivation


SAT Algorithms



O i i f L l S h


20/95

Organization of Local Search

Local search is incomplete; usually it cannot prove

unsatisability Very effective in specic contexts

Example:

(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

O i ti f L l S h
http://find/


21/95




Example:

(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4) Start with (possibly random) assignment:

x 4 = 0 , x 1 = x 2 = x 3 = 1 And repeat a number of times:

O g i ti f L l S h
http://find/


22/95




Example:



http://find/


23/95




Example:



If not all clauses satised, ip variable (e.g. x 4)

http://find/


24/95




Example:



If not all clauses satised, ip variable (e.g. x 4)

http://find/


25/95




Example:



If not all clauses satised, ip variable (e.g. x 4) Done if all clauses satised

http://find/


26/95




Example:



If not all clauses satised, ip variable (e.g. x 4) Done if all clauses satised

Repeat (random) selection of assignment a number of times

Outline
http://find/


27/95

Outline

Motivation


SAT Algorithms

Incomplete Algorithms

Local Search


Outline
http://find/


28/95

Outline

Motivation


SAT Algorithms


Local Search

Complete AlgorithmsBasic RulesResolution

Stalmarcks MethodRecursive LearningBacktrack Search (DPLL)Conict-Driven Clause Learning (CDCL)

Pure Literals
http://find/


29/95

Pure Literals

A literal is pure if only occurs as a positive literal or as a

negative literal in a CNF formula Example: = ( x 1 x 2) (x 3 x 2) (x 4 x 5) (x 5 x 4 )

x 1 and x 3 and pure literals

Pure literal rule:Clauses containing pure literals can be removed from theformula (i.e. just assign pure literals to the values that satisfythe clauses)

For the example above, the resulting formula becomes: = ( x

4 x

5)

(x 5

x 4)

A reference technique until the mid 90s; nowadays seldomused

Unit Propagation
http://find/


30/95

Unit Propagation

Unit clause rule:Given a unit clause, its only unassigned literal must beassigned value 1 for the clause to be satised

Example: for unit clause (x 1 x 2 x 3), x 3 must be assignedvalue 0

Unit propagationIterated application of the unit clause rule

(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

Unit Propagation
http://find/


31/95

Unit Propagation




(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

Unit Propagation
http://find/


32/95

Unit Propagation




(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

Unit Propagation
http://find/


33/95

p g




(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

Unit Propagation
http://find/


34/95

p g




(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

Unit Propagation
http://find/


35/95

p g




(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

Unit Propagation
http://find/


36/95

p g




(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

(x 1 x 2 x 3) (x 1 x 3 x 4) (x 1 x 2 x 4)

Unit propagation can satisfy clauses but can also unsatisfyclauses. Unsatised clauses create conicts.

Outline
http://find/


37/95

Motivation


SAT Algorithms


Local SearchComplete Algorithms

Basic RulesResolution


Resolution
http://find/


38/95

Resolution rule: If a formula contains clauses (x ) and (x ), then one

can infer ( )

(x ) (x ) ( )

Resolution is a sound and complete rule

Resolution
http://find/


39/95

Resolution forms the basis of a complete algorithm for SAT Iteratively apply the following steps: [Davis&Putnam60]

Select variable x Apply resolution rule between every pair of clauses of the form

( x ) and ( x ) Remove all clauses containing either x or x Apply the pure literal rule and unit propagation

Terminate when either the empty clause or the empty formula(equivalently, a formula containing only pure literals) is derived

Resolution An Example
http://find/


40/95

(x 1 x 2 x 3) (x 1 x 2 x 3) (x 2 x 3) (x 3 x 4 ) (x 3 x 4)

http://find/


41/95

(x 1 x 2 x 3) (x 1 x 2 x 3) (x 2 x 3) (x 3 x 4 ) (x 3 x 4)

(x 2 x 3) (x 2 x 3 ) (x 3 x 4) (x 3 x 4 )

http://find/


42/95

(x 1 x 2 x 3) (x 1 x 2 x 3) (x 2 x 3) (x 3 x 4 ) (x 3 x 4)

(x 2 x 3) (x 2 x 3 ) (x 3 x 4) (x 3 x 4 )

(x 3 x 3) (x 3 x 4) (x 3 x 4)

http://find/


43/95

(x 1 x 2 x 3) (x 1 x 2 x 3) (x 2 x 3) (x 3 x 4 ) (x 3 x 4)

(x 2 x 3) (x 2 x 3 ) (x 3 x 4) (x 3 x 4 )

(x 3 x 3) (x 3 x 4) (x 3 x 4) (x 3 x 4) (x 3 x 4)

http://find/


44/95

(x 1 x 2 x 3) (x 1 x 2 x 3) (x 2 x 3) (x 3 x 4 ) (x 3 x 4)

(x 2 x 3) (x 2 x 3 ) (x 3 x 4) (x 3 x 4 )

(x 3 x 3) (x 3 x 4) (x 3 x 4) (x 3 x 4) (x 3 x 4)

(x 3)

Formula is SAT

Outline
http://find/


45/95

Motivation


SAT Algorithms





Stalmarcks Method
http://find/


46/95

Recursive application of the branch-merge rule to each

variable with the goal of identifying common assignments = ( a b )(a c )(b d )(c d )

(a = 0 ) (b = 1) (d = 1)UP (a = 0) = {a = 0 , b = 1 , d = 1}

(a = 1 ) (c = 1) (d = 1)UP (a = 1) = {a = 1 , c = 1 , d = 1}

UP (a = 0 ) UP (a = 1 ) = {d = 1}

Any assignment to variable a implies d = 1.Hence, d = 1 is a necessary assignment!

Recursion can be of arbitrary depth

Outline
http://find/


47/95

Motivation


SAT Algorithms





Recursive Learning
http://find/


48/95

Recursive evaluation of clause satisability requirements for

identifying common assignments = (a b )(a c )(b d )(c d )

(a = 1 ) (c = 1) (d = 1)UP (a = 1) = {a = 1 , c = 1 , d = 1}

(b = 1 ) (d = 1)UP (b = 1) = {b = 1 , d = 1}

UP (a = 1 ) UP (b = 1 ) = {d = 1}

Every way of satisfying (a b ) implies d = 1.Hence, d = 1 is a necessary assignment!

Recursion can be of arbitrary depth

Outline
http://find/


49/95

Motivation


SAT Algorithms





Historical Perspective I
http://find/


50/95

In 1960, M. Davis and H. Putnam proposed the DPalgorithm:

Resolution used to eliminate 1 variable at each step Applied the pure literal rule and unit propagation

Original algorithm was inefficient

Historical Perspective II


51/95

In 1962, M. Davis, G. Logemann and D. Loveland proposedan alternative algorithm:

Instead of eliminating variables, the algorithm would split on agiven variable at each step

Also applied the pure literal rule and unit propagation The 1962 algorithm is actually an implementation of

backtrack search Over the years the 1962 algorithm became known as the

DPLL (sometimes DLL) algorithm

Basic Algorithm for SAT DPLL
http://find/


52/95

Standard backtrack search At each step:

[DECIDE] Select decision assignment [DEDUCE] Apply unit propagation and (optionally) the pure

literal rule [DIAGNOSIS] If conict identied, then backtrack If cannot backtrack further, return UNSAT Otherwise, proceed with unit propagation

If formula satised, return SAT

Otherwise, proceed with another decision

An Example of DPLL
http://find/


53/95

= ( a b d ) (a b e ) (b d e ) (a b c d ) (a b c d )

(a b c e ) (a b c e )

An Example of DPLL
http://find/


54/95


(a b c e ) (a b c e )

a

An Example of DPLL
http://find/


55/95


(a b c e ) (a b c e ) b

conflict

a

An Example of DPLL
http://find/


56/95


(a b c e ) (a b c e )

conflict

b

a

An Example of DPLL
http://find/


57/95


(a b c e ) (a b c e )

conflict

a

c

b

An Example of DPLL
http://find/


58/95


(a b c e ) (a b c e )

conflict

a

c

b

An Example of DPLL
http://find/


59/95


(a b c e ) (a b c e ) b

conflict

a

c

An Example of DPLL
http://find/


60/95


(a b c e ) (a b c e ) b

solution

a

c

b

conflict

Outline
http://find/


61/95

Motivation

What is Boolean Satisability?SAT Algorithms





CDCL SAT Solvers
http://find/


62/95

Introduced in the 90s

[Marques-Silva&Sakallah96][Bayardo&Schrag97] Inspired on DPLL Must be able to prove both satisability and unsatisability

New clauses are learnt from conicts Structure of conicts exploited (UIPs) Backtracking can be non-chronological Efficient data structures [Moskewicz&al01]

Compact and reduced maintenance overhead

Backtrack search is periodically restarted [Gomes&al98]

Can solve instances with hundreds of thousand variables andtens of million clauses

CDCL SAT Solvers
http://find/


63/95

Introduced in the 90s

[Marques-Silva&Sakallah96][Bayardo&Schrag97] Inspired on DPLL Must be able to prove both satisability and unsatisability

New clauses are learnt from conicts Structure of conicts exploited (UIPs) Backtracking can be non-chronological Efficient data structures

Compact and reduced maintenance overhead

Backtrack search is periodically restarted

Can solve instances with hundreds of thousand variables andtens of million clauses

Clause Learning
http://find/


64/95

During backtrack search, for each conict learn new clause,which explains and prevents repetition of the same conict

= ( a b ) (b c d ) (b e ) (d e f ) . . .

Clause Learning
http://find/


65/95


= ( a b ) (b c d ) (b e ) (d e f ) . . .

Assume decisions c = 0 and f = 0

Clause Learning
http://find/


66/95


= ( a b ) (b c d ) (b e ) (d e f ) . . .

Assume decisions c = 0 and f = 0 Assign a = 0 and imply assignments

Clause Learning
http://find/


67/95


= ( a b ) (b c d ) (b e ) (d e f ) . . .

Assume decisions c = 0 and f = 0 Assign a = 0 and imply assignments

Clause Learning


68/95


= ( a b ) (b c d ) (b e ) (d e f ) . . .

Assume decisions c = 0 and f = 0 Assign a = 0 and imply assignments A conict is reached: (d e f ) is unsatised

Clause Learning
http://find/


69/95


= ( a b ) (b c d ) (b e ) (d e f ) . . .

Assume decisions c = 0 and f = 0 Assign a = 0 and imply assignments A conict is reached: (d e f ) is unsatised (a = 0) (c = 0) (f = 0) ( = 0)

Clause Learning
http://goforward/http://find/http://goback/


70/95


= ( a b ) (b c d ) (b e ) (d e f ) . . .

Assume decisions c = 0 and f = 0 Assign a = 0 and imply assignments A conict is reached: (d e f ) is unsatised (a = 0) (c = 0) (f = 0) ( = 0) ( = 1) (a = 1) (c = 1) (f = 1)

Clause Learning
http://find/


71/95


= ( a b ) (b c d ) (b e ) (d e f ) . . .

Assume decisions c = 0 and f = 0 Assign a = 0 and imply assignments A conict is reached: (d e f ) is unsatised (a = 0) (c = 0) (f = 0) ( = 0) ( = 1) (a = 1) (c = 1) (f = 1)

Learn new clause (a c f )

Non-Chronological Backtracking
http://find/


72/95

During backtrack search, for each conict backtrack to one of

the causes of the conict

= ( a b ) (b c d ) (b e ) (d e f )(a c f ) (a g ) (g b ) (h j ) ( i k )

http://find/


73/95


the causes of the conict

= ( a b ) (b c d ) (b e ) (d e f )(a c f ) (a g ) (g b ) (h j ) ( i k )

Assume decisions c = 0 , f = 0 , h = 0 and i = 0

http://find/


74/95


the causes of the conict = ( a b ) (b c d ) (b e ) (d e f )

(a c f ) (a g ) (g b ) (h j ) ( i k )

Assume decisions c = 0 , f = 0 , h = 0 and i = 0 Assignment a = 0 caused conict learnt clause (a c f )

implies a = 1

http://find/


75/95



(a c f ) (a g ) (g b ) (h j ) ( i k )


implies a = 1

http://find/


76/95



(a c f ) (a g ) (g b ) (h j ) ( i k )


implies a = 1

http://find/


77/95



(a c f ) (a g ) (g b ) (h j ) ( i k )


implies a = 1 A conict is again reached: (d e f ) is unsatised

http://find/


78/95



(a c f ) (a g ) (g b ) (h j ) ( i k )


implies a = 1 A conict is again reached: (d e f ) is unsatised (c = 0) (f = 0) ( = 0)

http://find/


79/95



(a c f ) (a g ) (g b ) (h j ) ( i k )



( = 1) (c = 1) (f = 1)

http://find/


80/95



(a c f ) (a g ) (g b ) (h j ) ( i k )



( = 1) (c = 1) (f = 1)

Learn new clause (c f )


c
http://find/


81/95

c

i

h

f

a

(c f )(a c f )


c
http://find/


82/95

c

i

h

f

a

(c f )(a c f )

Learnt clause: (c f )

Need to backtrack, givennew clause

Backtrack to most recentdecision: f = 0

Clause learning andnon-chronologicalbacktracking are hallmarksof modern SAT solvers

Most Recent Backtracking Scheme

c
http://find/


83/95

c

i

h

f

a

(a c f )


c
http://find/


84/95

c

i

h

f

a

(a c f )


c
http://find/


85/95

a

c

i

h

a

(a c f )

f (a c f )

Learnt clause: (a c f ) No need to assign a = 1 -

backtrack to most recentdecision: f = 0

Search algorithm is nolonger a traditionalbacktracking scheme

Unique Implication Points (UIPs)c
http://find/


86/95

a i

h

b

g d

conflict f

e

Exploit structure from the implication graph To have a more aggressive backtracking policy

Identify additional clauses to be learnt[Marques-Silva&Sakallah96]

Create clauses (a c f ) and ( i f )

Imply not only a = 1 but also i = 0 1st UIP scheme is the most efficient [Zhang&al01]

Create only one clause ( i f ) Avoid creating similar clauses involving the same literals

Clause deletion policies
http://find/


87/95

Keep only the small clauses [Marques-Silva&Sakallah96] For each conict record one clause Keep clauses of size K Large clauses get deleted when become unresolved

Keep only the relevant clauses [Bayardo&Schrag97] Delete unresolved clauses with M free literals

Keep only the clauses that are used [Goldberg&Novikov02] Keep track of clauses activity

Data Structures

Key point: only unit and unsatised clauses must be detected
http://find/


88/95

Key point: only unit and unsatised clauses must be detectedduring search

Formula is unsatised when at least one clause is unsatised Formula is satised when all the variables are assigned and

there are no unsatised clauses

In practice: unit and unsatised clauses may be identied

using only two references Standard data structures ( adjacency lists):

Each variable x keeps a reference to all clauses containing aliteral in x

Lazy data structures (watched literals): For each clause, only two variables keep a reference to the

clause, i.e. only 2 literals are watched

Standard Data Structures (adjacency lists)

literals0 = 4
http://find/


89/95

size = 5literals1= 0

size = 5

literals1= 0literals0 = 5

size = 5literals1= 1literals0 = 4

unit

satisfied

unsatisfied

Each variable x keeps a reference

to all clauses containing a literal inx If variable x is assigned, then all

clauses containing a literal in x are evaluated

If search backtracks, then allclauses of all newly unassignedvariables are updated

Total number of references is L,where L is the number of literals

Lazy Data Structures (watched literals)

unresolved For each clause, only two
http://find/


90/95

satisfied

after backtracking to level 4

unresolved

unit

@5 @3 @1

@1@7@3@5

@5 @7 @7 @1

@1@3

@3

@1@3variables keep a reference to theclause, i.e. only 2 literals arewatched

If variable x is assigned, onlythe clauses where literals in x are watched need to be

evaluated If search backtracks, thennothing needs to be done

Total number of references is2 C , where C is the number

of clauses In general L 2 C , inparticular if clauses are learnt

Search Heuristics

Standard data structures : heavy heuristics
http://find/


91/95

Standard data structures : heavy heuristics DLIS: Dynamic Large Individual Sum [Marques-Silva99]

Selects the literal that appears most frequently in unresolvedclauses

Lazy data structures : light heuristics VSIDS: Variable State Independent Decaying Sum

[Moskewicz&al01] Each literal has a counter, initialized to zero When a new clause is recorded, the counter associated with

each literal in the clause is incremented The unassigned literal with the highest counter is chosen at

each decision

Other variations Counters updated also for literals in the clauses involved in

conicts [Goldberg&Novikov02]

Restarts I
http://find/


92/95

Plot for processor verication instance with branchingrandomization and 10000 runs

More than 50% of the runs require less than 1000 backtracks A small percentage requires more than 10000 backtracks

Run times of backtrack search SAT solvers characterized byheavy-tail distributions

Restarts II
http://find/


93/95

solutioncutoff cutoff

Repeatedly restart the search each time a cutoff is reached Randomization allows to explore different paths in search tree

Resulting algorithm is incomplete Increase the cutoff value Keep clauses from previous runs

cutoff

solution

clausesnew

Conclusions
http://find/


94/95

The ingredients for having an efficient SAT solver Mistakes are not a problem

Learn from your conicts ... and perform non-chronological backtracking

Restart the search Be lazy!

Lazy data structures Low-cost heuristics

Thank you!
http://find/


95/95
http://find/

sat solving

Documents