constrainedness including slides from toby walsh
TRANSCRIPT
![Page 1: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/1.jpg)
Constrainedness
Including slides from Toby Walsh
![Page 2: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/2.jpg)
Constraint satisfaction
• Constraint satisfaction problem (CSP) is a triple <V,D,C> where:– V is set of variables– Each X in V has set of values, D_X
• Usually assume finite domain• {true,false}, {red,blue,green}, [0,10], …
– C is set of constraints
Goal: find assignment of values to variables to satisfy all the constraints
![Page 3: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/3.jpg)
Constraint solver
• Tree search– Assign value to variable– Deduce values that must be removed from future/unassigned
variables• Constraint propagation
– If any future variable has no values, backtrack else repeat
• Number of choices– Variable to assign next, value to assign
Some important refinements like nogood learning, non-chronological backtracking, …
![Page 4: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/4.jpg)
Constraint propagation
• Arc-consistency (AC)– A binary constraint r(X1,X2) is AC iff
for every value for X1, there is a consistent value (often called support) for X2 and vice versa
E.g. With 0/1 domains and the constraint X1 =/= X2Value 0 for X1 is supported by value 1 for X2Value 1 for X1 is supported by value 0 for X2…
– A problem is AC iff every constraint is AC
![Page 5: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/5.jpg)
Tree search
• Backtracking (BT)
• Forward checking (FC)
• Backjumping (BJ, CBJ, DB)
• Maintaining arc-consistency (MAC)
• Limited discrepancy search (LDS)
• Non-chronological backtracking & learning
• Probing
• …
![Page 6: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/6.jpg)
Modelling
• Choose a basic model
• Consider auxiliary variables– To reduce number of
constraints, improve propagation
• Consider combined models– Channel between views
• Break symmetries
• Add implied constraints– To improve propagation
![Page 7: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/7.jpg)
Propositional Satisfiability
• SAT– does a truth assignment exist that
satisfies a propositional formula?– special type of constraint
satisfaction problem• Variables are Boolean• Constraints are formulae
– NP-complete
• 3-SAT– formulae in clausal form with 3
literals per clause– remains NP-complete
(x1 v x2) & (-x2 v x3 v -x4)
x1/ True, x2/ False, ...
![Page 8: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/8.jpg)
Random 3-SAT
• Random 3-SAT– sample uniformly from space
of all possible 3-clauses– n variables, l clauses
• Which are the hard instances?– around l/n = 4.3
What happens with larger problems?
Why are some dots red and others blue?
![Page 9: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/9.jpg)
Random 3-SAT
• Varying problem size, n
• Complexity peak appears to be largely invariant of algorithm– backtracking algorithms
like Davis-Putnam– local search procedures like
GSAT
What’s so special about 4.3?
![Page 10: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/10.jpg)
Random 3-SAT
• Complexity peak coincides with solubility transition
– l/n < 4.3 problems under-constrained and SAT
– l/n > 4.3 problems over-constrained and UNSAT
– l/n=4.3, problems on “knife-edge” between SAT and UNSAT
![Page 11: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/11.jpg)
• Shape– “Sharp” (in a technical sense) [Friedgut 99]
• Location– 2-SAT occurs at l/n=1 [Chavatal & Reed 92, Goerdt
92]
– 3-SAT occurs at 3.26 < l/n < 4.598
Theoretical results
![Page 12: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/12.jpg)
“But it doesn’t occur in X?”
• X = some NP-complete problem
• X = real problems
• X = some other complexity class
![Page 13: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/13.jpg)
“But it doesn’t occur in X?”
• X = some NP-complete problem
• Phase transition behaviour seen in:– TSP problem (decision not optimization)– Hamiltonian circuits (but NOT a complexity peak)– number partitioning– graph colouring– independent set– ...
![Page 14: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/14.jpg)
“But it doesn’t occur in X?”
• X = real problems
• Phase transition behaviour seen in:– job shop scheduling problems– TSP instances from TSPLib– exam timetables @ Edinburgh– Boolean circuit synthesis – Latin squares (alias sports scheduling)– ...
![Page 15: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/15.jpg)
“But it doesn’t occur in X?”
• X = some other complexity class
• Phase transition behaviour seen in:– polynomial problems like arc-consistency– PSPACE problems like QSAT and modal K– ...
![Page 16: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/16.jpg)
Algorithms at the phase boundary
What do we understand about problem hardness at the phase
boundary?
How can this help build better algorithms?
![Page 17: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/17.jpg)
N
Sol )(log1 2
Kappa
dnessconstraine of measure a kappa,
|)(|log
solutions ofnumber expected the
space statein that solutions ofset the
space state that of size the||
problem a of space state
2
SN
Sol
Sol
S
S
Defined for an ensemble of problems
![Page 18: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/18.jpg)
N
Sol )(log1 2 Kappa (motivation)
![Page 19: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/19.jpg)
|)(|log
)(log1
2
2
S
Sol
Kappa
When every state is a solution 0|)(|log
)(log1
2
2
S
Sol
When every state is not a solution |)(|log
)0(log1
2
2
S
When there is a unique solution 1|)(|log
)1(log1
2
2 S
![Page 20: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/20.jpg)
|)(|log
)(log1
2
2
S
Sol
Kappa
When every state is a solution 0|)(|log
)(log1
2
2
S
Sol
When every state is not a solution |)(|log
)0(log1
2
2
S
When there is a unique solution 1|)(|log
)1(log1
2
2 S
Easy Soluble
Easy Insoluble
On the knife edge
Hard
![Page 21: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/21.jpg)
An Example: random CSP’s
Each of n variables has a uniform domain size mThere is a probability p1 of a constraint between a pair of variablesThere is a probability p2 that a pair of values conflict (when a constraint exists)
21,,, ppmn
tightnessconstraint
graph constraint ofdensity
sizedomain uniform
variablesofnumber
2
1
p
p
m
n
![Page 22: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/22.jpg)
An Example: random CSP’s 21,,, ppmn
21
2
2/)1(22
)2/)1((2
1
1log
2
1
)(log
)1((log1
)1(
||
1
1
pp
n
m
pm
pmSol
mS
m
n
nnpn
nnpn
n
|)(|log
)(log1
2
2
S
Sol
![Page 23: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/23.jpg)
An Example: random CSP’s 21,,, ppmn
21 1
1log
2
1
pp
nm
|)(|log
)(log1
2
2
S
Sol
Can rearrange this formula so that we find the value of p2 such that we are at the phase transition, i.e. when kappa = 1
Then perform experiments to see if it is accurate predictori.e. we put theory to the test
![Page 24: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/24.jpg)
![Page 25: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/25.jpg)
![Page 26: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/26.jpg)
Kappa as a heuristic
Vvv
Ccc
m
p
)(log
)1(log
2
2
Minimise that
![Page 27: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/27.jpg)
An Example: not random CSP’s
Vvv
Ccc
Ccc
Vvv
Vvv
m
p
pmSol
mS
)(log
)1(log
)1(
||
2
2
|)(|log
)(log1
2
2
S
Sol
variablea of sizedomain
constraint a of tightness
sconstraint ofset in the constraint a
variablesofset in the variablea
v
c
m
p
Cc
Vv
![Page 28: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/28.jpg)
An Example: not random CSP’s
Vvv
Ccc
m
p
)(log
)1(log
2
2
|)(|log
)(log1
2
2
S
Sol
variablea of sizedomain
constraint a of tightness
sconstraint ofset in the constraint a
variablesofset in the variablea
v
c
m
p
Cc
Vv
![Page 29: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/29.jpg)
Kappa is general (3 examples)
n
la
)2
11(log2 SAT with n variables, l clauses, a literals/clause
)1
(log
m
m
n
em Graph colouring, n vertices, e edges, m colours
n
lm m )(log
)1(
Number partitioning, n numbers, range (0,l], m bags with same sum
![Page 30: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/30.jpg)
Kappa as a heuristic
Vvv
Ccc
m
p
)(log
)1(log
2
2
Minimise that
![Page 31: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/31.jpg)
Vvv
Ccc
m
p
)(log
)1(log
2
2
Minimise that
• assume we only measure domain size (denominator)• assume we only measure top line (numerator)• assume we measure top and bottom line (numerator and denominator
But that’s costly to do.Is there a low cost surrogate?
![Page 32: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/32.jpg)
Looking inside search
• Three key insights– constrainedness “knife-
edge”– backbone structure– 2+p-SAT
• Suggests branching heuristics– also insight into
branching mistakes
![Page 33: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/33.jpg)
Constrainedness knife-edge
kappa against depth/n
![Page 34: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/34.jpg)
Constrainedness knife-edge
• Seen in other problem domains– number partitioning, …
• Seen on “real” problems– exam timetabling (alias graph colouring)
• Suggests branching heuristic– “get off the knife-edge as quickly as possible”– minimize or maximize-kappa heuristicsmust take into account branching rate, max-kappa often
therefore not a good move!
![Page 35: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/35.jpg)
Minimize constrainedness
• Many existing heuristics minimize-kappa– or proxies for it
• For instance– Karmarkar-Karp heuristic for number partitioning– Brelaz heuristic for graph colouring– Fail-first heuristic for constraint satisfaction– …
• Can be used to design new heuristics– removing some of the “black art”
![Page 36: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/36.jpg)
Backbone
• Variables which take fixed values in all solutions– alias unit prime implicates
• Let fk be fraction of variables in backbone– l/n < 4.3, fk vanishing
(otherwise adding clause could make problem unsat)
– l/n > 4.3, fk > 0discontinuity at phase
boundary!
![Page 37: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/37.jpg)
Backbone
• Search cost correlated with backbone size– if fk non-zero, then can easily assign variable “wrong”
value– such mistakes costly if at top of search tree
• Backbones seen in other problems– graph colouring– TSP– …
Can we make algorithms that identify and exploit the backbone structure of a problem?
![Page 38: Constrainedness Including slides from Toby Walsh](https://reader035.vdocuments.net/reader035/viewer/2022062423/56649e9f5503460f94ba17fc/html5/thumbnails/38.jpg)
2+p-SAT
• 2-SAT is polynomial (linear) but 3-SAT is NP-complete– 2-SAT, unlike 3-SAT, has no
backbone discontinuity
• Morph between 2-SAT and 3-SAT– fraction p of 3-clauses
– fraction (1-p) of 2-clauses
• 2+p-SAT maps from P to NP– p>0, 2+p-SAT is NP-complete