set 5: constraint satisfaction problems chapter 6 r&nkkask/fall-2016...
TRANSCRIPT
![Page 1: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/1.jpg)
ICS-271:Notes 5: 1
Set 5: Constraint Satisfaction Problems
Chapter 6 R&N
ICS 271 Fall 2016
Kalev Kask
![Page 2: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/2.jpg)
ICS-271:Notes 5: 2
Outline
• The constraint network model
– Variables, domains, constraints, constraint graph, solutions
• Examples:
– graph-coloring, 8-queen, cryptarithmetic, crossword puzzles, vision
problems,scheduling, design
• The search space and naive backtracking,
• The constraint graph
• Consistency enforcing algorithms
– arc-consistency, AC-1,AC-3
• Backtracking strategies
– Forward-checking, dynamic variable orderings
• Special case: solving tree problems
• Local search for CSPs
![Page 3: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/3.jpg)
ICS-271:Notes 5: 3
![Page 4: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/4.jpg)
ICS-271:Notes 5: 4
A Bred green
red yellow
green red
green yellow
yellow green
yellow red
Constraint Satisfaction
Example: map coloring
Variables - countries (A,B,C,etc.)
Values - colors (e.g., red, green, yellow)
Constraints: e tc . ,ED D , AB ,A
C
A
B
D
E
F
G
![Page 5: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/5.jpg)
ICS-271:Notes 5: 5
![Page 6: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/6.jpg)
ICS-271:Notes 5: 6
![Page 7: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/7.jpg)
ICS-271:Notes 5: 7
![Page 8: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/8.jpg)
ICS-271:Notes 5: 8
![Page 9: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/9.jpg)
ICS-271:Notes 5: 9
Varieties of constraints
• Unary constraints involve a single variable,
– e.g., SA ≠ green
• Binary constraints involve pairs of variables,
– e.g., SA ≠ WA
• Higher-order constraints involve 3 or more variables,
– Clauses in boolean (propositional) logic
– e.g., cryptoarithmetic column constraints
![Page 10: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/10.jpg)
ICS-271:Notes 5: 10
Hard vs Soft Constraints
• Hard constraints : must be satisfied
– Satisfaction problem
• Soft constraints : capture preferences
– Optimization problem
![Page 11: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/11.jpg)
ICS-271:Notes 5: 11
Sudoku
Each row, column and major block must be all different
“Well posed” if it has unique solution: 27 constraints
![Page 12: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/12.jpg)
ICS-271:Notes 5: 12
Sudoku
Each row, column and major block must be all different
“Well posed” if it has unique solution: 27 constraints
2 34 62
Constraint propagation
•Variables: 81 slots
•Domains = {1,2,3,4,5,6,7,8,9}
•Constraints: •27 not-equal
![Page 13: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/13.jpg)
ICS-271:Notes 5: 13
![Page 14: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/14.jpg)
ICS-271:Notes 5: 15
![Page 15: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/15.jpg)
ICS-271:Notes 5: 16
A network of constraints
• Variables
–
• Domains
– of discrete values:
• Binary constraints:
– Rij which represent the list of allowed pairs of values, Rij is a subset
of the Cartesian product: .
• Constraint graph:
– A node for each variable and an edge for each constraint
• Solution:
– An assignment of a value from its domain to each variable such that
no constraint is violated.
• A network of constraints represents the relation of all solutions.
n,....,XX1
n,....,DD1
ji xDD
},,),(|){( 1 jjiiijjin DxDxRxx,....,xxsol
1X
5X
4X3X
2X
![Page 16: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/16.jpg)
ICS-271:Notes 5: 17
Example : The 4-queen problem
Q
Q QQ
Place 4 Queens on a chess board of
4x4 such that no two queens reside in
the same row, column or diagonal.
Standard CSP formulation of the problem:
• Variables: each row is a variable.
1X
4X
3X
2X
1 2 3 4
• Domains: }.4,3,2,1{iD
• Constraints: There are = 6 constraints involved:42( )
)}2,4)(1,4)(1,3)(4,2)(4,1)(3,1{(12 R
)}3,4)(1,4)(4,3)(2,3)(3,2)(1,2)(4,1)(2,1{(13 R
)}3,4)(2,4)(4,3)(2,3)(1,3)(4,2)(3,2)(1,2)(3,1)(2,1{(14 R
)}2,4)(1,4)(1,3)(4,2)(4,1)(3,1{(23 R
)}3,4)(1,4)(4,3)(2,3)(3,2)(1,2)(4,1)(2,1{(24 R
)}2,4)(1,4)(1,3)(4,2)(4,1)(3,1{(34 R
• Constraint Graph :
1X
2X4X
3X
![Page 17: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/17.jpg)
ICS-271:Notes 5: 19
Class scheduling/Timetabling
![Page 18: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/18.jpg)
ICS-271:Notes 5: 20
Search vs. Inference
• Search :
– In the space of partial variable-value assignments
– Key idea : conditioning
• Inference :
– Derive new information implied by values/constraints
– Key idea : local consistency
![Page 19: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/19.jpg)
ICS-271:Notes 5: 21
But only dn unique assignments
![Page 20: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/20.jpg)
ICS-271:Notes 5: 22
![Page 21: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/21.jpg)
ICS-271:Notes 5: 23
The search space
• Definition: given an ordering of the variables
– a state:
• is an assignment to a subset of variables that is consistent.
– Operators:
• add an assignment to the next variable that does not violate any
constraint. How to choose next variable?
– Goal state:
• a consistent assignment to all the variables.
n,....,XX1
![Page 22: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/22.jpg)
ICS-271:Notes 5: 24
![Page 23: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/23.jpg)
ICS-271:Notes 5: 25
![Page 24: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/24.jpg)
ICS-271:Notes 5: 26
![Page 25: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/25.jpg)
ICS-271:Notes 5: 27
![Page 26: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/26.jpg)
ICS-271:Notes 5: 28
Backtracking
• Complexity of extending a partial solution:
– Complexity of consistent: O(e log t), t bounds #tuples, e bounds #constraints
– Complexity of selectvalue: O(e k log t), k bounds domain size
![Page 27: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/27.jpg)
ICS-271:Notes 5: 30
The effect of variable ordering
z divides x, y and t
![Page 28: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/28.jpg)
ICS-271:Notes 5: 31
A coloring problem
![Page 29: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/29.jpg)
ICS-271:Notes 5: 32
Backtracking Search for a Solution
![Page 30: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/30.jpg)
ICS-271:Notes 5: 33
Backtracking Search for a Solution
![Page 31: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/31.jpg)
ICS-271:Notes 5: 34
Backtracking Search for All Solutions
![Page 32: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/32.jpg)
ICS-271:Notes 5: 35
Summary so far
• Constraint Satisfaction Problems
– variables, domains, constraints
– solution = assignment to all variables such that all constraints are satisfied
– constraint graph
• Search vs Inference:
– conditioning vs local consistency.
• Search
– Backtracking search :
• In the space of partial assignments;
• add consistent var=value assignment at each search space node.
• If not possible, backtrack
– Effect on search space size on var/val order :
• Variable order – most constrained variable first
• Value order – least constraining value first
![Page 33: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/33.jpg)
ICS-271:Notes 5: 36
The Minimal network:
Example: the 4-queen problem
![Page 34: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/34.jpg)
ICS-271:Notes 5: 37
Inference (Approximation) algorithms
• Arc-consistency (Waltz, 1972)
• Path-consistency (Montanari 1974, Mackworth 1977)
• k-consistency (Freuder 1982)
• Key Idea :
– Transform the network into smaller and smaller (but equivalent)
networks by tightening the domains/constraints.
![Page 35: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/35.jpg)
ICS-271:Notes 5: 38
Arc-consistency
32,1,
32,1, 32,1,
1 X, Y, Z, T 3
X Y
Y = Z
T Z
X T
X Y
T Z
32,1,
=
![Page 36: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/36.jpg)
ICS-271:Notes 5: 39
1 X, Y, Z, T 3
X Y
Y = Z
T Z
X T
X Y
T Z
=
1 3
2 3
• Incorporated into backtracking search
• Constraint programming languages powerful
approach for modeling and solving combinatorial
optimization problems.
Arc-consistency
![Page 37: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/37.jpg)
ICS-271:Notes 5: 40
Arc-consistency algorithm
domain of X domain of Y
Arc is arc-consistent if for any value of there exist a matching value of
Algorithm Revise makes an arc consistent
Begin
1. For each a in Di if there is no value b in Dj that is consistent with a then delete a
from the Di.
End.
Revise is , k is the number of values in each domain.
)( ji ,XXiX
iX
)( ji ,XX
)O(k 2
![Page 38: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/38.jpg)
ICS-271:Notes 5: 41
Algorithm AC-3
• Begin
– 1. Q <--- put all arcs in the queue in both directions
– 2. While Q is not empty do,
– 3. Select and delete an arc from the queue Q
• 4. Revise
• 5. If Revise changes (reduces) the domain of Xi then add to the
queue all arcs that go to Xi ((Xl,Xi for all l except i).
– 6. end-while
• End
• Complexity:
– Processing an arc requires O(k^2) steps
– There is edges
– The number of times each arc can be processed is 2·k
– Total complexity is
)( ji ,XX
)( ji ,XX
)O(ek 3
e
![Page 39: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/39.jpg)
ICS-271:Notes 5: 42
Sudoku
Each row, column and major block must be all different
“Well posed” if it has unique solution: 27 constraints
2 34 62
Constraint propagation
•Variables: 81 slots
•Domains = {1,2,3,4,5,6,7,8,9}
•Constraints: •27 not-equal
![Page 40: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/40.jpg)
ICS-271:Notes 5: 43
Sudoku
Each row, column and major block must be all different
“Well posed” if it has unique solution
Path-consistency or
3-consistency
4-consistency and
i-consistency in geeral
![Page 41: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/41.jpg)
ICS-271:Notes 5: 44
The Effect of Consistency Level
• After arc(2)-consistency z=5 and l=5 are
removed
• After path(3)-consistency
– R’_zx
– R’_zy
– R’_zl
– R’_xy
– R’_xl
– R’_yl
Tighter networks yield smaller search spaces
![Page 42: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/42.jpg)
ICS-271:Notes 5: 45
• Before search: (reducing the search space)
– Arc-consistency, path-consistency, k-consistency
– Variable ordering (fixed)
• During search:
– Look-ahead schemes:
• Variable ordering (Choose the most constraining variable)
• Value ordering (Choose the least restricting value)
– Look-back schemes:
• Backjumping
• Constraint recording
• Dependency-directed backtracking
Improving Backtracking O(exp(n))
![Page 43: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/43.jpg)
ICS-271:Notes 5: 46
![Page 44: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/44.jpg)
ICS-271:Notes 5: 47
Look-ahead: Variable and value orderings
• Intuition:
– Choose a variable that will detect failures early
– Choose a value least likely to yield a dead-end
– Approach: apply propagation at each node in the search tree
• Forward-checking
– Check each unassigned variable separately
• Maintaining arc-consistency (MAC)
– Apply full arc-consistency
![Page 45: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/45.jpg)
ICS-271:Notes 5: 48
![Page 46: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/46.jpg)
ICS-271:Notes 5: 49
![Page 47: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/47.jpg)
ICS-271:Notes 5: 50
![Page 48: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/48.jpg)
ICS-271:Notes 5: 51
![Page 49: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/49.jpg)
ICS-271:Notes 5: 52
![Page 50: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/50.jpg)
ICS-271:Notes 5: 53
![Page 51: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/51.jpg)
ICS-271:Notes 5: 54
![Page 52: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/52.jpg)
ICS-271:Notes 5: 55
![Page 53: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/53.jpg)
ICS-271:Notes 5: 60
![Page 54: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/54.jpg)
ICS-271:Notes 5: 62
Summary so far
• Constraint Satisfaction Problems
– variables, domains, constraints
– solution = assignment to all variables such that all constraints are satisfied
– constraint graph
• Search vs Inference:
– conditioning vs local consistency.
• Search
– In the space of partial assignments; add var=val at each search space node.
– Effect on search space size on var/val order :
• Variable order – most constrained variable first
• Value order – least constraining value first
• Inference (local consistency)
– Prune domains/constraints
– Arc/path/k-consistency
• Combining search and local consistency to improve efficiency
– Forward checking (FC)
– Maintaining Arc Consistency (MAC)
![Page 55: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/55.jpg)
ICS-271:Notes 5: 63
Forward-checking on Graph-coloring
)( 2ekO
)(
)(
3
2
ekO
ekOFW overhead:
MAC overhead:
![Page 56: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/56.jpg)
ICS-271:Notes 5: 64
Algorithm DVO (DVFC)
![Page 57: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/57.jpg)
ICS-271:Notes 5: 65
Propositional Satisfiability
• If Alex goes, then Becky goes:
• If Chris goes, then Alex goes:
• Query:
Is it possible that Chris goes to the party but Becky does not?
Example: party problem
) (or, BA BA
) (or, ACA C
e?satisfiabl
Is
C B, A,C B,Atheorynalpropositio
![Page 58: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/58.jpg)
ICS-271:Notes 5: 66
Unit Propagation
• Arc-consistency for CNFs.
• Involve a single clause and a single literal
• Example:
• Formula:
– C must be true
– A must be true
– B must be false and true
– formula is not satisfiable
AA , )( CC
C B, A,C B,A
![Page 59: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/59.jpg)
ICS-271:Notes 5: 67
Look-ahead for SAT(Davis-Putnam, Logeman and Laveland, 1962)
![Page 60: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/60.jpg)
ICS-271:Notes 5: 68
Look-ahead for SAT: DPLL
Only enclosed area will be explored with unit-propagation
Backtracking look-ahead with
Unit propagation=
Generalized arc-consistency
(Davis-Putnam, Logeman and Laveland, 1962)
example: (┐A V B) ˄ (┐C V A) ˄ (A V B V D) ˄ (C)
![Page 61: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/61.jpg)
ICS-271:Notes 5: 70
Look-back: Backjumping / Learning
• Backjumping:
– In deadends, go back to the most recent culprit.
• Learning:
– constraint-recording, no-good recording.
– good-recording
![Page 62: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/62.jpg)
ICS-271:Notes 5: 72
Backjumping
• (X1=r,x2=b,x3=b,x4=b,x5=g,x6=r,x7={r,b})
• (r,b,b,b,g,r) conflict set of x7
– Conflict set : a partial assignment that cannot be extended to the next variable
• (r,-,b,b,g,-) c.s. of x7
• (r,-,b,-,-,-,-) minimal conflict-set
• Leaf deadend: (r,b,b,b,g,r)
– No-good : a partial assignment that cannot be extended to a solution
– No-good = internal deadend
• Every conflict-set is a no-good
– But not other way around (x1=r is a no-good)
?
![Page 63: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/63.jpg)
ICS-271:Notes 5: 74
![Page 64: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/64.jpg)
ICS-271:Notes 5: 75
![Page 65: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/65.jpg)
ICS-271:Notes 5: 76
![Page 66: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/66.jpg)
ICS-271:Notes 5: 77
4. Arc-consistency in tree-structured CSPs makes search backtrack-free
![Page 67: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/67.jpg)
ICS-271:Notes 5: 78
![Page 68: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/68.jpg)
ICS-271:Notes 5: 79
The cycle-cutset method
• An instantiation can be viewed as blocking cycles in the graph
• Given an instantiation to a set of variables that cut all cycles (a
cycle-cutset) the rest of the problem can be solved in linear time by
a tree algorithm.
• Complexity (n number of variables, k the domain size and C the
cycle-cutset size):
)( 2knkO C
![Page 69: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/69.jpg)
ICS-271:Notes 5: 80
Tree Decomposition
T
WA
NT
SA
NT
SA
Q
SA
Q
NSW
SA NSW
V
Complexity is O(n exp(w)) where w bounds
the number of variables in a cluster. Known
as the treewidth.
![Page 70: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/70.jpg)
ICS-271:Notes 5: 81
Local Search for CSPs
• Local Search
• Works with complete variable assignments (called a state)
• Has a cost function associated with each state
• E.g. min-conflicts – number of constraints violated
• Considers only immediate neighborhood for next state
• E.g. change the value of one variable
• Operators
• Greedy heuristic : pick a value for a variable that leads to
greatest reduction in total cost – can get stuck in local optima
• Random moves : occasionally make a random move
• Sideways moves, re-starts, constraint re-weighting, …
• Incomplete
• Often fast
• orders of magnitude faster than systematic e.g. DFS
![Page 71: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/71.jpg)
ICS-271:Notes 5: 83
![Page 72: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/72.jpg)
ICS-271:Notes 5: 84
GSAT – local search for SAT(Selman, Levesque and Mitchell, 1992)
1. For i=1 to MaxTries
2. Select a random assignment A
3. For j=1 to MaxFlips
4. if A satisfies all constraints, return A
5. else flip a variable to maximize the score
6. (number of satisfied constraints; if no variable
7. assignment increases the score, flip at random)
8. end
9. end
Greatly improves hill-climbing by adding
restarts and sideway moves
![Page 73: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/73.jpg)
ICS-271:Notes 5: 85
WalkSAT (Selman, Kautz and Cohen, 1994)
With probability p
random walk – flip a variable in some unsatisfied constraint
With probability 1-p
perform a hill-climbing step
Adds random walk to GSAT:
Randomized hill-climbing often solves large and hard satisfiable problems
![Page 74: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/74.jpg)
ICS-271:Notes 5: 86
More Stochastic Search:
Simulated Annealing, Constraint Reweighting
• Simulated annealing:
– A method for overcoming local minimas
– Allows bad moves with some probability:
• With some probability related to a temperature parameter T the
next move is picked randomly.
– Theoretically, with a slow enough cooling schedule, this algorithm
will find the optimal solution. But so will searching randomly.
• Breakout method (Morris, 1990): adjust the weights of the violated
constraints
![Page 75: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/75.jpg)
ICS-271:Notes 5: 87
Problem Hardness
![Page 76: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/76.jpg)
ICS-271:Notes 5: 88
![Page 77: Set 5: Constraint Satisfaction Problems Chapter 6 R&Nkkask/Fall-2016 CS271/slides/05-constraint-satisfaction.pdf · – Backtracking search : • In the space of partial assignments;](https://reader033.vdocuments.net/reader033/viewer/2022041916/5e69d40a169121176c58379b/html5/thumbnails/77.jpg)
ICS-271:Notes 5: 89
Game of Mastermind