solve your problem faster - by changing the model barbara smith

Solve Your Problem Faster Solve Your Problem Faster - by changing the model- by changing the model

Barbara Smith

ERCIM/CologNet Workshop

Background AssumptionsBackground Assumptions• A constraint programming tool

providing:• a systematic search algorithm • combined with constraint propagation• a set of pre-defined constraints

• A problem that can be represented as a finite domain constraint satisfaction or optimization problem

• Caveat: the experiences described are based on ILOG Solver


What do I mean by Faster?What do I mean by Faster?

• e.g.`Peaceable Armies of Queens’ problem• place m black & m white queens on a

chessboard so that the black queens don’t attack the white queens, and maximize m

• Optimal solution for an 11 x 11 board:


Results for 8 x 8 boardResults for 8 x 8 boardBacktracks

to find

optimal

Backtracks toprove

optimality

Time (sec.)

Basic model 5270 12,002,608 2100Eliminate symmetry

5270 938,652 240

New model 4676 478,012 72

Variable order-ing heuristic

2973 44,276 12

Just changing the model makes a huge difference to the time to solve the problem


What are the options?What are the options?

• Given a CSP representation of the problem: • if there is symmetry in the CSP, eliminate it• find a related representation to use instead/as

well• add variables – to express different aspects of

the problem• add constraints

• to relate new variables to old• to prune dead-ends earlier

• change the search strategy• (other possibilities won’t be considered)


Symmetry in the CSPSymmetry in the CSP

• A symmetry transforms any (full or partial) assignment into another so that consistency/inconsistency is preserved

• Symmetry causes wasted search effort: after exploring choices that don’t lead to a solution, symmetrically equivalent choices will be explored

• Especially difficult if there is no solution, or if all solutions are wanted, or in optimization problems


Symmetry Breaking During Search (SBDS)Symmetry Breaking During Search (SBDS)

• See Gent & Smith, ECAI’2000

A = set of assignmentsmade so far

var = val var != val

+ g(var!= val) for any unbroken symmetry g, i.e. if g(A) is (or will be) true

On backtracking to a choice point, add symmetry breaking constraints to the 2nd branch explored


Symmetries of ‘armies of queens’Symmetries of ‘armies of queens’

• Variable s[i,j] represents the square i, j• The values are b, w or e (black, white or empty)

• 7 chess board symmetries: x (horizontal reflection), y, d1, d2, r90, r180, r270 + bw (swap black & white queens) + 7 combinations • x(s[i,j]=v) s[n-i+1,j]=v• bw(s[i,j] = v) s[i,j] = v’ , (v’ = b if v = w, w if v = b,

e if v = e ) • bw_x(s[i,j]=v) s[n-i+1,j]=v’ , etc.

• 15 symmetry functions are needed in all• each takes a constraint as input, e.g. s[i,j]=v, and

returns the symmetric constraint, e.g. s[i,j] = v’


Armies of Queens – 8 x 8 BoardArmies of Queens – 8 x 8 Board

All symmetries swapping colours now broken on

this branch

s[1,2]=w

s[1,2] != w

s[1,1]=w

s[1,1] != w

d1: s[1,1]!=w

bw_x: s[8,1]!=b, etc.

x: s[8,1]!=w

bw: s[1,1]!=b

r90: s[1,8]!=w

x: if s[8,1]=w then s[8,2]!=w , etc.


Results for 8 x 8 boardResults for 8 x 8 boardFails to

findoptimal

Fails to prove

optimality

Time (sec.)

Basic model 5270 12,002,608 2100Eliminate symmetry

5270 938,652 240


Optimizing SONET RingsOptimizing SONET Rings• See Sherali & Smith*, ‘Improving Discrete Model

Representations via Symmetry Considerations’ (*no relation)

• Transmission over optical fibre networks• Known traffic demands between pairs of client nodes• A node is installed on a SONET ring using an ADM

(add-drop multiplexer) • If there is traffic demand between 2 nodes, there

must be a ring that they are both on • Rings have capacity limits (number of ADMs, i.e.

nodes, & traffic)• Satisfy demands using the minimum number of

ADMs


Example & Optimal SolutionExample & Optimal Solution

• Up to 7 rings, maximum capacity 5 ADMs

• Ignore traffic capacity (for now)


A CSP Model A CSP Model

• Variables: xij = 1 if node i is on ring j

• At most 5 nodes on any ring:

• If there is a demand between nodes k and l :

• Minimize

5j

ijx

1j

ljkj xx

ji

ijx,


Symmetry in the SONET CSPSymmetry in the SONET CSP

• The rings are indistinguishable (only numbered for the purposes of the CSP model)

• We can eliminate the symmetry using SBDS just by describing all transpositions of pairs of rings:• e.g. r12(x[i,j] = v) x[i,2]=v if j =1

x[i,1]=v if j =2 x[i,j]=v otherwise

• That’s all – we can forget about symmetry (e.g. when choosing the variable ordering)


Three Alternative ModelsThree Alternative Models

• Whether a given node is on a given ring:• xij = 1 if node i is on ring j

• Which ring(s) each node is on:• Ni = set of rings node i is on

• Which nodes are on each ring• Rj = set of nodes on ring j

• In principle, any of these 3 sets of variables could be the basis of a complete CSP model


Dual Models from Boolean VariablesDual Models from Boolean Variables

• Given a CSP with Boolean variables xijkl…

we can form a new set of variables with one less subscript:

• e.g. xijkl… = 1 corresponds to yjkl… = i (an integer variable) or i Yjkl… (a set variable), depending on whether one or several possible values i are associated with each combination of j,k,l,…

• if the Boolean variables have n subscripts, we can derive n sets of dual variables


Which Model to Choose?Which Model to Choose?

• We don’t need to choose just one set of variables – we can use them all at once

• We then need new channelling constraints to link the sets of variables:• (xij = 1) = (i Rj) = (j Ni)• (see Cheng, Choi, Lee & Wu, Constraints, 1999)

• But we should not combine all 3 complete models

• Adding variables & channelling constraints doesn’t cost much – duplicated constraints do


Why add more variables?Why add more variables?

• Express each problem constraint in whichever way is easiest /most natural/ propagates best• e.g. gives better results than

• (Often easiest /most natural/ propagates best are the same thing)

• We can observe effects of search on different aspects of the model, & express them • develop implied constraints, search

strategies, etc.

1j

ljkj xx1lk NN


Possible Variables in the SONET ProblemPossible Variables in the SONET Problem• Whether a given node is on a given ring • Which ring(s) each node is on • Which nodes are on each ring • How many nodes are on each ring • How many rings each node is on • The total number of nodes on all rings • The total of the number of rings each node is on

• Which demand pairs are on each ring• Which ring(s) each demand pair is on• How many rings are used• ………


Choosing the Search VariablesChoosing the Search Variables

• We need to choose a set of variables such that an assignment to each one, satisfying the constraints, is a complete solution to the problem

• Assume we pass the search variables to the search algorithm in a list or array• the order determines a static variable

ordering


Possible ChoicesPossible Choices

• Use just one set of variables, e.g. xij – the others are just for constraint propagation

• Use two (or more) sets of variables (of the same type) e.g. Rj ,Ni

• interleave them in a sensible (static) order• or use a dynamic ordering applied to both sets of

variables

• Use an incomplete set of variables first, to reduce the search space before assigning a complete set• e.g. decide how many rings each node is on (search

variables |Ni|) and then which rings each node is on (xij )

• All three possibilities are useful – the first will be used in the SONET problem (and later the third)


Implied ConstraintsImplied Constraints• Constraints which can be derived from

the existing constraints, and so don’t eliminate any solutions

• We only want useful implied constraints: • they reduce search: i.e. at some point

during search, a partial assignment must be tried which is inconsistent with the implied constraint but would otherwise not fail immediately

• they reduce running time – the overhead of propagating extra constraints must be less than the savings in search


How to Find Useful Implied ConstraintsHow to Find Useful Implied Constraints

• Identify obviously wrong partial assignments that may/do occur during search• Try to predict them by contemplation/intuition• Observe the search in progress• Having many variables in the model enables

observing/thinking about many possible aspects of the search

• But we still need to check empirically that new constraints do reduce both search and running time


Implied Constraints: SONETImplied Constraints: SONET

• A node with degree in the demand graph > 4 must be on more than 1 ring (|Ni| > 1)

• If a pair of connected nodes have more than 3 neighbours in total, at least one of the pair must be on more than 1 ring (|Nk|+|Nl| > 2)


Optimality ConstraintsOptimality Constraints

• In optimizing, if we know that for any solution with a particular characteristic, there must be another solution at least as good, we can add constraints forbidding it

• e.g.• no ring should have just one node on it• any two rings must have more than 5 nodes in

total (otherwise we could merge them)

• Derive these in the same way as implied constraints


Variable Ordering HeuristicsVariable Ordering Heuristics

• Armies of Queens:• Place a white queen next where it will

attack fewest additional squares

• SONET problem:• add a node to a ring with spare capacity;

choose the node connected to most nodes already on the ring; of those, the node connected to most nodes still to be placed


Finding a Good Solution v. Proving Optimality Finding a Good Solution v. Proving Optimality

• Armies of Queens• the heuristic finds an optimal solution for

every case where this is known• but… it’s hopeless for proving optimality • the `anti-heuristic’ (place a white queen

where it will attack most additional squares) is much better!


Finding a Good Solution v. Proving Optimality: SONETFinding a Good Solution v. Proving Optimality: SONET

• The heuristic finds near-optimal solutions quite quickly, but is no good for proving optimality

• How could we prove this solution is optimal: {2,3,4,9,12}, {1,3,7,8,10}, {4,5,6,7,10}, {1,8,11,12,13} ?• e.g. show that we cannot reduce the number of times that

any of 3, 4, 7, 8, 12 appear without having another node appear twice instead

• Introduce variables ni = |Ni| i.e. the number of rings that node i is on

Two-stage search: find a good solution, then start search again, assigning ni variables first and then xij variables


What do I mean by Faster?What do I mean by Faster?

• The basic model (just the xij variables, no symmetry-breaking) can only solve small problems (7 nodes, 8 demand pairs)

• The final model can solve the problems in the Sherali & Smith paper (13 nodes, 24 demand pairs)• 3 full sets of variables + others• SBDS • implied constraints• variable ordering heuristic• two-stage search process


Some AdviceSome Advice

• Eliminate symmetry• Use lots of variables, with channelling constraints • But don’t express the same problem constraints

twice• Add constraints that make explicit what you know

about satisfying/optimal solutions• But only if they reduce search and running time• Learn from solving the problem by hand and

observing the search • If finding good solutions is easy and proving

optimality is hard, consider using a different strategies for each stage


ConclusionsConclusions

• Can we list “10 Steps to Successful Modelling”? • New problems still often lead to new ideas about

modelling• But some patterns do recur frequently

• e.g. models representing dual viewpoints

• We are beginning to automate some aspects of modelling

• e.g. symmetry, implied constraints

• Still a long way to go before building a good model of a problem is straightforward

• e.g. we often can’t tell if model A is better than model B without trying them both

• More research is still needed...


Selected ReferencesSelected References• Symmetry Breaking During Search in Constraint

Programming, I. P. Gent and B. M. Smith, Proceedings ECAI'2000, pp. 599-603, 2000.

• Models and Symmetry Breaking for `Peaceable Armies of Queens’, K.E Petrie, B. M. Smith & I. P. Gent, APES Report APES-50-2002, May 2002.

• Improving Discrete Model Representations via Symmetry Considerations, H. D. Sherali & J. C. Smith, Man.Sci. (47) pp. 1396-1407, 2001.

• Increasing Constraint Propagation by Redundant Modeling: an Experience Report, B. M. W. Cheng, K. M. F. Choi, J. H. M. Lee & J. C. K. Wu, Constraints (4) pp. 167-192, 1999.

solve your problem faster - by changing the model barbara smith

Documents

ring j

problem slide

ercimcolognet workshop

combination of j

ring r j

complete csp model slide

csp model variables

variable ordering slide