exhaustive search and a first approach to avoiding it
DESCRIPTION
Exhaustive Search and a first approach to avoiding it. Brute Force Methods. guarantee best fitness value is found feasible for ‘small’ data sets only. SAT satisfiability problem. set of propositions prop[n] fitness function logical expression based on propositions boolean fitness(prop) - PowerPoint PPT PresentationTRANSCRIPT
Exhaustive Search and a first approach to avoiding it
Brute Force Methods
guarantee best fitness value is found feasible for ‘small’ data sets only
SAT satisfiability problem
set of propositions prop[n] fitness function
logical expression based on propositionsboolean fitness(prop)
try all 2n combinations of T/F for propositions
boolean fitness(boolean b[]) //fitness functionboolean [] prop = new boolean[n];boolean satisfied = sat(n-1);
boolean sat (int index) // find a solution{ if (index==0) //base case
{prop[0] = true;if fitness(prop) return true;prop[0] = false;return fitness(prop);
}prop[index] = true; // recursiveif (sat(index-1)) return true;
prop[index] = false;return sat(index-1);
} ...
..T ..F
.FT.TT .TF .FF
FFTFTT FTF FFFTFTTTT TTF TFF
2
1
0
...
efficiency without risk
pruning the tree: suppose fitness(prop) is
(p0 \/ ~p1) /\ (~p2)
partial evaluation
...
..T ..F
.FT.TT .TF .FF
FFTFTT FTF FFFTFTTTT TTF TFF
...
p2
p1
p0
nodes can be ignored: fitness can not be true
TSP travelling salesman directed edges, complete graph:
exhaustive search is the permutation problem
enumerate all n! permutations of n distinct items (cities) {0,1,2,3,…,n-1}
1
2
0
3
ij 0 1 2 3
0 .
1 .
2 .
3 .
rank in path array
3 1 4 2
0 1 2 3D[i][j]
TSP travelling salesman O(nn)input(V) // number of cities (Vertices)int[V] rIP // rankInPath, init to 0==not rankedvisit(1) // generate visiting sequences
void visit(int rank){ for (city = 0 to V-1) if (rIP[city] == 0)
// not yet visited { rIP[city] = rank; if (rank == V) fitness(rIP) // base case else visit(rank+1) //recursive case rIP[city] = 0 }}
fitness(int[] p)
calculate pathlength
from D[i][j]
if bestpath, save p
fitness(int[] p)
calculate pathlength
from D[i][j]
if bestpath, save p
efficiency without risk
fix first city O((n-1)n-1) use sets instead of searching array
O((n-1)!) == O((n-1)n) keep partial fitness values
reduce cost of fitness evaluation apply branch and bound (pruning)
BUT… still O(en)
Branch and bound
...
..T ..F
.FT.TT .TF .FF
FFTFTT FTF FFFTFTTTT TTF TFF
...
...
01
012
0123 0132
0
013
02
021
0213 0231
023
03
031
0312 0321
032
pathlength = 845
592 693 519
549126 274
Variations on TSP
undirected edges: D[i][j] == D[j][i] incomplete graph: some D[i][j] = null Euclidean distances (on an airplane)
cities are located on plane at (xi,yi)
D[i][j] is computed from coordinates:D[i][j] = D[j][i]
= sqrt((xi-xj)2 + (yi-yj)2)
other data structures, efficiencies
Continuousproblem spaces
Where isheight of land?1. what scale to sample?
x∈(0,1], y∈(0,1]interval length:0.1: 100 data points0.01: 10,0000.001: 1,000,000
--- x --
- y
Continuousproblem spaces
Where isheight of land?2. constraints –
ignore waterfewer data points but constraints must
be tested
--- x --
- y
Continuousproblem spaces
Where isheight of land?3. where to locate
sample --- x --
- y
Continuousproblem spaces - NLP
Non-Linear Programming problems
Typical problems are functions of multiple variables over domains of each
Maximize f(x1,x2,x3,…,xn)
for x1 D∈ 1, x2 D∈ 2, x3 D∈ 3,…, xn D∈ n
NLP problems are NP complete*Linear Programming problems are polynomial
solvable with provable optimum in O(nk)
Linear Programming*-Simplex Method
Fitness function and constraints all linearfitness:
example constraints:
*assuming you are familiar with approximation methods, p.69-76
€
f (X) = a0x0 + a1x
1 + a2x2 + ...+ anx
n
€
x i ≥ 0;0 ≤ i ≤ n
b0x0 + b1x1 + b2x2 + ...+ bnxn ≤ b
c0x0 + c1x1 + c2x2 + ...+ cnxn ≥ c
d0x0 + d1x1 + d2x2 + ...+ dnxn = d
Linear Programming
with these restrictions:feasible solution spaceis a convex polyhedron-> solution is at a vertex2D e.g.:
€
f (X) = a0x0 + a1x
1 + a2x2 + ...+ anx
n
€
x i ≥ 0;0 ≤ i ≤ n
b0x0 + b1x1 + b2x2 + ...+ bnxn ≤ b
c0x0 + c1x1 + c2x2 + ...+ cnxn ≥ c
d0x0 + d1x1 + d2x2 + ...+ dnxn = d
Simplex Method
starts at a vertex (usually xi = 0 ∀i)
and “walks along edges” to adjacentvertex, until optimum is reachedapprox O(n4)
Local Search in Space of Complete Solutions
Complete or Partial Solution Space?
...
..T ..F
.FT.TT .TF .FF
FFTFTT FTF FFFTFTTTT TTF TFF
2
1
0
...
space of partial
solutions
FFTFTT FTF FFFTFTTTT TTF TFF
space of complete solutions
Partial Solution Space
...
..T ..F
.FT.TT .TF .FF
FFTFTT FTF FFFTFTTTT TTF TFF
2
1
0
...
space of partial
solutions
• algorithms are based on tree traversals• must have a partial fitness function• performance is improved by tree ‘pruning’• coming soon to a lecture near you...
Complete Solution Space
...
..T ..F
.FT.TT .TF .FF
FFTFTT FTF FFFTFTTTT TTF TFF
...
space of complete solutions
• start with a (random or guided) solution• move to other solutions seeking better fitness• performance is improved by concentrating on
good solutions
Local Search
-ideal solution for convex functions
-partial solution for complex functions
(finds local optimum)
Local search algorithm
localSearch(point P in domain)evaluate fP = fitness(P)
repeat until no improvement in fitness transform P to a neighbouring point P’ evaluate fP’
if fP’ better than fP, P=P’
return P
1 DimensionalConvex space Example
Marking schemes
-25
0
25
50
75
100
0 6 12 18 24 30 36
attendance
grade
Marking schemes
-25
0
25
50
75
100
0 6 12 18 24 30 36
attendance
grade
localSearch(point h=19)evaluate fh = 20(19)-192=19
repeat until no improvement in fitness transform h to a neighbouring point h’ evaluate fh’
if fh’ better than fh, h=h’
return h = 10
m = 20h - h2m = 20h - h2
in this domain, neighbouring points of h are (h+1), (h-1)
h fh
18 3619 1917 5116 6415 7514 8413 9112 9411 9910 1009 99
2 Dimensionalspaces
4 local search neighbours
--- x --
- y
Would 8 neighbours be better?
Would 8 neighbours be better?
Local search with SAT: representation
Any logical expression can be written inConjunctive Normal Form (CNF) a conjunction (/\) of disjunctions (\/) of
propositions and negations:(~P1 \/ P3) /\ (P2 \/ ~P3) /\ (P4 \/ P1)
(A \/ B) -> C= ~(A \/ B)\/ C= (~A /\ ~B) \/ C= (~A \/ C) /\ (~B \/ C)
Local searchwith SAT
What is a “better”point in search space?
What is a “neighbouring” point?
localSearch(point P in domain)evaluate fP = fitness(P)repeat until no better fitness transform P to neighbour pt P’ evaluate fP’ if fP’ better than fP , P=P’return P
localSearch(point P in domain)evaluate fP = fitness(P)repeat until no better fitness transform P to neighbour pt P’ evaluate fP’ if fP’ better than fP , P=P’return P
P1 P2 P3 P4 (~P1 \/ P3) (P2 \/ ~P3) (P4 \/ ~P1) fitness
T F T F T F F F
FITNESS: number of
TRUE disjuncts
FITNESS: number of
TRUE disjuncts
11
neighbour has one proposition with different
boolean value
neighbour has one proposition with different
boolean value
Local searchwith SAT
P1 P2 P3 P4 (~P1 \/ P3) (P2 \/ ~P3) (P4 \/ ~P1) fitness
T F T F T F F 1
localSearch(point P in domain)evaluate fP = fitness(P)repeat until no better fitness transform P to neighbour pt P’ evaluate fP’ if fP’ better than fP , P=P’return P
localSearch(point P in domain)evaluate fP = fitness(P)repeat until no better fitness transform P to neighbour pt P’ evaluate fP’ if fP’ better than fP , P=P’return P
P1 P2 P3 P4 fitness
F F T F 2
P1 P2 P3 P4 fitness
T T T F 2
P1 P2 P3 P4 fitness
T F F F 1
P1 P2 P3 P4 fitness
T F T T 2
Local search with SATbut SAT space is not convex…
procedure GSAT // Michalewicz & Fogelbegin
for i = 1 to MAX_TRIES doT random truth assignmentfor j = 1 to MAX_FLIPS do
if T satisfies fitness return (T)flip a proposition truth value
endendreturn (“no solution found”)
end
Local searchwith TSP
what is a “better”point in search space?
EASY – shorter path
what is a “neighbouring” point?i.e., what change to transform a path to a similar path?
MANY possible definitions of transformation operators.
localSearch(point P in domain)evaluate fP = fitness(P)repeat until no better fitness transform P to neighbour pt P’ evaluate fP’ if fP’ better than fP , P=P’return P
localSearch(point P in domain)evaluate fP = fitness(P)repeat until no better fitness transform P to neighbour pt P’ evaluate fP’ if fP’ better than fP , P=P’return P
TSP and local search some neighbourhood definitions depend
on the type of TSP.e.g., 2-Opt:change two non-adjacent edges:ABCDEFG to ABEDCFG
pathlength =pathlength –BC –EF + BE + CFfails when?TSP with directed edges: DE ≠ ED, CD ≠ DC
AB
C
D
E
FG
Neighbourhood/Search tradeoff
TSP neighbourhood definitions
swap adjacent cities O(n)
2-Opt: change two non-adjacent edges O(n2)
3-Opt: change among three non adjacent edges
δ-path (Lin-Kernighan)
AB
C
D
E
FG
AB
C
D
E
3-Opt O(n3)
THREE possible (2-Opt) neighbours:
FOUR possible neighbours:
B C D E F G H J KA
B
E D C H G F
J KAF G H
H G F
C D E
E D C
B
C D E H G F
J KAE D C
H G F
F G H
E D C
δ-path (Lin-Kernighan)
based on 2-Opt with 1 city (A) fixed.
remove KA and consider all δ-paths
(e.g., by adding KE)- record best 2-Opt tour if any (e.g. EF FA)- BUT move to next tour according to best
δ-path length (e.g. including FA)
B C D E F G H J KA A
Lin-Kernighan local search
initial tour T, BestT = Tstart city A, last city L=Kadded_list =null; // added edges cannot be removeddeleted_list=null; // deleted edges can’t be re-addedrepeat
find best 2-Opt neighbour tour (maybe change BestT)move to tour with best δ-path < BestT if anyput added edge in added_list (e.g., KE)put deleted edge in deleted_list(e.g., EF)L = F
until no moves (all edges added or no short δ-path)
B C D E F G H J KA A
Variations on local searchp, point in search space, fitness f(p)neighbourhood Np = {p1,…,pk}
(1) find first pi with f(pi)> f(p)
or(2) find best pj such that f(pj) ≥ f(pi) 1≤ i ≤ k
… pk
p2
p1 f(p)
(2)
(1)