A Simple Example

“The Gene is by far the most sophisticated program around.”

- Bill Gates, Business Week, June 27, 1994

Genetic Algorithms - History

• Pioneered by John Holland in the 1970’s• Got popular in the late 1980’s• Based on ideas from Darwinian Evolution• Can be used to solve a variety of problems

that are not easy to solve using other techniques

The Genetic Algorithm

• Directed search algorithms based on the mechanics of biological evolution

• Developed by John Holland, University of Michigan (1970’s)– To understand the adaptive processes of natural

systems– To design artificial systems software that retains

the robustness of natural systems

The Genetic Algorithm (cont.)

• Provide efficient, effective techniques for optimization and machine learning applications

• Widely-used today in business, scientific and engineering circles

Evolution in the real world• Each cell of a living thing contains chromosomes - strings of

DNA• Each chromosome contains a set of genes - blocks of DNA• Each gene determines some aspect of the organism (like eye

colour)• A collection of genes is sometimes called a genotype• A collection of aspects (like eye colour) is sometimes called a

phenotype• Reproduction involves recombination of genes from parents

and then small amounts of mutation (errors) in copying • The fitness of an organism is how much it can reproduce

before it dies• Evolution based on “survival of the fittest”

Start with a Dream…

• Suppose you have a problem• You don’t know how to solve it• What can you do?• Can you use a computer to somehow find a

solution for you?• This would be nice! Can it be done?

A dumb solution

A “blind generate and test” algorithm:

RepeatGenerate a random possible solutionTest the solution and see how good it is

Until solution is good enough

Can we use this dumb idea?

• Sometimes - yes:– if there are only a few possible solutions– and you have enough time– then such a method could be used

• For most problems - no:– many possible solutions– with no time to try them all– so this method can not be used

A “less-dumb” idea (GA)

Generate a set of random solutionsRepeat

Test each solution in the set (rank them)Remove some bad solutions from setDuplicate some good solutions

make small changes to some of them

Until best solution is good enough

How do you encode a solution?

• Obviously this depends on the problem!• GA’s often encode solutions as fixed length

“bitstrings” (e.g. 101110, 111111, 000101)• Each bit represents some aspect of the

proposed solution to the problem• For GA’s to work, we need to be able to

“test” any string and get a “score” indicating how “good” that solution is

Silly Example - Drilling for Oil

• Imagine you had to drill for oil somewhere along a single 1km desert road

• Problem: choose the best place on the road that produces the most oil per day

• We could represent each solution as a position on the road

• Say, a whole number between [0..1000]

Where to drill for oil?

0 500 1000

Road

Solution2 = 900Solution1 = 300

Digging for Oil

• The set of all possible solutions [0..1000] is called the search space or state space

• In this case it’s just one number but it could be many numbers or symbols

• Often GA’s code numbers in binary producing a bitstring representing a solution

• In our example we choose 10 bits which is enough to represent 0..1000

Convert to binary string

512 256 128 64 32 16 8 4 2 1

900 1 1 1 0 0 0 0 1 0 0

300 0 1 0 0 1 0 1 1 0 0

1023 1 1 1 1 1 1 1 1 1 1

In GA’s these encoded strings are sometimes called “genotypes” or “chromosomes” and the individual bits are sometimes called “genes”

Drilling for Oil

0 1000

Road

Solution2 = 900 (1110000100)Solution1 = 300 (0100101100)

O I

L

Location

30

5

Summary

We have seen how to:• represent possible solutions as a number• encoded a number into a binary string• generate a score for each number given a function of

“how good” each solution is - this is often called a fitness function

• Our silly oil example is really optimisation over a function f(x) where we adapt the parameter x

Search Space

• For a simple function f(x) the search space is one dimensional.

• But by encoding several values into the chromosome many dimensions can be searched e.g. two dimensions f(x,y)

• Search space an be visualised as a surface or fitness landscape in which fitness dictates height

• Each possible genotype is a point in the space• A GA tries to move the points to better places

(higher fitness) in the the space

Fitness landscapes

Search Space

• Obviously, the nature of the search space dictates how a GA will perform

• A completely random space would be bad for a GA

• Also GA’s can get stuck in local maxima if search spaces contain lots of these

• Generally, spaces in which small improvements get closer to the global optimum are good

The GA Cycle of Reproduction

reproduction

population evaluation

modification

discard

deleted members

parents

children

modifiedchildren

evaluated children

Population

Chromosomes could be:– Bit strings (0101 ... 1100)– Real numbers (43.2 -33.1 ... 0.0 89.2) – Permutations of element (E11 E3 E7 ... E1 E15)– Lists of rules (R1 R2 R3 ... R22 R23)– Program elements (genetic programming)– ... any data structure ...

population

Reproduction

reproduction

population

parents

children

Parents are selected at random with selection chances biased in relation to chromosome evaluations.

Chromosome Modification

modificationchildren

• Modifications are stochastically triggered• Operator types are:

– Mutation– Crossover (recombination)

modified children

Back to the (GA) AlgorithmGenerate a set of random solutionsRepeat

Test each solution in the set (rank them)Remove some bad solutions from setDuplicate some good solutions

make small changes to some of them

Until best solution is good enough

Adding Sex - Crossover

• Although it may work for simple search spaces our algorithm is still very simple

• It relies on random mutation to find a good solution

• It has been found that by introducing “sex” into the algorithm better results are obtained

• This is done by selecting two parents during reproduction and combining their genes to produce offspring

Adding Sex - Crossover

• Two high scoring “parent” bit strings (chromosomes) are selected and with some probability (crossover rate) combined

• Producing two new offspring (bit strings)• Each offspring may then be changed

randomly (mutation)

Selecting Parents

• Many schemes are possible so long as better scoring chromosomes more likely selected

• Score is often termed the fitness• “Roulette Wheel” selection can be used:

– Add up the fitness's of all chromosomes– Generate a random number R in that range– Select the first chromosome in the population

that - when all previous fitness’s are added - gives you at least the value R

Example population

No. Chromosome Fitness1 1010011010 12 1111100001 23 1011001100 34 1010000000 15 0000010000 36 1001011111 57 0101010101 18 1011100111 2

Roulette Wheel Selection

1 2 3 1 3 5 1 2

0 18

21 3 4 5 6 7 8

Rnd[0..18] = 7

Chromosome4

Parent1

Rnd[0..18] = 12

Chromosome6

Parent2

Crossover - Recombination

1010000000

1001011111

Crossover single point - random

1011011111

1010000000

Parent1

Parent2

Offspring1

Offspring2

With some high probability (crossover rate) apply crossover to the parents. (typical values are 0.8 to 0.95)

Mutation

1011011111

1010000000

Offspring1

Offspring2

1011001111

1000000000

Offspring1

Offspring2

With some small probability (the mutation rate) flip each bit in the offspring (typical values between 0.1

and 0.001)

mutate

Original offspring Mutated offspring

Back to the (GA) Algorithm

Generate a population of random chromosomesRepeat (each generation)

Calculate fitness of each chromosomeRepeat

Use roulette selection to select pairs of parentsGenerate offspring with crossover and mutation

Until a new population has been produced

Until best solution is good enough

Many Variants of GA

• Different kinds of selection (not roulette)– Tournament– Elitism, etc.

• Different recombination– Multi-point crossover– 3 way crossover etc.

• Different kinds of encoding other than bitstring– Integer values– Ordered set of symbols

• Different kinds of mutation

Many parameters to set

• Any GA implementation needs to decide on a number of parameters: Population size (N), mutation rate (m), crossover rate (c)

• Often these have to be “tuned” based on results obtained - no general theory to deduce good values

• Typical values might be: N = 50, m = 0.05, c = 0.9

Why does crossover work?

• A lot of theory about this and some controversy

• Holland introduced “Schema” theory• The idea is that crossover preserves “good

bits” from different parents, combining them to produce better solutions

• A good encoding scheme would therefore try to preserve “good bits” during crossover and mutation

Genetic Programming

• When the chromosome encodes an entire program or function itself this is called genetic programming (GP)

• In order to make this work encoding is often done in the form of a tree representation

• Crossover entials swaping subtrees between parents

Genetic Programming

It is possible to evolve whole programs like this but only small ones. Large programs with complex functions present big problems

Implicit fitness functions

• Most GA’s use explicit and static fitness function (as in our “oil” example)

• Some GA’s (such as in Artificial Life or Evolutionary Robotics) use dynamic and implicit fitness functions - like “how many obstacles did I avoid”

• In these latter examples other chromosomes (robots) effect the fitness function

Problem

• In the Travelling Salesman Problem (TSP) a salesman has to find the shortest distance journey that visits a set of cities

• Assume we know the distance between each city• This is known to be a hard problem to solve because

the number of possible routes is N! where N = the number of cities

• There is no simple algorithm that gives the best answer quickly

Problem

• Design a chromosome encoding, a mutation operation and a crossover function for the Travelling Salesman Problem (TSP)

• Assume number of cities N = 10• After all operations the produced chromosomes

should always represent valid possible journeys (visit each city once only)

• There is no single answer to this, many different schemes have been used previously

Considering the GA Technology“Almost eight years ago ... people at Microsoft wrote a program [that] uses some genetic things for finding

short code sequences. Windows 2.0 and 3.2, NT, and almost all Microsoft

applications products have shipped with pieces of code created by that

system.”

- Nathan Myhrvold, Microsoft Advanced Technology Group, Wired, September 1995

Benefits of Genetic Algorithms

• Concept is easy to understand• Modular, separate from application• Supports multi-objective optimization• Good for “noisy” environments• Always an answer; answer gets better with

time• Inherently parallel; easily distributed

Benefits of Genetic Algorithms (cont.)

• Many ways to speed up and improve a GA-based application as knowledge about problem domain is gained

• Easy to exploit previous or alternate solutions• Flexible building blocks for hybrid applications• Substantial history and range of use

When to Use a GA

• Alternate solutions are too slow or overly complicated• Need an exploratory tool to examine new approaches• Problem is similar to one that has already been

successfully solved by using a GA• Want to hybridize with an existing solution• Benefits of the GA technology meet key problem

requirements

Some GA Application TypesDomain Application Types

Control gas pipeline, pole balancing, missile evasion, pursuit

Design semiconductor layout, aircraft design, keyboardconfiguration, communication networks

Scheduling manufacturing, facility scheduling, resource allocation

Robotics trajectory planning

Machine Learning designing neural networks, improving classificationalgorithms, classifier systems

Signal Processing filter design

Game Playing poker, checkers, prisoner’s dilemma

CombinatorialOptimization

set covering, travelling salesman, routing, bin packing,graph colouring and partitioning

Conclusions

Question: ‘If GAs are so smart, why ain’t they rich?’

Answer: ‘Genetic algorithms are rich - rich in application across a large and growing number of disciplines.’

- David E. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning

Using Genetic Algorithms [GAs] to both design composite materials and aerodynamic shapes for race cars and regular means of transportation (including aviation) can return combinations of best materials and best engineering to provide faster, lighter, more fuel efficient and safer vehicles for all the things we use vehicles for. Rather than spending years in laboratories working with polymers, wind tunnels and balsa wood shapes, the processes can be done much quicker and more efficiently by computer modeling using GA searches to return a range of options human designers can then put together however they please.

Getting the most out of a range of materials to optimize the structural and operational design of buildings, factories, machines, etc. is a rapidly expanding application of GAs. These are being created for such uses as optimizing the design of heat exchangers, robot gripping arms, satellite booms, building trusses, flywheels, turbines, and just about any other computer-assisted engineering design application. There is work to combine GAs optimizing particular aspects of engineering problems to work together, and some of these can not only solve design problems, but also project them forward to analyze weaknesses and possible point failures in the future so these can be avoided

Robotics involves human designers and engineers trying out all sorts of things in order to create useful machines that can do work for humans. Each robot's design is dependent on the job or jobs it is intended to do, so there are many different designs out there. GAs can be programmed to search for a range of optimal designs and components for each specific use, or to return results for entirely new types of robots that can perform multiple tasks and have more general application. GA-designed robotics just might get us those nifty multi-purpose, learning robots we've been expecting any year now since we watched the Jetsons as kids, who will cook our meals, do our laundry and even clean the bathroom for us

Evolvable hardware (EH) is a new field about the use of evolutionary algorithms (EA) to create specialized electronics without manual engineering. It brings together reconfigurable hardware, artificial intelligence, fault tolerance and autonomous systems. Evolvable hardware refers to hardware that can change its architecture and behavior dynamically and autonomously by interacting with its environment.

In its most fundamental form an evolutionary algorithm manipulates a population of individuals where each individual describes how to construct a candidate circuit. Each circuit is assigned a fitness, which indicates how well a candidate circuit satisfies the design specification. The evolutionary algorithm uses stochastic operators to evolve new circuit configurations from existing ones. Done properly, over time the evolutionary algorithm will evolve a circuit configuration that exhibits desirable behavior.

Each candidate circuit can either be simulated or physically implemented in a reconfigurable device. Typical reconfigurable devices are field-programmable gate arrays (for digital designs) or field-programmable analog arrays (for analog designs). At a lower level of abstraction are the field-programmable transistor arrays that can implement either digital or analog designs.

In many cases conventional design methods (formulas, etc.) can be used to design a circuit. But in other cases the design specification doesn't provide sufficient information to permit using conventional design methods. For example, the specification may only state desired behavior of the target hardware.

The fitness of an evolved circuit is a measure of how well the circuit matches the design specification. Fitness in evolvable hardware problems is determined via two methods::extrinsic evolution: all circuits are simulated to see how they performintrinsic evolution : physical tests are run on actual hardware.In extrinsic evolution only the final best solution in the final population of the evolutionary algorithm is physically implemented, whereas with intrinsic evolution every individual in every generation of the EA's population is physically realized and tested.

Evolvable hardware problems fall into two categories: original design and adaptive systems. Original design uses evolutionary algorithms to design a system that meets a predefined specification. Adaptive systems reconfigure an existing design to counteract faults or a changed operational environment.

Do you find yourself frustrated by slow LAN performance, inconsistent internet access, a FAX machine that only sends faxes sometimes, your land line's number of 'ghost' phone calls every month? Well, GAs are being developed that will allow for dynamic and anticipatory routing of circuits for telecommunications networks. These could take notice of your system's instability and anticipate your re-routing needs. Using more than one GA circuit-search at a time, soon your interpersonal communications problems may really be all in your head rather than in your telecommunications system. Other GAs are being developed to optimize placement and routing of cell towers for best coverage and ease of switching, so your cell phone and blackberry will be thankful for GAs too.

New applications of a GA known as the "Traveling Salesman Problem" or TSP can be used to plan the most efficient routes and scheduling for travel planners, traffic routers and even shipping companies. The shortest routes for traveling. The timing to avoid traffic tie-ups and rush hours. Most efficient use of transport for shipping, even to including pickup loads and deliveries along the way. The program can be modeling all this in the background while the human agents do other things, improving productivity as well! Chances are increasing steadily that when you get that trip plan packet from the travel agency, a GA contributed more to it than the agent did

On the security front, GAs can be used both to create encryption for sensitive data as well as to break those codes. Encrypting data, protecting copyrights and breaking competitors' codes have been important in the computer world ever since there have been computers, so the competition is intense. Every time someone adds more complexity to their encryption algorithms, someone else comes up with a GA that can break the code. It is hoped that one day soon we will have quantum computers that will be able to generate completely indecipherable codes. Of course, by then the 'other guys' will have quantum computers too, so it's a sure bet the spy vs. spy games will go on indefinitely

The de novo design of new chemical molecules is a burgeoning field of applied chemistry in both industry and medicine. GAs are used to aid in the understanding of protein folding, analyzing the effects of substitutions on those protein functions, and to predict the binding affinities of various designed proteins developed by the pharmaceutical industry for treatment of particular diseases. The same sort of GA optimization and analysis is used for designing industrial chemicals for particular uses, and in both cases GAs can also be useful for predicting possible adverse consequences. This application has and will continue to have great impact on the costs associated with development of new chemicals and drugs

The development of microarray technology for taking 'snapshots' of the genes being expressed in a cell or group of cells has been a boon to medical research. GAs have been and are being developed to make analysis of gene expression profiles much quicker and easier. This helps to classify what genes play a part in various diseases, and further can help to identify genetic causes for the development of diseases. Being able to do this work quickly and efficiently will allow researchers to focus on individual patients' unique genetic and gene expression profiles, enabling the hoped-for "personalized medicine" we've been hearing about for several years

In the current unprecedented world economic meltdown one might legitimately wonder if some of those Wall Street gamblers made use of GA-assisted computer modeling of finance and investment strategies to funnel the world's accumulated wealth into what can best be described as dot-dollar black holes. But then again, maybe they were simply all using the same prototype, which hadn't yet been de-bugged. It is possible that a newer generation of GA-assisted financial forecasting would have avoided the black holes and returned something other than bad debts the taxpayers get to repay. Who knows

Those who spend some of their time playing computer Sims games (creating their own civilizations and evolving them) will often find themselves playing against sophisticated artificial intelligence GAs instead of against other human players online. These GAs have been programmed to incorporate the most successful strategies from previous games - the programs 'learn' - and usually incorporate data derived from game theory in their design. Game theory is useful in most all GA applications for seeking solutions to whatever problems they are applied to, even if the application really is a game