alexandria acm sc | introduction to genetic algorithms
TRANSCRIPT
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
1/45
INTRODUCTION TOGENETIC
ALGORITHMS
BY
AHMED MEDHAT OTHMAN
@amedhat3
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
2/45
AGENDA
1. Introduction
2. Natural Inspired Computing
3. Classical Computation vs. bio-inspired computing
4. Evolution in the real world5. Collaborative discussion
6. Problem solving
7. Genetic Algorithms (GA)
8. Anatomy of a GA
9. Advantages of GA
10. Limitations of GA
11. Related techniques12. Summary
2
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
3/45
INTRODUCTION
3
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
4/45
INTRODUCTION
4
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
5/45
INTRODUCTION
5
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
6/45
INTRODUCTION
6
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
7/45
INTRODUCTION
7
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
8/45
INTRODUCTION
8
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
9/45
INTRODUCTION
The only intelligent systems on this planet are biological. Biological intelligences are designed by natural
evolutionary processes.
They often work together in groups, swarms, or flocks. They don't appear to use logic, mathematics, complex
planning, complicated modeling of their environment.
They can achieve complex information processing andcomputational tasks that current artificial intelligences
find very challenging indeed.
9
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
10/45
NATURAL INSPIRED
COMPUTING
10
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
11/45
NATURAL INSPIRED
COMPUTING
In other words Biologically Inspired Computing. Biological organisms cope with the demands of their
environments.
They uses solutions quite unlike the traditional human-engineered approaches to problem solving.
They exchange information about what theyve discoveredin the places they have visited.
Bio-inspired computing is a field devoted to tacklingcomplex problems using computational methods modeled
after design principles encountered in nature.
11
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
12/45
CLASSICAL COMPUTATION
VS.
BIO-INSPIRED COMPUTING
12
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
13/45
CLASSICAL COMPUTATION
VS.
BIO-INSPIRED COMPUTING
Classical computing is good at: Number-crunching Thought-support (glorified pen-and-paper) Rule-based reasoning Constant repetition ofwell-definedactions.
Classical computing is bad at:
Pattern recognition Robustness to damage Dealing with vague and incomplete information; Adapting and improving based on experience
13
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
14/45
CLASSICAL COMPUTATION
VS.
BIO-INSPIRED COMPUTING
Bio-inspired computing takes a more evolutionaryapproach to learning.
In traditional AI, intelligence is often programmed fromabove. The Programmer create the program and imbues it
with its intelligence.
Bio-inspired computing, on the other hand, takes a morebottom-up, decentralized approach.
Bio-inspired computing often involve the method ofspecifying a set of simple rules, a set of simple organisms
which adhere to those rules.
14
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
15/45
EVOLUTION IN THE
REAL WORLD
Each cell of a living thing containschromosomes - strings ofDNA.
Each chromosome contains a set ofgenes - blocks of DNA.
Each gene determines some aspectof the organism (like eye colour).
A collection ofgenes is sometimescalled a genotype.
A collection ofaspects (like eyecharacteristics) is sometimes called
a phenotype.
15
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
16/45
EVOLUTION IN THE
REAL WORLD
Reproduction involves recombination of genes from parentsand then small amounts ofmutation (errors) in copying.
The fitness of an organism is how much it can reproducebefore it dies.
Evolution based on survival of the fittest.
16
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
17/45
COLLABORATIVE
DISCUSSION
What we can learn from nature? Applications of nature in the engineering field.
17
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
18/45
PROBLEM SOLVING
Suppose you have a problem.
You dont know how to solve it. What can you do? Can you use a computer to somehow find a solution? This would be nice! Can it be done?
18
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
19/45
PROBLEM SOLVING
Brute-Force Solution
A blind generate and test algorithm:
Repeat
Generate a random possible solution
Test the solution and see how good it is
Until solution is good enough
19
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
20/45
PROBLEM SOLVING
Can we use this Brute-Force idea?
Sometimes -YES: if there are only a few possible solutions and you have enough time then such a method couldbe used
For most problems - NO: many possible solutions with no time to try them all so this method cannotbe used
20
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
21/45
PROBLEM SOLVING
Search Techniques
Calculus Base
TechniquesGuided random search
techniques
Enumerative
Techniques
BFSDFS Dynamic
Programming
Tabu Search HillClimbing
SimulatedAnnealing
EvolutionaryAlgorithms
Genetic
Programming
Genetic
Algorithms
Fibonacci Sort
21
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
22/45
GENETIC ALGORITHMS
22
Initialize Population
satisfy constraints ?
Evaluate Fitness
Select Survivors
Output Results
Randomly Vary Individuals
Yes
No
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
23/45
GENETIC ALGORITHMS
How do you encode a solution?
Obviously this depends on the problem! GAs often encode solutions as fixed length
bitstrings (e.g. 101110, 111111, 000101)
Each bit represents some aspect of the proposed solutionto the problem
For GAs to work, we need to be able to test any stringand get a score indicating how good that solution is.
23
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
24/45
GENETIC ALGORITHMS
The set of all possible solutions [0..1000] is called thesearch space orstate space.
In this case its just one number but it could be manynumbers.
Often GAs code numbers in binary producing a bitstringrepresenting a solution.
We choose 1,0 bits which is enough to represent 0..1000
24
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
25/45
GENETIC ALGORITHMS
Example: encoding 4 parameters Param1 value = 1000 = 8 Param2 value = 1011 = 11 Etc.,
25
Binary Representation
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
26/45
GENETIC ALGORITHMS
Search Space
For a simple function f(x) the search space is onedimensional.
But by encoding several values into the chromosomemany dimensions can be searched e.g. two dimensionsf(x,y).
Search space can be visualised as a surface orfitnesslandscape in which fitness dictates height.
Each possible genotype is a point in the space. A GA tries to move the points to better places (higher
fitness) in the the space.
26
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
27/45
GENETIC ALGORITHMS
Fitness landscapes
27
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
28/45
GENETIC ALGORITHMS
Implicit fitness functions
Most GAs use explicit and static fitness function Some GAs (such as in Artificial Life or Evolutionary Robotics) use
dynamic and implicit fitness functions - like how many obstacles
did I avoid
Individuals fitness
Average fitness of population
28
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
29/45
GENETIC ALGORITHMS
Example - Drilling for Oil
Imagine you have to drill for oil somewhere along a single1 km desert road.
Problem: choose the best place on the road that producesthe most oil per day.
We could represent each solution as a position on theroad.
Say, a whole number between [0..1000]
29
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
30/45
GENETIC ALGORITHMS
Where to drill for oil?
30
0 500 1000
Road
Solution2 = 900Solution1 = 300
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
31/45
GENETIC ALGORITHMS
In GAs these encoded strings are sometimes calledgenotypesor chromosomes and the individual bits are
sometimes called genes
31
512 256 128 64 32 16 8 4 2 1
900 1 1 1 0 0 0 0 1 0 0
300 0 1 0 0 1 0 1 1 0 0
1023 1 1 1 1 1 1 1 1 1 1
Convert to binary string
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
32/45
GENETIC ALGORITHMS
32
0 1000
Road
Solution2 = 900
(1110000100)
Solution1 = 300
(0100101100)
OI
L
Location
30
5
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
33/45
ANATOMY OF GA
Selecting Parents
Many schemes are possible so long as better scoringchromosomes more likely selected
Score is often termed the fitness Roulette Wheel selection can be used:
Add up the fitness's of all chromosomesGenerate a random number R in that range
Select the first chromosome in the population that - when allprevious fitnesss are added - gives you at least the value R
33
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
34/45
ANATOMY OF GA
34
Example population
No. Chromosome Fitness1 1010011010 12
1111100001
2
3 1011001100 34 1010000000 15 0000010000 36 1001011111 57 0101010101 18 1011100111 2
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
35/45
ANATOMY OF GA
35
Roulette Wheel Selection
1 2 3 1 3 5 1 2
0 18
21 3 4 5 6 7 8
Rnd[0..18] = 7
Chromosome4
Parent1
Rnd[0..18] = 12
Chromosome6
Parent2
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
36/45
ANATOMY OF GA
36
Crossover - Recombination
0100101100
1110000100
Crossoversingle point -
random
0100000100
1110101100
Parent1
Parent2
Offspring1
Offspring2
With some high probability (crossover rate) applycrossover to the parents.
(typical values are 0.8 to 0.95)
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
37/45
Why does crossover work?
A lot of theory about this and some controversy The idea is that crossover preserves good bits from
different parents, combining them to produce better
solutions
A good encoding scheme would therefore try to preservegood bits during crossover and mutation
37
ANATOMY OF GA
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
38/45
ANATOMY OF GA
38
Mutation
1011011111
1010000000
Offspring1
Offspring2
1011001111
1000000000
Offspring1
Offspring2
mutate
Original offspring Mutated offspring
With some small probability (the mutation rate) flip each bit in
the offspring (typical values between 0.1 and 0.001)
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
39/45
ANATOMY OF GA
39
Many Variants of GA
Different kinds of selection (not roulette) Tournament Elitism, etc.
Different recombination Multi-point crossover 3 way crossover etc.
Different kinds of encoding other than bitstringInteger values
Ordered set of symbols Different kinds of mutation
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
40/45
ADVANTAGES OF GA
Concepts are easy to understand Genetic Algorithms are intrinsically parallel. Always an answer; answer gets better with time Inherently parallel; easily distributed Less time required for some special applications Chances of getting optimal solution are more
40
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
41/45
LIMITATIONS OF GA
The population considered for the evolution should bemoderate or suitable one for the problem (normally 20-30 or
50-100)
Crossover rate should be 80%-95% Mutation rate should be low i.e. 0.5%-1% assumed as best The method of selection should be appropriate Writing of fitness function must be accurate
41
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
42/45
RELATED TECHNIQUES
Genetic programming Evolutionary programming Swarm intelligence
Ant colony optimization Particle swarm optimization Intelligent Water Drops Bees algorithm
Neural Networks42
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
43/45
SUMMARY
43
Representation Binary strings
Recombination N-point or uniform
Mutation Bitwise bit-flipping with fixedprobability
Parent selection Fitness-Proportionate
Survivor selection All children replace parents
Speciality Emphasis on crossover
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
44/45
4
4
-
7/27/2019 Alexandria ACM SC | Introduction to Genetic Algorithms
45/45
THANKS
Contact information Email: [email protected] Website: http://amedhat.info Twitter: @amedhat3 Linkedin: linkedin.com/in/amedhat
5