darwinian invention & problem solving by means of genetic programming koza book series presented...

Darwinian Invention & Darwinian Invention & Problem Solving by means of Problem Solving by means of

Genetic ProgrammingGenetic Programming

Koza Book SeriesKoza Book Series

Presented by:Presented by:

Luke WissmannLuke Wissmann

Luke Wissmann--Darwinian Invention using MachinesLuke Wissmann--Darwinian Invention using Machines

Can Machines be Creative?Can Machines be Creative?

““No machine can compare with a man’s hands. No machine can compare with a man’s hands. Machinery gives speed, power, complete Machinery gives speed, power, complete

uniformity, and precision, but it cannot give uniformity, and precision, but it cannot give creativity, adaptability, freedom, heterogeneity. creativity, adaptability, freedom, heterogeneity. These the machine is incapable of, hence the These the machine is incapable of, hence the superiority of the hand, which no amount of superiority of the hand, which no amount of

rationalismrationalism can negate.” can negate.”

-Soetsu Yanagi (1972)-Soetsu Yanagi (1972)

How can computers be How can computers be made to do what needs to made to do what needs to

be done, without being be done, without being told exactly how to do it?told exactly how to do it?

Paraphrasing Arthur Samuel (1959)


John KozaJohn Koza

• University of Michigan

• Set up a company that built computer systems for state lotteries that printed instant lottery tickets.

• More than 800 papers have been published on GP since 1992 and, 51 PhD students are preparing theses on the subject. (’98)

Consulting Professor (Medical Informatics)Stanford Biomedical Informatics Department of MedicineSchool of Medicine

Department of Electrical EngineeringSchool of EngineeringStanford University


Presentation OutlinePresentation Outline

Koza VideoKoza Video Description of Genetic ProgrammingDescription of Genetic Programming Design of Analog Circuits using GP’sDesign of Analog Circuits using GP’s

Koza BooksKoza Books A:I ratio as a metric of machine creativityA:I ratio as a metric of machine creativity Koza’s 8 Human-Competitive Criteria Koza’s 8 Human-Competitive Criteria Koza’s SuccessesKoza’s Successes Artificial Invention—Necessary Components?Artificial Invention—Necessary Components? Back to the LiteratureBack to the Literature


Koza Book SeriesKoza Book Series

Book 1: Introduced GP principlesBook 1: Introduced GP principlesBook 2: Automatically defined functions Book 2: Automatically defined functions

and scalabilityand scalabilityBook 3: Darwinian invention and problem Book 3: Darwinian invention and problem

solvingsolvingBook 4: Routine invention problem solvingBook 4: Routine invention problem solving


Koza’s Answer: 16 AttributesKoza’s Answer: 16 Attributes1.1. Starts with ‘what needs to be done’Starts with ‘what needs to be done’2.2. Tells us ‘how to do it’Tells us ‘how to do it’3.3. Produces computer programProduces computer program4.4. Automatic determination of program sizeAutomatic determination of program size5.5. Reuses codeReuses code6.6. Reuses parametersReuses parameters7.7. Internal storageInternal storage8.8. Iterations, loops, recursionsIterations, loops, recursions9.9. Self-organization of hierarchiesSelf-organization of hierarchies10.10. Automatic determination of program architectureAutomatic determination of program architecture11.11. Wide range of program constructsWide range of program constructs12.12. Well-definedWell-defined13.13. Program independentProgram independent14.14. Wide applicabilityWide applicability15.15. ScalabilityScalability16.16. Competitive with human-produced resultsCompetitive with human-produced results

How can computers be made How can computers be made to do what needs to be done, to do what needs to be done,

without being told exactly without being told exactly how to do it?how to do it?

Interesting!!!!


Human-Competitive ResultsHuman-Competitive Results

to a million to a million significant digits?significant digits?

Speed of calculation?Speed of calculation? Balancing a Balancing a

checkbook?checkbook?

We are not talking about deterministic calculations & Behavior!!!

I

A


The 8 Human-Competitive CriteriaThe 8 Human-Competitive Criteria

1.1. Result was or could be patentedResult was or could be patented2.2. Equal or better (EoB) than published scientific result at Equal or better (EoB) than published scientific result at

that timethat time3.3. EoB than result placed in internationally paneled EoB than result placed in internationally paneled

scientific databasescientific database4.4. Result publishable now regardless of sourceResult publishable now regardless of source5.5. EoB than most recent solution to ‘big’ problemEoB than most recent solution to ‘big’ problem6.6. EoB than result considered an achievement within field EoB than result considered an achievement within field

in pastin past7.7. Solves problem of indisputable difficulty within fieldSolves problem of indisputable difficulty within field8.8. Holds it own or wins regulated competitionHolds it own or wins regulated competition

All measure the worth of a result outside the field of artificial intelligence


36 Human-Competitive Results36 Human-Competitive Results(May 2003)(May 2003)

21 previously patented results21 previously patented results 6 after January 20006 after January 2000

AlgorithmsAlgorithms Information science, biologyInformation science, biology

Soccer-playing programSoccer-playing program CircuitsCircuits Controllers and tuning rulesControllers and tuning rules Yagi-Uda antennaYagi-Uda antenna Classifier programClassifier program


Necessary Elements Necessary Elements for Artificial Invention (GP style)for Artificial Invention (GP style)

Design that can be created in a Design that can be created in a hierarchical fashionhierarchical fashion

Behavior of elements must be well Behavior of elements must be well understoodunderstood

Elements can be described quantitativelyElements can be described quantitativelyQuantitative performance metricsQuantitative performance metricsSimulation techniqueSimulation technique


Putting A & B together…….Putting A & B together…….

“Learning, then, in the most general sense of the word, consists of putting two and two or A and B together…. But let

me repeat that such lightning inductions based on a single case are possible only if MA and MB are well established,

flexible matrices which the subject knows ‘inside out’.”

-Arthur Koestler (The Act of Creation)


Inventors as CriticsInventors as Critics

“If the shortcomings of things are what drive their evolution, then inventors must be among technology’s

severest critics. They are, and it is the inventor’s unique ability not only to realize what is wrong with

existing artifacts but also to see how such wrong may be righted in order to provide increasingly more

sophisticated gadgets and devices.”

From: The Evolution of Useful Things,

Henry Petroski (1992)

CreativityBlocks

Perceptual Blocks-Stereotyping-Isolating the problem-Limiting the solution space-Not using various viewpoints-Sensory saturation-Limiting sensory inputs

Emotional Blocks-Fear of failure-Inability to tolerate ambiguity-Judging instead of generating ideas-Inability to ‘sleep-on-it’-No challenge or interest-Over-motivation to succeed-No imagination-Too much imagination-Inability to distinguish reality from fantasy

Cultural and Environmental-Fantasize and be playful-Use intuition-Use non-traditional approach-Money and science can’t solve all problems-Trust and value others-Limit distractions-Find support

Intellectual and Expressive-Use the most efficient language-Be flexible in the way you decideto solve the problem-Use the right information-Build language skills(mathematical, symbolic, etc.)

Alternate Languages-Mathematics-Visual Thinking-Others

Source: Conceptual Blockbusting, by James Adams

Can computers be creative?Can computers be creative?

Harold Cohen or AARON?

http://crca.ucsd.edu/~hcohen/


An idea or invention can be creative

An individual (or computer) can be intelligent or adept.

Confusion????Confusion????

Behavior X


Invention vs. DesignInvention vs. Design

Invention—process of discovering a principleInvention—process of discovering a principle

Design—process of applying that principleDesign—process of applying that principleSource: Pye (1964)

Do GP’s discover principles or apply principles?


A Slight Change…….A Slight Change…….

Idea/Invention

FieldDomain

Product/Process

Selective Agent(s)Drawings &Prototypes

Mapping

CreativityTriad

InnovationTriad


“To remaining skeptics who doubt the inventive competence of genetics and evolution, I say read this book and change your mind or risk the strong possibility that your doubts will soon cause you significant intellectual embarrassment.”

—David E. Goldberg, University of Illinois


When Machines are able to When Machines are able to Discover Discover andand Apply Principles …..Apply Principles …..


SummarySummary

Genetic programmingGenetic programming It’s It’s COOL!!COOL!! It can be used to routinely invent (design)It can be used to routinely invent (design)The solutions can compete with those of humansThe solutions can compete with those of humansThere seems to be a list of necessary elementsThere seems to be a list of necessary elements It gets past some common barriers to creative It gets past some common barriers to creative

solutionssolutionsNew Shirt for Koza FundNew Shirt for Koza Fund


ReferencesReferences

Pye, D. (1964). Pye, D. (1964). The Nature of DesignThe Nature of Design. . Reinhold: New York, NY.Reinhold: New York, NY.

Yanagi, S. (1972). Yanagi, S. (1972). The Unkown Craftsman: The Unkown Craftsman: A Japanese Insight into BeautyA Japanese Insight into Beauty (Adapted (Adapted by Bernard Leach). Kodansha by Bernard Leach). Kodansha International: New York, NY.International: New York, NY.

Petroski, H. (1992). Petroski, H. (1992). The Evolution of Useful The Evolution of Useful ThingsThings. Vintage: New York, NY.. Vintage: New York, NY.

darwinian invention & problem solving by means of genetic programming koza book series presented...

Documents

darwinian invention

literatureluke wissmann

luke wissmannluke wissmann

scientific result

patented result

problem solvingbook

computer systems

gp principlesbook