emergence & the information processing paradigm (alt: ants vs clocks!)

Emergence & The Information Processing Paradigm(alt: Ants vs Clocks!)

Deepak Kumar

Computer Science

Bryn Mawr College

Emergence

Information Processing

Problem Solving(Investigation of Processes)

• Given a dictionary of english words, what are all the anagram classes? (e.g. earthling, haltering, lathering)

• Given the details of a tropical depression, can you predict if it will become a hurricane? What path will it take?

• Can you play the game of chess in a way that guarantees a win or a draw?

• How does the mind work?

• What is the most optimal way to get from here to there?

• What is the square root of 42?

• If the Fed. Raises the short term interest rates? What impact will it have on international currencies?

• What is the meaning of life?

Investigation: Asking Questions

Given: A problem/process.

• Does it have a model?• Is the model solvable?• Is it computable?• What is the best algorithm for it?• Write a computer program that implements the

algorithm.• Is the program equivalent to the model?• Does the model lend any new insights into the

problem/process?

Given: A ProblemDoes it have a model?

Is the model solvable?

Is it computable?

What is the best algorithm?

Write a program thatimplements the algorithm thatmodels the problem

Is the program correct?

Do the solutions offered bythe program work in the realworld?

Does the model/program offerany new insights into the problem?

Information Processing

Program thatimplements themodel that solvesthe problem.

Information Processing

Program thatimplements themodel that solvesthe problem.

Computers & Clocks• 1 flop = 1 floating-point operation (+, -, *, /) per second.• 1 megaflop = 1 MF = 106 flops• 1 gigaflop = 1 GF = 109 flops• 1 teraflop = 1 TF = 1012 flops• 1 petaflop = 1 PF = 1015 flops

The fastest PC can give a peak performance of 1GF/second

World’s fastest computer operates at approximately 35 TF/sec(as of 11/16/2003): NEC’s Earth Simulator at the Earth Simulator Center in Yokohama, Japan

Examining a single move of a chess game is roughly equivalent to 1000 FLOPS.

To make one move the computer will have to examine approximately 1065 board situations.

This will take the Earth Simulator approximately 1050 years to make one move!

Computability: Problems that can be solved by algorithms (Turing Machines)

Does it have a model?

Is the model solvable?

Is it computable?

What is the best algorithm?

Write a program thatimplements the algorithm thatmodels the problem

Is the program correct?

Do the solutions offered bythe program work in the realworld?

Does the model/program offerany new insights into the problem?

Complexity Theory: Resources required during computation to solve a given problem

(time & space)Does it have a model?

Is the model solvable?

Is it computable?

What is the best algorithm?

Write a program thatimplements the algorithm thatmodels the problem

Is the program correct?

Do the solutions offered bythe program work in the realworld?

Does the model/program offerany new insights into the problem?

What is an algorithm?

A set of instructions arranged in a specific order is a procedure.

Similar to a recipe, process, method, technique, procedure, routine, rigmarole, except the word “algorithm” connotes just a little something different.

An algorithm is a finite, definite, effective procedure, with some output.

Donald Knuth: The Art of Computer Programming, Volume 1: Fundamental Algorithms, 3 rd edition, 1997.

Algorithm (origins)Abu ‘Abd Allah Muhammad ibn Musa al-Khwarizmi

(Father of Abdullah, Mohammad, son of Moses, native of Khwarizm)

Kitab al jabr wa’l-muqabala (Rules of restoring and equating)

Finitethere must be an end to it within a reasonable time

DefinitePrecisely definable in clearly understood terms, no “pinch of salt” type vagaries, or possible ambiguities

EffectiveIt must be possible to actually carry out the steps

ProcedureThe sequence of specific steps

Outputunless there is something coming out of the computation, the result will be unknown!

Algorithm (properties)

Problem Size

• Time complexity of a problem is the number of steps that it takes to solve an instance of the problem as a function of the size of the input. i.e. if the input is of size, n, it will take f(n) steps to solve it.

Program thatimplements themodel that solvesthe problemIn f(n) steps.

Problem size = 4

Complexity Classes(# steps it takes to solve a problem)

• Constant time: O(1)• Logarithmic time: O(log n)• Linear time: O(n)• Quadratic Time: O(n2)• Polynomial time (P): O(nk)• Non-deterministic Polynomial Time (NP): O(nk)

on some inputs.• Exponential time: O(2n)• Exponential time (in general): O(2p(n))

# timesteps

Solving hard problems: Options

• solve small problem sizes

• use heuristics• get a close enough

answer!

• Emergence!

Emergence as a methodology for solving problems

• What kinds of problems?no models exist, unsolvable models, uncomputable models, exponential problems!

• How? “many ants are better than one”Train ‘em!

The Emergence Approach to Problem Solving

TrainingRegime

ProblemInstance

Solution“ANTS”

Key Idea: Known solutions to problem instances are used in training.

On “Graduation”

ProblemInstance

Solution“ANTS”

The system will generalize to solve other problem instances!

Examples: Neural Networks

ProblemInstance

Solution

SupervisedTraining

Examples: Self-Organizing Maps

ProblemInstance

Solution

Unsupervised Training!

Examples: Genetic Programming

ProblemInstances

Solutions

FitnessFunction

Emergence & Problem Solving

Are solutions to severalproblem instances known?

Given a solution, is it possibleto quantify the “quality” of the solution?

Identify the training regimeand implement it.

Does the solution offered by thetrained program work in the real world?

Does the resulting program offer anyinsights into the problem?

Some characteristics of emergent problem solving

• Rather than starting from a model, start with a set of known problem-solution instances

• Solutions may not be exact, but characterized as “best” solutions under the circumstances

• Training takes time• Once trained, all solutions take the same

amount of time, independent of the size of the problem!

• Emergent solutions can beat the clock!• Emergent Computation = Surreal computation?