mining the computational universe

Mining the computable universeMost prominent findings

Fractal Dimensions versus Time Complexity

Mining the computational universeSlowdown and speed-up phenomena in the micro-cosmos of

small Turing machines

Joost J. Joosten1, Fernando Soler Toscano2 and Hector Zenil3

Dept. Logica, Historia i Filosofia de la Ciencia, Universitat de Barcelona for 1,Grupo de Logica, Lenguaje e Informacion, Universidad de Sevilla for 2, Wolfram

Research for 3

February 21, 2011Utrecht

Joost J. Joosten1, Fernando Soler Toscano2 and Hector Zenil3 Mining the computational universe

Small Turing MachinesThe Experiment

I An experimental investigation to foundational questions incomputer science.

I Just mine the universe of all possible computers of some kindand explore what’s out there

I In particular we are interested in a relation between

I Size of the computers vs.I Time they need to perform a particular task

I Our expectations are to find some trade-offs:

I Increasingly more complex computers are capable to do thingsmore efficiently until the optimal time performance(complexity class) is attained

I Size of the computers vs.

I Time they need to perform a particular task

I The computers that we study are small Turing machines.

I Let us briefly state our definition of a Turing Machine

I A Turing Machine (TM) consists of a head moving over aone-way infinite tape consisting of infinitely many discrete cells

I The head can be in a finite number s of states

I Each cell of the tape contains one of k possible letters ofsome finite alphabet

I We call Turing Machines with these characteristics s, k TuringMachines

I Our TMs operate in discrete time-steps

I At each time step the machine evaluates

what symbol is on the current tape cell (i.e., wherethe head currently is) and,

what state the head is in

I Depending on this, the machine looks up in its defining tablewhat action it should take

I An action consist of

I writing a new (possible the same) symbol on the current cellI move the head either left or rightI go to a new (possible the same) state

I An action consist ofI writing a new (possible the same) symbol on the current cell

I move the head either left or rightI go to a new (possible the same) state

I An action consist ofI writing a new (possible the same) symbol on the current cellI move the head either left or right

I go to a new (possible the same) state

I An action consist ofI writing a new (possible the same) symbol on the current cellI move the head either left or rightI go to a new (possible the same) state

I We employ a standard (NKS) and concise way to depict smallTMs

I Instead of letters of an alphabet we will also speak of colors

TM number 323

State 1:

State 2:

I We employ a standard (NKS) and concise way to depict smallTMs

I Instead of letters of an alphabet we will also speak of colors

TM number 323

State 1:

State 2:

TM number 323

State 1:

State 2:

I We look at all 2,2 TMs and see what functions they cancompute,

I once we fix a definition of what computing a function means.

I Next we look at all 3,2 TMs and see which functions theycompute.

I We compare runtimes of TMs in 2,2 to equivalent TMs in 3,2.

I Equivalent meaning, computing the same function.

I Input is represented unary in the sense that input n isrepresented by n + 1 consecutive black cells

I The computations starts with the head in the first cell(counting from the end of the tape) in State 1

I The computation halts when the head ”drops off” the tape

I The tape content after halting is called the output

I We say that two TMs compute the same function when on allfirst 21 inputs 0, 1, . . . 20 the output is an identical tapeconfiguration

I We overcome the Halting Problem and Rice’s Theorem by thefollowing pragmatic approach

I A TM τ on input x diverges by definition if it has not haltedafter 1000 steps

I Two TMs are computing the same function if they, using ouralternative definition of divergence, are the same on the first21 inputs

I There is strong – both theoretical and experimental –evidence that error margins drop exponentially (in nr. of stepsor nr. of inputs)

Here we see what function is computed by the 2,2 TM 323

fH0L= 1fH7L= 170 fH14L= 21845

fH1L= 2 fH8L= 341fH15L= 43690

fH2L= 5 fH9L= 682 fH16L= 87381

fH3L= 10fH10L= 1365 fH17L= 174762

fH4L= 21 fH11L= 2730fH18L= 349525

fH5L= 42 fH12L= 5461 fH19L= 699050

fH6L= 85 fH13L= 10922fH20L= 1398101

We will omit the input sequence and only show the outputsequence to represent a function.

In case of our TM 323 we get the following graphical representation

The Halting ProblemThe Extensionality ProblemSpeed-up versus Slow-downClusteringTypes of computation

I The computed output data really gives rise to an interestingmicro-cosmos of small Turing Machines

I First we noticed that 1000 steps were not enough todetermine a function on the first 21 inputs

I We filled out were we saw the divergence was evidentlynon-genuine

I This was a very time-consuming process for the 74 differentfunctions in 2,2 and 4012 functions in 3,2

I We observed a theoretical result of Calude’s that a TM eithernever halts or halts very fast

I However, we saw in 2,2 some very pronounced”phase-transitions” in these halting distributions

I We would expect these ”phase-transitions” to be lesspronounced in higher spaces s, 2 with s > 2

I To our utter surprise, this was NOT the case

I Rudimentary manifestations of (presumably) differentcomplexity classes.

I This exponential decay also justifies our pragmatic approachtowards the Halting Problem

I And allows an error estimation in “practical” situations

I Main explanation of the phase transition lies in the fact thatthe bulk of the TMs are linear performers.

I Together with the fact that there are not too many differentkind of linear performers around.

I We observed a similar phenomenon when looking at theamount of initial input values needed to determine a function

For 2,2 Only three different outputs needed to determine a 2,2 TMcomputable function completely!

For 3,2 2 4 6 8outputs

functionsNumber of characterized functions

I These findings are interesting and sustain our pragmaticapproach to extensionality

I Questions:

I is the derivative (difference) also exponential?

I For a s, k TM space, let I(s, k) be the minimal initial segmentdetermining all s, k-TM computable functions:

What kind of function is I(s, k) and what is its asymptotics?

I What are the practical bearings of all this?

I Questions:

I On average, functions are computed much more SLOWLY in3,2 than in 2,2

I If we supposed that “chances” of speed-up versus slow-downon the level of algorithms were fifty-fifty,

I then the probability that we observed at most 122instantiations of speed-up would be in the order of 10−108.

I On average we observed slow-down

I We think this fact could be related to issues of stability inevolution (stasis)

I For certain organisms the change on more efficiency is sosmall when mutating that the species will be stable

I Currently we are looking for an adequate biological corpus totest this hypothesis

I Also, the asymptotic behavior moves to the slower part of thespectrum

i f TH2,2L TH3,2L

1 O@NullD,1 O@NullD,1

2 O@nD,46O@1D,129 O@nD,1130 OAn2E,6

OAn3E,8

3 O@nD,9O@1D,2 O@nD,94 OAn2E,12

O@ExpD,5

4 O@nD,5 O@nD,61 OAn2E,9 O@ExpD,4

I Also, the asymptotic behavior moves to the slower part of thespectrum

i f TH2,2L TH3,2L

1 O@NullD,1 O@NullD,1

2 O@nD,46O@1D,129 O@nD,1130 OAn2E,6

OAn3E,8

3 O@nD,9O@1D,2 O@nD,94 OAn2E,12

O@ExpD,5

i f TH2,2L TH3,2L

5 O@nD,2 O@nD,133 OAn2E,2

6 O@nD,3 O@1D,7 O@nD,61 OAn3E,1

7 O@1D,4 O@nD,5 O@ExpD,3O@1D,46 O@nD,32 OAn2E,6

O@ExpD,17

8 O@nD,2 O@nD,34

i f TH2,2L TH3,2L

9 O@nD,1 O@nD,34

i f TH2,2L TH3,2L

13 O@1D,2 O@1D,12

14 O@1D,5 O@1D,23 O@nD,8

15 O@1D,3 O@1D,11

16 O@1D,3 O@1D,9

i f TH2,2L TH3,2L

17 OAn2E,1 OAn2E,13

18 O@nD,1 O@nD,12

20 OAn2E,1 OAn2E,11

i f TH2,2L TH3,2L

21 OAn2E,1 OAn2E,11

22 O@nD,1 O@nD,14

23 O@1D,3 O@1D,9

24 OAn2E,1 OAn2E,12

i f TH2,2L TH3,2L

26 O@nD,4 O@nD,14

27 O@1D,1 O@1D,6

28 O@1D,1 O@1D,7

i f TH2,2L TH3,2L

29 O@1D,39 O@1D,107

30 O@1D,1 O@1D,7

31 O@1D,3 O@1D,25

32 O@1D,1 O@1D,5 O@nD,1

i f TH2,2L TH3,2L

33 O@1D,9 O@1D,9 O@nD,7 O@ExpD,3

34 O@1D,23 O@1D,58 O@nD,13 O@ExpD,1

36 O@nD,1 O@1D,3 O@nD,19

i f TH2,2L TH3,2L

37 O@nD,1 O@nD,12

38 O@1D,1 O@1D,23 O@nD,1

39 O@1D,1 O@1D,16

40 O@nD,1 O@1D,3 O@nD,6

i f TH2,2L TH3,2L

41 O@1D,1 O@1D,23

42 O@1D,4 O@1D,42 O@nD,1

43 O@1D,2 O@1D,16

44 O@1D,1 O@1D,22 O@nD,1

i f TH2,2L TH3,2L

45 O@1D,1 O@1D,8

46 O@1D,1 O@1D,14 O@nD,2

47 O@nD,1 O@1D,26 O@nD,57

48 O@nD,1 O@nD,32

i f TH2,2L TH3,2L

49 O@nD,1 O@1D,14 O@nD,17

50 O@nD,1 O@nD,15

51 O@nD,1 O@nD,15

52 O@nD,1 O@nD,12

i f TH2,2L TH3,2L

53 O@1D,1 O@1D,10

54 O@1D,3 O@1D,70

55 O@1D,3 O@1D,17 O@nD,1

56 O@1D,6 O@1D,35 O@nD,7

i f TH2,2L TH3,2L

57 O@1D,1 O@1D,21 O@nD,4 O@ExpD,2

58 O@1D,1 O@1D,22

59 O@1D,1 O@1D,15 O@nD,7

60 O@nD,1 O@1D,1 O@nD,37

i f TH2,2L TH3,2L

61 O@nD,1 O@1D,3 O@nD,45

62 O@nD,1 O@1D,15 O@nD,20

63 O@nD,1 O@nD,11

64 O@nD,1 O@1D,2 O@nD,31

i f TH2,2L TH3,2L

65 O@nD,1 O@nD,21

66 O@1D,1 O@1D,20

67 O@1D,1 O@1D,25

68 O@1D,1 O@1D,11

i f TH2,2L TH3,2L

69 O@1D,2 O@1D,16

70 O@nD,1 O@1D,3 O@nD,4

71 O@nD,1 O@1D,1 O@nD,20

72 O@1D,1 O@1D,4

i f T�2,2� T�3,2�71 O�n�,1 O�1�,1 O�n�,20

72 O�1�,1 O�1�,4

73 O�n�,1 O�n�,10

74 O�n�,1 O�1�,2 O�n�,12

We see the whole asymptotic spectrum shift to the right

I The clustering that caused the “phase transitions” are alsowitnessed at the level of the individual functions

I Both on time usage and on space usage

I The clustering that caused the “phase transitions” are alsowitnessed at the level of the individual functions

I Both on time usage and on space usage

I We think the most informative average is the harmonicaverage in this case

I The harmonic average of the runtimes of the TMs computingon particular function gives us:

I the typical/average amount of information computed by arandom TM in a time unit.

I Let the TMs computing a function be {TM1, . . . , TMn withruntimes t1, . . . , tn}

I If we let TM1 run for 1 time unit, next TM2 for 1 time unitand finally TMn for 1 time unit

I then the amount of information of the output computed is1/t1 + ...+ 1/tn.

I Which gives rise to the harmonic average n1x1

+...+ 1xn

I Constant time behavior like the head (almost) immediatelydropping off the tape;

I Linear behavior like running to the end of the tape and thenback again as Rule 2240;

I Iterative behavior like using each black cell to repeat a certainprocess as in Rule 1447;

I Localized computation like in Rule 2205;

I Recursive computations like in Rule 1351.

Iterative computation in Rule 1447

Linear computations in Rule 2240

Localized computations in Rule 2205

Recursion in Rule 1351

Exponential space

Fractal dimensionMain results

We can assign a fractal dimension to the limiting space-time tapeconfiguration:

I The intuition for fractal dimension is as follows.

I Suppose we use boxes of size r to cover a spatial figure F

I Let N(F , r) denote the number of boxes of size r needed to tocover F

I for a line the “volume” is: V ∼ r · N(F , r)

I for a surface the “volume” is: V ∼ r2 · N(F , r)

I for a volume the “volume” is: V ∼ r3 · N(F , r)

I We define the fractal dimension intuitively as that number dsuch that the “volume” of F is: V ∼ rd · N(F , r)

I Solving for d , the V disappears and we obtain:

d = lim r ↓ 0log(N(F , r))

log( 1r )

I Standard definition: dF := limr↓0log(N(F ,r))

log( 1r )

where F is to be

covered by boxes of size r

I But in the case of a TM progression F is not always welldefined

I But we can slightly modify the definition to still use it.

I However, direct calculations show that convergence is slow

I It is known that the fractal dimension of such figures isactually in general uncomputable

log( 1r )

where F is to be

log( 1r )

where F is to be

log( 1r )

where F is to be

log( 1r )

where F is to be

In 2,2 TM space we checked that for a TM τ we have:

I τ runs in super-polynomial time iff limiting fractal dimensionof progression equals 1.

I τ runs in linear time iff limiting fractal dimension ofprogression equals 2.

I τ runs in between linear time and super-polynomial time ifflimiting fractal dimension of progression is strictly between 1and 2.

The results are also partially proven.

mining the computational universe

Documents