tutorial 5: information, nanoscience and complexity

Tutorial 5: Information, Nanoscience and Tutorial 5: Information, Nanoscience and ComplexityComplexity

Question1: [Binary and Decimal Numbering System]Write the number 2500 in binary.

Read

in

th

is

dir

ecti

on

!

The number 2500 in binary is 100111000100.

2500

201250 20

This reads as we have the divisor 1250 and the remainder 0 when 2500 is divided by 2.

Conversion from base 10 to 2

625

21312

20156

2078 2039 21

0 Stop until you get 0!

19 219 214 202 201 21

Question1: [Continue]Also, what is the number 1100100 (binary code) in decimal?

Conversion from base 2 to 10

1 1 0 0 1 0 0

26 25 24 23 22 21 20

Right to left!

0 x 20 + 0 x 21 + 1 x 22 + 0 x 23 + 0 x 24 + 1 x 25 + 1 x 26

= 0 x 1 + 0 x 2 + 1 x 4 + 0 x 8 + 0 x 16 + 1 x 32 + 1 x 64

= 0 + 0 + 4 + 0 + 0 + 32 + 64

= 100

Question2: [Number of bits]How many bits does one need to encode any position in a string of 800 symbols?

The position of the symbols goes like 1, 2, 3, 4, 5, ….. 800. To determine the number of bits needed to encode any position, we only need to determine the number of bits required to encode the highest position. The highest position in this case is 800.

The smallest power of 2 greater than 800 is 210 = 1,024. Hence the minimal number of bits needed to encode a position in a string of 800 symbols is 10.

Question3: [Shannon entropy and Optimal Coding]Suppose you are given a five-sided biased die with a probability of 1/8 to obtain a 1 or 2, and probability of ¼ to obtain a 3, 4, or 5. What is an optimal code for transmitting the throws of this die?To solve this problem, we have to first determine the number of bits required to encode each of the symbols respectively according to the formula –log2P. Next, based on these, design the coding scheme wisely without ambiguity. These will be the optimal code to transmit this information.

Dice Probability Length of code Example of code

1 1/8 -log2(1/8) =3 111

2 1/8 -log2(1/8) =3 110

3 1/4 -log2(1/4) =2 10

4 1/4 -log2(1/4) =2 01

5 1/4 -log2(1/4) =2 00

Average number of bits per throw,

25.224

133

8

12

log2

i

ii ppn

The coding scheme is NOT unique. We can swap the “0” and “1” globally and this doesn’t not change the code. Also, which face belongs to which code of the same length is arbitrary so we can just choose.

Question4: [Information Theory (past year exam)](i) How many bits will I minimally need to transmit the outcomes of 300 throws of an 8 sided die? (2 marks)

984.1

128

172

64

16

32

15

16

14

8

13

4

12

2

11

log2

i

ii ppn

There are eight different sides, one could use three bits per throw (since 23=8, we can count from 1 to 8). So 300 throws would require 900 bits. BUT, this is the minimal bits needed when the dices and throw are unbiased! (Of course we can do better if we assume the dices are biased).

9002.595984.1300

300

nN

Suppose the dice/throw are biased with following frequencies:

In this question, we assume the throw are unbiased since probability is not given.

Question4: [continue](ii) How many bits does one need to encode any position in a string of 60,000 symbols? (2 marks)

The smallest power of 2 greater than 60,000 is 216 = 65,536. Hence the minimal number of bits needed to encode a position in a string of 60,000 symbols is 16.(iii) Suppose you are given a four-sided biased die with a probability of 1/8 to obtain a “1” or a “2”, a probability of 1/4 to obtain a “3” and a probability of 1/2 to obtain a “4”. What is the optimal code for transmitting the throws of this die? (4 marks)

Based on these, we know that the optimal codes for the symbol “1”, “2”, “3” and “4” are 3 bits, 3 bits, 2 bits and 1 bit respectively.

Question4: [continue]

Optimal CodingTo design the optimal coding scheme, we always start from the simplest one, i.e. the one requires the minimum number of bits.We could choose the following coding scheme:

“4” = 0, “3” = 10, “2” = 110, “1” = 111

Of course, the coding scheme is not unique. We could also choose:

“4” = 1, “3” = 01, “2” = 001, “1” = 000“4” = 1, “3” = 01, “2” = 000, “1” = 001

or

However, the following scheme is not correct:“4” = 0, “3” = 11, “2” =

110, “1” = 111Let’s say we have the string 110. Should be decode it to be the symbol “2” or “34”?

Therefore, the design of the optimal code must be careful so that no ambiguity would be caused!

Question5: [Turing Machine]What is a Turing machine? Design a Turing machine that can write the word “Life” its tape.

1. There is an infinite tape with squares on it.2. Each square may contain a symbol from a finite

alphabet but there can be only finitely many non-blank square on the tape.

3. There is a read-write head which is positioned somewhere on the tape.

4. The machine is in one of a finite number of states.5. There is a set of rules that tell the machine what to do

given a certain state and a certain symbol read from the tape.

A Turing machine is a very simple machine, but, logically speaking, has all the power of any digital computer. It may be described as follows:

Question5: [continue]

The rule is of the following form:

(current state, current symbol, new state, new symbol, left/right)

This rule means that if the Turing machine is now in current state, and the symbol under the read/write head is current symbol, change its internal state to new state, replace the symbol on the tape at its current position by new symbol, and move the read/write head one square in the given direction (left or right).

Question5: [Turing Machine (continue)]

0Head

State

Tape

Rules are defined in this format: (current state, current symbol, new state, new symbol, left/right)

S

Start with the symbol S on the tape and in state zero.Then apply the rules until the machine stops.

> = Right, - = write or do nothing


0Head

State

Tape


S

R = Right, - = write or do nothing

Let us go through this algorithm step by step

In the beginning (this is also called the initial condition) we have:

Step 0 - Read Apply rule 1.Rules:

1) 0 , S , 1 , L , R

2) 1, - , 2 , I , R

3) 2, - , 3 , F, R

4) 3, - , 4 , E ,R

5) 4, -, - , -, -


1Head

State

Tape


L


Step 1 - Move Right

Rules:

1) 0 , S , 1 , L , R

2) 1, - , 2 , I , R

3) 2, - , 3 , F, R

4) 3, - , 4 , E ,R

5) 4, -, - , -, -


1Head

State

Tape


L


Step 2 - Read Apply rule 2

Rules:

1) 0 , S , 1 , L , R

2) 1, - , 2 , I , R

3) 2, - , 3 , F, R

4) 3, - , 4 , E ,R

5) 4, -, - , -, -


2Head

State

Tape


L


Step 3 - Move Right

IRules:

1) 0 , S , 1 , L , R

2) 1, - , 2 , I , R

3) 2, - , 3 , F, R

4) 3, - , 4 , E ,R

5) 4, -, - , -, -


2Head

State

Tape


L



IRules:

1) 0 , S , 1 , L , R

2) 1, - , 2 , I , R

3) 2, - , 3 , F, R

4) 3, - , 4 , E ,R

5) 4, -, - , -, -


3Head

State

Tape


L


Step 5-Move Right

I F

Rules:

1) 0 , S , 1 , L , R

2) 1, - , 2 , I , R

3) 2, - , 3 , F, R

4) 3, - , 4 , E ,R

5) 4, -, - , -, -


4Head State

Tape


L



I F E

Rules:

1) 0 , S , 1 , L , R

2) 1, - , 2 , I , R

3) 2, - , 3 , F, R

4) 3, - , 4 , E ,R

5) 4, -, - , -, -


4Head State

Tape


L



I F E

The head will stay there without doing anything at this position after applying Rule 5.

Rules:

1) 0 , S , 1 , L , R

2) 1, - , 2 , I , R

3) 2, - , 3 , F, R

4) 3, - , 4 , E ,R

5) 4, -, - , -, -

Question6: [Turing Machine (past year exam)](i) Look at the figure: In which direction will the Turing machine above move next? (1 mark)

Since the current state is ‘1’ and current symbol is ‘2’, we apply rule 4 and the head moves to left.(ii) Change the rules such that when starting from S in state 0, the machine will forever keep on moving to the right. (2 marks)

Rules:

1) 0, S or 0 or 1 , 1 , - , R (*)

2) 0, 2 , 0 , - , R

3) 2, 1 , 1 , 0 , R

4) 1, 1 or 2 , 0 , - , R (*)

Rules:

1) 0, S or All, 0 , - , R (*)

2) 0, 2 , 0 , - , R

3) 2, 1 , 1 , 0 , R

4) 1, 1 or 2 , 0 , - , L

Some possibilities

Rules:

1) 0, S , 0 , - , R (*)

2) 0, 0 or 1 or 2 , 0 , - , R (*)

3) 2, 1 , 1 , 0 , R

4) 1, 1 or 2 , 0 , - , L

Question6: [Continue](iii) What is the minimum number of rules that we need to implement (ii)? (1 mark)

(iv) Design a Turing machine that can write the word ‘NUS’ and give its rules? (4 marks)

Tape Tape

0Head

State

S

Rules:

1) 0, S , 1 , N , R

2) 1, - , 2 , U , R

3) 2, - , 3 , S , R

4) 3, - , - , - , -

3Head

State

N U S

Rules:

1) 0, S or All, 0 , - , R (*)

2) 0, 2 , 0 , - , R

3) 2, 1 , 1 , 0 , R

4) 1, 1 or 2 , 0 , - , LWe need 1 rules minimally, namely modify rule 1. But,

this is absolutely trivial case! (acceptable generally speaking).

Rules:

1) 0, S or 0 or 1 , 1 , - , R

2) 0, 2 , 0 , - , R

3) 2, 1 , 1 , 0 , R

4) 1, 1 or 2 , 0 , - , R

non-trivial cases, we need 2 rules

Question11: [Numbering Scheme in Cellular Automata]Consider the elementary cellular automata with only next nearest neighbour (i.e. figure 20.24 in text book), discuss the numbering scheme in cellular automata.

If we say that black is one and grey is zero, then we can see that the TOP is just counting from 0 to 7.

Val

ue 1

Val

ue 4

Val

ue 2

Val

ue 1

Val

ue 4

Val

ue 2

= 4

= 3

The first thing to notice is that the top is always the same.

This is the part that changes.

03 2 1

47 6 5

Question11: [Continue]How about the bottom? We can do exactly the same thing but

since we have 8 boxes on the bottom it’s counting from 0 to 255.

Value 128 Value 64 Value 16Value 32

Value 8 Value 4 Value 2 Value 1 “Rule 0”

“Rule 255” 64+16+8+2= “Rule 90”

Question11: [Continue]

Now, it would be easier to determine the numbering scheme if we are given the rules.

2+16+64+128 = “Rule 210”

Value 128 Value 64 Value 16Value 32

Value 8 Value 4 Value 2 Value 1

What is the numbering scheme for the following rule?

numbering scheme is equivalent to the “rule”. (but normally can be given in either binary or decimal). (see next slide)

http://cell-auto.com/wolframnumbers/


How to design the rule from the given numbering scheme (say in decimal)?

First, we convert the numbering scheme (i.e. 210) into its binary.

Add the binary number ‘0’ from the leftmost till the number of bits is 8!

‘0’ grey‘1’ black

Color the bottom boxes according to the binary numbers.


Example 1: Design the rule for the numbering scheme 100.

100 20

50 20

25 21

12 20

6 20

3 21

1 21

0

0 1 1 0 0 1 0 0

the extra ‘0’ bit

0 1 1 0

0 1 0 0


Example 2: Design the rule for the numbering scheme 210.

210 20

105 21

52 20

26 20

13 21

6 20

3 21

1 1 0 1 0 0 1 0

1 1 0 1

0 0 1 0

1 21

0

Exercise: Can you work out the elementary automata with second nearest neighbor with 2 possible states? Hint: There are 25=32 of them, allow us to count from 0-31. How about the lower box? We have (20=1, 21=2,……. 231=2,147,483,648)

Question13: [Cellular Automata (past year exam)]We have the following cellular automaton rule:(i) What rule is this? (i.e. give the number such as “rule 90”).

value 2value 32value 64value 128 value 16 value 8 value 1value 4

(ii) Draw the next four steps in figure 1 using the rule from step (i).

Rule 132

Answer: “Rule 132”

132411281

Question13: [Continue](iii) What rule was used to make figure 2?

So, we have Rule 110.110214181321641 (iv) What is notable about the rule used in figure 2?This is an universal Turning machine and thus can act perform universal computation. (the proof is difficult!) (v) If, for the cellular automaton considered here, we allow the cells to have 3 states instead of 2 states, how many rules will there be? (5 marks in total)

value 2

value 32value 64value 128 value 16

value 8 value 1value 4

For 3 states and nearest neighbors case,

123 106255.733

rulesN

tutorial 5: information, nanoscience and complexity

Documents