1 turing’s thesis. 2 turing’s thesis: any computation carried out by mechanical means can be...
TRANSCRIPT
1
Turing’s Thesis
2
Turing’s thesis:
Any computation carried outby mechanical meanscan be performed by a Turing Machine
(1930)
3
Computer Science Law:
A computation is mechanical if and only ifit can be performed by a Turing Machine
There is no known model of computationmore powerful than Turing Machines
4
Definition of Algorithm:
An algorithm for functionis a Turing Machine which computes
)(wf
)(wf
5
When we say:
There exists an algorithm
Algorithms are Turing Machines
We mean:
There exists a Turing Machinethat executes the algorithm
6
Turing Machines are “hardwired”
they executeonly one program
A limitation of Turing Machines:
Real Computers are re-programmable
7
Encoding Turing Machine
8
Alphabet Encoding
Symbols: a b c d
Encoding: 1 11 111 1111
9
State Encoding
States: 1q 2q 3q 4q
Encoding: 1 11 111 1111
Head Move Encoding
Move:
Encoding:
L R
1 11
10
Transition Encoding
Transition: ),,(),( 21 Lbqaq
Encoding: 10110110101
separator
11
Machine Encoding
Transitions:
),,(),( 21 Lbqaq
Encoding:
10110110101
),,(),( 32 Rcqbq
110111011110101100
separator
12
A Turing Machine is described with a binary string of 0’s and 1’s
The set of Turing machines forms a language:
each string of the language isthe binary encoding of a Turing Machine
Therefore:
13
Language of Turing Machines
L = { 010100101,
00100100101111,
111010011110010101, …… }
(Turing Machine 1)
(Turing Machine 2)
……
14
Countable Sets
15
Infinite sets are either: Countable
or
Uncountable
16
Countable set:
There is a one to one correspondencebetween elements of the setand Natural numbers
Any finite set
Any Countably infinite set:
or
17
Example:
Even integers: ,6,4,2,0
The set of even integersis countable
Positive integers:
Correspondence:
,4,3,2,1
n2 corresponds to 1n
18
Example: The set of rational numbersis countable
Rational numbers: ,87,43,21
19
Naïve Proof
Rational numbers: ,31,21,11
Positive integers:
Correspondence:
,3,2,1
Doesn’t work:
we will never count numbers with nominator 2:
,32,22,12
20
Better Approach
11
21
31
41
12
22
32
13
23
14
21
11
21
31
41
12
22
32
13
23
14
22
Rational Numbers: ,22,31,12,21,11
Correspondence:
Positive Integers: ,5,4,3,2,1
23
We proved:
the set of rational numbers is countable by describing an enumeration procedure
24
Definition
An enumeration procedure for is aTuring Machine that generatesall strings of one by one
Let be a set of strings S
S
S
and
Each string is generated in finite time
25
EnumerationMachine for
,,, 321 sss
Ssss ,,, 321
Finite time: ,,, 321 ttt
strings
Soutput
(on tape)
26
Enumeration MachineConfiguration
Time 0
0q
Time
sq
1x 1s#1t
27
Time
sq
3x 3s#3t
Time
sq
2x 2s#2t
28
If for a set there is an enumeration procedure, then the set is countable
Observation:
29
Example:
The set of all stringsis countable
},,{ cba
We will describe an enumeration procedure
Proof:
30
Naive procedure:
Produce the strings in lexicographic order:
aaaaaa
......
Doesn’t work: strings starting with will never be produced
b
aaaa
31
Better procedure:
1. Produce all strings of length 1
2. Produce all strings of length 2
3. Produce all strings of length 3
4. Produce all strings of length 4
..........
Proper Order
32
Produce strings in Proper Order:
aaabacbabbbccacbcc
aaaaabaac......
length 2
length 3
length 1abc
33
Theorem: The set of all Turing Machinesis countable
Proof:
Find an enumeration procedure for the set of Turing Machine strings
Any Turing Machine can be encodedwith a binary string of 0’s and 1’s
34
1. Generate the next binary string of 0’s and 1’s in proper order
2. Check if the string describes a Turing Machine if YES:YES: print string on output tape if NO:NO: ignore string
Enumeration Procedure:
Repeat
35
Uncountable Sets
36
A set is uncountable if it is not countable
Definition:
37
Theorem:
Let be an infinite countable set
The powerset of is uncountable S2 S
S
38
Proof:
Since is countable, we can write S
},,,{ 321 sssS
Elements of S
39
Elements of the powerset have the form:
},{ 31 ss
},,,{ 10975 ssss
……
40
We encode each element of the power setwith a binary string of 0’s and 1’s
1s 2s 3s 4s
1 0 0 0}{ 1s
Powersetelement
Encoding
0 1 1 0},{ 32 ss
1 0 1 1},,{ 431 sss
41
Let’s assume (for contradiction) that the powerset is countable.
Then: we can enumerate the elements of the powerset
42
1 0 0 0 0
1 1 0 0 0
1 1 0 1 0
1 1 0 0 1
Powerset element Encoding
1t
2t
3t
4t
43
Take the powerset elementwhose bits are the complements in the diagonal
44
1 0 0 0 0
1 1 0 0 0
1 1 0 1 0
1 1 0 0 1
1t
2t
3t
4t
New element: 0011
(birary complement of diagonal)
45
The new element must be someof the powerset
it
However, that’s impossible:
the i-th bit of must be the complement of itself
from definition of it
it
Contradiction!!!
46
Since we have a contradiction:
The powerset of is uncountable S2 S
47
An Application: Languages
Example Alphabet : },{ ba
The set of all Strings:
},,,,,,,,,{},{ * aabaaabbbaabaababaS infinite and countable
48
Example Alphabet : },{ ba
The set of all Strings:
},,,,,,,,,{},{ * aabaaabbbaabaababaS infinite and countable
A language is a subset of :
},,{ aababaaL
S
49
Example Alphabet : },{ ba
The set of all Strings:
},,,,,,,,,{},{ * aabaaabbbaabaababaS infinite and countable
The powerset of contains all languages:
}},,,}{,{},{},{{2 aababaabaaS 1L 2L 3L 4L
uncountable
S
50
Languages: uncountable
Turing machines: countable
1L 2L 3L kL
1M 2M 3M
?
There are more languagesthan Turing Machines
51
There are some languages not acceptedby Turing Machines
(These languages cannot be described by algorithms)
Conclusion:
52
Languages Accepted by Turing Machines
Languages not accepted by Turing Machines
kL
53
Recursively Enumerable and
Recursive Languages
54
Definition:
A language is recursively enumerableif some Turing machine accepts it
55
For string :
Let be a recursively enumerable languageL
and the Turing Machine that accepts itM
w
Lwif then halts in a final state M
Lwif then halts in a non-final stateM
or loops forever
56
Definition:
A language is recursiveif some Turing machine accepts itand halts on any input string
In other words: A language is recursive if there is a membership algorithm for it
57
For string :
Let be a recursive languageL
and the Turing Machine that accepts itM
w
Lwif then halts in a final state M
Lwif then halts in a non-final stateM
58
Facts:
1. There is a specific language which is not recursively enumerable (not accepted by any Turing Machine)
2. There is a specific language which is recursively enumerable but not recursive
59
Recursive
Recursively Enumerable
Non Recursively Enumerable
60
A Language which is not
Recursively Enumerable
61
We want to find a language thatis not Recursively Enumerable
This language is not accepted by anyTuring Machine
62
Consider alphabet }{a
Strings: ,,,, aaaaaaaaaa
1a 2a 3a 4a
63
Consider Turing Machinesthat accept languages over alphabet }{a
They are countable:
,,,, 4321 MMMM
64
Example language accepted by
},,{)( aaaaaaaaaaaaML i
},,{)( 642 aaaML i
iM
Alternative representation
1a 2a 3a 4a 5a 6a 7a
)( iML
0 1 1 10 0 0
65
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
66
Consider the language
)}(:{ iii MLaaL
L consists from the 1’s in the diagonal
67
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
},,{ 43 aaL )}(:{ iii MLaaL
68
Consider the language
)}(:{ iii MLaaL
L consists of the 0’s in the diagonal
)}(:{ iii MLaaL
L
69
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
},,{ 21 aaL )}(:{ iii MLaaL
70
Theorem:
Language is not recursively enumerableL
71
Proof:
is recursively enumerableL
Assume for contradiction that
There must exist some machinethat accepts
kML
LML k )(
72
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
Question: ?1MMk
73
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
Answer: 1MMk )(
)(
11
1
MLa
MLa k
74
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
Question: ?2MMk
75
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
Answer: 2MMk )(
)(
22
2
MLa
MLa k
76
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
Question: ?3MMk
77
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
Answer: 3MMk )(
)(
33
3
MLa
MLa k
78
Similarly: ik MM
)(
)(
ii
ki
MLa
MLa
)(
)(
ii
ki
MLa
MLa
for any i
Because either:
or
79
Therefore, the machine cannot exist kM
Therefore, the languageis not recursively enumerable
L
End of Proof
80
Observation:
There is no algorithm that describes L
(otherwise would be accepted by some Turing Machine)
L
81
Recursive
Recursively Enumerable
Non Recursively Enumerable
L
82
A Language which is Recursively Enumerable
and not Recursive
83
We want to find a language which
There is a Turing Machine that accepts the language
The machinedoesn’t halt on some input
Is recursively enumerable
But notrecursive
84
We will prove that the language
)}(:{ iii MLaaL
Is recursively enumerablebut not recursive
85
1a 2a 3a 4a
)( 1ML
0 1 10
)( 2ML
)( 3ML
01 0 1
0 1 11
)( 4ML 0 10 0
},,{ 43 aaL
86
The language
Theorem:
)}(:{ iii MLaaL
is recursively enumerable
87
Proof:
We will give a Turing Machine thataccepts L
88
Turing Machine that accepts LFor any input string w
• Compute , for which iaw• Find Turing machine iM
(using an enumeration procedure for Turing Machines)
• Simulate on inputiMia
• If accepts, then accept iM w
i
End of Proof
89
Observation:
)}(:{ iii MLaaL
)}(:{ iii MLaaL
Recursively enumerable
Not recursively enumerable
(Thus, also not recursive)
90
Theorem:
The language )}(:{ iii MLaaL
is not recursive
91
Proof:
Assume for contradiction that is recursive
Then is recursive:
L
L
Take the Turing Machine that accepts M L
halts on any input:M
If accepts then rejectIf rejects then accept
MM
92
Therefore:
L is recursive
But we know:
L is not recursively enumerable
thus, not recursive
CONTRADICTION!!!!
93
Therefore, is not recursiveL
End of Proof
94
L
Recursive
Recursively Enumerable
Non Recursively Enumerable
L
95
The Chomsky Hierarchy(Formal Language Hierarchy)
96
Context-Sensitive Grammars:
and: |||| vu
Productionsvu
String of variablesand terminals
String of variablesand terminals
97
Non-recursively enumerable
Recursively-enumerable
Recursive
Context-sensitive
Context-free
Regular
The Chomsky Hierarchy
98
Decidability
99
Consider problems with answer YES or NO
Examples:
• Does Machine have three states ?M
• Is string a binary number? w
• Does DFA accept any input? M
100
A problem is decidable if some Turing machinedecides (solves) the problem
Decidable problems:
• Does Machine have three states ?M
• Is string a binary number? w
• Does DFA accept any input? M
101
Turing MachineInputprobleminstance
YES
NO
The Turing machine that decides (solves) a problem answers YES or NO for each instance of the problem
102
The machine that decides (solves) a problem:
• If the answer is YES then halts in a yes state
• If the answer is NO then halts in a no state
These states may not be final states
103
YES states
NO states
Turing Machine that decides a problem
YES and NO states are halting states
104
Some problems are undecidable:
which means:there is no Turing Machine thatsolves all instances of the problem
A simple undecidable problem:
The membership problem
105
The Membership Problem
Input: •Turing MachineM
•String w
Question: Does accept ? M w
?)(MLw
106
Theorem:
The membership problem is undecidable
Proof: Assume for contradiction thatthe membership problem is decidable
(there are and for which we cannotdecide whether )
M w)(MLw
107
Thus, there exists a Turing Machinethat solves the membership problem
H
HM
w
YES
NO
M accepts w
M rejects w
108
Let be a recursively enumerable language L
Let be the Turing Machine that acceptsM L
We will prove that is also recursive: L
we will describe a Turing machine thataccepts and halts on any inputL
109
M accepts ?wNO
YESM
w
Hacceptw
Turing Machine that acceptsand halts on any input
L
reject w
110
Therefore, L is recursive
But there are recursively enumerablelanguages which are not recursive
Contradiction!!!!
Since is chosen arbitrarily, every recursively enumerable language is also recursive
L
111
Therefore, the membership problem is undecidable
END OF PROOF
112
Another famous undecidable problem:
The halting problem
113
The Halting Problem
Input: •Turing MachineM
•String w
Question: Does halt on input ? M w
114
If the halting problem was decidable thenevery recursively enumerable languagewould be recursive
115
Theorem:
The halting problem is undecidable
Proof: Assume for contradiction thatthe halting problem is decidable
116
There exists Turing Machinethat solves the halting problem
H
HM
w
YES M halts on w
Mdoesn’t halt on
wNO
117
Let be a recursively enumerable language L
Let be the Turing Machine that acceptsM L
We will prove that is also recursive: L
we will describe a Turing machine thataccepts and halts on any inputL
118
M halts on ?wYES
NOM
w
Run with input
Mw
Hreject w
acceptw
rejectw
Turing Machine that acceptsand halts on any input
L
Halts on final state
Halts on non-final state
119
Therefore L is recursive
But there are recursively enumerablelanguages which are not recursive
Contradiction!!!!
Since is chosen arbitrarily, every recursively enumerable language is also recursive
L
120
Therefore, the halting problem is undecidable
END OF PROOF