theoreticalfoundations ofcomputerscience

28

Upload: others

Post on 04-May-2022

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: TheoreticalFoundations ofComputerScience

Theoretical Foundationsof Computer Science

Wilhelmiina HämäläinenLecture material 15.1. 2005Department of Computer ScienceUniversity of Joensuu

Page 2: TheoreticalFoundations ofComputerScience

Chapter 1

Introduction

Theory of computation considers how to classify problems according to theirsolvability, di�culty and e�ciency before they are solved. Theory of computationis classically divided into two parts:

1. In Theory of computability we study, which problems can in principle besolved by computers and how di�cult the given problem is. The di�culty isde�ned by quite coarse level according to how di�cult model of computationis requires. In addition theory of computability gives good advice how to solvee�ciently some special type of problems.

2. In Theory of computational complexity we study, how e�ciently theproblem can be solved. Theory of computational complexity reminds anal-ysis of algorithms, but we don't determine time and space complexity of anindividual algorithm, but the worst case complexity class of the problem it-self. Theory of computational complexity gives also good aids for reducing aproblem into other already known problems.

In this course, we consider the �rst part of theory of compuation, namely theory ofcomputability. The content is computational problems and corresponding modelsof computation (mechanical models for solving).

Look at the following comic (The assistant's Nightmare):

1

Page 3: TheoreticalFoundations ofComputerScience

2 CHAPTER 1. INTRODUCTION

no efficientsolution

efficientsolution

solvableproblem

unsolvableproblem

totallyunsolvablesolvable

partially

computationalproblem

non−computationalproblem

problem

Figure 1.1: Classi�cation of problems.

Page 4: TheoreticalFoundations ofComputerScience

3

1. �Frustrating to check all misspellings. I should make a checking machine.�

2. And so is done. �Well, Machine, let's work. All strings which may appear inthe code are in the list.�

3. But what happens...? �Crazy machine! Here is di�erent number of beginningand end parenthesis. I'll give you a stack so you will remember better.�

4. After a while. �This is going well. Good, that I invented to put also if-elsestructures into the stack. I'll just have to test the programs.�

5. But: �Oh! It still fell down. I still must make the machine better.�

Page 5: TheoreticalFoundations ofComputerScience

4 CHAPTER 1. INTRODUCTION

6. �Ok, now you have an in�nite tape. You'll simulate the program with theinput while I lay on my leaves.�

7. The time pasts. �Hmm, it seems I have slept quite a while. Is the computationstill going on?�

8. �Stupid machine! There is an in�nite loop. I guess I have to return back towork myself.�

• v.s. Chomsky hierarchy of languages

kielet

kontekstittomat kielet

kontekstilliset kielet

tyyppi 0

tyyppi 1

tyyppi 2

tyyppi 3

äärelliset

ratkeamattomat ongelmat

esim. pysähtymisongelma

tunnistus:

tunnistus:

Turing−kone + ääretöntyönauha (pysähtyy aina)

universaali Turing−kone(pysähtyy ’kyllä’−tapauksessa

tunnistus:pinoautomaatti

säännölliset kielet

tunnistus:äärellinen automaatti

RAM−koneohj.kielet

,

Turing−kone +äärell. työnauha

luonnoll. kielet?

(rajall. muisti)

)

esim. palindromit

rekursiivisesti numeroituvatkielet

rekursiivisetkielet

rajoittamattomatkielet

Figure 1.2: Chomsky language hierarchy + the class of recursive languages.

Page 6: TheoreticalFoundations ofComputerScience

1.1. AUTOMATA 5

• type 3 Regular languages (special case �nite languages)

• type 2 Context-free languages

• type 1 Context-sensitive languages

• type 0 Unrestricted languages = Recursive languages + Recursively countablelanguages

• outside unsolvable problems

1.1 Automata

Three kinds of automata, which correspond models of computation. The simplerautomaton you need, the easier the problem!

Common for all automata• at each time step the automaton is in one state

• the automaton moves from one state to another according to next characterread and some rules

Finite automaton

• reads the next input character

Page 7: TheoreticalFoundations ofComputerScience

6 CHAPTER 1. INTRODUCTION

• no extra memory

• works in linear time O(n)

• applications: pattern recognition, simple checkings in compilers (e.g. if vari-ables and functions are de�ned correctly, if a number is integer/decimal num-ber), simple machines like washing machine, tra�c lights, lift, bank automa-ton, ...

Push-down automaton

• reads the next input character

• has a stack memory

• can read and write auxiliary symbols on stack

• works in polynomial (cubic) time O(n3)

• applications: compilers, parsing and generating simple natural language, arith-metic calculator, interpreting html-pages visually, ...

Page 8: TheoreticalFoundations ofComputerScience

1.1. AUTOMATA 7

Turing machine

• reads and writes next character on working tape = in�nite memory

• can move forward or backward

• time and space requirements can be anything!

• aplications: can solve any problem the computers can solve!

Page 9: TheoreticalFoundations ofComputerScience

8 CHAPTER 1. INTRODUCTION

q q’a

q q’

q q’

a ,B/C

a/b,D

D=right or left

Figure 1.3: In �nite automaton, we move from state q to state q′ by reading thenext input character a. In push-down automaton, we read also the top character Bfrom stack, and write C instead. In Turing machine, we read a from working tape,write b instead, and move on working tape either to right or left.

1.2 Mathematical concepts• logical symbols

• sets

• relations

• functions

• countability

• proof methods

1.2.1 Logical symbols

Let P and Q be propositions i.e. truth valued sentences, which describe some events.E.g. P=�The Moon is cheese�, Q=�Napoleon lives in the Moon�.

• ¬P : P is false (not P , !P )

Page 10: TheoreticalFoundations ofComputerScience

1.2. MATHEMATICAL CONCEPTS 9

• P ∨ Q: either P or Q is true (or both) (in programming languages P or Q,P ||Q)

• P ∧Q: both P and Q are true (P and Q, P&&Q)

• P ⇒ Q: implication �if P , then Q� (≡ ¬P ∨Q)

• P ⇔ Q: � equivalence �P if and only if Q� (≡ (P ⇒ Q) ∧ (Q ⇒ P ))

• In addition we often need ∀ (universal quanti�er, �for all�) and ∃ (existencequanti�er, �exists�, �for some�)E.g. ∀x, x ∈ N �for all natural numbers x ...�, ∃x, x ∈ N �for some naturalnumber x ...�

1.2.2 Sets

• set=a collection of elements or memberse.g. A = {a1, a2, ..., an} or Σ = {a, b, c, ..., z}mark ai ∈ A (�ai belongs to set A�)

• special case empty set ∅• A ⊆ B: A is the subset of B

A ⊆ B ⇔ ∀x(x ∈ A → x ∈ B)

proper subset A ⊂ B: A ⊆ B ∧ A 6= B

• A ∪B: union of A and B

A ∪B = {x|x ∈ A ∨ x ∈ B}

• A ∩B: intersection of A and B

A ∩B = {x|x ∈ A ∧ x ∈ B}

• A \B: A subtraction B

A \B = {x|x ∈ A ∧ x /∈ B}

• A = E \ A: complement of A in E

Page 11: TheoreticalFoundations ofComputerScience

10 CHAPTER 1. INTRODUCTION

• A×B: cartesian product of A and B:

A×B = {(x, y)|x ∈ A ∧ y ∈ B},in which < x, y > is an ordered pair.

• P(A): the power set of A

P(A) = {X|X ⊆ A}e.g. if A = {a, b, c}, then P(A) =

{∅, {a}, {b}, {c}, {a, b}, {a, c}, {b, c}, {a, b, c}}

1.2.3 Relations and functionsRelation

E.g. a (binary) relation between A and B is de�ned as a subset of A×B:

R = {(a, b)|a ∈ A ∧ b ∈ B ∧R(a, b)}

• E.g. A and B are natural numbers and R is a successor relation:

R(a, b), if and only if b = a + 1.

Then the relation consists of pairs {(0, 1), (1, 2), (2, 3), ...}.• inverse relation of R ⊆ A×B R−1 ⊆ B × A is relation

R−1 = {(b, a)|(a, b) ∈ R}

Function

A relation between A and B f ⊆ A×B is a function or mapping from set Ato set B, if the following conditions hold:

1. For each element of A there is a mapping in B i.e.

∀x ∈ A ∃y ∈ B (y = f(x))

2. For each element of A there is only one mapping in B i.e.

∀x ∈ A, ∀y1, y2 ∈ B ((x, y1) ∈ f ∧ (x, y2) ∈ f ⇒ y1 = y2.

Page 12: TheoreticalFoundations ofComputerScience

1.2. MATHEMATICAL CONCEPTS 11

AB

f(A)

f(x1)=f(x2)x2

x1

.

.

.

f

Figure 1.4: A=de�nition set, B=goal set, f(A)=value set

• Denoted y = f(x) ⇔ (x, y) ∈ f.

Injection, surjection, bijection

Function f : A → B is

injection, if∀x1, x2 ∈ A(f(x1) = f(x2) ⇒ x1 = x2)

surjection, if∀y ∈ B∃x ∈ A(y = f(x))

bijection, if f is both surjection and bijectionNOTICE! f is bijection ⇔ f has inverse function

• e.g.1 f : Z→ Z+ ∪ {0}f(x) = |x| surjection, not injection

• e.g.2 f : Z+ → Qf(x) = 1

xinjection, not surjection

• e.g.3 f : R+ → R+

f(x) = x2 bijection, inverse mapping f(y) =√

y

Page 13: TheoreticalFoundations ofComputerScience

12 CHAPTER 1. INTRODUCTION

1.2.4 Countability

Set X is countable, if

1. X is �nite or

2. There exists a bijection f : N→ X, for which

X = {f(n)|n ∈ N}.

• intuitively: the elements of X can be ordered and indexed by natural numbers:X = {x0, x1, x2, ...}.

• e.g. set of ood numbers X = {1, 3, 5, ...} is countable, because we can de�nea bijection f : N→ X

f(n) = 2n + 1

• If X is not countable it is uncountable.

• e.g. set of real numbers R and power set P(N) of natural numbers N areuncountable

1.2.5 Proof methodsMathematical induction

We want to show that P (n) holds for all natural numbers.Two parts:

1. Case n = 0: Prove that P (0) holds.2. Induction step: Prove that for all n P (n) → P (n + 1)

• e.g. Claim: Σni=1i = n2+n

2for all n ≥ 0

Proof:1. n = 0 Σn

i=1i = 0 = 02+02

2. Induction assumption: ∃k ∈ N such that the claim holds for all n ≤kCase n = k + 1 : Σk+1

i=1 i = 1 + 2 + 3 + ... + k + (k + 1)Σk

i=1i = k2+k2

, soΣk+1

i=1 i = k2+k2

+ (k + 1) = (k+1)2+(k+1)2

2

Page 14: TheoreticalFoundations ofComputerScience

1.2. MATHEMATICAL CONCEPTS 13

0 1 2 3 4 5

2 30 1−3 −2 −1

N

Z

Figure 1.5: How to numerate integer numbers.

Page 15: TheoreticalFoundations ofComputerScience

14 CHAPTER 1. INTRODUCTION

Undirect Proof (Contradiction method or Proof by antithesis)We want to prove �if P, then Q�. Let's suppose P and make an antithesis ¬Q.If we can conclude a contradiction, the claim is true.

• Notice! We don't know, what is the contradiction, we want to reach.• Notice! Implication P ⇒ Q is true in every case but when P ∧ ¬Q.• e.g. Let's suppose U is an in�nite set and S is a �nite subset of it and T

is the complement of S in U .Claim: T is in�nite.Proof: Antithesis: T is �nite. Because both S and T are �nite, also U is�nite, which means contradiction (U is in�nite). 2

• How would you prove the claim: �In the world there are at least twopeople with exactly same number of hairs in their heads�?

Contraposition

We want to prove �if P, then Q�. Instead we prove an equivalent claim �if notQ, then not P� (P ⇒ Q ≡ ¬Q ⇒ ¬P )

• ∼ a special case of antithesis method: Suppose P and ¬Q and try toconclude ¬P � now we know, what is the contradiction we are lookingfor (P ∧ ¬P ).

Proving existential and universal claims�There exists x ∈ X, for which...� and �For all x ∈ X ...� can sometimes beenproved directly.

• Existial claim ∃x ∈ XP (x): construct such x (guess, produce, invent aproducing algorithm etc.) Notice! You have to show that the wantedproperty really holds.

• e.g. �In the Himalaya there is a mountain, which is higher than any othermountain in the world.�

• Universal claim ∀x ∈ XP (x): select an arbitrary x from X and showthat the wanted property P (X) holds for it.

• e.g. Let S = {x ∈ R|(x2 − 3x + 2 ≤ 0)} and T = {x ∈ R|1 ≤ x ≤ 2}.Claim: S = T .

Counter examples

Page 16: TheoreticalFoundations ofComputerScience

1.3. EXCURSION: TRANSFINITE ORDINAL NUMBERS 15

• Claim ∀x ∈ XP (x) can be invalidated by giving any counter example inX

• e.g. claim �All cats are black.�

Litterature

Solow, Daniel: How to read and write proofs. An introduction to mathematicalthought process. John Wiley & Sons, 1982. (Easy reading guide book for con-structing proofs!)

1.3 Excursion: Trans�nite ordinal numbers

The trans�nite ordinal numbers mean in�nite numbers which still can be ordered.E.g. David Hilbert, Georg Cantor, Kurt Gödel and Rudy Rucker have studiedproblems dealing with such numbers.

ω Omega

ω or ℵ0 (aleph-zero) is the greatest normal ordinal number, i.e. lim n. ω can bede�ned: ω is the �rst such number a, for which a + 1 = a. E.g. 1 + ω=ω, 2ω = ω.

We can still de�ne working sum and product operations for ω. Let's decide thatω + 1= the follower of ω and limn→ω(ω + n) = ω + ω = 2ω.

Now lim(ω×2+n) = ω×3, lim(ω×n) = ω×ω = ω2. In the same way ω3, ω4, ..., ωω.

ω2 is the �rst such ordinal number a that ω + a = a and ωω is the �rst such ordinalnumber a that ω × a = a. (ω × ωω = ωω+1 = ωω, because 1 + ω = ω).

Notice! Sum and product operations for the trans�nite numbers are not invertible!

ε0 Epsilon-zero

Even greater numbers can be generated by nested exponents ωωωω...

. ε0 is the �rstsuch ordinal number a that ωa = a.

Page 17: TheoreticalFoundations ofComputerScience

16 CHAPTER 1. INTRODUCTION

We can describe ε0 better by a new operation tetration ab, which means a-timesexponent b i.e. bbb...b

, in which b appears a times. Very fast great numbers, i.e.310 = 10milliard, 2ω = ωω, 3ω = ωωω . Now ε0 =ω ω.Even greater numbers by nested tetration operations!

ℵ1 Aleph-one

The �rst ordinal number which is mahtavampi? than ω. I.e. there doesn't existany bijection, which would function ℵ1 elements into ω elements.

Hilbert's Hotel

Let's think about Hilbert's Hotel, in which there is in�nite number of rooms, num-bered by 0, 1, 2, 3, .... Hilbert's Hotel is such a strange hotel that even if it is full itcan take new visitors. Let's suppose for example that there is one visitor in eachroom and a new visitor arrives to the hotel. How can we give her/him a room?Easily! We put the new visitor into room 0, which we get empty by moving theguest 0 into room 1, which we get empty by moving the guest 1 into room 2 and soon.What about if there comes a group of in�nite number of guests at once? Now wecan put all the previous guests into even rooms and new guests into odd rooms.In the same way we can �t ω2, ωω or even ε0 quests. However the capacity of thehotel has a limit: ℵ1. ℵ1 is the �rst such number that we cannot �t such number ofguests into the hotel by any ordering.

Cantor's proof

Cantor showed 1873 that there are at least ℵ1 points in the mathematical space.(Usually we say that the set of real numbers is uncountable). Cantor used a verysmart technique in his proof, so called Cantor's diagonalization method:Claim: The set of real numbers R is uncountable.Proof: It's enough to study some subset of R and show that it is uncountable. Let'sselect interval ]0, 1[ and make an antithesis:Antithesis: The interval ]0, 1[ is countable. It means that we can number all realnumbers x, 0 < x < 1 by natural numbers. Let's suppose that the numbering

Page 18: TheoreticalFoundations ofComputerScience

1.3. EXCURSION: TRANSFINITE ORDINAL NUMBERS 17

is done by a bijection r, which gives the real number x a number of order r(x)(r :]0, 1[→ N). Let's suppose that we can represent all real numbers of the interval]0, 1[ as an in�nite matrix M , in which every natural number corresponds some realnumber represented with in�nite precision. The beginning of the matrix could befollowing:

r(1) : .141592...r(2) : .333333...r(3) : .718281...r(4) : .414213...r(5) : .500000...

Let's now construct a new real number in the following way: We read the digitsin the diagonal of the matrix in order and change every digit to something else.I.e. if the desimal representation of the new number is .d1d2d3d4... the ith desimaldi 6= M [i][i] (i.e. the ith desimal of the ith row).

The beginning of the number could be for example .02719.... However this numbercannot appear in the matrix, because it di�ers from each number in the matrix:from number 1 in �rst digit, from number 2 in second digit, from number 3 in thirddigit and so on. So there cannot exist such ordering function r.

Literature

Rucker, Rudy: White Light, or, What is Cantor's Continuum Problem? Ace Books,New York, 1982. (Imaginative science �ction novel, in which we play with in�nitiesand also visit Hilbert's Hotel.)Rucker, Rudy: In�nity and the Mind. The Science and Philosophy of In�nite. (Aneasy reading scienti�c book going on the themes of the White Light.)

Lem, Stanislaw: N. Ya. Vilenkin, Stories about Sets. Academic Press, New York,1968. (A collection of shortstories, one of them about Hilbert's Hotel.)

Hofstadter, Douglas R.: Gödel, Esser, Bach: An Eternal Golden Braid. VintageBooks, New York, 1989. (Chapter XIII shows by Cantor's diagonalization argumentthat there exists problems which are algoritmically unsolvable. Also otherwise suit-able reading for this course!)

Page 19: TheoreticalFoundations ofComputerScience

18 CHAPTER 1. INTRODUCTION

Page 20: TheoreticalFoundations ofComputerScience

Chapter 2

Computational problems

• Computational problem ∼ any task which can be modelled such way that itcan be solved by a computer

• E.g.: multiplying integers, ordering library cards, managing course database

• the solving program is one representation

• a more general representation is easier to analyze

2.0.1 Problem: The MIU-system

In the logic school of Kissastan the cat students study MIU-stystem, in which all theclauses are constructed from three letters: M , I and U . There is only one axiom,MI, in the system. New clauses (theorems) can be derived from the previous clauses(the axiom or theorems) by the following rules:

1. If you posses a string, whose last letter is I, you can add on a U at the end.

2. Suppose you have Mx (in which x can be any string, also an empty string).You can derive a string Mxx.

3. If III occurs in some string, you may replace it by U .

4. If UU occurs in some string, you can drop it from the string.

The rules can be applied freely, when ever they �t the axiom or already derivedtheorems, but you are not allowed to do anything else (that's why the system is

19

Page 21: TheoreticalFoundations ofComputerScience

20 CHAPTER 2. COMPUTATIONAL PROBLEMS

called formal).

The cat students should derive from the axiom MI a theorem MU . Can you per-form the reasoning?

So MIU-system can be thought as some kind of language game, in which we havean alphabet {M, I, U} and a grammar to construct words. Let's represent the rulesmore formally:

xI → xIUMx → MxxxIIIy → xUyxUUy → xy,

in which x and y can be any strings.

Let's consider the set of all possible strings Σ∗, which consists of strings {ε,M, I, U,MM,MI,MU, IM, II, IU, UM,UI, UU,MMM,MMI, MMU,MIM, ...}. Now we can makedi�erent questions about the strings. E.G. problem π1(x): �What strings can beproduced from a given string x by the rules of the system?� or π2(x): �Can youproduce string x from MI by the rules of the system?� The answer for the �rstquestion is a set of strings (or possibly an empty set), in fact a subset of Σ∗, whilethe answer of the latter one is simply �yes� or �no�. Such yes/no-problems arecalled decision problems. Formally we can de�ne a decision problem π as a mappingπ : Σ∗ → {0, 1}. So it associates to each string of the alphabet either answer 1 or 0.

We can also ask, which strings of Σ∗ does the decision problem πA accept? (i.e.for which x π(x) = 1 or the inverted relation π−1(1) = x?) The answer set A iscalled a formal language and the corresponding decision problem πA the recognitionproblem of language A.

Problem: Which words do the following languages consist of?

• All strings which can be produced from M .

• All strings which can be produced from U .

• All strings which can be produced from MU

(The idea of the MIU-system is introduced in the book by Hofstadter, Gödel, Escher,Bach.)

Page 22: TheoreticalFoundations ofComputerScience

21

2.0.2 Formalization• the problem has potentially an in�nite set of cases (�input�)

• the solution is an algorithm, which associates to each case its answer (�out-put�).

• e.g. multiplying integers

� cases: all possible pairs of integers� an answer for a given pair: the product of the integers� solution of the problem: any algorithm for multiplying integers

• each case and its answer must be �nitely representable.

• Computational problem = a mapping from the set of cases to the set of answers

2.0.3 Finite representation• all information has to represented by bits in the last hand

• natural to allow also other symbols

• De�nition: ��nite representation� = a string in some alphabet.

2.0.4 Some concepts• Alphabet a nonempty, �nite set of characters or symbols. e.g. binary alphabet{0, 1} and latin alphabet {A,B,. . . ,Z}.

• String ordered queue of characters E.g. �01001�, �000�,�LTE�,�XYZZY�

• Length of x: the number of characters. Mark |x|. E.g.

|01001| = |XYZZY| = 5, |000| = |OTE| = 3.

• Empty string ε, length |ε| = 0.

• Catenation strings written together. E.g.

(i) CATFISH = CATFISH;(ii) if x = 00 and y = 11, then xy = 0011 and yx = 1100;

Page 23: TheoreticalFoundations ofComputerScience

22 CHAPTER 2. COMPUTATIONAL PROBLEMS

(iii) for all x xε = εx = x;(iv) for all x, y |xy| = |x|+ |y|.

• The set of all strings of the alphabet Σ Σ∗. E.g. Σ = {0, 1}, Σ∗ ={ε, 0, 1, 00, 01, 10, . . .}.

2.0.5 Decision problems and formal languages

A computational problem π is a mapping

π : Σ∗ → Γ∗,

in which Σ and Γ are alphabets

Decision problems are a subclass of computational problems, for which the answeris �yes� or �no�. Formally:

A decision problem is a mapping π : Σ∗ → {0, 1}.

E.g. �is a given number prime?� can be represented as a mapping in Σ = {0, 1, 2, . . . , 9}

π : Σ∗ → {0, 1}, π(x) =

{1, if x is prime;0, if x is not prime.

Each decision problem de�nes a formal language and vice versa:

• For each decision problem π : Σ∗ → {0, 1} there is a set of strings

Aπ = {x ∈ Σ∗ | π(x) = 1},

i.e. those cases for which the answer is �yes�

• for each set of string A ⊆ Σ∗ there is a decision problem

πA : Σ∗ → {0, 1}, πA(x) =

{1, if x ∈ A;0, if x /∈ A.

• formal language in Σ = arbitrary set of strings A ⊆ Σ∗

• recognition problem of language A = decision problem πA associated to A

Page 24: TheoreticalFoundations ofComputerScience

23

2.0.6 Solvability

• the program P solves the computational problem π, if for each input x theprogram P computes and outputs the value π(x).

• Can all possible computational problems be solved by computer?

• No: the set of all possible strings (possible programs) is countable, but theset of all possible decision problems is uncountable (We cannot put ℵ guestsinto ω rooms.)

• Notice! The result is independent of the programming language used!

Theorem 1 For any alphabet Σ the set of all strings Σ∗ is countable.

Proof: Let Σ = {a1, a2, ..., an}. Let's �x an �alphabetic order� e.g. a1 < a2 < ... <an.

The strings of Σ∗ can be ordered in (canonical order):

1. �rst list strings, whose length is 0 (= ε), then those, whose lengt is 1 (=a1, a2, . . . , an), then those, whose length is 2, and so on

2. in each length group the strings are listed in alphabetic order

• for each natural number n there is astring of Σ∗ and vice versa → Σ∗ iscountable

Page 25: TheoreticalFoundations ofComputerScience

24 CHAPTER 2. COMPUTATIONAL PROBLEMS

Bijection f : N→ Σ∗ is:

0 7→ ε

1 7→ a1

2 7→ a2

... ...n 7→ an

n + 1 7→ a1a1

n + 2 7→ a1a2

... ...2n 7→ a1an

2n + 1 7→ a2a1

... ...3n 7→ a2an

... ...n2 + n 7→ anan

n2 + n + 1 7→ a1a1a1

n2 + n + 2 7→ a1a1a2

... ... 2

Theorem 2 2 Any set of decision problems of Σ is uncountable.

*Proof: (Cantor's diagonalization argument.)Lets mark the collection of all decision problems of Σ by Π

Π = {π | π is mapping Σ∗ → {0, 1}}.

Antithesis: Suppose that Π is countable, i.e. there exists a numbering

Π = {π0, π1, π2, . . .}.Let the strings of Σ∗be in canonical order x0, x1, x2, . . ..Let's construct a new decision problem π:

π : Σ∗ → {0, 1}, π(x) =

{1, jos πi(xi) = 0;0, jos πi(xi) = 1.

Page 26: TheoreticalFoundations ofComputerScience

25

Because π ∈ Π, π = πk for some k ∈ N.Then

π(xk) =

{1, if πk(xk) = π(xk) = 0;0, if πk(xk) = π(xk) = 1.

CONTRADICTION. The claim that the set Π is countable, is false. 2

π↘ π0 π1 π2 π3 · · ·

x0

16 0 0 0 1 · · ·

x1 006 1 0 0 · · ·

x2 1 106 1 1 · · ·

x3 0 0 016 0 · · ·

... ... ... ... ... . . .

• we can solve only a small part of all existing computational problems by anyprograms

• all �strong enough� programming languages solve just the same class of solv-able problems (Church�Turing thesis).

• most computational problems are absolutely unsolvable.

• also interesting practical problems

• e.g. halting problem: given a program P and its input x; we should decide, ifthe computation of P completes with input x, or does it stay in in�nite loop

2.0.7 Excursion: The unsolvability of the halting problem

Interpreted in C-formalism: �There doesn't exist a total (always halting) C-program,which would solve, if the given C-program P halts with input w�.

Let's suppose we could write a total C-funcion

int H(char *p, char *w),

Page 27: TheoreticalFoundations ofComputerScience

26 CHAPTER 2. COMPUTATIONAL PROBLEMS

which gets value 1, if the function represented by string p halts with input w, and0 otherwise. Let's write another C-function H:

void H(char *p){if H(p, p) while (1) ;

}

Let's mark the code of H by h and study the computation of H by its own descrip-tion. Contradiction:

H(h) halts ⇔ H(h, h) = 0 ⇔ H(h) doesn't halt.

⇒ such total halting tester program H cannot exist.

The comic about Omnipotency and existency crisis

One day the Great Guru decided to test Turing. �Here is a machine, which can tellabout all machines, if they halt or not. And look! It will always halt itself.�

�Let's study you.First I make a copy of you.�

Turing decided to change the copy a little bit. �You are a very good copy, but stilleat this.� �Hmm. If H says that P halts I'll loop.�

�Ok, machine. Tell me, if this sister of yours halts in her selfrespection.�

The machine thought and thought. �Hmm... If I say that it halts it doesn't halt.��And if I say it doesn't halt it halts.�

�I am sorry I have an existency crisis.� �I cannot exist. Goodbye!� And so themachine vanished into air.

Page 28: TheoreticalFoundations ofComputerScience

27