csci 317 discrete structures for computer science 5. countability

5. Countability and decidability D. Keil Discrete Structures 2/14

David M. Keil, Framingham State University CSCI 317 Discrete Structures for Computer Science

5. Countability and decidability

1. Countable sets

2. Incompleteness

3. Recursive function theory

4. Undecidable problems

5. Non-well-founded sets and coinduction

1 D. Keil Discrete Structures for Computer Science 5. Countability 2/14

D. Keil Discrete Structures for Computer Science 5. Countability 2/14 2

• What does analyzing infinity

have to do with computing?

• Are all true assertions provable?

• Are all computational

problems solvable?

Inquiry



5. Distinguish countable from

uncountable sets, applying the

diagonal proof method in number

theory, logic, and computability

Reading: Epp, Sec. 6.4; Ch. 7; handouts

Topic objective

Subtopic outcomes


5.1a Prove that a set is countable*

5.1b Prove that a set is uncountable*

5.2 Describe the Incompleteness Theorem

and its proof

5.3 Explain primitive recursion and its

correspondence with computability

5.4 Prove that a problem is undecidable

5.5 Define a non-well-founded

set coinductively


• Are all infinite sets the same size?

• What does it mean to say that two

sets are the same size?


1. Countable sets

Infinite sets


• We define countable and uncountable

sets, beginning with a discussion

of functions

• Cantor showed that the real numbers

are uncountable, using a diagonal proof

• Later we will examine other

diagonal proofs


Cardinalities of sets


• Cardinality of a set: its number of elements

• Finite set one whose cardinality may be

expressed as a natural number; i.e., for

which a bijection with {0, 1, …, n} exists

• Examples: natural numbers up to a given n;

strings in k over finite alphabet

• Infinite set: one not finite

• Example: Card(N) = |N| = Card (ODD)

• The natural numbers ℕ can be arranged in a

particular order; we say they are countable

• Definition: A set is countable iff there is a

bijection between it and ℕ

• Countable sets tend to be sets of finite objects,

defined inductively

• Example: Natural numbers divisible by 10:

bijection {(0, 0), (1, 10), (2, 20), …}

D. Keil Discrete Structures for Computer Science 5. Countability 2/14

Countable sets

8


Cardinalities and countability • We say that sets have the same cardinality iff

there is a bijection (1-to-1 correspondence)

between their sets of elements

• Example with infinite sets:

Card (EVEN) = Card (ℕ)

Proof: EVEN0 = 0, EVEN1 = 2, etc.

• Example:

Card (ℚ) = Card (ℕ)

Proof: Zig-zag through a table of rationals


The rational numbers are countable

• The rationals may be enumerated as below,

where the numerator is the row and the

denominator is the column :



Countability of strings • A language is a set of strings

• The set of strings *, over finite alphabet ,

is countable; hence any language is countable

• A language may be represented as an infinite

bit sequence: a 1 bit in a certain bit location

denotes membership in the language:

{0, 10, 11, 100} = 10101100…

• Are the Java programs countable?

Propositional-logic formulas?


The set ℝ of real numbers • Intuitively, the real numbers express analog

or continuous quantities; e.g., they give all

possible weights or distances.

• We may consider the set ℝ, of reals, as

corresponding to all the points on a line

segment, line, plane, space, or space/time

• Another definition: The reals are numbers

that can be expressed with an infinite series

of digits after the decimal or binary point



Theorem: Card (ℝ) > Card (ℕ) (Cantor)

Proof:

1. Suppose ℝ were countable

2. Then the interval [0, 1) could be enumerated

as the following infinite binary fraction:

r1 = 0 . b1,1 b1,2 b1,3 …

r2 = 0 . b2,1 b2,2 b2,3 …

…

3. Now, consider real s = 0. b1,1 b2,2 b3,3 …

That is, for all n, bit n is bn,n (bitwise

negation of the diagonal of r)


Let bi,j be the

ith bit of rj

Cantor’s thm. (cont’d)

4. Now, since for every i, the ith bit of s is

different from the ith bit of ri, so s differs

from every element of r, so there is no

element of sequence r that matches s

5. Hence r does not contain s, which is

clearly a real number

6. Hence we have a contradiction and must

reject the supposition

7. Hence ℝ is not countable



Cantor’s proof, illustrated Supposed enumeration of ℝ:

r1 = (0, 0, 0, 0, 0, 0, 0, ...)

r2 = (1, 1, 1, 1, 1, 1, 1, ...)

r3 = (0, 1, 0, 1, 0, 1, 0, ...)

r4 = (1, 0, 1, 0, 1, 0, 1, ...)

r5 = (1, 1, 0, 1, 0, 1, 1, ...)

r6 = (0, 0, 1, 1, 0, 1, 1, ...)

r7 = (1, 0, 0, 0, 1, 0, 0, ...)

...

Real number s that differs

with every element in

enumeration in at least

one bit:

s = (1, 0, 1, 1, 1, 0, 1, ...)


Another simple diagonal proof • Let be “the set of all sets” ; hence

• Theorem: “set of all sets” is an inconsistent idea

• Proof:

1. Consider = { A : A A } (sets not members of themselves)

2. ( ) ( ) , ( ) ( )

3. These are contradictions hence reject #1

• Note role of “Spoiler” instance

• We say that , are not well founded (NWF)



How many functions exist? • Start with the set of functions

{ f : {0} {0,1}}

• This is a set of two functions:

{{(0,1)}, {(0,0)}}

• The set { f : {0, 1} {0,1}} has four fns,

{ f : {0, 1, 2} {0,1}} has eight, etc.

• There are 2|ℕ| predicates f : ℕ {0,1}

• Compare with the cardinality of the natural

numbers; the cardinality of reals


Can all predicates on ℕ be

computed in Java? 1. Enumerate all Java methods that take an integer

parameter and return 0 or 1, as J1, J2, J3, … in a

bitwise ordering

2. Consider the predicate f : ℕ {0,1} s.t. f (n) = 1

if Jn(n) = 0 or Jn(n) hangs, f (n) = 0 if Jn(n) = 1

3. The predicate f differs in its behavior from each

element of our enumeration of Java methods

4. Hence f is a predicate not computed by any

Java method



Results • ℕ is countable (cardinality 0, aleph-null)

• ℝ is uncountable (cardinality 1)

• The set of Java functions is countable

using their bit representations

• The set of predicates f : ℕ {0,1} is

uncountable

• There are 20 = 1 predicates on ℕ

• The sets of natural numbers are

uncountable (see handout)


Subtopic outcomes


5.1a Prove that a set is countable*

5.1b Prove that a set is uncountable*



• What is logic?

• What is a theorem?

• Are all true assertions theorems?

• Are there limits to the power

of logic?

2. Incompleteness

Formal systems


• Consist of formal expressions of

–Axioms

–Rules of inference

• Theorems may be derived by application

of rules of inference starting with axioms

• Proofs are finite applications of rules of

inference, leading to theorems

• Theory: a synonym for a formal system


Soundness and completeness


• Sound (consistent): a system of rules that

produces from set S only sentences that

logically follow from S

• Examples:

• Modus Ponens: (p (p q) ) q

• Modus Tollens: ((p q) q) p

• Complete: a system that can infer from S

every sentence that follows from S

Gödel’s result


• Gödel showed, with a self-referential

assertion, that no system of logic can be

both consistent and complete

• The proof method is similar to the

diagonal proof

• This resolved one of the main research

problems announced by David Hilbert in

the late 19th century


Gödel’s incompleteness theorem • Kurt Gödel, 1931, at age 25, in Vienna,

challenged accepted mathematical ideas as

developed in Russell and Whitehead’s

Principia Mathematica

• He showed that no formal axiomatic system

of logic could be both consistent

and complete

• Since predicate calculus was proven to be

consistent, Gödel’s theorem showed that

some truths are unprovable in the system


General-purpose definitions • Consistency: Attribute of a system in which

no false assertion can be proven

(p) ((⊢ p) p)

• Completeness: Attribute of a system in

which every true assertion can be proven

(p) ( p ⊢ p)

• Gödel number: A value that uniquely

encodes a logical assertion or sequence of

assertions; e.g., a proof



Intuition for Gödel’s proof • Gödel needed a counter-example to the

claim that a system can be both complete

and consistent

• He showed that something like the

following is true but unprovable:

“This assertion has no proof ”

• This assertion here seems indeterminate

• But the assertion actually used has a

precise meaning


Definitions related to theorem • Proves(x, y, z): The assertion that the sequence of

symbols with Gödel number x is a valid proof that

y(z) is true, where y is the formula with Gödel

number y, and z is a natural number.

• Let G(y) be the assertion that (x)Proves(x, y, y)

i.e., G(y) means that what the formula with Gödel

number y asserts about y is unprovable

• Example: Where y encodes the predicate even,

G(y) is the assertion that one cannot prove that the

encoding of the predicate even is an even number



Gödel’s theorem

“(x) Proves (x, #(G), #(G))” is true

in predicate calculus, but cannot be

proven in the predicate calculus

• Or: What G asserts about predicate G is true but unprovable

• Or: There is no proof that G(#(G)) is true, but G(#(G)) is nevertheless true


Proof of Gödel’s theorem 1. Suppose the theorem’s claim is false and there

exists a proof in predicate calculus that (x)

Proves(x, #(G), #(G)).

2. Then the claim is true, by consistency of

predicate calculus.

3. Therefore by its own assertion it is unprovable

in predicate calculus.

4. This contradicts step 1, hence claim holds.

Note: The above proof goes outside the formal

system of predicate calculus.



A diagonal way to see the proof • Think of an infinite truth table, with all formulas on

one variable enumerated down the left side and

arguments to the formulas as column headers

• There is a true in column c of row r iff fr(c) is true

• Let q be the Gödel number of the formula (fq(q))

that asserts the unprovability of its argument

• Then there must be a true in col. q of row q, because

if fq(q) = false then formula q is not unprovable,

hence provable, hence true, a contradiction

• But if fq(q) = true, then fq(q) is both true and (as it

claims) unprovable


Incompleteness and computing

• Using a similar diagonal method, Turing

proved a special case of Gödel’s theorem

• Given a computable function f, executing an

algorithm to compute y = f (x) proves that

f (x) = y

• But for some functions f, for some x,

y = f (x), but there is no proof that f (x) = y

• This is the same as saying that no algorithm

computes f ; example: the Halting Problem



Gödel’s theorem and computation


• The concept of a formal system has close

connection to computation

• This is because mechanical checking, not

higher mathematical reasoning, is used to

verify logical assertions

• Original statement of Gödel’s theorem referred

only to a specific formal system (Principia);

• But with theoretical results about computabil-

ity in 1930s, it was clear that Gödel’s proof

applies to formal systems in general

Larger significance • Gödel’s theorem was part of a series of results

establishing some limits of knowledge, e.g.

- Einstein’s relativity results

- Heisenberg’s Uncertainty Principle

- Turing’s discovery of undecidable problems

• It marked the failure of Russell’s, Whitehead’s

and Hilbert’s effort to reduce mathematics to

first-order logic and set theory

• A category of valid assertions is established

that we cannot know are valid



Misinterpretations of theorem


• It does not assert that statement G is true,

only that Consistent(S) G

• Only applies to systems with a certain

amount of arithmetic

• G is a “provable fixpoint” where f (x) = x

• Formal systems have limited ability to

prove lower bounds on Kolmogorov

complexity of strings (Chaitin)

Misapplications of the theorem


• It is said that it is often misused as an

analogy, metaphor, or source of inspiration

• Theorem does not say that every consistent

formal system is incomplete; only systems

with a certain amount of arithmetic


Subtopic outcome


5.2 Describe the Incompleteness

Theorem

• What problems are solvable?

• Is there a way to identify a large

set of functions that are

algorithmically computable?

• Will that help define what

computation is?


3. Recursive function theory


Functions and computation


• Functions, with their statically fixed inputs and

outputs, are algorithmic computing problems

• Services, in the form of streams of output

interleaved dynamically with streams of inputs,

are the problems of sequential interaction

• Missions, consisting of specifications of

multiple possibly asynchronous input/output

streams, are the problems of multi-stream

interactive systems

• The set of functions that are recursively

definable, extended slightly, is exactly the

same set of functions that are computable

• Some models of computation: Turing

machines, the Java language, random-

access machines

• Results are due to K. Gödel and S. Kleene

and are related to work on the lambda

calculus (A. Church)

40

Overview: -recursive functions



The basic primitive

recursive functions on N

• zero(x) = 0

• succ(x) = x + 1

• pred(x) = x – 1 for x > 0

• projk(x1, … xn) = xk , for any k

• The above functions are computable by

simple programs and Turing machines


Composition of primitive

recursive functions • Definition: If h(x) = f (g(x)) then h is said

to be obtained from f and g by composition

• Theorem: If f and g are computable, and h

is obtained from f and g by composition,

then h is computable

• Proof: A program that combines the

programs for f and g in sequence, passing

Pf’s output to Pg as input, computes h



43

Primitive recursion • Definition: With k fixed, for all x 0, if

h(0) = k; h(x+1) = g(x, h(x))

Then h is said to be obtained from g by

primitive recursion

• Example:

factorial(0) = 1

factorial(x+1) = g(x, factorial(x))

where g(x, y) = (x+1)y

Thus factorial is obtained from multiplication by

primitive recursion (fact (x + 1) = (x+1)fact(x))


44

Computability of primitive

recursive functions • Theorem: If g is algorithmically

computable, and h is obtained from g by

primitive recursion, then h is computable

• Proof: The algorithm on the next slide

computes h(x) where

h(0) = k

h(x+1) = g(x, h(x)) for all x > 0



45

Computability of primitive recursive functions, cont’d f(x)

y k

z 0

while x > 0 y g(z, y)

z z + 1 x x ∸ 1

return y

Discussion: y gets value of h(0), then h(1), h(2), etc., up to h(x)


46

Minimalization of predicates

• Let miny (P(x1, … , xn, y)) be the smallest

value of y such that P(x1, … , xn, y) is true

• Proper minimalization is applied when the

function miny (P(x1, … , xn, y)) is total

• Examples:

–Find smallest prime number that has

“00000” in its binary expression

–Find the shortest Java program that

outputs its own code



-recursive functions

• Define (mu), or bounded minimalization:

y (f (x1, x2, y)) =

min { y | f (x1, x2, y) = 0 }

• Definition: The -recursive functions are

the primitive recursive functions and the

functions constructible from PR functions

by application of minimalization.


Computability and -recursion


• Theorem: Function y (f (x1, x2, y)) is

computable iff f is -recursive

• Proof: If f (x,y) is computable, then an

algorithm also exists that increments y

starting at 0 until condition

f (x,y) = 0 is satisfied.


Computability and strings


• An algorithmic problem is a function

f : N N …

• or * *, or * {F, T} for some

alphabet such as = {0, 1}

• The -recursive functions may thus be

considered to include the computable

functions * *, or * {F, T} where

natural encodings-decodings are used

between * and N

Computability


The following are equivalent,

for function f :

• f is computed by some algorithm

• f is computed by some Turing machine

• f is a -recursive function


Subtopic outcome


5.4 Prove that a problem is

undecidable

• Are not all problems solvable?

• Is automatic software to verify

software possible?


4. Undecidable problems


Functions that are not computable


• The functions on natural numbers are

uncountable (see Subtopic 5.1)

• The programs in any language are a subset

of {0, 1}*, i.e., bit strings

• The set of finite bit strings is countable

• Hence there are problems or functions

for which there exists no program P that

solves or computes

• Some functions are uncomputable; problems

where is an uncomputable predicate are

called undecidable

• Recognizing undecidable and intractable

problems (See T6) can save unbounded effort

• Notation:

– P is the function computed by program P

– #(P) N encodes P’s source code

– f (x) means that function f is defined for x

– f (x) means that f is undefined for x

Undecidability and intractability



55

The Halting Problem • Consider the decision problem or language

HALT, consisting of the set of pairs (M,x) s.t. the

TM with description M halts on input x

• Is there a TM that decides HALT?


56

Theorem: HALT is undecidable Proof:

1. Suppose HALT is decidable.

2. Then construct TM S from HALT-

decider H, where S makes a copy

of its input M, feeds M and M to H,

loops forever if H(M,M), halts if H(M,M)

3. Consider whether S halts on input S. If S(S) then

S(S) and conversely -- a contradiction.

4. Since S is clearly constructible except for

component H, therefore H cannot be constructed

5. Hence HALT is undecidable.



Undecidability of HALT problem • Let HALT(P, x) =

true if program P halts on input x (P(x))

false otherwise

• If HALT is decidable by some program H, then a

program S can be constructed, with input x, that

halts iff H determines that the program with code x

loops forever on input x

• Consider how S behaves on input #(S):

– S halts (S(#(S))) if S hangs on input #(S)

– S hangs (S(#(S))) if S halts on input #(S)

• Since S hangs iff S halts, S, hence H, cannot exist


58

Program behavior is undecidable • Not only is the problem of deciding whether a

given program halts uncomputable…

• …but also no programs exist to decide any

useful (nontrivial) property of programs

• By “nontrivial” we mean any that holds for

some programs but not all programs

• Examples: – Is L(M) infinite? Regular? CF? – Is x in L(M)? – Is L(M1) a subset of L(M2)



Subtopic outcome


5.4 Prove that a problem

is undecidable


5. Non-well-founded sets and coinduction

• What happens to recursion when

there is no base case?

• How can infinitely large structures

be defined?

• How are infinite processes modeled?


Streams


• An infinite stream of inputs to a reactive

system may be imagined, together with the

corresponding infinite stream of outputs

• Sets of streams express interactive behavior

• Coinduction defines sets of infinite objects as

induction defines sets of finite ones

ProgramEnvironmentinputs

Interaction stream

outputs

Inductively defined sets • (i) 0 is a natural number;

(ii) Every n ℕ has a unique successor, n; (iii) i and ii are the only ways to obtain a

natural number (Peano)

• Set of expressions using numerals, +, and ( ):

expression

numeral |

numeral + expression | ( expression )

• The definition of such a set may use itself

• But such a set does not contain itself



Strings are defined inductively • A string is a sequence of symbols over a

finite alphabet

• Induction defines countable sets of

finite objects

• Example: Strings over alphabet

* = {} { ax | a , x *}

• Sets of strings are countable, e.g., by

numbering them according to bit

representation: , 0, 1, 01, 00, 10, 11, …


Well-founded sets • (B. Russell) Let the village hair stylist be the

person who cuts the hair of everyone who

doesn’t cut their own hair

• Question: Who cuts the hair-stylist’s hair?

• Similar Questions: Is there a set of all sets?

If so, does it contain itself ? Is there a set of all

sets that don't contain themselves?

• In classical (well founded) set theory, it is

meaningless to say a set belongs or doesn't

belong to itself (Foundation Axiom)



Alternative axioms of set theory


• Foundation axiom: When a set is defined

by a predicate, the specification must say

that the set is a subset of a known set

• That is, no set may belong to itself

• Anti-Foundation axiom: asserts that a set

equation has a unique solution if it enables

a set to belong to itself directly or indi-

rectly (e.g., A = {B }, B = {C}, C = {A})

Streams and

non-well-founded sets • Example: A = { B, C }; B = { A, D }

• Every stream of characters is a character,

followed by a stream of characters

• Streams contain streams, which contain

streams, which contain streams, ...

• Ex.: The set of all bit streams {0,1}

consists of any infinite sequence that

consists of 0 or 1 followed by a stream



Streams and coinduction • Induction defines countable sets of finite objects:

* = {} { ax | a , x *}

• Coinduction

– is a dual of induction (recursion)

– lacks a base case

– is used to define sets of infinite objects

• The set of streams over an alphabet, expressing

ongoing processes such as interaction, is

defined coinductively:

= { ax | a , x }


Subtopic outcome


5.5 Define a non-well-founded

set coinductively


Barwise, Moss. Vicious Circles, 1996

Finsler, 1920s (NWF sets)

M. Davis, R. Sigal, E. Weyuker. Computability, Complexity, and

Languages. Academic Press, 1994.

John W. Dawson. Logical Dilemmas: The Life and Work of Kurt

Gödel. A. K. Peters, 1997.

S. Epp. Discrete Mathematics with Applications. Brooks/Cole, 2011.

T. Franzen. Gödel’s Theorem: An Incomplete Guide to its Use and

Abuse. AK Peters, 2005.

Kurt Gödel. On Formally Undecidable Propositions of the Principia

Mathematica and Related Systems. 1931.

Ernest Nagel and James R. Newman. Gödel’s Proof. New York

University Press, 1958.


References

csci 317 discrete structures for computer science 5. countability

Documents