cdt314 faber formal languages, automata and models of computation lecture 13

11

CDT314

FABER

Formal Languages, Automata and Models of Computation

Lecture 13

Mälardalen University

2012

2

Content

Alan Turing and Hilbert Program Universal Turing Machine Chomsky Hierarchy DecidabilityReducibilityUncomputable FunctionsRice’s TheoremChurch-Turing ThesisComputation beyond Turing ModelInteractive Computing, Persistent TM’s (Dina Goldin/Peter Wegner)

3

TURING MACHINES

Based on C Busch, RPI, Models of Computation

4

Turing’s "Machines". These machines are humans who calculate.

(Wittgenstein)

A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine.

(Turing)

5

............Tape

Read-Write head

Control Unit

Standard Turing Machine

6

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

The Tape

7

............

Read-Write head

1. Reads a symbol

2. Writes a symbol

3. Moves Left or Right

The head at each time step:

8

ExampleTime 0

............ a a cb

Time 1............ a b k c

1. Reads a2. Writes k3. Moves Left

9

Head starts at the leftmost position

of the input string

............

Blank symbol

head

a b ca

Input string

The Input String

#####

10

1q 2qLba ,

Read WriteMove Left

1q 2qRba ,

Move Right

States & Transitions

11

............ # a b caTime 1

1q 2qRba ,

............ a b cbTime 2

1q

2q

# # # #

# # # # #

12

Determinism

1q

2qRba ,

Allowed Not Allowed

3qLdb ,

1q

2qRba ,

3qLda ,

No lambda transitions allowed in standard TM!

Turing Machines are deterministic

13

Formal Definitions for

Turing Machines

14

Transition Function

1q 2qRba ,

),,(),( 21 Rbqaq

15

Turing Machine

),#,,,,,( 0 FqQM

Transition

functionInitial

stateblank

Final

states

States

Input

alphabetTape

alphabet

16

Universal Turing Machine

17

A limitation of Turing Machines:

Better are reprogrammable machines.

Turing Machines are “hardwired”

they execute

only one program

18

Solution: Universal Turing Machine

• Reprogrammable machine

• Simulates any other Turing Machine

Characteristics:

19

Universal Turing Machine

simulates any other Turing Machine M

Input of Universal Turing Machine

• Description of transitions of M

• Initial tape contents of M

20

Universal

Turing

Machine

Description of Three tapes

MTape Contents of

Tape 2

States of M

Tape 3

M

Tape 1

21

We encode/describe Turing Machine as a string of symbols.

M

Description of M

Tape 1

22

Alphabet Encoding

Symbols: a b c d

Encoding: 1 11 111 1111

Tape Contents Encoding

23

State Encoding

States: 1q 2q 3q 4q

Encoding: 1 11 111 1111

Head Move Encoding

Move:

Encoding:

L R

1 11

24

Transition Encoding

Transition: ),,(),( 21 Lbqaq

Encoding: 10110110101

separator

25

Machine Encoding

Transitions:

),,(),( 21 Lbqaq

Encoding:

10110110101

),,(),( 32 Rcqbq

110111011110101100

separator

26

Tape 1 contents of Universal Turing Machine:

encoding of the simulated machine

as a binary string of 0’s and 1’s

M

27

A Turing Machine is described

with a binary string of 0’s and 1’s.

The set of Turing machines forms a language:

Each string of the language is

the binary encoding of a Turing Machine.

Therefore:

28

Language of Turing Machines

L = { 010100101,

00100100101111,

111010011110010101,

…… }

(Turing Machine 1)

(Turing Machine 2)

……

(Turing Machine 3)

29

The Chomsky Hierarchy

30Non-recursively enumerable

Recursively-enumerable

Recursive

Context-sensitive

Context-free

Regular

The Chomsky Language Hierarchy

31

Unrestricted Grammars

Productions

vu

String of variables

and terminals

String of variables

and terminals

Recursively Enumerable Languages

32

Example of unrestricted grammar

dAc

cAaB

aBcS

33

A language is recursively enumerable

if and only if it is generated by an

unrestricted grammar.

L

Theorem

34

Context-Sensitive Grammars

and |||| vu

Productions

vu

String of variables

and terminals

String of variables

and terminals

35

The language }{ nnn cba

is context-sensitive:

aaAaaaB

BbbB

BbccAc

bAAb

aAbcabcS

|

|

36

A language is context sensitive

if and only if

it is accepted by a Linear-Bounded automaton.

L

Theorem

37

Linear Bounded Automata (LBAs)

are the same as Turing Machines

with one difference:

The input string tape space

is the only tape space allowed to use.

38

[ ]a b c d e

Left-end

marker

Input string

Right-end

marker

Working space

in tape

All computation is done between end markers.

Linear Bounded Automaton (LBA)

39

There is a language which is context-sensitive

but not recursive.

Observation

40

Decidability

41

Consider problems with answer YES or NO.

Examples

• Does Machine have three states ?M

• Is string a binary number? w

• Does DFA accept any input? M

42

A problem is decidable if some Turing machine

solves (decides) the problem.

Decidable problems:

• Does Machine have three states ?M

• Is string a binary number? w

• Does DFA accept any input? M

43

Turing MachineInput

problem

instance

YES

NO

The Turing machine that solves a problem

answers YES or NO for each instance.

44

The machine that decides a problem:

• If the answer is YES

then halts in a yes state

• If the answer is NO

then halts in a no state

These states may not be the final states.

45

YES

NO

Turing Machine that decides a problem

YES and NO states are halting states

46

Difference between

Recursive Languages (“Acceptera”) and Decidable problems (“Avgöra”)

The YES states may not be final states.

For decidable problems:

47

Some problems are undecidable:

There is no Turing Machine that

solves all instances of the problem.

48

A famous undecidable problem:

The halting problem

49

The Halting Problem

Input: • Turing Machine M• String w

Question: Does halt on ? wM

50

Theorem

The halting problem is undecidable.

Proof

Assume to the contrary that

the halting problem is decidable.

51

There exists Turing Machine

that solves the halting problemH

HM

w

YES M halts on w

Mdoesn’t

halt onwNO

52

H

wwM 0q

yq

nq

Input:

initial tape contents

Encoding

of M wString

YES

NO

Construction of H

53

Construct machine H

returns YES then loop forever. HIf

returns NO then halt.HIf

54

H

wwM 0q

yq

nq NO

aq bq

H

Loop forever

YES

55

HConstruct machine

Input:

If M halts on input Mw

Then loop forever

Else halt

Mw (machine )M

56

Mw MM wwcopy

Mw H

H

57

HRun machine with input itself

Input:

If halts on input

Then loop forever

Else halt

Hw ˆ (machine )H

H Hw ˆ

58

on input H Hw ˆ

If halts then loops forever.

If doesn’t halt then it halts.

:

H

H

CONTRADICTION !

59

This means that

The halting problem is undecidable.

END OF PROOF

60

Another proof of the same theorem:

If the halting problem was decidable then

every recursively enumerable language

would be recursive.

Recursive vs. Recursively Enumerable Languages

http://www.cs.colostate.edu/~massey/Teaching/cs301/RestrictedAccess/Slides/301lecture23.pdf

61

Theorem

The halting problem is undecidable.

Proof

Assume to the contrary that

the halting problem is decidable.

62

There exists Turing Machine

that solves the halting problem.

H

HM

w

YES M halts on w

Mdoesn’t

halt onwNO

63

Let be a recursively enumerable language. L

Let be the Turing Machine that accepts .M L

We will prove that is also recursive: L

We will describe a Turing machine that

accepts and halts on any input.L

64

M halts on ?wYES

NOM

w

Run

with input

Mw

Hreject w

accept w

reject w

Turing Machine that accepts

and halts on any input

L

Halts on final state

Halts on non-final

state

65

Therefore L is recursive.

But there are recursively enumerable

languages which are not recursive.

Contradiction!

Since is chosen arbitrarily, we have

proven that every recursively enumerable

language is also recursive.

L

66

Therefore, the halting problem is undecidable.

END OF PROOF

67

A simple undecidable problem:

The Membership Problem

68

The Membership Problem

Input: • Turing Machine M

• String w

Question: Does accept ? M w

69

Theorem

The membership problem is undecidable.

Proof

Assume to the contrary that

the membership problem is decidable.

70

There exists a Turing Machine

that solves the membership problem

H

HM

w

YES M accepts w

NO M rejects w

71

Let be a recursively enumerable language. L

Let be the Turing Machine that accepts .M L

We will prove that is also recursive: L

We will describe a Turing machine that

accepts and halts on any input.L

72

M accepts ?wNO

YESM

w

Haccept w

Turing Machine that accepts

and halts on any input

L

reject w

73

Therefore, L is recursive.

But there are recursively enumerable

languages which are not recursive.

Contradiction!

Since is chosen arbitrarily, we have

proven that every recursively enumerable

language is also recursive.

L

74

Therefore, the membership problem

is undecidable.

END OF PROOF

75

Reducibility

76

Problem is reduced to problemA B

If we can solve problem then

we can solve problem .

BA

B

A

77

If is decidable then is decidable.B A

If is undecidable then is undecidable.A B

Problem is reduced to problemA B

78

Example

the halting problem

reduced to

the state-entry problem.

79

The state-entry problem

Inputs:

Question:

M•Turing Machine

•State q

•String w

Does M enter state q

on input ?w

80

Theorem

The state-entry problem is undecidable.

ProofReduce the halting problem to

the state-entry problem.

81

Suppose we have an algorithm (Turing Machine)

that solves the state-entry problem.

We will construct an algorithm

that solves the halting problem.

82

Algorithm for

state-entry

problem

M

w

q

YES

NO

entersM q

doesn’t

enterM q

Assume we have the state-entry algorithm:

83

Algorithm for

Halting problem

M

w

YES

NO

halts onM w

doesn’t

halt onM w

We want to design the halting algorithm:

84

Modify input machine M• Add new state q

• From any halting state add transitions to q

M q

halting statesSingle

halt state

M

85

M halts

M halts on state q

if and only if

86

Algorithm for halting problem

Inputs: machine and stringM w

2. Run algorithm for state-entry problem

with inputs: M wq, ,

1. Construct machine with state M q

87

Generate

M M

w

M qw

State-entry

algorithm

Halting problem algorithm

YES

NO

YES

NO

88

Since the halting problem is undecidable,

it must be that the state-entry problem

is also undecidable.

END OF PROOF

We reduced the halting problem

to the state-entry problem.

89

Another example

The halting problem

reduced to

the blank-tape halting problem.

90

The blank-tape halting problem

Input: MTuring Machine

Question: Does M halt when started with

a blank tape?

91

ProofReduce the halting problem to the

blank-tape halting problem.

Theorem

The blank-tape halting problem is undecidable.

92

Suppose we have an algorithm

for the blank-tape halting problem.

We will construct an algorithm

for the halting problem.

93

Algorithm for

blank-tape

halting problemM

YES

NO

halts on

blank tape

M

doesn’t halt

on blank tape

M

Assume we have the

blank-tape halting algorithm

94

Algorithm for

halting problem

M

w

YES

NO

halts onM w

doesn’t

halt onM w

We want to design the halting algorithm:

95

wMConstruct a new machine

• On blank tape writes w• Then continues execution like M

wM

Mthen write w

step 1 step2

if blank tape execute

with input w

96

M halts on input string

wM halts when started with blank tape.

if and only if

w

97

Algorithm for halting problem

1. Construct wM

2. Run algorithm for

blank-tape halting problem

with input wM

Inputs: machine and stringM w

98

Generate

wMMw

Blank-tape

halting

algorithm

Halting problem algorithm

YES

NOwM

YES

NO

99

Since the halting problem is undecidable,

the blank-tape halting problem is

also undecidable.

END OF PROOF

We reduced the halting problem

to the blank-tape halting problem.

100

Summary of Undecidable Problems

Halting Problem

Does machine halt on input ?M w

Membership problem

Does machine accept string ?M w

Is a string member of a

recursively enumerable language ?)Lw(In other words:

101

Does machine halt when starting

on blank tape?

Blank-tape halting problem

M

State-entry Problem:

Does machine enter state

on input ?

Mw

q

102

Uncomputable Functions

103

Uncomputable Functions

A function is uncomputable if it cannot

be computed for all of its domain.

Domain Rangef

104

An uncomputable function:

)(nfmaximum number of moves until

any Turing machine with states

halts when started with the blank tape.

n

Example

105

TheoremFunction is uncomputable.)(nf

Then the blank-tape halting problem

is decidable.

ProofAssume to the contrary that

is computable.)(nf

106

Algorithm for blank-tape halting problem

Input: machine M

1. Count states of : M m

2. Compute )(mf

3. Simulate for steps

starting with empty tape

M )(mf

If halts then return YES

otherwise return NO

M

107

Therefore, the blank-tape halting

problem must be decidable.

However, we know that the blank-tape

halting problem is undecidable.

Contradiction!

108

Therefore, function is uncomputable.)(nf

END OF PROOF

109

Rice’s Theorem

110

Non-trivial property of

recursively enumerable languages:

Any property possessed by some (not all)

recursively enumerable languages.

Definition

111

Some non-trivial properties of

recursively enumerable languages:

• is emptyL

L• is finite

L• contains two different strings

of the same length

112

Rice’s Theorem

Any non-trivial property of

a recursively enumerable language

is undecidable.

113

Rice’s Theorem

If is a set of Turing-acceptable languages that contain some but not all such languages, no TM can decide for an arbitrary Turing-acceptable language L if L belongs to or not.

114

Exempel

Givet en Turingmaskin M, kan man avgöra om alla strängar som accepteras av M börjar och slutar på samma tecken?

115

Oavgörbart

Problemet handlar om en icke-trivial språkegenskap. Det finns TM:er vars accepterade strängar har egenskapen i fråga, och det finns TM:er vars accepterade strängar inte har egenskapen.

116

Formellt:

= { L | TM accepterbara språk vars strängar börjar

och slutar på samma tecken. }

117

Now we will prove some non-trivial properties

without using Rice’s theorem.

118

Theorem

For any recursively enumerable language Lit is undecidable whether it is empty.

Proof

We will reduce the membership problem

to the problem of deciding whether

is empty.

L

119

Membership problem:

Does machine accept string ?wM

120

Algorithm for

empty language

problem

M

YES

NO

Assume we have the empty language algorithm:

Let be the machine that accepts M L

)(ML

)(ML

empty

not empty

LML )(

121

Algorithm for

membership

problem

M

w

YES

NO

acceptsM w

rejectsM w

We will design the membership algorithm:

122

First construct machine : wM

When enters a final state,

compare original input string with . wM

Accept if original input is

the same as .w

123

Lw

)( wML is not empty

if and only if

}{)( wML w

124

Algorithm for membership problem

Inputs: machine and string M w

1. Construct wM

2. Determine if is empty )( wML

YES: then )(MLw

NO: then )(MLw

125

construct

wM

Check if

)( wML

is empty

YES

NO

M

w

NO

YES

Membership algorithm

wM

126

Since the membership problem is undecidable,

the empty language problem is

also undecidable.

END OF PROOF

We reduced the empty language problem

to the membership problem.

127

Decidability…continued…

128

Theorem

For a recursively enumerable language Lit is undecidable to determine whether

is finite. L

Proof

We will reduce the halting problem

to the finite language problem.

129

Assume we have the finite language algorithm:

Algorithm for

finite language

problem

M

YES

NO

)(ML

)(ML

finite

not finite

Let be the machine that accepts M L

LML )(

130

We will design the halting problem algorithm:

Algorithm for

Halting problem

M

w

YES

NO

halts onM w

doesn’t

halt onM w

131

First construct machine .wM

When enters a halt state,

accept any input (infinite language).

M

Initially, simulates on input . M w

Otherwise accept nothing (finite language).

132

M halts on

)( wML is not finite.

if and only if

w

133

Algorithm for halting problem:

Inputs: machine and string M w

1. Construct wM

2. Determine if is finite )( wML

YES: then doesn’t halt on M wNO: then halts on M w

134

construct

wM

Check if

)( wMLis finite

YES

NO

M

w

NO

YES

Machine for halting problem

135

Since the halting problem is undecidable,

the finite language problem is

also undecidable.

END OF PROOF

We reduced the finite language problem

to the halting problem.

136

Theorem

For a recursively enumerable language Lit is undecidable whether contains

two different strings of same length.

L

ProofWe will reduce the halting problem

to the two strings of equal length- problem.

137

Assume we have the two-strings algorithm:

Let be the machine that accepts M LLML )(

Algorithm for

two-strings

problem

M

YES

NO

)(ML

)(ML

contains

doesn’t

contain

two equal length strings

138

We will design the halting problem algorithm:

Algorithm for

Halting problem

M

w

YES

NO

halts onM w

doesn’t

halt onM w

139

First construct machine . wM

When enters a halt state,

accept symbols or .

M

Initially, simulates on input . M w

a b

(two equal length strings)

140

M halts on

wM

if and only if

w

accepts and a b

(two equal length strings)

141

Algorithm for halting problem

Inputs: machine and string M w

1. Construct wM

2. Determine if accepts

two strings of equal lengthwM

YES: then halts on M w

NO: then doesn’t halt on M w

142

construct

wM

Check if)( wML

has two

equal length

strings

YES

NO

M

w

YES

NO

Machine for halting problem

143

Since the halting problem is undecidable,

the two strings of equal length problem is

also undecidable.

END OF PROOF

We reduced the two strings of equal length -

problem to the halting problem.

144

Church-Turing Thesis*

*Source: Stanford Encyclopaedia of Philosophy

145

A Turing machine is an abstract representation of a computing device.

It is more like a computer program (software)

than a computer (hardware).

146

LCMs [Logical Computing Machines:

Turing’s expression for Turing machines]

were first proposed by Alan Turing,

in an attempt to give

a mathematically precise definition

of "algorithm" or "mechanical procedure".

147

The Church-Turing thesis concerns an effective or mechanical method in logic and mathematics.

148

A method, M, is called ‘effective’ or ‘mechanical’ just in case:

1. M is set out in terms of a finite number of exact instructions (each instruction being expressed by means of a finite number of symbols);

2. M will, if carried out without error, always produce the desired result in a finite number of steps;

3. M can (in practice or in principle) be carried out by a human being unaided by any machinery except for paper and pencil;

4. M demands no insight or ingenuity on the part of the human being carrying it out.

149

Turing’s thesis: LCMs [logical computing machines; TMs] can do anything that could be described as "rule of thumb" or "purely mechanical". (Turing 1948)

He adds: This is sufficiently well established that it is now agreed amongst logicians that "calculable by means of an LCM" is the correct accurate rendering of such phrases.

150

Turing introduced this thesis in the course of arguing that the Entscheidungsproblem, or decision problem, for the predicate calculus - posed by Hilbert (1928) - is unsolvable.

151

Church’s account of the Entscheidungsproblem

By the Entscheidungsproblem of a system of symbolic logic is here understood the problem to find an effective method by which, given any expression Q in the notation of the system, it can be determined whether or not Q is provable in the system.

152

Church’s thesis: A function of positive integers is effectively calculable only if recursive.

153

Misunderstandings of the Turing Thesis

Turing did not show that his machines can solve any problem that can be solved "by instructions, explicitly stated rules, or procedures" and nor did he prove that a universal Turing machine "can compute any function that any computer, with any architecture, can compute".

154

Turing proved that his universal machine can compute any function that any Turing machine can compute; and he put forward, and advanced philosophical arguments in support of, the thesis here called Turing’s thesis.

155

A thesis concerning the extent of effective methods - procedures that a human being unaided by machinery is capable of carrying out - has no implication concerning the extent of the procedures that machines are capable of carrying out, even machines acting in accordance with ‘explicitly stated rules’.

156

Among a machine’s repertoire of atomic operations there may be those that no human being unaided by machinery can perform.

157

Turing introduces his machines as an idealised description of a certain human activity, the tedious one of numerical computation, which until the advent of automatic computing machines was the occupation of many thousands of people in commerce, government, and research establishments.

158

Turing’s "Machines". These machines are humans who calculate. (Wittgenstein)

A man provided with paper, pencil, and rubber, and subject to strict discipline, is in effect a universal machine. (Turing)

Computation Beyond Turing Model

“Traditionally, the dynamics of computing systems, their unfolding behavior in space and time has been a mere means to the end of computing the function which specifies the algorithmic problem which the system is solving. In much of contemporary computing, the situation is reversed: the purpose of the computing system is to exhibit certain behaviour. (…) We need a theory of the dynamics of informatic processes, of interaction, and information flow, as a basis for answering such fundamental questions as: What is computed? What is a process? What are the analogues to Turing completeness and universality when we are concerned with processes and their behaviours, rather than the functions which they compute?” (Abramsky, 2008)

Abramsky, S. (2008) Information, processes and games. In Philosophy of Information; van Benthem, J., Adriaans, P., Eds.; North Holland: Amsterdam, The Netherlands; pp. 483–549.

159

Computation Beyond Turing Model

According to Abramsky, there is the need for second generation models of computation, and in particular there is the need for process models such as Petri nets, Process Algebra, and similar.

The first generation models of computation were originating from problems of formalization of mathematics and logic, while processes or agents, interaction, and information flow are genuine product of the modern development of computing.

In the second generation models of computation, previous isolated systems with limited interactions with the environment are replaced by processes or agents for which the interactions with each other and with the environment are fundamental.

160

Computation Beyond Turing Model

As a result of interactions among agents and with the environment, complex behavior emerges.

The basic building block of this interactive approach is the agent, and the fundamental operation is interaction.

The ideal features sought for are computational qualities of computational agents such as biological organisms such as resilience and self-* capabilities.

This approach works at both macro-scale (such as processes in operating systems, software agents on the Internet, transactions, etc.) and on micro-scale (program implementation down to hardware).

161

162

 Interaction: Conjectures, Results, and Myths

Dina GoldinUniv. of Connecticut, Brown University

http://cs.brown.edu/people/dqg/

Additional reading

163

Fundamental Questions Underlying Theory of Computation

What is computation?

How do we model it?

164

SHARED WISDOM(from our undergraduate Theory of Computation courses)

computation: finite transformation of input to output

input: finite size (e.g. string or number)

closed system: all input available at start, all output generated at end

behavior: functions, transformation of input data to output data

Church-Turing thesis: Turing Machines capture this (algorithmic) notion of computation

Mathematical worldview:

All computable problems are function-based.

165

“The theory of computability and non-computability [is] usually referred to as the theory of recursive functions... the notion of TM has been made central in the development."

Martin Davis, Computability & Unsolvability, 1958

“Of all undergraduate CS subjects, theoretical computer science has changed the least over the decades.”

SIGACT News, March 2004

“A TM can do anything that a computer can do.”Michael Sipser, Introduction to the Theory of Computation, 1997

THE MATHEMATICAL WORLDVIEW

166

The Operating System Conundrum

Real programs, such as operating systems and word processors, often receive an unbounded amount of input over time, and never "finish" their task. Turing machines do not model such ongoing computation well…

[TM entry, Wikipedia]

If a computation does not terminate,

it’s “useless” – but aren’t OS’s

useful??

167

Rethinking Shared Wisdom:(what do computers do?)

computation: finite transformation of input to output

input: finite-size (string or number)

closed system: all input available at start, all output generated at end

behavior: functions, algorithmic transformation of input data to output data

Church-Turing thesis: Turing Machines capture this (algorithmic) notion of computation

computation: ongoing process which performs a task or delivers a service

dynamically generated stream of input tokens (requests, percepts, messages)

open system: later inputs depend on earlier outputs and vice versa (I/O entanglement, history dependence)

behavior: processes, components, control devices, reactive systems, intelligent agents

Wegner’s conjecture: Interaction is more powerful than algorithms

168

Example: Driving home from work

Algorithmic input: a description of the world (a static “map”)

Output: a sequence of pairs of #s (time-series data)- for turning the wheel- for pressing gas/break

Similar to classic AI search/planning problems.

169

But… in a real-world environment, the output depends on every grain of sand in the road (chaotic behavior).

Can we possibly have a map that’s detailed enough?

Worse yet… the domain is dynamic. The output depends on weather conditions, and on other drivers and pedestrians.

We can’t possibly be expected to predict that in advance!

Nevertheless the problem is solvable!

Google “autonomous vehicle research”

Driving home from work (cont.)

?

170

Driving home from work (cont.)

The problem is solvable interactively.

Interactive input: stream of video camera images, gathered as we are driving

Output: the desired time-series data, generated as we are driving

similar to control systems, or online computation

A paradigm shift in the conceptualization of computational problem solving.

171

• Rethinking the mathematical worldview

• Persistent Turing Machines (PTMs)

• PTM expressiveness

• Sequential Interaction – Sequential Interaction Thesis

• The Myth of the Church-Turing Thesis

– the origins of the myth

• Conclusions and future work

OUTLINE

172

Sequential Interaction

Sequential interactive computation:

System continuously interacts with its environment by alternately accepting an input string and computing a corresponding output string.

Examples:- method invocations of an object instance

in an OO language- a C function with static variables- queries/updates to single-user databases- recurrent neural networks- control systems; online computation; transducers; dynamic

algorithms; embedded systems, etc.

173

Sequential Interaction Thesis

• Universal PTM: simulates any other PTM– Need additional input describing the PTM (only once)

• Example: simulating Answering Machine(simulate AM, will-do), (record hello, ok), (erase, done), (record John, ok),(record Hopkins, ok), (playback, John Hopkins), …

Simulation of other sequential interactive systems is analogous.

174

CHURCH-TURING THESIS REVISITED

Church-Turing Thesis:

Whenever there is an effective method for obtaining the values of a mathematical function, the function can be computed by a Turing Machine.

Common Reinterpretation (Strong Church-Turing Thesis)

A TM can do (compute) anything that a computer can do.

The equivalence of the two is a myth!

This myth has been dogmatically accepted by the CS community.

The function-based behavior of algorithms does not capture all forms of computation.

Turing himself would have denied it.

In the same paper where he introduced what we now call Turing Machines, he also introduced choice machines, as a distinct model of computation.

Choice machines extend Turing Machines to interaction by allowing a human operator to make choices during the computation.

175

ORIGINS OF THE CHURCH-TURING THESIS MYTH

A TM can do anything that a computer can do.

Based on several claims:

1. A problem is solvable if there exists a Turing Machine for computing it.

2. A problem is solvable if it can be specified by an algorithm.

3. Algorithms are what computers do.

Each claim is correct in isolation, provided we understand the underlying assumptions.

Together, they induce an incorrect conclusion

TMs = solvable problems = algorithms = computation

176

Deconstructing the Turing Thesis Myth (1)

TMs = solvable problems

• Assumes:All computable problems are function-based.

• Reasons:

– Theory of Computation started as a field of mathematics; mathematical principles were adopted for the fundamental notions of computation, identifying computability with the computation of functions, as well as with Turing Machines.

– The batch-based operation of original computers did

suggest other conceptualizations of computation.

177

Deconstructing the Turing Thesis Myth (2)

solvable problems = algorithms

Assumes:- Algorithmic computation is also function based;

i.e., the computational role of an algorithm is to transform input data to output data.

Reasons: – Original (mathematical) meaning of “algorithms”

E.g. Euclid’s greatest common divisor algorithm

– Original (Knuthian) meaning of “algorithms”

“An algorithm has zero or more inputs, i.e., quantities which are given to it initially before the algorithm begins.“ [Knuth’68]

178

Deconstructing the Turing Thesis Myth (3)

algorithms = computation

Reasons: – The ACM Curriculum (1968): Adopted algorithms as the central

concept of CS without explicit agreement on the meaning of this term.

179

Deconstructing the Turing Thesis Myth (3)

algorithms = computation

Reasons: – Textbooks: When defining algorithms, the assumption of their

closed function-based nature was often left implicit, if not forgotten

“An algorithm is a recipe, a set of instructions or the specifications of a process for doing something. That something is usually solving a problem of some sort.”

[Rice&Rice’69]

“An algorithm is a collection of simple instructions for carrying out some task. Commonplace in everyday life, algorithms sometimes are called procedures or recipes.” [Sipser’97]

180

• Rethinking the mathematical worldview

• Persistent Turing Machines (PTMs)

• PTM expressiveness

• Sequential Interaction

• The Myth of the Church-Turing Thesis

• Conclusions and future work

Outline

181

The Shift to Interaction in CS

Computation = transforming input to output

Computation = carrying out a task over time

Logic and search in AI Intelligent agents, partially observable environments, learning

Procedure-oriented programming

Object-oriented programming

Closed systems Open systems

Compositional behavior Emergent behavior

Rule-based reasoning Simulation, control, semi-Markov processes

Algorithmic Interactive

182

The Interactive Turing Test

• From answering questions to holding discussions.• Learning from -- and adapting to -- the questioner.• “Book intelligence” vs. “street smarts”.

“It is hard to draw the line at what is intelligence and what is environmental interaction. In a sense, it does not really matter which is which, as all intelligent systems must be situated in some world or other if they are to be useful entities.” [Brooks]

183

Many other interactive models– Reactive and embedded systems– Dataflow, I/O automata [Lynch], synchronous languages,

finite/pushdown automata over infinite words– Interaction games [Abramsky], online algorithms [Albers]– TM extensions: on-line Turing machines [Fischer], interactive Turing

machines [Goldreich]...

Concurrency Theory– Focuses on communication (between concurrent agents/processes)

rather than computation [Milner]– Orthogonal to the theory of computation and TMs.

What makes PTMs unique?– Provably more expressive than TMs.– Bridging the gap between concurrency theory (labeled transition

systems) and traditional TOC.

Modeling Interactive Computation: PTMs in Perspective

184

Theory of Sequential Interactionconjecture: notions analogous to computational complexity, logic, and recursive functions can be developed for sequential interaction computation

Multi-stream interaction–From hidden variables to hidden interfacesconjecture: multi-stream interaction is more powerful than sequential interaction [Wegner’97]

Formalizing indirect interaction–Interaction via persistent, observable changes to the common environment–In contrast to direct interaction (via message passing)conjecture: direct interaction does not capture all forms of multi-agent behaviors (GDC: except in case when environment is represented as an agent in an multi-agent model)

Future Work: 3 conjectures

185

Referenceshttp://www.cse.uconn.edu/~dqg/papers/

[Wegner’97] Peter WegnerWhy Interaction is more Powerful than AlgorithmsCommunications of the ACM, May 1997

[EGW’04] Eugene Eberbach, Dina Goldin, Peter Wegner Turing's Ideas and Models of Computationbook chapter, in Alan Turing: Life and Legacy of a Great Thinker, Springer 2004

[I&C’04] Dina Goldin, Scott Smolka, Paul Attie, Elaine SondereggerTuring Machines, Transition Systems, and InteractionInformation & Computation Journal, 2004

[GW’04] Dina Goldin, Peter WegnerThe Church-Turing Thesis: Breaking the Mythpresented at CiE 2005, Amsterdam, June 2005 to be published in LNCS