1

Language Recognition (11.4)and Turing Machines (11.5) Longin Jan LateckiTemple University

Based on slides by Costas Busch from the coursehttp://www.cs.rpi.edu/courses/spring05/modcomp/and …

2

Three Equivalent Representations

Finite automata

Regularexpressions

Regular languages

Each can

describethe others

Kleene’s Theorem: For every regular expression, there is a deterministic finite-state automaton that defines the same language, and vice versa.

3

EXAMPLE 1

Consider the language { ambn | m, n N}, which is represented by the regular expression a*b*.

A regular grammar for this language can

be written as follows: 

S | aS | B B b | bB.

4

Regular Expression

Regular Grammar

a* S | aS(a+b)* S | aS | bSa* + b* S | A | B

A a | aAB b | bB

a*b S b | aSba* S bA

A | aA(ab)* S | abS

5

NFAs Regular grammarsThus, the language recognized by FSA

is a regular language Every NFA can be converted into a corresponding regular grammar and vice versa.Each symbol A of the grammar is associated with a non-terminal node of the NFA sA, in particular, start symbol

S is associated with the start state sS.

Every transition is associated with a grammar production: T(sA,a) = sB A aB.

Every production B is associated with final state sB.

See Ex. 3, p. 771, and Ex. 4, p. 772.

6

Kleene’s Theorem

LanguagesGenerated byRegular Expressions

LanguagesRecognizedby FSA

7

LanguagesGenerated byRegular Expressions

LanguagesRecognizedby FSA

LanguagesGenerated byRegular Expressions

LanguagesRecognizedby FSA

We will show:

8

Proof - Part 1

r)(rL

For any regular expression the language is recognized by FSA (= is a regular language)

LanguagesGenerated byRegular Expressions

LanguagesRecognizedby FSA

Proof by induction on the size of r

9

Induction BasisPrimitive Regular Expressions: a,,

NFAs

)()( 1 LML

)(}{)( 2 LML

)(}{)( 3 aLaML

regularlanguages

a

10

Inductive Hypothesis

Assume for regular expressions andthat and are regular languages

1r 2r

)( 1rL )( 2rL

11

Inductive StepWe will prove:

1

1

21

21

*

rL

rL

rrL

rrL

Are regular Languages

12

By definition of regular expressions:

11

11

2121

2121

**

rLrL

rLrL

rLrLrrL

rLrLrrL

13

)( 1rL )( 2rLBy inductive hypothesis we know: and are regular languages

Regular languages are closed under: *1

21

21

rLrLrL

rLrL Union Concatenation

Star

We need to show:

This fact is illustrated in Fig. 2 on p. 769.

14

Therefore:

** 11

2121

2121

rLrL

rLrLrrL

rLrLrrL

Are regularlanguages

And trivially: ))(( 1rL is a regular language

15

Proof - Part 2

LanguagesGenerated byRegular Expressions

LanguagesRecognizedby FSA

Lr LrL )(

For any regular language there is a regular expression with

Proof by construction of regular expression

16

Since is regular take the NFA that accepts it

LM

LML )(

Single final state

17

From construct the equivalentGeneralized Transition Graph in which transition labels are regular

expressions

M

Example:

a

ba,

cM

a

ba

c

18

Another Example:

ba a

b

b0q 1q 2q

ba,a

b

b0q 1q 2q

b

b

19

Reducing the states:ba

ab

b0q 1q 2q

b

0q 2q

babb*

)(* babb

20

Resulting Regular Expression:

0q 2q

babb*

)(* babb

*)(**)*( bbabbabbr

LMLrL )()(

21

In GeneralRemoving states:

iq q jqa b

cde

iq jq

dae* bce*dce*

bae*

22

The final transition graph:

0q fq

1r

2r

3r4r

*)*(* 213421 rrrrrrr

LMLrL )()(

The resulting regular expression:

23

DFA - regular languagesPush down automata - Context-freeBounded Turing M’s - Context sensitiveTuring machines - Phrase-structure

Models of computing

24

Foundations

The theory of computation and the practical application it made possible — the computer — was developed by an Englishman called Alan Turing.

25

Alan Turing1912 (23 June): Birth, Paddington, London1931-34: Undergraduate at King's College, Cambridge University1932-35: Quantum mechanics, probability, logic1936: The Turing machine, computability, universal machine1936-38: Princeton University. Ph.D. Logic, algebra, number theory 1938-39: Return to Cambridge. Introduced to German Enigma cipher machine1939-40: The Bombe, machine for Enigma decryption

1939-42: Breaking of U-boat Enigma, saving battle of the Atlantic1946: Computer and software design leading the world.1948: Manchester University1949: First serious mathematical use of a computer1950: The Turing Test for machine intelligence1952: Arrested as a homosexual, loss of security clearance1954 (7 June): Death (suicide) by cyanide poisoning, Wilmslow, Cheshire.

—from Andrew Hodges http://www.turing.org.uk/turing/

26

The Decision Problem

In 1928 the German mathematician, David Hilbert (1862-1943), asked whether there could be a mechanical way (i.e. by means of a fully specifiable set of instructions) of determining whether some statement in a formal system like arithmetic was provable or not.In 1936 Turing published a paper the aim of which was to show that there was no such method. “On computable numbers, with an application to the Entscheidungs problem.” Proceedings of the London Mathematical Society, 2(42):230-265.

27

The Turing MachineIn order to argue for this claim, he needed a clear concept of “mechanical procedure.”His idea — which came to be called the Turing machine — was this:

(1) A tape of infinite length

(2) Finitely many squares of the tape have a single symbol from a finite language.

(3) Someone (or something) that can read the squares and write in them.

(4) At any time, the machine is in one of a finite number of internal states.

(5) The machine has instructions that determine what it does given its internal state and the symbol it encounters on the tape. It can change its internal state; change the symbol on

the square; move forward; move backward; halt (i.e. stop).

28

01

1

1

1

Current state = 1

If current state = 1and current symbol = 0then new state = 10new symbol = 1move right

0

29

11

1

1

1

Current state = 10

If current state = 1and current symbol = 0then new state = 10new symbol = 1move right

30

11

1

1

1

Current state = 10

If current state = 1and current symbol = 0then new state = 10new symbol = 1move right

31

FunctionsIt is essential to the idea of a Turing machine that it is not a physical machine, but an abstract one — a set of procedures.

It makes no difference whether the machine is embodied by a person in a boxcar on a track, or a person with a paper and pencil, or a smart and well-trained flamingo.

32

Turing’s TheoremIn the 1936 paper Turing proved that there are “general-purpose” Turing machines that can compute whatever any other Turing machine.This is done by coding the function of the special-purpose machine as instructions of the other machine — that is by “programming” it. This is called Turing’s theorem.These are universal Turing machines, and the idea of a coding for a particular function fed into a universal Turing machine is basically our conception of a computer and a stored program. The concept of the universal Turing machine is just the concept of the computer as we know it.

33

First computers: custom computing machines

1946 -- Eniac: the control is hardwired manually foreach problem.

Control

Input tape (read only)

Output tape (write only)

Work tape (memory)

1940: VON NEUMANN: DISTINCTION BETWEEN DATA AND INSTRUCTIONS

34

Can Machines Think?In “Computing machinery and intelligence,” written in 1950, Turing asks whether machines can think.He claims that this question is too vague, and proposes, instead, to replace it with a different one.That question is: Can machines pass the “imitation game” (now called the Turing test)? If they can, they are intelligent. Turing is thus the first to have offered a rigorous test for the determination of intelligence quite generally.

35

The Turing TestThe game runs as follows. You sit at a computer terminal and have an electronic conversation. You don’t know who is on the other end; it could be a person or a computer responding as it has been programmed to do.If you can’t distinguish between a human being and a computer from your interactions, then the computer is intelligent.Note that this is meant to be a sufficient condition of intelligence only. There may be other ways to be intelligent.

36

Artificial Intelligence

37

The Church-Turning ThesisTuring, and a logician called Alonzo Church (1903-1995), independently developed the idea (not yet proven by widely accepted) that whatever can be computed by a mechanical procedure can be computed by a Turing machine.This is known as the Church-Turing thesis.

38

AI: The ArgumentWe’ve now got the materials to show that AI is possible:

P1: Any function that can be computed by a mechanical procedure can be computed by a Turing machine. (Church-Turing thesis)

P2: Thinking is nothing more than the computing of functions by mechanical procedures (i.e., thinking is symbol manipulation). (Functionalist-Computationalist thesis)

C1: Therefore, thinking can be performed by a Turing machine.

P3: Turing machines are multiply realizable. In particular, they can be realized by computers, robots, etc.

It is possible to build a computer, robot, etc. that can think. That is, AI is possible.

39

Turing Machines

40

The Language Hierarchy

*aRegular Languages

Context-Free Languagesnnba Rww

nnn cba ww?

**ba

?

41

*aRegular Languages

Context-Free Languagesnnba Rww

nnn cba ww

**ba

Languages accepted byTuring Machines

42

A Turing Machine

............Tape

Read-Write headControl Unit

43

The Tape

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

44

............

Read-Write head

The head at each time step:

1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right

45

............

Example:Time 0

............Time 1

1. Reads2. Writes

a a cb

a b k c

ak

3. Moves Left

46

............Time 1

a b k c

............Time 2

a k cf

1. Reads2. Writes

bf

3. Moves Right

47

The Input String

............

Blank symbol

head

a b ca

Head starts at the leftmost positionof the input string

Input string

48

............

Blank symbol

head

a b ca

Input string

Remark: the input string is never empty

49

States & Transitions

1q 2qLba ,

Read Write Move Left

1q 2qRba ,

Move Right

50

Example:

1q 2qRba ,

............ a b ca

Time 1

1qcurrent state

51

............ a b caTime 1

1q 2qRba ,

............ a b cbTime 2

1q

2q

52

............ a b caTime 1

1q 2qLba ,

............ a b cbTime 2

1q

2q

Example:

53

............ a b caTime 1

1q 2qRg,

............ ga b cbTime 2

1q

2q

Example:

54

Determinism

1q

2qRba ,

Allowed Not Allowed

3qLdb ,

1q

2qRba ,

3qLda ,

No lambda transitions allowed

Turing Machines are deterministic

55

Partial Transition Function

1q

2qRba ,

3qLdb ,

............ a b ca

1q

Example:

No transitionfor input symbol c

Allowed:

56

Halting

The machine halts if there areno possible transitions to follow

57

Example:

............ a b ca

1q

1q

2qRba ,

3qLdb ,

No possible transition

HALT!!!

58

Final States

1q 2q Allowed

1q 2q Not Allowed

• Final states have no outgoing transitions

• In a final state the machine halts

59

Acceptance

Accept Input If machine halts in a final state

Reject Input

If machine halts in a non-final state or If machine enters an infinite loop

60

Turing Machine Example

A Turing machine that accepts the language:*aa

0q

Raa ,

L,1q

61

aaTime 0

0q

a

0q

Raa ,

L,1q

62

aaTime 1

0q

a

0q

Raa ,

L,1q

63

aaTime 2

0q

a

0q

Raa ,

L,1q

64

aaTime 3

0q

a

0q

Raa ,

L,1q

65

aaTime 4

1q

a

0q

Raa ,

L,1q

Halt & Accept

66

Rejection Example

0q

Raa ,

L,1q

baTime 0

0q

a

67

0q

Raa ,

L,1q

baTime 1

0q

a

No possible TransitionHalt & Reject

68

Infinite Loop Example

0q

Raa ,

L,1q

Lbb ,

A Turing machine for language *)(* babaa

69

baTime 0

0q

a

0q

Raa ,

L,1q

Lbb ,

70

baTime 1

0q

a

0q

Raa ,

L,1q

Lbb ,

71

baTime 2

0q

a

0q

Raa ,

L,1q

Lbb ,

72

baTime 2

0q

a

baTime 3

0q

a

baTime 4

0q

a

baTime 50q

a

Infinite loop

73

Because of the infinite loop:

•The final state cannot be reached

•The machine never halts

•The input is not accepted

74

Another Turing Machine Example

Turing machine for the language }{ nnba

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

75

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

ba

0q

a bTime 0

76

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

bx

1q

a b Time 1

77

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

bx

1q

a b Time 2

78

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

2q

a b Time 3

79

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

2q

a b Time 4

80

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

0q

a b Time 5

81

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

1q

x b Time 6

82

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

1q

x b Time 7

83

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx x y

2q

Time 8

84

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx x y

2q

Time 9

85

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

0q

x y Time 10

86

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

3q

x y Time 11

87

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

3q

x y Time 12

88

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

4q

x y

Halt & Accept

Time 13

89

If we modify the machine for the language }{ nnba

we can easily construct a machine for the language }{ nnn cba

Observation:

90

Formal Definitionsfor

Turing Machines

91

Transition Function

1q 2qRba ,

),,(),( 21 Rbqaq

92

1q 2qLdc ,

),,(),( 21 Ldqcq

Transition Function

93

Turing Machine:

),,,,,,( 0 FqQM

States

Inputalphabet

Tapealphabet

Transitionfunction

Initialstate

blank

Finalstates

94

Configuration

ba

1q

a

Instantaneous description:

c

baqca 1

95

yx

2q

a bTime 4

yx

0q

a bTime 5

A Move: aybqxxaybq 02

96

yx

2q

a bTime 4

yx

0q

a bTime 5

bqxxyybqxxaybqxxaybq 1102

yx

1q

x bTime 6

yx

1q

x bTime 7

97

bqxxyybqxxaybqxxaybq 1102

bqxxyxaybq 12Equivalent notation:

98

Initial configuration: wq0

ba

0q

a b

wInput string

99

The Accepted Language

For any Turing Machine M

}:{)( 210 xqxwqwML f

Initial state Final state

100

Standard Turing Machine

• Deterministic

• Infinite tape in both directions

•Tape is the input/output file

The machine we described is the standard:

101

Computing Functionswith

Turing Machines

102

A function )(wf

Domain: Result Region:

has:

D

Dw

S

Swf )()(wf

103

A function may have many parameters:

yxyxf ),(

Example: Addition function

104

Integer Domain

Unary:

Binary:

Decimal:

11111

101

5

We prefer unary representation:

easier to manipulate with Turing machines

105

Definition:A function is computable ifthere is a Turing Machine such that:

fM

Initial configuration Final configuration

Dw Domain

0q

w

fq

)(wf

final stateinitial state

For all

106

)(0 wfqwq f

Initial Configuration

FinalConfiguration

A function is computable ifthere is a Turing Machine such that:

fM

In other words:

Dw DomainFor all

107

Example

The function yxyxf ),( is computable

Turing Machine:

Input string: yx0 unary

Output string: 0xy unary

yx, are integers

108

0

0q

1 1 1 1

x y

1 Start

initial state

The 0 is the delimiter that separates the two numbers

109

0

0q

1 1 1 1

x y

1

0

fq

1 1

yx

11

Start

Finish

final state

initial state

110

0

fq

1 1

yx

11Finish

final state

The 0 helps when we usethe result for other operations

111

0q

Turing machine for function

1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

yxyxf ),(

112

Execution Example:

11x

11y 0

0q

1 1 1 1

Time 0x y

Final Result

0

4q

1 1 1 1yx

(2)

(2)

113

0

0q

1 1Time 0

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

1 1

114

0q

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

01 11 1Time 1

115

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

0

0q

1 1 1 1Time 2

116

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

1q

1 11 11Time 3

117

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

1q

1 1 1 11Time 4

118

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

1q

1 11 11Time 5

119

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

2q

1 1 1 11Time 6

120

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

3q

1 11 01Time 7

121

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

3q

1 1 1 01Time 8

122

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

3q

1 11 01Time 9

123

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

3q

1 1 1 01Time 10

124

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

3q

1 11 01Time 11

125

0q 1q 2q 3qL, L,01

L,11

R,

R,10

R,11

4q

R,11

4q

1 1 1 01

HALT & accept

Time 12

126

Another Example

The function xxf 2)( is computable

Turing Machine:

Input string: x unary

Output string: xx unary

x is integer

127

0q

1 1

x

1

1

fq

1 1

x2

11

Start

Finish

final state

initial state

128

Turing Machine Pseudocode for xxf 2)(

• Replace every 1 with $

• Repeat:• Find rightmost $, replace it with 1

• Go to right end, insert 1

Until no more $ remain

129

0q 1q 2q

3q

R,1$

L,1

L,

R$,1 L,11 R,11

R,

Turing Machine for xxf 2)(

130

0q 1q 2q

3q

R,1$

L,1

L,

R$,1 L,11 R,11

R,

Example

0q

1 1

3q

1 11 1

Start Finish

131

Another Example

The function ),( yxf

is computable 0

1 yx

yx

if

if

132

Turing Machine for

Input: yx0

Output: 1 0or

),( yxf0

1 yx

yx

if

if

133

Turing Machine Pseudocode:

Match a 1 from with a 1 from x y

• Repeat

Until all of or is matchedx y

• If a 1 from is not matched erase tape, write 1 else erase tape, write 0

x)( yx

)( yx

134

Combining Turing Machines

135

Block Diagram

TuringMachineinput output

136

Example:

),( yxf0

yx yx

yx

if

if

Comparer

Adder

Eraser

yx,

yx,

yx

yx

yx

0