computability, turing machines and lambda calculus

Programming Languages and Paradigms

Functional Programming Background(computability, turing machines, lambda calculus)

Edward (Ned) Blurock

Functional Programming Functional Programming

COMPUTABILITY

Will an algorithm be able to compute the task?Is the task I want impossible?Is it possible to formulate an impossible task?What is the consequences for other algorithms

Functional Programming Computability

Computability

Computability vs Complexity

• Computability refers to whether of not in principle it is possible to evaluate f(n) by following a set of instructions.

• We are not for the moment worried if this computation requires 1,000,000 or more consecutive steps.

• The latter refers to complexity which we will return to later.

Functional Programming

Computability

EnsheidungsproblemDecision problem

Hilbert 1928

The Entscheidungsproblem asks for an algorithm

that takes as input a statement of a first-order logic and

answers "Yes" or "No" according to whether the statement is universally valid

(valid in every structure satisfying the axioms).

An algorithm to ask whether something is trueIs it computable

Computability

EnsheidungsproblemDecision problem

Hilbert 1928

• Before the question could be answered, the notion of "algorithm" had to be formally defined. – This was done by Alonzo Church in 1936 with the

concept of "effective calculability" based on his λ calculus

– and by Alan Turing in the same year with his concept of Turing machines.

– It was recognized immediately by Turing that these are equivalent models of computation.

Computability

Alan Turing (1912 – 1954)

Alonzo Church (1903-1995)

Turing Machine Lambda calculusTwo mathematical ways to ask questions about

“computability”Functional Programming

Computability

Equivalent Computers

z z zz z z z

), X, L

2: look for (

#, 1, -

), #, R

(, #, L

(, X, R

#, 0, -

Finite State Machine

Turing Machine

term = variable | term term | (term) | variable . term

y. M v. (M [y v]) where v does not occur in M.

(x. M)N M [ x N ]

Lambda Calculus

Computability

Alan Turing, 1936

• "On computable numbers, with an application to the Entscheidungsproblem [decision problem]”

• addressed a previously unsolved mathematical problem posed by the German mathematician David Hilbert in 1928:

Computability

Turing, 1936

Is there, in principal, any definite mechanical method or process

by which all mathematical questions

could be decided?

The Decision ProblemFunctional Programming

Computability

Turing, 1936

• To answer the question (in the negative), • Turing proposed a simple abstract computing

machine, • modeled after a mathematician with a pencil,

an eraser, and a stack of pieces of paper• He asserted that any function is a computable

function if it is computable by one of these abstract machines.

Computability

Turing, 1936

• He then investigated whether there were some mathematical problems which could not be solved by any such abstract machine.

• Such abstract computing machines are now called "Turing machines".

• One particular Turing machine, called a "Universal Turing Machine", served as the model for designing actual programmable computers.

Computability

Why a Turing Machine

Because Turing machines are very simple compared with computers in practical use,

conceptually easier to prove impossibility results

for Turing machines. These impossibility results

apply as well to all known computersFunctional Programming

Computability

Turing, 1936

• Startling result of Turing's 1936 paper• assertion that there are well-defined

problems that cannot be solved by any computational procedure.

Computability

What is Computable?Computation is usually modelled as a mapping from inputs to outputs, carried out by a formal “machine,” or program, which processes its input in a sequence of steps.

An “effectively computable” function is one that can be computed in a finite amount of time using finite resources.

………………………

outputEffectively

computable function

Computability

Unsolvable

• problems are formulated as functions– unsolvable functions: noncomputable;

• Problems formulated as predicates– Undecidable.

Using Turing's concept of the abstract machine, we would say that a function is noncomputable

if there exists no Turing machine that could compute it.

Turing Machine

TURING MACHINEComputability

............Tape

Read-Write headControl Unit

Functional Programming Turing Machine

Turing Machine

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

The tape

Turing Machine

............

Read-Write head

The head at each time step:

1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right

Turing Machine

............

Example: Time 0

............Time 1

a a cb

a b k c

1. Reads a2. Writes k 3. Moves Left

Turing Machine

............Time 1

a b k c

............Time 2

a k cf

1. Reads b2. Writes f3. Moves Right

Turing Machine

Add 1 to a sequence of ones

If 1, then move right If 1, then move left

If 0, replace with 1

Start: 0 0 0 1 1 1 1 0 0 0 4 ones in a row

End: 0 0 0 1 1 1 1 1 0 0 5 ones in a row

Turing Machine

So what?

You can add a 1 to a sequence of ones

From a set of a few (a manageable number) of primitive operations

If a number is represented by a sequence of onesthen you have the x+1 operator, i.e addition

This is a/(the most) basic mathematical operator

Powerful enough to represent complex mathematics

You can derive other (all) mathematic operations and objects

Turing Machine

Powerful “language”From a set of a few (a manageable number) of

primitive operations

Easier enough To formulate Proofs about algorithms

Powerful enough To represent complex mathematics

All the ingredients are there Only have to deal with a few basic operations

Not worried about complexity…. Worried about, for example, “can” we compute?Functional Programming

Turing Machine (pt 2)

TURING MACHINE (PT 2)Computability

The Input String

............

Blank symbol

a b ca

Head starts at the leftmost positionof the input string

Input string

States & Transitions

1q 2qLba ,

Read Write Move Left

1q 2qRba ,

Move Right

Turing machine for the language }{ nnba

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

a bTime 0

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

a b Time 1

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

a b Time 2 b

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

a b Time 3

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

a b Time 4

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

a b Time 5

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

x b Time 6

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

x b Time 7

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

yx x y

Time 8

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

yx x y

Time 9

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

x y Time 10

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

x y Time 11

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

x y Time 12

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Laa ,Lyy ,

Ryy ,4q

Halt & Accept

Time 13

LAMBDA CALCULUSComputability

Functional Programming Lambda Calculus

Lambda Calculus

Alonzo Church (1903-1995)

Another formulationofcomputability

Lambda Calculus

Equivalent Computers

z z zz z z z ...

Turing Machine

y. M v. (M [y v]) where v does not occur in M.

(x. M)N M [ x N ]

Lambda Calculus

What is Calculus?• In High School:

d/dx xn = nxn-1 [Power Rule]d/dx (f + g) = d/dx f + d/dx g [Sum Rule]

Calculus is a branch of mathematics that deals with limits and the differentiation and integration of functions of one or more variables...

Lambda Calculus

Real Definition• A calculus is just a bunch of rules for

manipulating symbols.• People can give meaning to those symbols,

but that’s not part of the calculus.• Differential calculus is a bunch of rules for

manipulating symbols. There is an interpretation of those symbols corresponds with physics, slopes, etc.

Lambda Calculus

-calculusAlonzo Church, 1940

Lambda Calculus

-calculus

variable . term

F(variable) = term

Lambda Calculus

• Once we have precise and formal rules for manipulating symbols, we can use it to reason with.

• Since we can interpret the symbols as representing computations, we can use it to reason about programs.

Lambda Calculus

Universal Computer• Lambda Calculus can simulate a Turing Machine

– Everything a Turing Machine can compute, Lambda Calculus can compute also

• Turing Machine can simulate Lambda Calculus– Everything Lambda Calculus can compute, a Turing

Machine can compute also• Church-Turing Thesis: this is true for any other

mechanical computer also

Lambda Calculus

Normal Steps• Turing machine:

– Read one square on tape, follow one FSM transition rule, write one square on tape, move tape head one square

• Lambda calculus:– One beta reduction

• Your PC:– Execute one instruction (?)

• What one instruction does varies

Lambda Calculus

-Reduction (the source of all computation)

(x. M)N

This is the function definitionM is an expression with the variable x

N is used at the value of x

N substitutes x in the expression M

Lambda Calculus

(x. x)a = a

-Reduction (the source of all computation)

Identity operationFunction gives the argument as output

(x. xx)a = aa(y.(x. xy)a)b =(y.ay)b=ab (y.(x. xy) z.zy)b =(y.(z.zy)y)b = (y. yy)b =bb

Lambda Calculus

-calculusUseful and expression enough that a programming language was

modeled after it

LISP was developed from -calculus

not the other way round.)

This was the “first” general language ofArtificial intelligence

AlsoThe “first” functional programming language

Lambda Calculus

Expression Reductions

Reductions are used in the lambda calculus to reduce a statement to its simplest possible form.

This is, essentially, what a computation in lambda calculus does.

If you were to give a lambda calculus expression to "lambda calculus" calculator, it would use a combination of alpha, beta, and eta reductions to find the simplest reduction of your expression.

https://classes.soe.ucsc.edu/cmps112/Spring03/readings/lambdacalculus/reductions.html

Lambda Calculus

Expression Reduction

( x. x x) ( y. y)𝛌 𝛌--> x x [ y. 𝛌

y / x]== ( y. y) ( y. y)𝛌 𝛌

--> y [ y. y / 𝛌y]

== y. y𝛌

How the original expression isevaluated

Is determined the

operational semanticsof lambda calculus (there are many ways to evaluate expressions)

Operational Semantics: how the substitutions are made:Key concepts:

specification of free variablesspecification of bound variables

Β-reduction a formalization of simple substitution

Name Capture

Functional Programming Lambda Calculus

Care must be taken to avoid “name capture”

of bound variables.

For example, this reduction would be “wrong”:

(λx. (λy. x)) y λy. y⇒because the outer y is presumably different

from the inner y bound by the lambda. In cases like these, the bound variable is renamed to obtain the “correct” behavior:

(λx. (λy. x)) y λ⇒ z. y

http://www.cs.yale.edu/homes/hudak/CS201S08/lambda.pdf

Α-reduction is a formalization of renaming of variables (without this, β-reduction can give the wrong answer)

Examples

EXAMPLES OF LAMBDA CALCULUSComputability

Examples

syntax

• the identity function:– 𝛌x.x

• 2 notational conventions:• applications associate to the left (like in ML): • “y z x” is “(y z) x”• the body of a lambda extends as far as possible to the

right:• “𝛌x.x z.x z x𝛌 ” is “ x.(x z.(x z x))𝛌 𝛌 ”

Examples

terminology

• 𝛌x.t

• 𝛌x.x y

the scope of x is the term t(this is the part of the expression

where x can be substituted)

x is bound in the term x.x y𝛌(x is one of the variables that can be substituted)

y is free in the term x.x y𝛌(y is free to be substituted

into the expression .. In this case x)

Examples

Example

(λx. x x) ( y. y)𝛌--> x x [ y. y 𝛌 /

x]== ( y. 𝛌 y) ( y. y𝛌 )--> y [ y. y 𝛌 / y]== y. y𝛌

Substitute ( y. y) into x𝛌Substituted

Substitute ( y. y𝛌 ) into y

Substituted

Examples

Another example

( x. 𝛌 x x) ( x. x x𝛌 )--> x x [ x. x x𝛌 /x]== ( x. x x) ( x. x x)𝛌 𝛌

In other words, it is simple to write non terminating computations

in the lambda calculuswhat else can we do?

Substitute x. x x 𝛌 into every x

The result is the same as the beginning

computability, turing machines and lambda calculus

Education

6. introduction to the lambda calculus. © o. nierstrasz ps...

1 turing machines and effective computability

lambda calculus and lisp pz03j. lambda calculus the lambda...

transparency no. p3c3-1 formal language and automata theory...

continuous models of computation: computability, complexity...

an introduction to feedback turing computability

turing computability and information...

cs 4820: limits of computability · famous and inﬂuential...

ece 695 numerical simulations lecture 2: computability and...

introduction to theory of computability€¦ · the...

turing versus gödel on computability and the mind ·...

formalizing computability theory via partial recursive...

logic in action - staff.fnwi.uva.nl · a brief history of...

ian. king. lambda calculus == turing machine? imperative...

computability and complexity · lambda calculus (...

turing oracle machines, online computing, and three...

ordered structures and computability -...

cs5371 theory of computation lecture 10: computability...

generic computability, turing degrees, and asymptotic...

computability theory turing machines professor msc. ivan a....