computability, turing machines and lambda calculus

Programming Languages and Paradigms

Functional Programming Background(computability, turing machines, lambda calculus)

Edward (Ned) Blurock

Functional Programming Functional Programming

COMPUTABILITY

Will an algorithm be able to compute the task?Is the task I want impossible?Is it possible to formulate an impossible task?What is the consequences for other algorithms

Functional Programming Computability

Computability

Computability vs Complexity

• Computability refers to whether of not in principle it is possible to evaluate f(n) by following a set of instructions.

• We are not for the moment worried if this computation requires 1,000,000 or more consecutive steps.

• The latter refers to complexity which we will return to later.

Functional Programming

Computability

EnsheidungsproblemDecision problem

Hilbert 1928

The Entscheidungsproblem asks for an algorithm

that takes as input a statement of a first-order logic and

answers "Yes" or "No" according to whether the statement is universally valid

(valid in every structure satisfying the axioms).

An algorithm to ask whether something is trueIs it computable


Computability

EnsheidungsproblemDecision problem

Hilbert 1928

• Before the question could be answered, the notion of "algorithm" had to be formally defined. – This was done by Alonzo Church in 1936 with the

concept of "effective calculability" based on his λ calculus

– and by Alan Turing in the same year with his concept of Turing machines.

– It was recognized immediately by Turing that these are equivalent models of computation.


Computability

Alan Turing (1912 – 1954)

Alonzo Church (1903-1995)

Turing Machine Lambda calculusTwo mathematical ways to ask questions about

“computability”Functional Programming

Computability

Equivalent Computers

z z zz z z z

1

Start

HALT

), X, L

2: look for (

#, 1, -

), #, R

(, #, L

(, X, R

#, 0, -

Finite State Machine

...

Turing Machine

term = variable | term term | (term) | variable . term

y. M v. (M [y v]) where v does not occur in M.

(x. M)N M [ x N ]

Lambda Calculus


Computability

Alan Turing, 1936

• "On computable numbers, with an application to the Entscheidungsproblem [decision problem]”

• addressed a previously unsolved mathematical problem posed by the German mathematician David Hilbert in 1928:


Computability

Turing, 1936

Is there, in principal, any definite mechanical method or process

by which all mathematical questions

could be decided?

The Decision ProblemFunctional Programming

Computability

Turing, 1936

• To answer the question (in the negative), • Turing proposed a simple abstract computing

machine, • modeled after a mathematician with a pencil,

an eraser, and a stack of pieces of paper• He asserted that any function is a computable

function if it is computable by one of these abstract machines.


Computability

Turing, 1936

• He then investigated whether there were some mathematical problems which could not be solved by any such abstract machine.

• Such abstract computing machines are now called "Turing machines".

• One particular Turing machine, called a "Universal Turing Machine", served as the model for designing actual programmable computers.


Computability

Why a Turing Machine

Because Turing machines are very simple compared with computers in practical use,

conceptually easier to prove impossibility results

for Turing machines. These impossibility results

apply as well to all known computersFunctional Programming

Computability

Turing, 1936

• Startling result of Turing's 1936 paper• assertion that there are well-defined

problems that cannot be solved by any computational procedure.


Computability

What is Computable?Computation is usually modelled as a mapping from inputs to outputs, carried out by a formal “machine,” or program, which processes its input in a sequence of steps.

An “effectively computable” function is one that can be computed in a finite amount of time using finite resources.

………………………

input

yes

no

outputEffectively

computable function


Computability

Unsolvable

• problems are formulated as functions– unsolvable functions: noncomputable;

• Problems formulated as predicates– Undecidable.

Using Turing's concept of the abstract machine, we would say that a function is noncomputable

if there exists no Turing machine that could compute it.


Turing Machine

TURING MACHINEComputability


............Tape

Read-Write headControl Unit

Functional Programming Turing Machine

Turing Machine

............

Read-Write head

No boundaries -- infinite length

The head moves Left or Right

The tape

OR


Turing Machine

............

Read-Write head

The head at each time step:

1. Reads a symbol 2. Writes a symbol 3. Moves Left or Right


Turing Machine

............

Example: Time 0

............Time 1

a a cb

a b k c

1. Reads a2. Writes k 3. Moves Left

a


Turing Machine

............Time 1

a b k c

............Time 2

a k cf

1. Reads b2. Writes f3. Moves Right

b


Turing Machine

Add 1 to a sequence of ones

If 1, then move right If 1, then move left

Loop

If 0, replace with 1

Start: 0 0 0 1 1 1 1 0 0 0 4 ones in a row

End: 0 0 0 1 1 1 1 1 0 0 5 ones in a row


Turing Machine

So what?

You can add a 1 to a sequence of ones

From a set of a few (a manageable number) of primitive operations

If a number is represented by a sequence of onesthen you have the x+1 operator, i.e addition

This is a/(the most) basic mathematical operator

Powerful enough to represent complex mathematics

You can derive other (all) mathematic operations and objects


Turing Machine

Powerful “language”From a set of a few (a manageable number) of

primitive operations

Easier enough To formulate Proofs about algorithms

Powerful enough To represent complex mathematics

All the ingredients are there Only have to deal with a few basic operations

Not worried about complexity…. Worried about, for example, “can” we compute?Functional Programming

Turing Machine (pt 2)

TURING MACHINE (PT 2)Computability



The Input String

............

Blank symbol

head

a b ca

Head starts at the leftmost positionof the input string

Input string



States & Transitions

1q 2qLba ,

Read Write Move Left

1q 2qRba ,

Move Right



Turing machine for the language }{ nnba

0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

ba

0q

a bTime 0



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

bx

1q

a b Time 1



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

x

2q

a b Time 2 b



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

2q

a b Time 3



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

2q

a b Time 4



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

0q

a b Time 5



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

1q

x b Time 6



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

1q

x b Time 7



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx x y

2q

Time 8



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx x y

2q

Time 9



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

0q

x y Time 10



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

3q

x y Time 11



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

3q

x y Time 12



0q 1q 2q3q Rxa ,

Raa ,Ryy ,

Lyb ,

Laa ,Lyy ,

Rxx ,

Ryy ,

Ryy ,4q

L,

yx

4q

x y

Halt & Accept

Time 13


LAMBDA CALCULUSComputability

Functional Programming Lambda Calculus

Lambda Calculus

Lambda Calculus

Alonzo Church (1903-1995)

Another formulationofcomputability


Lambda Calculus

Equivalent Computers

z z zz z z z ...

Turing Machine


y. M v. (M [y v]) where v does not occur in M.

(x. M)N M [ x N ]

Lambda Calculus


Lambda Calculus

What is Calculus?• In High School:

d/dx xn = nxn-1 [Power Rule]d/dx (f + g) = d/dx f + d/dx g [Sum Rule]

Calculus is a branch of mathematics that deals with limits and the differentiation and integration of functions of one or more variables...


Lambda Calculus

Real Definition• A calculus is just a bunch of rules for

manipulating symbols.• People can give meaning to those symbols,

but that’s not part of the calculus.• Differential calculus is a bunch of rules for

manipulating symbols. There is an interpretation of those symbols corresponds with physics, slopes, etc.


Lambda Calculus

-calculusAlonzo Church, 1940



Lambda Calculus

-calculus

variable . term

F(variable) = term


Lambda Calculus

Why?

• Once we have precise and formal rules for manipulating symbols, we can use it to reason with.

• Since we can interpret the symbols as representing computations, we can use it to reason about programs.


Lambda Calculus

Universal Computer• Lambda Calculus can simulate a Turing Machine

– Everything a Turing Machine can compute, Lambda Calculus can compute also

• Turing Machine can simulate Lambda Calculus– Everything Lambda Calculus can compute, a Turing

Machine can compute also• Church-Turing Thesis: this is true for any other

mechanical computer also


Lambda Calculus

Normal Steps• Turing machine:

– Read one square on tape, follow one FSM transition rule, write one square on tape, move tape head one square

• Lambda calculus:– One beta reduction

• Your PC:– Execute one instruction (?)

• What one instruction does varies


Lambda Calculus

-Reduction (the source of all computation)

(x. M)N

This is the function definitionM is an expression with the variable x

N is used at the value of x

N substitutes x in the expression M


Lambda Calculus

(x. x)a = a

-Reduction (the source of all computation)

Identity operationFunction gives the argument as output

(x. xx)a = aa(y.(x. xy)a)b =(y.ay)b=ab (y.(x. xy) z.zy)b =(y.(z.zy)y)b = (y. yy)b =bb


Lambda Calculus

-calculusUseful and expression enough that a programming language was

modeled after it

LISP was developed from -calculus

not the other way round.)

This was the “first” general language ofArtificial intelligence

AlsoThe “first” functional programming language


Lambda Calculus

Expression Reductions


Reductions are used in the lambda calculus to reduce a statement to its simplest possible form.

This is, essentially, what a computation in lambda calculus does.

If you were to give a lambda calculus expression to "lambda calculus" calculator, it would use a combination of alpha, beta, and eta reductions to find the simplest reduction of your expression.

https://classes.soe.ucsc.edu/cmps112/Spring03/readings/lambdacalculus/reductions.html

Lambda Calculus

Expression Reduction


( x. x x) ( y. y)𝛌 𝛌--> x x [ y. 𝛌

y / x]== ( y. y) ( y. y)𝛌 𝛌

--> y [ y. y / 𝛌y]

== y. y𝛌

How the original expression isevaluated

Is determined the

operational semanticsof lambda calculus (there are many ways to evaluate expressions)

Operational Semantics: how the substitutions are made:Key concepts:

specification of free variablesspecification of bound variables

Β-reduction a formalization of simple substitution

Name Capture

Functional Programming Lambda Calculus

Care must be taken to avoid “name capture”

of bound variables.

For example, this reduction would be “wrong”:

(λx. (λy. x)) y λy. y⇒because the outer y is presumably different

from the inner y bound by the lambda. In cases like these, the bound variable is renamed to obtain the “correct” behavior:

(λx. (λy. x)) y λ⇒ z. y

http://www.cs.yale.edu/homes/hudak/CS201S08/lambda.pdf

Α-reduction is a formalization of renaming of variables (without this, β-reduction can give the wrong answer)

Examples

EXAMPLES OF LAMBDA CALCULUSComputability


Examples

syntax

• the identity function:– 𝛌x.x

• 2 notational conventions:• applications associate to the left (like in ML): • “y z x” is “(y z) x”• the body of a lambda extends as far as possible to the

right:• “𝛌x.x z.x z x𝛌 ” is “ x.(x z.(x z x))𝛌 𝛌 ”


Examples

terminology

• 𝛌x.t

• 𝛌x.x y

the scope of x is the term t(this is the part of the expression

where x can be substituted)

x is bound in the term x.x y𝛌(x is one of the variables that can be substituted)

y is free in the term x.x y𝛌(y is free to be substituted

into the expression .. In this case x)


Examples

Example

(λx. x x) ( y. y)𝛌--> x x [ y. y 𝛌 /

x]== ( y. 𝛌 y) ( y. y𝛌 )--> y [ y. y 𝛌 / y]== y. y𝛌

Substitute ( y. y) into x𝛌Substituted

Substitute ( y. y𝛌 ) into y

Substituted


Examples

Another example

( x. 𝛌 x x) ( x. x x𝛌 )--> x x [ x. x x𝛌 /x]== ( x. x x) ( x. x x)𝛌 𝛌

In other words, it is simple to write non terminating computations

in the lambda calculuswhat else can we do?

Substitute x. x x 𝛌 into every x

The result is the same as the beginning


computability, turing machines and lambda calculus

Education