Download - Penrose’s Mistake
Penrose’s Mistake
Uri Sharir
Computability Seminar – Prof. Nachum DershowitzTel Aviv University
December 11, 2013
2
Outline
• Brief history of AI• Lucas and Gödel• Penrose and Turing• Summary
3
What is AI?
artificial intelligence n 1 : a branch of computer science dealing with the simulation of intelligent behavior in computers 2 : the capability of a machine to imitate intelligent human behavior
Merriam-Webster
4
Two Kinds of AI “Strong AI”
An appropriately programmed computer is a mind
The human mind is in reality some kind of physical symbol-manipulation system
“Weak AI” Physical symbol systems have the necessary
and sufficient means for intelligent action Allows for the possibility that the mind is not
mechanical, but claims that it can (theoretically, at least) be simulated by a Turing machine
5History: Myths and Antiquity Throughout human history,
people have used technology to emulate themselves
Greece: Hephaestus’ golden robots,
Talos, Pandora Pygmalion’s Galatea 384-322 BC: Aristotle
describes the syllogism 1st century AD: Heron of
Alexandria’s automata China:
King Mu of Zhou and Yan Shi’s “artificial actor”
6
History: Alchemy, Fiction and Hoaxes ~800: Jabir ibn Hayyan’s
Takwin 1206: Al-Jazari’s
programmable automata ~1500: Paracelsus’
homunculus ~1580: Rabbi Judah Loew ben
Bezalel of Prague’s Golem 1769: Wolfgang von
Kempelen’s The Turk 1818: Mary Shelley’s
Frankenstein 1920: Karel Čapek’s Rossum’s
Universal Robots (R.U.R.)
7
History: Formal Reasoning
1275: Ramon Llull’s Ars Magna
Early 17th century: René Descartes: bodies of animals are nothing more than complex machines (but not mind or soul)
1641: Thomas Hobbes in Leviathan: “reason is nothing but reckoning”
1672: Gottfried Leibniz: Stepped Reckoner Characteristica universalis
8
History: Formal Reasoning 1854: George Boole’s Boolean algebra
“Investigate the fundamental laws of those operations of the mind by which reasoning is performed, to give expression to them in the symbolic language of a calculus”
1913: Bertrand Russell and Alfred North Whitehead publish Principia Mathematica
1920s and 1930s: David Hilbert’s program Alonzo Church’s lambda calculus The Turing machine Kurt Gödel’s incompleteness theorems
Within certain limits, any mathematicalreasoning could be mechanized
9
History: The Birth of AI 1943:
Norbert Wiener's Cybernetics Warren Sturgis McCulloch and
Walter Pitts publish “A Logical Calculus of the Ideas Immanent in Nervous Activity”
1950: Alan Turing proposes the Turing
Test Isaac Asimov published his
Three Laws of Robotics 1951: first working AI programs 1952-1962: Arthur Samuel’s
checkers programs
10
History: The Birth of AI 1955: Allen Newell and Herbert A. Simon
created the “Logic Theorist”, eventuallyproving 38 of the first 52 theorems inPrincipia Mathematica (and finding newand more elegant proofs for some) Simon: they had “solved the venerable mind/body
problem, explaining how a system composed of matter can have the properties of mind”
Later John Searle will call this the “Strong AI Hypothesis” 1956: The first Dartmouth College summer AI
conference is organized by John McCarthy, Marvin Minsky,Nathan Rochester (IBM) andClaude Shannon
11
History: The Golden Years (1956-1974) Several fields of work:
Reasoning as search▪ Stanford’s Shakey
Natural language▪ Joseph Weizenbaum’s ELIZA
Micro-worlds▪ Terry Winograd’s SHRDLU
Optimism: 1958: “Within ten years a digital computer will be the world's chess champion”;
“within ten years a digital computer will discover and prove an important new mathematical theorem” (H.A. Simon and Allen Newell)
1965: “Machines will be capable, within twenty years, of doing any work a man can do” (H.A. Simon)
1967: “Within a generation ... the problem of creating ‘artificial intelligence’ will substantially be solved” (Marvin Minsky)
1970: “In from three to eight years we will have a machine with the general intelligence of an average human being” (Marvin Minsky)
Funding: 1963: MIT received a $2.2M grant from the newly created Advanced Research
Projects Agency (later DARPA)
A*<P,O,I,G>
12
History: The 1st AI Winter (1974-1980) Expectations set too high… Limited computing power (AI requires computer
power in the same way that aircraft require horsepower)
Intractability and combinatorial explosion (search or scaling toy micro-worlds)
Common sense and reasoning (general knowledge too vast)
Moravec’s paradox: proving theorems and solving geometry problems is comparatively easy for computers, but a supposedly simple task like recognizing a face or crossing a room without bumping into anything is extremely difficult
The end of funding…
13
Critiques from Across the Campus 1961: John Lucas in “Minds, Machines and
Gödel”: Turing’s and (more so) Gödel’s theorems prove
that “Mechanism is false”: minds cannot be explained as machines
1965, 1972: Hubert Dreyfus in Alchemy and AI and What Computers Can't Do: Challenges the four philosophical assumptions
AI researchers treated as axioms:▪ Biological (brains are composed of on/off switches)▪ Psychological (minds operate according to formal rules)▪ Epistemological (all knowledge can be formalized)▪ Ontological (the world consists of independent facts)
1980: John Searle’s Chinese room argument
14
Outline
• Brief history of AI• Lucas and Gödel• Penrose and Turing• Summary
15
John Randolph Lucas (1929-) Follows Gödel’s proof:
Assume a consistent formal system F, which is strong enough to produce simple arithmetic (contains ℕ, + and ×)
Consider the Epimenides-inspired formulag = “This formula is unprovable in F”
If g is provable in F, we get a contradiction:▪ g is false in F (by its definition)▪ g is true in F (since F is sound)
Then g must be unprovable in F, and thus g is true So all interesting consistent formal systems are incomplete:
“contain unprovable, though perfectly meaningful, formulae … we, standing outside the system, can see to be true”
In particular, this “must apply to cybernetical machines” “It follows that no machine can be a complete or adequate
model of the mind, that minds are essentially different from machines”
16
Lucas’ Argument in Other Words Assume that for any human H there exists at least one
(deterministic) formal logical system F(H) which reliably predicts H’s actions in all circumstances
For any logical system F, a sufficiently skilled mathematical logician (equipped with a sufficiently powerful computer if necessary) can construct some statements G(F) which are true but unprovable in F (Gödel)
If a human M is such a logician, then if M is given F(M), he or she can construct G(F(M)) and determine that they are true (which F(M) can’t)
Hence F(M) does not reliably predict M’s actions in all circumstances Lucas goes further to state the anti-deterministic view that this
implies that M has free will And it is implausible that the qualitative difference between
mathematical logicians and the rest of the population is such that the former have free will and the latter do not
17
More from Lucas “We are trying to produce a model of the mind which is
mechanical – which is essentially ‘dead’ – but the mind, being in fact ‘alive’, can always go one better”
“Thanks to Gödel's theorem, the mind always has the last word” Adding to the system additional axioms or rules to prove its
Gödel formula wouldn’t help, because the “strengthened” system would have a Gödel formula of its own
It’s like a game between the mechanist (M) and Lucas (L): M goes first: he produces a definite mechanical model of
the mind L points to something that it cannot do, but the mind can M is free to modify his example, but each time he does so,
L is entitled to look for defects in the revised model If M can devise a model that L cannot find fault with,
his thesis is established, otherwise it is not proven And since M necessarily cannot, his thesis is refuted X
18
Objections to Lucas1. Consistency Gödel’s theorem (and Lucas’ argument) applies only to
consistent systems If F is inconsistent, we can prove
anything, including g and ¬g(inconsistent systems are not incomplete)
Two possible responses to Lucas:1. We can’t establish our own consistency
▪ This chimes with Gödel’s second incompleteness theorem: one cannot prove the consistency of a formal system from within the system itself
▪ We might be able to prove our consistency drawing considerationsfrom outside the system, but who’s to say they are consistent?
2. We are in fact inconsistent Turing machines▪ Lucas responds that we are fallible but not systematically
inconsistent▪ Otherwise we would believe anything, but we tend to
discard inconsistencies whenever we notice them
19
Objections to Lucas2. Complexity Paul Benacerraf in God, the Devil, and Gödel (1967):
It is not easy to construct a Gödel formula In order to do so for any given formal system one must have a
solid understanding of the algorithm at work in the system The formal system the human mind might implement is likely to
be extremely complex (Hofstadter shares this view) So Lucas’ argument is a disjunction:
▪ Either no formal system encodes all human arithmetical capacity▪ Or any system which does has no axiomatic specification
which human beings can comprehend Lucas’ response:
He could be helped in producing the Gödel formulafor any given formal system
It might be difficult, but we could at leastin principle determine what the Gödelformula is for any given system
20
Objections to Lucas3. The Whiteley Sentence C.H. Whiteley (1962) claimed humans have similar
limitations to the one that Lucas’ argument attributes to machines (so perhaps we are not different from machines after all)
The Whiteley sentence:w = “Lucas cannot consistently assert this formula”
If w is true, then if Lucas asserts it he is inconsistent So one of two options:
Lucas is inconsistent (invalidating his own argument) Lucas cannot utter w on pain of inconsistency, in which case w is true
and Lucas is incomplete Counter-response:
Lucas can recognize that w is true, as there’s a point of view from which he can understand how the sentence tricks him
From this point of view Lucas can appreciate that he can’t assert the sentence, and consequently he can recognize its truth
21
Objections to Lucas4. Idealizations Lucas’ argument sets up a hypothetical scenario
involving an idealized mind and an idealized machine Some believe that once these idealizations are
rejected, Lucas’ argument falters: The output of any human mind is finite, and can thus be
programmed (Lucas: but this is not modelling in principle; actuality instead of potentiality)
Add a Gödelizing operator, generating an infinite numberof Gödel formulae (Lucas: this new system will have atransfinite Gödel formula)
Most people do not understand Gödel’s theorem, so all Lucas has shown is that a handful of competent mathematical logicians are not machines (Lucas: it only means that the argument is more obvious regarding people who do understand Gödel’s theorem)
22
Objections to Lucas5. Lucas Arithmetic David Lewis in “Lucas Against Mechanism” (1969):
“Peano arithmetic” is the arithmetic that machines can produce and “Lucas arithmetic” is the arithmetic that humans can produce
Lucas arithmetic will contain Gödel formulae while Peano arithmetic will not, so humans are not machines
But Lucas has not shown that he (or anyone else) can produce Lucas arithmetic in its entirety, which he must do if his argument is to succeed, so his argument is incomplete
Lucas’ response: All is required to disprove mechanism is to produce
a single theorem that a human can see is true but amachine cannot, which Gödel’s theorem does
23
Lucas’ Final Thought Lucas doesn't rule out the idea of intelligent computers:
“When we increase the complexity of our machines there may, perhaps, be surprises in store for us”
“Turing draws a parallel with a fission pile. Below a certain ‘critical’ size, nothing much happens: but above the critical size, the sparks begin to fly. So too, perhaps, with brains and machines…”
“Although it sounds implausible, it might turn out that above a certain level of complexity, a machine ceased to be predictable … and started doing things on its own account”
“It would begin to have a mind of its own when it was no longer entirely predictable … but was capable of doing things which we recognized as intelligent”
“But then it would cease to be a machine” “There would then be two ways of bringing new minds into the world:
▪ “The traditional way, by begetting children born of women…▪ “… and a new way by constructing very, very complicated systems of, say,
valves and relays”
24
Outline
• Brief history of AI• Lucas and Gödel• Penrose and Turing• Summary
25Sir Roger Penrose (1931-) English mathematical physicist, recreational
mathematician and philosopher Renowned for his work in mathematical
physics, in particular general relativity and cosmology
Some notable contributions: 1955: Moore-Penrose pseudoinverse 1950s: Penrose triangle 1959: Penrose stairs 1965: Penrose-Hawking
singularity theorems 1974: Penrose tiling
26Physics and Consciousness Penrose has written several books on the
connection between fundamental physics and (human) consciousness: 1989: The Emperor’s New Mind 1994: Shadows of the Mind 1997: The Large, the Small and the Human Mind
The known laws of physics are inadequate to explain the phenomenon of consciousness
Consciousness arises from quantum mechanics, and its description requires a new kind of physics
Bases his arguments on: The Emperor’s New Mind: Turing’s halting theorem
(limitations of computation) Shadows of the Mind: Gödel’s theorem
(limitations of formal systems)
27
The Emperor’s New Mind (1989)Turing’s Theorem
The Halting Problem…That’s the Question!
YesWill this
program ever halt?
No
28
Penrose’s Argument in a Nutshell1. Consider all current sound human knowledge about non-
termination2. Suppose we could reduce this knowledge to a (finite)
computer program3. Then we could create a self-referential version of this
program4. A contradiction to its correct performance can be derived
Penrose’s resolution: the second step is invalid;no program can incorporate what (finitely many)humans know
So although humans can emulate Turing machines, machines cannot simulate all humans
The human mind possesses super-Turingabilities, using undiscovered physical processes
29
Formally
Let be the set of all Turing machines mapping partially to
The domain of Mi is Let be the set of all r.e. sets
( is computable if ) Let
30
Formally Penrose’s claimed theorem: Proof:
Assume , then there is a fixed such that = Dk
So in particular(1) And by definition of ,(2) The conjunction of (1) and (2) is of the form
(¬A B)(B A), which implies A, i.e. Note that we (human mathematicians) have
convincingly demonstrated that, which means that Combining this with (1), we get
31
Where’s the Beef?
The logical error is the step“Mk(k) = was convincingly demonstrated”
What actually was demonstrated is But from assuming that holds, it cannot
be concluded that human mathematicians can find a proof for
32Turing’s Halting Problem in Brief Suppose a programming language which:
Supports programs as data Has some sort of conditional if ... then ... else ... Includes at least one non-terminating program loop
Consider the decision problem of determining whether a program X does not halt on itself:
? Suppose A is a program that claimed to return T for
(exactly) all such X Then A would fail to answer correctly regarding the
programC(Y ) := if A(Y ) then T else loop()
BecauseC(C) = T ⇔ A(C) = T ⇔ C(C) =
33
Restating Penrose Assume the following programs:
R▪ Can identify programs that diverge when fed themselves as input▪ But uses an undisclosed implementation (quantum mechanics?)
A▪ Incorporates all current, sound scientific knowledge on non-
termination (equivalently, incorporates all R’s knowledge)▪ Known implementation
G▪ “God”, an oracle that always has the correct answer
C▪ Applies Cantorian diagonalization to A to produce paradoxical behavior
K▪ Undisclosed program used by R (in its brain or logic circuitry) to
inspect programs like A
34
Restating Penrose
Let denote the set of one-input partial predicates in any standard model of computation:
For any , one can construct CA asCA(Y ) := if A(Y ) then T else loop()
is the total predicate:G(X) := [X(X) = ]
35
Restating Penrose Consider some partial predicate with the following
rule (not necessarily its only rule): return T if
program X is of the formX(Y ) := if Z(Y ) then T else loop(), where Z …andK[Z(X) ≠ T]
R is a special case of what Penrose believes humans are capable of
It answers T regarding the divergence of X(X) if it believes (via K) that the test Z(X) does not succeed
Specifically, R(CA) = T if K[A(CA) ≠ T] = T We don’t know what R will do if K returns F or does
not respond within some reasonable time frame
36
Restating Penrose The following facts are indisputable:
A(CA) = T ⇔ CA(CA) = T A(CA) ≠ T ⇔ CA (CA) = ⊥A(CA) = T ⇒ G(CA) = F A(CA) ≠ T ⇒ G(CA) = T
Then no A can always be right when asked CA
A(CA) ≠ G(CA) Since we assumed A incorporates R’s knowledge,
A(CA) = R(CA) ⇒ R(CA) ≠ G(CA) If A simulates R on CA, then R does not respond
properly So there are two options:
A(CA) ≠ R(CA) A does not simulate R on CA
A(CA) = R(CA) = ⊥ R’s knowledge is incomplete:K[A(CA) ≠ T] = ⊥
37
Options for R So R has two options:
I. R has super-Turing abilities
II. R is incomplete R(CA) = ⊥
What about other options, if R can err:III. R internalizes an untruth
K[A(CA) ≠ T ] = T but in fact ¬[A(CA) ≠ T]IV. R “acts without thinking”
K[A(CA) ≠ T ] ≠ T and in reality R(CA) = ¬G(CA) So if R gives the wrong answer on CA it’s either:
Because of the rule described earlier, and then K is unsound (III) Based on some other rule which pre-empts our rule, and then K
does not return T within some reasonable time frame (IV)
F T
F T
F T
A(CA) = R(CA)
I
IV
R(CA) = ⊥
K = T II
III
38
Getting Back to Penrose
Penrose believes (I) is the only correct option, and proposes a non-Turing-equivalent model for R
Although R is deterministic, it cannot be captured algorithmically, it is more G-like
CA(CA) = ?
39What Does This Mean for AI? If we disagree with Penrose’s option
(I), and believe that there may exist a program A that simulates R, we must choose between:II. R reasons unsoundly with KIII. R feels under pressure and answers
using a process other than KIV. R doesn’t answer at all
40(II) R Reasons Unsoundly with K Hilary Putnam:
Humans are inconsistent machines, believing falsehoods through unsound reasoning
Martin Davis: “No human mathematician can claim
infallibility. We all make mistakes!” “There is nothing in Gödel’s theorem to
preclude the mathematical powers of a human mind being equivalent to an algorithmic process that produces false as well as true statements”
41
(III) R Answers without K John McCarthy:
“Much of Penrose’s reasoning is nonmonotonic, e.g. preferring the simplest explanation”
“But his methodology doesn’t allow for nonmonotonic reasoning by the program”
There is an A = R which answers incorrectly, but Penrose looks at A′ which acts like R for which only monotonic K is consulted, and A’ ≠ R
J.R. Lucas: “Our inconsistencies are mistakes rather than set
policies” “They correspond to the occasional malfunctioning of a
machine, not its normal scheme of operations”
42
(IV) R Doesn’t Answer R could be “inadequate”, not giving an answer about CA, and
maybe that’s alright, cause we are also “inadequate” David Chalmers:
“Perhaps we are sound, but we cannot know unassailably that we are sound”
Hilary Putnam: “There is an obvious lacuna: the possibility of a program … which is not
simple enough to appreciate in a perfectly conscious way is overlooked” Geoffrey LaForte, Patrick J. Hayes, and Kenneth M. Ford:
“One can show … that Penrose’s notion of what it is to know oneself to be sound cannot itself be sound”
“Humans may be unable to know that they are consistent” Arnon Avron:
“We cannot fully analyze a complicated learning machine, let alone the human mind”
“One cannot establish one’s own self-consistency”
43
Strange Loop An interesting case: suppose A is designed
to parrot R when asked whether CA(CA) = Note that when anyone asks about CA, CA
consults A, which turns the question over to R So what does R answer?
R says “yes” to A A says “yes” to CA CA(CA) = T R says “no” to A A says “no” to CA CA(CA) =
No matter what R says, it will be wrong The only sound alternative for R is to
“take the Fifth”: R(CA) =
44
Shadows of the Mind (1994)Gödel’s Theorem
45
Penrose’s Gödelian Error Suppose:
A human H has a computational mind associated with the program P
H is made aware of P which we are using to “compute his brain” We may use a formal mathematical system F to logically deduce
the behavior of P, and hence H’s behavior, beliefs and thoughts So F encapsulates all of H’s knowledge and beliefs
Penrose argues: If H believes statement X, then since F encapsulates all of H’s
beliefs, it must be possible to prove X in the system F H will surely believe that the system F is sound So H will believe (by Gödel's theorem) that g is true But since g is not a theorem of F, it follows that F cannot after all
encapsulate all of H’s beliefs So H cannot have a computational mind
46
Penrose’s Gödelian Error Penrose’s confusion:
If H believes X (on day 1) then it is necessary that F deduces thatH believes X on day 1, but it is not necessary that F deduces X!
If on day 2 H changes his mind and believes the opposite ¬X, then it is necessary that F deduces that H believes ¬X on day 2 (and not ¬X)
If on day 3 H goes crazy then it is necessary that F deduces that H is crazy on day 3; it is not necessary that F deduces all sorts of crazy formulae
Although F cannot deduce g, there is no reason why F cannot deduce that H believes g
Gödel's theorem doesn’t say anything about what can be proved concerning the state of H’s mind!
The essential point: H’s beliefs do not form part of the deductive system F If H believes two contradictory statements X and Y, these statements
are not theorems of F and so it does not follow that F is inconsistent, as it would be if X and Y were theorems
47
Feferman’s Criticism Solomon Feferman has a common cause with Penrose:
Opposes the dominant computational model of the mind Human thought, and in particular mathematical thought, is
not achieved by mechanical application of algorithms Rather, by trial-and-error, insight and inspiration, in a
process that machines will never share with humans But has a lot of criticism:
Penrose extends his argument too far in areas such as mathematical soundness and consistency, providing ammunition for the computational-mind camp
Feferman finds several technical and logical errorsin Penrose’s reasoning
But then concludes that Penrose’s case wouldnot be altered by correcting these flaws
48
Feferman’s Criticism The computational-mind argument stresses the equivalence
between Turing machines and formal systems But the model of mathematical thought using formal systems
is closer to the nature of human (and particularly mathematical) thought
The Turing machine model: given a problem, human reason would plug away, applying the same algorithm indefinitely, in the hope of finding an answer
But it’s ridiculous to think that mathematics is done this way The basis of mathematical success is trial-and-error
reasoning, insight and inspiration, based on prior experience – but not on general rules
A mechanical approach is only appropriate after an initial proof has been arrived at, for checking and validating it
49
Outline
• Brief history of AI• Lucas and Gödel• Penrose and Turing• Summary
50
So What Does Gödel Have to Say? “So the following disjunctive conclusion is
inevitable: “Either mathematics is incompletable in this sense,
that its evident axioms can never be comprised in a finite rule, that is to say, the human mind (even within the realm of pure mathematics) infinitely surpasses the powers of any finite machine…”
“… or else there exist absolutely unsolvable diophantine problems of the type specified…”
Lucas: “It is clear that Gödel thought the second disjunct false”
51
And What About Turing? “There are certain things that [any digital computer]
cannot do” “If it is rigged up to give answers to questions as in
the imitation game, there will be some questions to which it will either give a wrong answer, or fail to give an answer at all however much time is allowed for a reply”
Ordinal logics as a way to “escape” Gödel: In pre-Gödel times it was thought by some that it would be
possible to carry this to such an extent that the necessity for intuition would be eliminated
Not all the steps in a proof are mechanical, some are intuitive
“If a machine is intelligent, it must also be fallible”
52
And What About Turing? “Although it is established that there are limitations to the powers of
any particular machine, it has only been stated, without any sort of proof, that no such limitations apply to the human intellect”
“Whenever one of these machines is asked the appropriate critical question, and gives a definite answer, we know that this answer must be wrong, and this gives us a certain feeling of superiority. Is this feeling illusory?”
“It is no doubt quite genuine, but I do not think too much importance should be attached to it”
“We too often give wrong answers to questions ourselves to be justified in being very pleased at such evidence of the fallibility on the part of the machines”
“Our superiority can only be felt on such an occasion in relation to the one machine over which we have scored our petty triumph”
“There would be no question of triumphing simultaneously over all machines”
“There might be men cleverer than any given machine, but then again there might be other machines cleverer again, and so on”
53
Final Thought AI began as a godly feat of endowing the inanimate with life Formal reasoning put the emphasis on mechanizing human
thought instead of body Rational thinking strived to eliminate the need for a “soul”
in describing the mind When AI was born, the mechanists adopted either
reductionism or holism, but in any case there was no “soul” But just at the same time, mathematics and philosophy
showed that there is still a gap between mind and machine What is this gap? The human soul? Undiscovered physics?
Can we ever actually know? So we now know the theoretical limits of AI… But it seems that how it will evolve is a question of what
you believe in
Thank You!