Hard problems

November 29. 2001

Administrivia

• Ps 8– Handed out Tuesday (12/4)

– Due Tuesday (12/11) or Tuesday (12/18) your choice

• Ps 9– Handed out Thursday (12/6)

– In 3 parts• Part a due 12/11

• Part b due 12/13

• Part c due 12/14

The big picture

• We built a computer• We built an operating system • We attached the computer to a network• We wrote a compiler • We designed algorithms• We attached the computer to a network

• Next, we want to see if there are limitations– Mathematical proofs that things cannot be done???

Simple unsolvable problem

This sentence is false.

Limitations of Computers?

• Possible impossible tasks for computers?

• Emotion

• Beating the world chess champion

• Creative thought

• Others?

Limitations of Computers?

• How to show that such tasks are impossible?

• Find a cleanly defined problem

• Prove a theorem that says it can’t be solved

• Theorem is proved using axioms (like logic gates) that are assembled (like state machines and computers) to justify the result

• The underlying field is called mathematical logic

Simple task --The “Hello” Assignment

• You write a computer program which outputs “Hello!” and stops.

• Assignment promises full credit for ANY program that outputs “Hello!” and then halts, and no credit otherwise.

• Assignment is to be done on a computer that is as fast as you like and has as much memory as you like.

The “Hello” Assignment

• Can I write a program that automatically grades your homework?

The “Hello” Assignment

• Can I write a program that automatically grades your homework?

• Sample homework

Function GradeHELLO (Program P)

Execute P, and store output in output

If (output = “Hello!”) ThenReturn “Pass”

ElseReturn “Fail”

End If

End Function

The “Hello” Assignment

• Possible glitch– What if the program never stops?

Program P

Do While (1 > 0)Print “Hello!”

End While

End Program

The “Hello” Assignment

• Student’s program runs forever!

• This means the grading program would run forever.

• It would never tell me that the student should fail…

Program P

Do While (1 > 0)Print “Hello!”

End While

End Program

The “Hello” Assignment

Can just stop the Program after 1 hour and fail the student if the program hasn’t stopped yet.

Program P

Do While (1 > 0)Print “Hello!”

End While

End Program

Program P

Print “Hello!”Do While (1 hour hasn’t passed)

Do NothingEnd While

End Program

The “Hello” Assignment• Other kind of program you could write

Program PInitialize x = 0Do While (x is not a proof of Fermat’s Last Theorem)

x = x + 1End While

Print “Hello!”

End Program

The “Hello” Assignment• Other kind of program you could write

Program PInitialize x = 0Do While (x is not a proof of Fermat’s Last Theorem)

x = x + 1End While

Print “Hello!”

End Program

Interpret x as a sequence of mathematicalstatements…

The “Hello” Assignment• Other kind of program you could write

• Grading program might say to fail you.

• But technically you deserve to Pass.

• Unless the proof is wrong

Program PInitialize x = 0Do While (x is not a proof of Fermat’s Last Theorem)

x = x + 1End While

Print “Hello!”

End Program

The “Hello” Assignment• We keep running in to problems…

• Could it be that writing a computer program to grade such a simple assignment is impossible?

• YES! Moreover we can prove that it is impossible.

The Halting Problem• Can we design a program that tells whether

other programs ever halt on their own input. So, for program P

• Output “halts” if P(P) eventually halts

• Output “never halts” if P(P) never halts

• Theorem (Alan Turing 1937): No computer program cansolve the halting problem.

Proof by Contradiction• Suppose there was a computer program

called HALT, which solved the halting problem.

• Consider the following program:Program NonConformist (Program P)If ( HALT(P) = “never halts” ) Then

Halt Else

Do While (1 > 0)Print “Hello!”

End WhileEnd If

End Program

Proof by Contradiction

• Does NonConformist(NonConformist) halt?

• Yes? That means HALT(NonConformist) = “never

halts”

• No? That means HALT(NonConformist) = “halts”

Program NonConformist (Program P)If ( HALT(P) = “never halts” ) Then

Halt Else

Do While (1 > 0)Print “Hello!”

End WhileEnd If

End Program

Contradiction!

Undecidability• Halting Problem:

Given a computer program P:

• Output “halts” if P(P) eventually halts

• Output “never halts” if P(P) never halts

• We’ve shown that the Halting Problem is Undecidable – no computer program can ever solve it, no matter how powerful the computer is.

Undecidability• Many other problems are undecidable, too.

• Moreover, the same ideas that we used to prove undecidability can be used to prove a very disturbing statement.

GÖDEL’S GÖDEL’S INCOMPLETENESS THEOREMINCOMPLETENESS THEOREM

• In 1931, Gödel stunned the world by Gödel stunned the world by proving that in proving that in anyany logical framework logical framework in which you can express basic facts in which you can express basic facts about numbers, about numbers, there exists a true there exists a true statement that cannot be proved in statement that cannot be proved in that framework.that framework.

• ““Not every true statement has a proof”Not every true statement has a proof”

GÖDEL’S GÖDEL’S INCOMPLETENESS THEOREMINCOMPLETENESS THEOREM

• In 1931, Gödel stunned the world by Gödel stunned the world by proving that in proving that in anyany logical framework logical framework in which you can express basic facts in which you can express basic facts about numbers, about numbers, there exists a true there exists a true statement that cannot be proved in statement that cannot be proved in that framework.that framework.

• But wait, if it can’t be proved, how do But wait, if it can’t be proved, how do we know it is true?we know it is true?

GÖDEL’S GÖDEL’S INCOMPLETENESS THEOREMINCOMPLETENESS THEOREM

• We can prove that it is true by We can prove that it is true by “jumping” outside the original logical “jumping” outside the original logical framework to a “larger” logical framework to a “larger” logical framework…framework…

GÖDEL’S GÖDEL’S INCOMPLETENESS THEOREMINCOMPLETENESS THEOREM

• We can prove that it is true by We can prove that it is true by “jumping” outside the original logical “jumping” outside the original logical framework to a “larger” logical framework to a “larger” logical framework…framework…

• But that larger logical framework also But that larger logical framework also has a true statement that cannot be has a true statement that cannot be proved!proved!

GÖDEL’S GÖDEL’S INCOMPLETENESS THEOREMINCOMPLETENESS THEOREM

• We can prove that it is true by We can prove that it is true by “jumping” outside the original logical “jumping” outside the original logical framework to a “larger” logical framework to a “larger” logical framework…framework…

• But that larger logical framework also But that larger logical framework also has a true statement that cannot be has a true statement that cannot be proved!proved!

• ……and so on, and so on…and so on, and so on…

• No matter when we stop, there will No matter when we stop, there will always be some true statement that always be some true statement that doesn’t have a proof!doesn’t have a proof!

GÖDEL’S GÖDEL’S INCOMPLETENESS THEOREMINCOMPLETENESS THEOREM

• Until Gödel came along, Until Gödel came along, Mathematicians were searching for the Mathematicians were searching for the “one true logical framework” in which “one true logical framework” in which everything could be proved. everything could be proved.

• Their faith was shattered by Gödel.Their faith was shattered by Gödel.

• For more on this and its philosophical For more on this and its philosophical implications, see implications, see Gödel, Escher, BachGödel, Escher, Bach by Douglas Hofstadter.by Douglas Hofstadter.

What Computers Can’t DoIn Your Lifetime

• We’ve now seen an example of a problem that computers can’t solve, even if computers have unlimited speed and unlimited time.

• Are there more real world problems (eg that arise in business, science, … ) that can be solved but take billions of years?

“Search” Problems

• We’ll focus on what we’ll call “search problems”.(In CS terms, we’ll be talking about what are termed NP problems)

• Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it.

Example: Coloring

Example: Coloring with 3 colors• Suppose we are given a

collections of circles.

• Some circles are connected to others.

• Rule: No connected circles can have the same color.

• You only have three colors (Green, Red, Yellow)

• Is there a valid coloring?

• Note: easy to check validity. Hard to find?

Search Problems:More Precise Definition

• Intuitively, a search problem is a problem where you are looking for something which you can recognize quickly if you find it.

• A bit more precisely, we require that there is a small circuit that can quickly check the validity of a solution.

• For coloring, it would just be a circuit that checks that for every pair of connected circles, the colors are different.

Example:Traveling Salesman

Problem• A salesman wants to visit n different cities.

• He knows the costs associated with flights between each city.

• Can he visit all the cities spending less than $B total?

• Note: Easy to check that a given flight plan visits all the cities and costs less than $B.

• Seems hard to find…

Search Problems…• Unfortunately, we don’t know how to solve

these and many other search problems with a computer in our lifetimes for large inputs.

• For many years, computer scientists wondered which search problems could be solved, and which couldn’t.

• Just because computer scientists couldn’t solve the Coloring problem for 40 years doesn’t mean it is impossible, right?

• Proof of Fermat’s theorem took over 300 years

Search Problems…• We still don’t know how to prove that it is

impossible to solve them quickly…

• But now we are much more confident that they really are impossible to solve quickly.

Breakthrough• Another surprise:

• In the early 70’s, Cook, Levin, and Karp showed us that if we can solve the Coloring problem quickly, then we can solve ALL search problems quickly!

• But general search problems are defined in terms of circuits, not colors…

Coloring Circuits?• Suppose we think of :

• Green as meaning True.

• Red as meaning False

• Yellow as meaning nothing

Coloring Circuits?

T F

X making aNOT gateby coloring

NOT X

T FX Y

Output

X Y

F F

F T

T F

T T

T FX Y

Output

X Y

F F

F T

T F

T T

TF

T FX Y

Output

X Y

F F

F T

T F

T T

T FX Y

Output

X Y

F F

F T

T F

T T

T FX Y

Output

X Y

F F

F T

T F

T T

T FX Y

Output

X Y

F F

F T

T F T

T T

X Y

F F F

F T T

T F T

T T T

T FX Y

Output

X Y OR

F F F

F T T

T F T

T T T

OR==

T FX Y

Output

Coloring !• And you thought coloring was for kids..

• In fact, we can encode any circuit into a collection of connected circles waiting to be colored.

• Thus, we can efficiently reduce any search problem to Coloring.

• If we can solve Coloring quickly, we can solve any search problem quickly!

Many more• Traveling Salesman Problem is also as “hard”

as any search problem.

• Search Problems with this property are called complete problems

• Although we don’t know how to prove that they are hard, we know that if we solve one of them, we solve all search problems!

• Gives us more confidence that they really are hard.

Wrap-up• Today we saw problems that computers

can’t solve at all, and also problems that computers probably can’t solve in our lifetimes.

• Too much bad news?

• Could it ever be useful to have hard problems?

• Yes! Next time: Cryptography