computation. binary numbers decimal numbers binary numbers

Computation

Binary Numbers

http://schoolscience.rice.edu/duker/robots/binarynumber.html

• Decimal numbers

• Binary numbers

http://schoolscience.rice.edu/duker/robots/binarynumber.html

Representing Text

• Decide how many characters we need to represent.

• Determine the required number of bits.

• Ascii: 7 bits. Can encode 27 = 128 different symbols.

Ascii

http://www.neurophys.wisc.edu/comp/docs/ascii.html

http://www.neurophys.wisc.edu/comp/docs/ascii.html

Representing Text

Four …

01000110 01101111 01110101 01110010

When We Need More Characters

简体字

What about things like:

When We Need More Characters

Answer: Unicode: 32 bits. Over 4 million characters.

http://www.unicode.org/charts/

简体字

What about things like:

http://www.unicode.org/charts/

Digital Images

RGB

The red channel

RGB

The green channel

RGB

Red Green Blue

Representing Pictures

Representing Sounds

Representing Programs

public static TreeMap<String, Integer> create() throws IOException public static TreeMap<String, Integer> create() throws IOException

{ Integer freq; String word; TreeMap<String, Integer> result = new TreeMap<String,

Integer>(); JFileChooser c = new JFileChooser(); int retval = c.showOpenDialog(null); if (retval == JFileChooser.APPROVE_OPTION)

{ Scanner s = new Scanner( c.getSelectedFile());while( s.hasNext() ){ word = s.next().toLowerCase(); freq = result.get(word); result.put(word, (freq == null ? 1 : freq + 1));}

} return result;}

}

The Roots of Modern Technology

5thc B.C. Aristotelian logic invented

1642 Pascal built an adding machine

1694 Leibnitz reckoning machine

The Roots, continued

1834 Charles Babbage’s Analytical Engine

Ada writes of the engine, “The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we know how to order it to perform.”

The picture is of a model built in the late 1800s by Babbage’s son from Babbage’s drawings.

The Roots: Logic

1848 George Boole The Calculus of Logic

chocolate

nuts

mint

chocolate and nuts and mint

Boolean Logic

P Q P P Q P Q P Q P Q

True True False True True True True

True False False True False False False

False True True True False True False

False False True False False True True

Using Boolean Logic

P Q P P Q P Q (P Q) Q P ((P Q) Q)

True True False True True True False


False True True True True True True

False False True False True True True

How Big Are the Truth Tables?

P Q RTrue True True

True True False

True False True

True False False

False True True

False True False

False False True

False False False

2n

0

200000

400000

600000

800000

1000000

1200000

1 3 5 7 9 11 13 15 17 19 21

Increased Power of Quantified Logic

• Bear(Smoky).

x (Bear(x) Animal(x)).

x (Animal(x) Bear(x)).

x (Animal(x) y (Mother-of(y, x))).

x ((Animal(x) Dead(x)) Alive(x)).

Using Quantified Logic (FOL)

We can represent what we know as a set of logical formulas and then manipulate them.

Facts in English(1) Marcus was a man.

(2) Marcus was a Pompeian.

(3) All Pompeians were Romans.

(4) Caesar was a ruler.

(5) All Romans were either loyal to Caesar or hated him.

(6) Everyone is loyal to someone.

(7) People only try to assassinate rulers they are not loyal to.

(8) Marcus tried to assassinate Caesar.

Facts in First Order Logic (FOL)(1) Marcus was a man.

man(Marcus)(2) Marcus was a Pompeian.

Pompeian(Marcus)(3) All Pompeians were Romans.

x Pompeian(x) Roman(x)(4) Caesar was a ruler.

ruler(Caesar)(5) All Romans were either loyal to Caesar or hated him.

x Roman(x) loyalto(x, Caesar) hate(x, Caesar)(6) Everyone is loyal to someone.

x y loyalto(x, y)(7) People only try to assassinate rulers they are not loyal to.

x y person(x) ruler(y) tryassassinate(x, y) loyalto(x, y)(8) Marcus tried to assassinate Caesar.

tryassassinate(Marcus, Caesar)

Search

Breadth-First Search

Is this a good idea?

Depth-First Search

Scalability

Solving hard problems requires search in a large space.

To play master-level chess requires searching about 8 ply deep. So about 358 or 21012 nodes must be examined.

Growth Rates of Functions

The Advent of the Computer

1945 ENIAC The first electronic digital computer

1949 EDVAC

The first stored program computer

Moore’s Law

http://www.intel.com/technology/mooreslaw/

http://www.intel.com/technology/mooreslaw/

How Much Computer Power Might It Take?

http://www.frc.ri.cmu.edu/~hpm/book97/ch3/index.html

http://www.frc.ri.cmu.edu/~hpm/book97/ch3/index.html

How Much Compute Power is There?

From Hans Moravec, Robot Mere Machine to Transcendent Mind 1998.

Kurweil’s Vision

http://www.pocket-lint.co.uk/news/news.phtml/12920/13944/Computers-match-humans-by-2030.phtml



Limits to What We Can Compute

Are there fundamentally uncomputable things?

• Does God exist?

• What’s the best way to run a country?

• Does this puzzle have a solution?

Mathematics in the Early 20th Century

1900 Hilbert’s program and the effort to formalize mathematics

1931 Kurt Gödel’s paper, On Formally Undecidable Propositions

1936 Alan Turing’s paper, On Computable Numbers with an application to the Entscheidungs problem

What Can Algorithms Do?

1. Can we make all true statements theorems?

2. Can we decide whether a statement is a theorem?

Gödel’s Incompleteness Theorem

Kurt Gödel showed, in the proof of his Incompleteness Theorem [Gödel 1931], that the answer to question 1 is no. In particular, he showed that there exists no decidable axiomatization of Peano arithmetic that is both consistent and complete.

The Entscheidungsproblem

Does there exist an algorithm to decide, given an arbitrary sentence w in first order logic, whether w is valid?

Given a set of axioms A and a sentence w, does there exist an algorithm to decide whether w is entailed by A?

Given a set of axioms, A, and a sentence, w, does there exist an algorithm to decide whether w can be proved from A?

Turing Machines

At each step, the machine must:

● choose its next state, ● write on the current square, and ● move left or right.

An Example

The Church-Turing Thesis

• Turing machines

• Lambda calculus

• Standard programming languages like Java

• Conway’s game of life

• DNA computing

All formalisms powerful enough to describe everything we think of as a computational algorithm are equivalent.

The Game of Life

Playing the game

The rules: ● A dead cell with exactly three live neighbors becomes

a live cell (birth). ● A live cell with two or three live neighbors stays alive

(survival). ● In all other cases, a cell dies or remains dead

(overcrowding or loneliness).

A game halts iff it reaches some stable configuration.

http://www.math.com/students/wonders/life/life.html

Back to the Entscheidungsproblem

1. Given a Turing machine M, we can construct a logical formula F that is true iff M ever halts.

2. If there were a solution to the Entscheidungsproblem, then we could determine the truth of F and thus be able to decide whether M ever halts.

3. But there is no procedure for determining (in the general case) whether M halts.

4. So there is no solution to the Entscheidungsproblem.

Theorem: The Entscheidungsproblem is unsolvable.

Proof: (A variant of Turing’s proof)

The Halting Problem

Turing machine, M

input string, w

Does M halt on w?Accept

Reject

An Example of a Similar Question

Does this program halt on all inputs?

times3(x: positive integer) = While x 1 do: If x is even then x = x/2. Else x = 3x + 1.

Let’s try it.

http://www.numbertheory.org/php/collatz.html

The Halting Problem Is Undecidable

Proof: If it were decidable, then some TM MH would decide it. MH would implement the specification:

halts(<M: string, w: string>) = If <M> is a Turing machine description and M halts on input w then accept. Else reject.

Trouble

Trouble(x: string) = if halts(x, x) then loop forever, else halt.

What is Trouble(<Trouble>)?

What is halts(<Trouble, Trouble>)?

● If halts reports that Trouble(<Trouble>) halts, Trouble ________.

● If halts reports that Trouble(<Trouble>) does not halt, Trouble __________.

Another Undecidable Problem

The Post Correspondence Problem

A PCP Instance With a Simple Solution

i X Y

1 b aab

2 abb b

3 aba a

4 baaa baba

A PCP Instance With a Simple Solution

i X Y

1 b aab

2 abb b

3 aba a

4 baaa baba

Solution: 3, 4, 1

Another PCP Instance

i X Y

1 11 011

2 01 0

3 001 110

A PCP Instance With No Simple Solution

i X Y

1 1101 1

2 0110 11

3 1 110

A PCP Instance With No Simple Solution

i X Y

1 1101 1

2 0110 11

3 1 110

Shortest solution has length 252.

A Tiling Problem

Given a finite set T of tiles of the form:

Is it possible to tile an arbitrary surface in the plane?

A Set of TilesThat Cannot Tile the Plane

Is the Tiling Problem Decidable?

Wang’s conjecture: If a given set of tiles can be used to tile anarbitrary surface, then it can always do so periodically. In otherwords, there must exist a finite area that can be tiled and thenrepeated infinitely often to cover any desired surface.

But Wang’s conjecture is false.

The Implications

• The Entscheidungs problem is undecidable.

• There’s no black box reasoning engine for FOL.

• Would quantum computing change the picture?

• Does undecidability doom our attempt to make artificial copies of ourselves?

Is Decidability Enough?

Boolean Logic, Again






How many steps would it take a deterministic Turing machine to examine this table?

Nondeterminism

Nondeterministically Deciding SAT






How many steps in a single path of the process of examining this table?

P and NP

• P – problems that are solvable deterministically in polynomial time.

• NP – problems that are solvable nondeterministically in polynomial time.

The Traveling Salesman Problem

Given n cities and the distances between each pair ofthem, find the shortest tour that returns to its starting pointand visits each other city exactly once along the way.

15

20

25

89

23

40

10

4

73

28


15

20

25

89

23

40

10

4

73

28

Given n cities:

Choose a first city nChoose a second n-1Choose a third n-2

… n!


Can we do better than n!

● First city doesn’t matter. ● Order doesn’t matter.

So we get (n-1!)/2.

The Growth Rate of n!

2 2 11 479001600

3 6 12 6227020800

4 24 13 87178291200

5 120 14 1307674368000

6 720 15 20922789888000

7 5040 16 355687428096000

8 40320 17 6402373705728000

9 362880 18 121645100408832000

10 3628800 19 2432902008176640000

11 39916800 36 3.61041


http://www.tsp.gatech.edu/history/pictorial/pictorial.html

Solving it Nondeterministically

15

20

25

89

23

40

10

4

73

28

SAT and Traveling Salesman

D ND

SAT 2n n

Traveling Salesman n! n

P and NP

• P – problems that are solvable deterministically in polynomial time.

• NP – problems that are solvable nondeterministically in polynomial time.

• NP-complete – NP problems that are at least as hard as every other NP-complete problem.

Growth Rates of Functions, Again

Does Complexity Doom AI?

computation. binary numbers decimal numbers binary numbers

Documents