ilja state2014expressivity

About what cannot be said.

Maarten Marx

University of Amsterdam

May 15, 2014

Abstract

It is not easy to think of things that you cannot express. Language seemslimitless. On the other hand, we often use expressions like ”I cannot findthe words to thank you/say how much I love you/express my sadness” etc.For artificial languages, it is often possible to give semanticcharacterizations of their expressive power. We can use these to show thatsome concept is not expressible. This is a powerful tool. Without it, wecan only say that until now we did not find a way to express the concept inthe language.But we do not know if this is because of the limits of ourknowledge or because of the limits of the language. In this presentation wemake these notions precise and show with examples how they can be usedin concrete situations.

Outline

1 Natural Languages

2 Expressing things succinctlyNumbers

Still used in some Georgian shops: tallyingRoman NumbersWhat we all know and use: decimal systemNumbers in computers: zeros and ones

Propositional logicBasis of all reasoning3 basic types of sentencesSpotting contradictions

3 Limits of expressivityFirst order logic and numbers

Integers and RationalsRationals and Reals

4 Take home message

Disclaimer

This talk contains lots of symbols

Please ask or slow me down if needed.

. . .

Natural Languages

For natural languages, linguists have given bounds on the expressivityof the morphology and the grammar.

E.g., Chomsky claimed that English morphology can be generated bya regular language and English grammar by a context-free language.

Chomsky has/had the idea that people are born with some kind oflanguage or grammar-instinct.

But then grammars cannot be arbitrarily difficult/expressive.

It is a sport among linguists to find counterexamples to proposedbounds.

And now we get formal. . .

Expressing things succinctly

Our aim

Show that different languages can express the same, but using lessresources.Less resources??? That just means ”easier or shorter”

Our method

We first look at representing numbers.Then at the basis of all reasoning: propositional logic.

Expressing quantities

We want to create a notation for expressing positive quantities:

1, 2, 3, 4, . . .

We evaluate our notation on how much work it is to write down anumber.

Still used in some Georgian shops: tallying

Definition: tallying

I , II , III , . . . , IIII , IIII I , . . . , IIII IIII

Evaluation

Clearly works, every number can be expressed

Uses really only one symbol: I

Size of the expression equals the quantity which is expressed

This is not handy for very rich people. . .

Roman Numbers

Definition

I , II , III , IIII , IV , . . . , IX , . . . ,D, . . . ,C , . . . , L, . . . ,M

Evaluation

Clearly works, every number can be expressed

Uses several different symbols

Value of a symbol depends on its position: compare eg IX and XI .

Size of the expression much smaller than the quantity which isexpressed

except for really large numbers (as M is the ”largest” symbol)

Decimal system

Definition

1, 2, 3, . . . , 10, 11, . . . , 100, . . . , 1.000, . . . , 1.000.000, . . .

Evaluation

Uses 10 different symbols

Value of a symbol depends on its position:

In 538,the 5 means 500,the 3 means 30,and the 8 means 8

To express a number n we need less than log10(n) + 1 digits.

For example, we express 1000 with 4 digits.

To express the enormous wealth of Ivanishvili we need only 10 or 11digits.

Very handy and efficient

Numbers in computers: zeros and ones

Definition

1, 10, 11, 100, 101, 110, 111, 1000, . . .

Evaluation

Used in every computer.

Value of a symbol depends on its position: In 101 stands the first 1for 22 and the last 1 for 20, and thus 101 means five.

With tallying, we used 1 symbol. In our decimal system we use 10symbols. Here we use just 2 symbols.

Question Is the zero-one system more like tallying or more like thedecimal system in terms of the size of the expressions?

Answer: more like the decimal system

Number of inhabitants of Georgia

Tallying: IIII IIII IIII IIII . . . IIII IIII IIII IIII IIII IIII (very very big)

Decimal: 4.000.000

Binary string with length at most log2(4.000.000), which is

log2(4 ∗ 1.000 ∗ 1.000) = log2(4) + log2(1.000) + log2(1.000) =2 + 10 + 10 = 22

Number of inhabitants of the world

Tallying: 8 billion: 2.000 times as long as that for Georgia

Decimal: 8 billion uses only 10 digits, about 1.5 times the number ofdigits for Georgia

Binary log2(Georgia) + log2(2.000) = 22 + 11 = 33. Also just about1.5 times longer.

Conclusion

Different ways of expressing the same notions

can differ very much in the ease or costs needed to express something.

Propositional logic

Our aim

Show that checking whether you state something that makes sense can bevery time consuming.

Our method

Employ elementary reasoning

Propositional logic: Basis of all reasoning

Atomic propositions

”It is hot.”

”This talk is too long.”

Represent them by symbols p, q, r , . . ..

Can be true or false. They are atoms. We do not look inside.

Connectives

not φ, φ and ψ, φ or ψ

Examples

p and not q

not ( p or q )

p or not (q and p)

p and p

3 basic types of sentences

Tautology

Is always true. Example

p or not p

Contingency

Can be true and can be false. Example p or q

Contradiction

Is always false. Example p and not p

3 basic types of sentences

Tautology

Is always true. Example p or not p

Contingency

Can be true and can be false. Example

p or q

Contradiction

Is always false. Example p and not p

3 basic types of sentences

Tautology

Is always true. Example p or not p

Contingency

Can be true and can be false. Example p or q

Contradiction

Is always false. Example

p and not p

3 basic types of sentences

Tautology

Is always true. Example p or not p

Contingency

Can be true and can be false. Example p or q

Contradiction

Is always false. Example p and not p

Ex falso quodlibet

We better be careful with stating something, because,

when we state a contradiction,

everything follows from it.

Ex falso quodlibet

We better be careful with stating something, because,

when we state a contradiction,

everything follows from it.

Example

Assume you have as true

A This talk is greatB This talk is not great

We show that ”Santa Claus exists” follows.

from A we can derive

C This talk is great or Santa Claus exists.

Combining B and C we get,

D Santa Claus exists.

Don’t worry, we can also deduce that Santa Clause does not exist.

Spotting contradictions

Question

Given a sentence φ in prop. logic, can we compute if φ is a contradiction?

Answer

Yes, but it may take a very long time.

Spotting contradictions

Question

Given a sentence φ in prop. logic, can we compute if φ is a contradiction?

Answer

Yes, but it may take a very long time.

Easy when formula justs lists possible worlds

Special shape

not only in front of atoms

formula is a disjunction of conjunctions

(p and q and not r) or ( p and q and not p) or (not q and . . . ) or . . .

Spotting contradictions

Is very easy and quick:

for each disjunct, check if it contains an atom and its negation;

if all are like that, the formula is a contradiction

otherwise it is not.

How long does that take?

We just walk through the formula from left to rightand store at most one disjunct in our memory.

But life is not always as simple

Rewrite formulas

Every formula φ can be rewritten into this special form,

without changing its meaning.

φ is thus a contradiction precisely if its special form is one.

Rewriting is expensive

In bad cases, the equivalent of φ in special form

is exponentially larger than φ.

Example: a φ of 400 symbols leads then to a special form of 2400

symbols.

This is very much. A gigabyte is 230 bytes. A terrabyte is 240 bytes.A petabyte 250

2400 is roughly a 1 with 120 zeros, which is roughly the number ofatoms in the entire universe. . .

What to learn from this all

Spotting contradictions is hard

In eassence, nobody knows if there is a truely better way of spottingcontradictions than the one just sketched.

Artificial Intelligence is all about trying to do things faster in specialcases.

Expressivity

Different languages may both express the same thing

but one can be much more efficient/handy/succinct than the other

No we will look at real limits of expressive power.

Showing that a language has limits

Can express and cannot express

In general it is harder to show that you (and nobody else too) cannotexpress something in a language

than to show that you can

in the latter case, you just do it

Example

Express ”exclusive or” in propositional logic.

p xor q is true precisly if exactly one of p and q is true

(p and not q) or (not p and q)

Showing that a language has limits

Can express and cannot express

In general it is harder to show that you (and nobody else too) cannotexpress something in a language

than to show that you can

in the latter case, you just do it

Example

Express ”exclusive or” in propositional logic.

p xor q is true precisly if exactly one of p and q is true

(p and not q) or (not p and q)

First order logic and numbers

Talk about individuals and relations among them

Add constants, variables, quantifiers, and relations to propositionallogic.

Example

There is no largest number

not∃x∀y . y ≤ x

Every donkey has a tail

∀ x. if DONKEY(x) then ∃ y. TAIL(x,y).

This wants Facebook: all friends of your friends are your own friends

if x Friend y and y Friend z then x Friend z .

First order logic and numbers

Talk about individuals and relations among them

Add constants, variables, quantifiers, and relations to propositionallogic.

Example

There is no largest number

not∃x∀y . y ≤ x

Every donkey has a tail

∀ x. if DONKEY(x) then ∃ y. TAIL(x,y).

This wants Facebook: all friends of your friends are your own friends

if x Friend y and y Friend z then x Friend z .

First order logic and numbers

Talk about individuals and relations among them

Add constants, variables, quantifiers, and relations to propositionallogic.

Example

There is no largest number

not∃x∀y . y ≤ x

Every donkey has a tail

∀ x. if DONKEY(x) then ∃ y. TAIL(x,y).

This wants Facebook: all friends of your friends are your own friends

if x Friend y and y Friend z then x Friend z .

Integers and Rationals

Integers Z: 1,2,3,4,. . . . Rationals Q: . . . , 12 , . . . ,

23 , . . . , 1, . . .

We speak about them in First Order Logic using only the relation <(smaller than).

Question: can we express the difference between integers andrationals in FOL?

Yes!

Between any two different numbers there is another number.

∀x∀y . x < y ⇒ ∃z(x < z & z < y)

Integers and Rationals

Integers Z: 1,2,3,4,. . . . Rationals Q: . . . , 12 , . . . ,

23 , . . . , 1, . . .

We speak about them in First Order Logic using only the relation <(smaller than).

Question: can we express the difference between integers andrationals in FOL?

Yes!

Between any two different numbers there is another number.

∀x∀y . x < y ⇒ ∃z(x < z & z < y)

Integers and Rationals

Integers Z: 1,2,3,4,. . . . Rationals Q: . . . , 12 , . . . ,

23 , . . . , 1, . . .

We speak about them in First Order Logic using only the relation <(smaller than).

Question: can we express the difference between integers andrationals in FOL?

Yes!

Between any two different numbers there is another number.

∀x∀y . x < y ⇒ ∃z(x < z & z < y)

Difference between finite and infinite

Question

Can we express the difference between all integers and a finite numberof integers?

Yes

”there is a largest number”

∃x . not∃y . x < y

The same question but only using the equality relation.

We need to know how many elements there are in the finite set. Say n

∃x1∃x2 . . . ∃xn+1(x1 6= x2&x1 6= x3& . . .)

Note Formula is not general, and can be very long.

Difference between finite and infinite

Question

Can we express the difference between all integers and a finite numberof integers?

Yes

”there is a largest number”

∃x . not∃y . x < y

The same question but only using the equality relation.

We need to know how many elements there are in the finite set. Say n

∃x1∃x2 . . . ∃xn+1(x1 6= x2&x1 6= x3& . . .)

Note Formula is not general, and can be very long.

Difference between finite and infinite

Question

Can we express the difference between all integers and a finite numberof integers?

Yes

”there is a largest number”

∃x . not∃y . x < y

The same question but only using the equality relation.

We need to know how many elements there are in the finite set. Say n

∃x1∃x2 . . . ∃xn+1(x1 6= x2&x1 6= x3& . . .)

Note Formula is not general, and can be very long.

Rationals and Reals

Reals

Reals, R: all numbers which you can write using decimal expansion.

Thus all rationals, but also√

2, π, . . .

Question

Can we express the difference between these two in FOL?

No!

To show that we need to play games.

Rationals and Reals

Reals

Reals, R: all numbers which you can write using decimal expansion.

Thus all rationals, but also√

2, π, . . .

Question

Can we express the difference between these two in FOL?

No!

To show that we need to play games.

Model comparison games

Rules

Two players (Spoiler and Duplicator) play on two models

Spoiler starts and picks a number in one model

Dupicator answers by picking a number in the other model.

these numbers are connected by a line.

and so on

Duplicator must always answer so that the lines do not cross.

if she cannot answer she lost, if she can continue forever she wins.

Games and Logic

Two models make the same formulas true if Duplicator can win anygame played on them.

What we learn from this

If you want to express the difference between R and Q you need alanguage stronger than FOL.

The difference is:

in R every upward bounded set of numbers has a least upper bound.

The least upper bound of {n ∈ Q | n < π} is π, but π does not existsin Q.

Note that we quantified over SETS, not just over elements.

Take home message

Expressivity of languages

Every language (artificial and natural) has limits to what it canexpress.

These limits say something about the complexity of the language.

Know your limits

If you cannot express what you want in some language, then

it is either your fault, orthe ”fault” (the limits) of the language.

If you know these limits, you can either search further, or acceptreality and be in peace.

Thank you

Thank you!