ilja state2014expressivity
TRANSCRIPT
![Page 1: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/1.jpg)
About what cannot be said.
Maarten Marx
University of Amsterdam
May 15, 2014
![Page 2: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/2.jpg)
Abstract
It is not easy to think of things that you cannot express. Language seemslimitless. On the other hand, we often use expressions like ”I cannot findthe words to thank you/say how much I love you/express my sadness” etc.For artificial languages, it is often possible to give semanticcharacterizations of their expressive power. We can use these to show thatsome concept is not expressible. This is a powerful tool. Without it, wecan only say that until now we did not find a way to express the concept inthe language.But we do not know if this is because of the limits of ourknowledge or because of the limits of the language. In this presentation wemake these notions precise and show with examples how they can be usedin concrete situations.
![Page 3: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/3.jpg)
Outline
1 Natural Languages
2 Expressing things succinctlyNumbers
Still used in some Georgian shops: tallyingRoman NumbersWhat we all know and use: decimal systemNumbers in computers: zeros and ones
Propositional logicBasis of all reasoning3 basic types of sentencesSpotting contradictions
3 Limits of expressivityFirst order logic and numbers
Integers and RationalsRationals and Reals
4 Take home message
![Page 4: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/4.jpg)
Disclaimer
This talk contains lots of symbols
Please ask or slow me down if needed.
. . .
![Page 5: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/5.jpg)
Natural Languages
For natural languages, linguists have given bounds on the expressivityof the morphology and the grammar.
E.g., Chomsky claimed that English morphology can be generated bya regular language and English grammar by a context-free language.
Chomsky has/had the idea that people are born with some kind oflanguage or grammar-instinct.
But then grammars cannot be arbitrarily difficult/expressive.
It is a sport among linguists to find counterexamples to proposedbounds.
![Page 6: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/6.jpg)
And now we get formal. . .
![Page 7: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/7.jpg)
Expressing things succinctly
Our aim
Show that different languages can express the same, but using lessresources.Less resources??? That just means ”easier or shorter”
Our method
We first look at representing numbers.Then at the basis of all reasoning: propositional logic.
![Page 8: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/8.jpg)
Expressing quantities
We want to create a notation for expressing positive quantities:
1, 2, 3, 4, . . .
We evaluate our notation on how much work it is to write down anumber.
![Page 9: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/9.jpg)
Still used in some Georgian shops: tallying
Definition: tallying
I , II , III , . . . , IIII , IIII I , . . . , IIII IIII
Evaluation
Clearly works, every number can be expressed
Uses really only one symbol: I
Size of the expression equals the quantity which is expressed
This is not handy for very rich people. . .
![Page 10: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/10.jpg)
Roman Numbers
Definition
I , II , III , IIII , IV , . . . , IX , . . . ,D, . . . ,C , . . . , L, . . . ,M
Evaluation
Clearly works, every number can be expressed
Uses several different symbols
Value of a symbol depends on its position: compare eg IX and XI .
Size of the expression much smaller than the quantity which isexpressed
except for really large numbers (as M is the ”largest” symbol)
![Page 11: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/11.jpg)
Decimal system
Definition
1, 2, 3, . . . , 10, 11, . . . , 100, . . . , 1.000, . . . , 1.000.000, . . .
Evaluation
Uses 10 different symbols
Value of a symbol depends on its position:
In 538,the 5 means 500,the 3 means 30,and the 8 means 8
To express a number n we need less than log10(n) + 1 digits.
For example, we express 1000 with 4 digits.
To express the enormous wealth of Ivanishvili we need only 10 or 11digits.
Very handy and efficient
![Page 12: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/12.jpg)
Numbers in computers: zeros and ones
Definition
1, 10, 11, 100, 101, 110, 111, 1000, . . .
Evaluation
Used in every computer.
Value of a symbol depends on its position: In 101 stands the first 1for 22 and the last 1 for 20, and thus 101 means five.
With tallying, we used 1 symbol. In our decimal system we use 10symbols. Here we use just 2 symbols.
Question Is the zero-one system more like tallying or more like thedecimal system in terms of the size of the expressions?
![Page 13: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/13.jpg)
Answer: more like the decimal system
Number of inhabitants of Georgia
Tallying: IIII IIII IIII IIII . . . IIII IIII IIII IIII IIII IIII (very very big)
Decimal: 4.000.000
Binary string with length at most log2(4.000.000), which is
log2(4 ∗ 1.000 ∗ 1.000) = log2(4) + log2(1.000) + log2(1.000) =2 + 10 + 10 = 22
Number of inhabitants of the world
Tallying: 8 billion: 2.000 times as long as that for Georgia
Decimal: 8 billion uses only 10 digits, about 1.5 times the number ofdigits for Georgia
Binary log2(Georgia) + log2(2.000) = 22 + 11 = 33. Also just about1.5 times longer.
![Page 14: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/14.jpg)
Conclusion
Different ways of expressing the same notions
can differ very much in the ease or costs needed to express something.
![Page 15: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/15.jpg)
Propositional logic
Our aim
Show that checking whether you state something that makes sense can bevery time consuming.
Our method
Employ elementary reasoning
![Page 16: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/16.jpg)
Propositional logic: Basis of all reasoning
Atomic propositions
”It is hot.”
”This talk is too long.”
Represent them by symbols p, q, r , . . ..
Can be true or false. They are atoms. We do not look inside.
Connectives
not φ, φ and ψ, φ or ψ
Examples
p and not q
not ( p or q )
p or not (q and p)
p and p
![Page 17: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/17.jpg)
3 basic types of sentences
Tautology
Is always true. Example
p or not p
Contingency
Can be true and can be false. Example p or q
Contradiction
Is always false. Example p and not p
![Page 18: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/18.jpg)
3 basic types of sentences
Tautology
Is always true. Example p or not p
Contingency
Can be true and can be false. Example
p or q
Contradiction
Is always false. Example p and not p
![Page 19: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/19.jpg)
3 basic types of sentences
Tautology
Is always true. Example p or not p
Contingency
Can be true and can be false. Example p or q
Contradiction
Is always false. Example
p and not p
![Page 20: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/20.jpg)
3 basic types of sentences
Tautology
Is always true. Example p or not p
Contingency
Can be true and can be false. Example p or q
Contradiction
Is always false. Example p and not p
![Page 21: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/21.jpg)
Ex falso quodlibet
We better be careful with stating something, because,
when we state a contradiction,
everything follows from it.
![Page 22: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/22.jpg)
Ex falso quodlibet
We better be careful with stating something, because,
when we state a contradiction,
everything follows from it.
![Page 23: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/23.jpg)
Example
Assume you have as true
A This talk is greatB This talk is not great
We show that ”Santa Claus exists” follows.
from A we can derive
C This talk is great or Santa Claus exists.
Combining B and C we get,
D Santa Claus exists.
Don’t worry, we can also deduce that Santa Clause does not exist.
![Page 24: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/24.jpg)
Spotting contradictions
Question
Given a sentence φ in prop. logic, can we compute if φ is a contradiction?
Answer
Yes, but it may take a very long time.
![Page 25: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/25.jpg)
Spotting contradictions
Question
Given a sentence φ in prop. logic, can we compute if φ is a contradiction?
Answer
Yes, but it may take a very long time.
![Page 26: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/26.jpg)
Easy when formula justs lists possible worlds
Special shape
not only in front of atoms
formula is a disjunction of conjunctions
(p and q and not r) or ( p and q and not p) or (not q and . . . ) or . . .
Spotting contradictions
Is very easy and quick:
for each disjunct, check if it contains an atom and its negation;
if all are like that, the formula is a contradiction
otherwise it is not.
How long does that take?
We just walk through the formula from left to rightand store at most one disjunct in our memory.
![Page 27: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/27.jpg)
But life is not always as simple
Rewrite formulas
Every formula φ can be rewritten into this special form,
without changing its meaning.
φ is thus a contradiction precisely if its special form is one.
Rewriting is expensive
In bad cases, the equivalent of φ in special form
is exponentially larger than φ.
Example: a φ of 400 symbols leads then to a special form of 2400
symbols.
This is very much. A gigabyte is 230 bytes. A terrabyte is 240 bytes.A petabyte 250
2400 is roughly a 1 with 120 zeros, which is roughly the number ofatoms in the entire universe. . .
![Page 28: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/28.jpg)
What to learn from this all
Spotting contradictions is hard
In eassence, nobody knows if there is a truely better way of spottingcontradictions than the one just sketched.
Artificial Intelligence is all about trying to do things faster in specialcases.
Expressivity
Different languages may both express the same thing
but one can be much more efficient/handy/succinct than the other
No we will look at real limits of expressive power.
![Page 29: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/29.jpg)
Showing that a language has limits
Can express and cannot express
In general it is harder to show that you (and nobody else too) cannotexpress something in a language
than to show that you can
in the latter case, you just do it
Example
Express ”exclusive or” in propositional logic.
p xor q is true precisly if exactly one of p and q is true
(p and not q) or (not p and q)
![Page 30: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/30.jpg)
Showing that a language has limits
Can express and cannot express
In general it is harder to show that you (and nobody else too) cannotexpress something in a language
than to show that you can
in the latter case, you just do it
Example
Express ”exclusive or” in propositional logic.
p xor q is true precisly if exactly one of p and q is true
(p and not q) or (not p and q)
![Page 31: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/31.jpg)
First order logic and numbers
Talk about individuals and relations among them
Add constants, variables, quantifiers, and relations to propositionallogic.
Example
There is no largest number
not∃x∀y . y ≤ x
Every donkey has a tail
∀ x. if DONKEY(x) then ∃ y. TAIL(x,y).
This wants Facebook: all friends of your friends are your own friends
if x Friend y and y Friend z then x Friend z .
![Page 32: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/32.jpg)
First order logic and numbers
Talk about individuals and relations among them
Add constants, variables, quantifiers, and relations to propositionallogic.
Example
There is no largest number
not∃x∀y . y ≤ x
Every donkey has a tail
∀ x. if DONKEY(x) then ∃ y. TAIL(x,y).
This wants Facebook: all friends of your friends are your own friends
if x Friend y and y Friend z then x Friend z .
![Page 33: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/33.jpg)
First order logic and numbers
Talk about individuals and relations among them
Add constants, variables, quantifiers, and relations to propositionallogic.
Example
There is no largest number
not∃x∀y . y ≤ x
Every donkey has a tail
∀ x. if DONKEY(x) then ∃ y. TAIL(x,y).
This wants Facebook: all friends of your friends are your own friends
if x Friend y and y Friend z then x Friend z .
![Page 34: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/34.jpg)
Integers and Rationals
Integers Z: 1,2,3,4,. . . . Rationals Q: . . . , 12 , . . . ,
23 , . . . , 1, . . .
We speak about them in First Order Logic using only the relation <(smaller than).
Question: can we express the difference between integers andrationals in FOL?
Yes!
Between any two different numbers there is another number.
∀x∀y . x < y ⇒ ∃z(x < z & z < y)
![Page 35: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/35.jpg)
Integers and Rationals
Integers Z: 1,2,3,4,. . . . Rationals Q: . . . , 12 , . . . ,
23 , . . . , 1, . . .
We speak about them in First Order Logic using only the relation <(smaller than).
Question: can we express the difference between integers andrationals in FOL?
Yes!
Between any two different numbers there is another number.
∀x∀y . x < y ⇒ ∃z(x < z & z < y)
![Page 36: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/36.jpg)
Integers and Rationals
Integers Z: 1,2,3,4,. . . . Rationals Q: . . . , 12 , . . . ,
23 , . . . , 1, . . .
We speak about them in First Order Logic using only the relation <(smaller than).
Question: can we express the difference between integers andrationals in FOL?
Yes!
Between any two different numbers there is another number.
∀x∀y . x < y ⇒ ∃z(x < z & z < y)
![Page 37: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/37.jpg)
Difference between finite and infinite
Question
Can we express the difference between all integers and a finite numberof integers?
Yes
”there is a largest number”
∃x . not∃y . x < y
The same question but only using the equality relation.
We need to know how many elements there are in the finite set. Say n
∃x1∃x2 . . . ∃xn+1(x1 6= x2&x1 6= x3& . . .)
Note Formula is not general, and can be very long.
![Page 38: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/38.jpg)
Difference between finite and infinite
Question
Can we express the difference between all integers and a finite numberof integers?
Yes
”there is a largest number”
∃x . not∃y . x < y
The same question but only using the equality relation.
We need to know how many elements there are in the finite set. Say n
∃x1∃x2 . . . ∃xn+1(x1 6= x2&x1 6= x3& . . .)
Note Formula is not general, and can be very long.
![Page 39: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/39.jpg)
Difference between finite and infinite
Question
Can we express the difference between all integers and a finite numberof integers?
Yes
”there is a largest number”
∃x . not∃y . x < y
The same question but only using the equality relation.
We need to know how many elements there are in the finite set. Say n
∃x1∃x2 . . . ∃xn+1(x1 6= x2&x1 6= x3& . . .)
Note Formula is not general, and can be very long.
![Page 40: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/40.jpg)
Rationals and Reals
Reals
Reals, R: all numbers which you can write using decimal expansion.
Thus all rationals, but also√
2, π, . . .
Question
Can we express the difference between these two in FOL?
No!
To show that we need to play games.
![Page 41: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/41.jpg)
Rationals and Reals
Reals
Reals, R: all numbers which you can write using decimal expansion.
Thus all rationals, but also√
2, π, . . .
Question
Can we express the difference between these two in FOL?
No!
To show that we need to play games.
![Page 42: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/42.jpg)
Model comparison games
Rules
Two players (Spoiler and Duplicator) play on two models
Spoiler starts and picks a number in one model
Dupicator answers by picking a number in the other model.
these numbers are connected by a line.
and so on
Duplicator must always answer so that the lines do not cross.
if she cannot answer she lost, if she can continue forever she wins.
Games and Logic
Two models make the same formulas true if Duplicator can win anygame played on them.
![Page 43: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/43.jpg)
What we learn from this
If you want to express the difference between R and Q you need alanguage stronger than FOL.
The difference is:
in R every upward bounded set of numbers has a least upper bound.
The least upper bound of {n ∈ Q | n < π} is π, but π does not existsin Q.
Note that we quantified over SETS, not just over elements.
![Page 44: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/44.jpg)
Take home message
Expressivity of languages
Every language (artificial and natural) has limits to what it canexpress.
These limits say something about the complexity of the language.
Know your limits
If you cannot express what you want in some language, then
it is either your fault, orthe ”fault” (the limits) of the language.
If you know these limits, you can either search further, or acceptreality and be in peace.
![Page 45: Ilja state2014expressivity](https://reader033.vdocuments.net/reader033/viewer/2022060116/557e621cd8b42ae5688b4c29/html5/thumbnails/45.jpg)
Thank you
Thank you!