intrinsically random is how complex object alphabet
TRANSCRIPT
15-252 Spring 2021 : Lecture 4
Kolmogorov complexity
How"
intrinsically complex"
or
" random"is
an object ?✓
Strings over a finitealphabet ⇐
* )
A - B3333.3333 Which string is most
-random ?
B =2718281828
C = 1764212643Which is most complex ?
A is a simple string
BIC look more random /complex .
'
Turns out B also has a simpledescriptions
B - first to digitsof e
C - I wrotewhat came to my
mind (sampled arandom
(o digitnumber!
Kolmogorov - Chaitin or descriptive complexB
aims to measure the complexity of a Anglobred
Complexity should berelated to the shortest
'description of th object.
"How compressible is the string"?
Intuition natural - but meaningful defy is
subtle & important to get
Beware of Berry's paradoxMsht
Need to define valid description method .
Description : partial function f : s*→s*
To describe x,we use
(⇐ 101137
a description p s -t fcp) = a
Complexity of a under this desc. method f
Cf Gc) : = min { Ipl : ftp)=x }
What is f ? what to pick I€ wanted complexity ofx to be intrinsic tox
Cf Cx) depends on f .
Imagines f(g) = 0210110110010110111f-Hock x # ogling IM
So can arbitrarily favor certain strip- watt
short descriptions .-3 Not GOOD-
GGIo.li. Develop a"
universal descriptions method"
L
-
↳"Best
' ' desc . method that givesthe
minimum description length to all strings .
Defy (Minimization) API"
f.no. EE, z*
minorities a partial fn g : E*→s* *
of F constant a set forced sees
Cf Cx) s agent t c
God , Minimal universal desc.
method U
which minorizes everyother potentillaHoned
description method .
Define Kolmogorov Complexity
Kcal := glad
#it?• What
'
does such a U look like ?
If we allow all partial fas then
no such U can exist .
Ulp,)=X , Ukip Hr.
. Utpal - kn .
. .
P, L Bec . . Lpn h . .
infinite. Seg .
Subsequence 't>
Pi ,s Pne L. .
. pin
Pi , > 21PM hlp , )= Ulpi,)
Piz > 21h21 hbhlz ? (Pid??Infinitely many strip Whare descriptions
of shalt the length under h compacduto
.
Description : partial recursive fuchsincomputable)
Z TM M s - t on input p forwhich
fb) is defined,Mon inputp
halts
with ftp) written on output tape .
Computable wayto map descriptors p
to the strings X=fCp)
theorem : F partial recursive fg U thatMinories every other partial seaman
' th.
Proof : U = a Universal Turing Machine
U on input (M ,w>
Simulates Mon wand does whatever
M does on w
K¥064 := min l l LM.ws/ : TM M
halt oninput
WI outputsx }
Note u ( LM ,w>) =x
Let's argue U minories everyother
partial recursive fa , say g.
Let TM N compute of .
For xEE* 8 let p be ashortest story .
Sit g (phx ,ie N Cpl = K
N on input phe159outputs x
How to describe x under U ?
Clearly U ( LN , p >) =x
KN , p> I = CN + Ipl
↳ const . dependsonN -
CTO encode pain ka, by, repeateach bit of a
twice, add 01 , thenwrite b - Ka,by a 2) at -141-12
8. FK Cola ) s ont Ipl= cent Cg Ge) ④
Deth (Kolmogorov Complexity) of a
string x is defined as
Kcal Could= mins KM.ws/ a TM M
halt
on input w& outputs x }
Note M can runon w fora long time,
doesn't factor intoKhed
=
Defy:(Incompressible ship )
A string xis said to be incompressible on
Kolmogorovrandom if Kcx) > la I
Note . Tx ,K Gc) I felt 04 )
(Justify this ? )
Use"
print x"
as the program
Lenya : F incompressible strip of every
length .
Pf . There are 2"
binary strings oflength n .
but only 211 binary strip of
length L n.
ID
Examples :
KCon ) s log n toll)( Specify n in binary 8 a
program that on input n'
in biliary
outputs on )K ( first n digits of e ) ← login toll)
K ( yon-M D = In ± OLD (
Exercise)
→ y is an incompressible spy'
og
length in
theorem :
↳ { a 1 kcal 31×1 }is undecidable .
[It is undecidable to tell ifa
Sony is compressible )
PI : Suppose 8 TM p that decides
path if a shiny is incompressible .large
enough .
Consider TM p'
: :
--Onn input Ln >:
• Russ P onall strings of length
is
in lexicographic order .
• For the first Amy? thatP reports to
be incompressible , outputZ
.
Lip ! Cns ) is a description of z
Ktp! an>71=04)+ log n
OTOH , KEE) z n .
Cog P says
z isincompressible,
→ n s log n told
a contradiction cos nis largeenough )
can use Kolmogorov complexity to prove
Godel first incompleteness theorem
For Any sufficiently expressive
consistent mathematical theory
Fai:m7:÷tm'T:mExpress statements like 4kcal > L
"
Using similar Berry paradox like
argument , can proc:
IL St Fx"Kcal > L
'
is not provable .
OTOH Ix at Ktx > L
(by counting )
⇒ A true statementthat can't
be
proud in a theorythatcan
express"Kfc)>L
"
type statements.