sharpening occam ’s razor with quantum mechanics

Sharpening Occam’s razor with Quantum

MechanicsSISSA Journal Club

Matteo Marcuzzi 8th April, 2011

Niclas Koppernigck(Copernicus)

Clausius Ptolemaeus(Ptolemy)

Tyge Brahe(Tychonis)

Describing Systems

Describing Systems

Johannes Kepler

Describing Systems

.3

2

cRT

Describing Systems

Algorithmic Abstraction

212

21

rmmGF

Describing Systems

Algorithmic AbstractionSame output

Describing Systems

Same outputDifferent intrinsic information!

Solar system

celestial objects

Sun FlaresPlanet Orography

MeteorologyPeople behaviour

Compton Scattering

Describing Systems

Same outputDifferent intrinsic information!

Much more memory

required!OCCAM’S RAZOR

Describing SystemsN Spin Chain

Up parity

1 spin-flip per second10 if even

if odd

0


Up parity10 if even

if odd

0

1 spin-flip per second

1


Up parity10 if even

if odd

0


1 0


Up parity10 if even

if odd

0


1 0 1


Up parity10 if even

if odd

0


1 0 1 0


Up parity10 if even

if odd

0


1 0 1 0 1


Up parity10 if even

if odd

0


1 0 1 0 1 0


Up parity10 if even

if odd

0


1 0 1 0 1 0 1


Up parity10 if even

if odd

0


1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

N bits needed

Describing SystemsHidden System

0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1 0 1

read x

return (x+1) mod 21-bit only!

Statistically equivalent output

N bits

Computational Mechanics

• Statistical equivalence

•Measure of complexity

• Pattern identification

0m


0m 0stm





0m 0stm 04 m





0m 0stm 04 m ?





�

21012 SSSSSS Stochastic Process

DiscreteStationary

iSRandom Variables A: Alphabet

edcbaA ,,,,

1,0'A

GCTAA ,,,''


�


DiscreteStationary


123 SSSS �

Pasts

210 SSSS Futures


�


DiscreteStationary


123 ssssA ��

Set of histories

210 ssssA Set of future strings


�


DiscreteStationary

Machine

000101000101110101101…

Statistical Equivalence

)()( sSsS MS

� PP

1,0A


�


DiscreteStationary

Machine

A�

1,0A…010100010

…1100111

…01010101

States

Partition R


�


DiscreteStationary

Machine

A�

States

Partition R

1R

2R

ijS RsRas ��P

a


�


DiscreteStationary

Machine

A�

1R

2R ijM

aij RRaT ,)( P Transition Rates

)(11

aT

)(12

aT

Rj

Ri


�


DiscreteStationary

OCCAM POOL

Computational MechanicsA little information theory

spspSHAs

log Shannon entropy

XSH Conditional entropy YXSH ,

XSHSHXSI : Mutual information

SSIE�

: Excess entropy

SH


Machine Cannot distinguish between them

SSHSH�

R

Partition RA�

We want to preserve information

SSHSH�

R


Machine

SSHSH�

R

Partition RA�


SSHSH�

R

with the least possible memory

0C Log(# states)minimize


Machine

SSHSH�

R

Partition RA�


SSHSH�

R


minimize RHC Statisticalcomplexity


SSHSH�

R


SSHSH�

R



OCCAM POOL Optimal partition


Optimal partition

We want to preserve informationwith the least possible memory


SSHSH�

R

)'()( ssss SS

�� PP

ε-machine

ε

'~ ss ��if

Causal States

(unique)

Computational Mechanics: Examples

2-periodic sequence

2-periodic, ends with

2-periodic, ends with

1p1p

A

B

I

initial state

Computational Mechanics: Examples

2-periodic sequence

1p1p

A

B

I

initial state21p

21precurrent transient

Computational Mechanics: Examples1D Ising model

p p

p

p

ijV transfer matrix

jij

i Vuujip

Computational Mechanics: Examples1D Next-nearest-neighbours Ising

p

p

p

p

p

p p

p

31 J

12 J

2.0T

2


p

p

p

p

p

p p

p

31 J

12 J

2.0T

23


p

p

p

p

p

p p

p

31 J

12 J

2.0T

23

1


p

p

p

p

p

p p

p

31 J

12 J

2.0T

negligible

1

1

8B


p

p

31 J

12 J

2.0T

1

1

period 3

period 18B

Sharpening the razor with QM

EC Statistical complexit

y

Excess entropy SSI

�: RH

EC Ideal system


,,AA�

ε

ε-machines are deterministic

ε

,,AA�

ε


1R

3R2R

4R

0,, )(44

)(34

)(24 blueblueblue TTT EC

,,AA�

ε


EC 0)( cijT

fixed i,c unique j

fixed j,c unique i

ideal


ε qεcausal state Ri system state i

symbol “s” symbol state s

siTSis

skik

,

)()(sijT

q-machine states

qεsystem state i

symbol state s

siTSis

skik

,

)(

q-machine states


CLASSICAL

QUANTUM

Prepare kS

Measure C.S.j t

2kSjt )(t

kjTProbability

tjS

)( iiq RS PP ip

qεsystem state i

symbol state s

siTSis

skik

,

)(

q-machine states


CLASSICAL

QUANTUM )( iiq RS PP ip

ii

i ppC log

logtrCq

iii

i SSp

qCC

E

E

qεsystem state i

symbol state s

siTSis

skik

,

)(

q-machine states


CLASSICAL

QUANTUM

ii

i ppC log

logtrCq

iii

i SSp

qCC

ijji SS

E

E

ks

sjk

sik TT

,

)()(Ideal system

E

E

qεsystem state i

symbol state s

siTSis

skik

,

)(

q-machine states


CLASSICAL

QUANTUM

qCC

ijji SS

Non-ideal systems

Quantum mechanics improves efficiency


single spinp

21

p

p

p

p1 p1 21

p

?21

21

C

E

qC

References

M. Gu, K. Wiesner, E. Rieper & V. Vedral - "Sharpening Occam's razor with Quantum Mechanics" - arXiv: quant-ph/1102.1994v2 (2011)

C. R. Shalizi & J. P. Crutchfield - "Computational Mechanics: Pattern and Prediction, Structure and Simplicity" - arXiv: cond-mat/990717v2 (2008)

D. P. Feldman & J. P. Crutchfield - "Discovering Noncritical Organization: Statistical Mechanical, Information Theoretic, and Computational Views of Patterns in One-Dimensional Spin Systems" - Santa Fe Institute Working Paper 98-04-026 (1998)

sharpening occam ’s razor with quantum mechanics

Documents

evenif odd01 spinflip

simplest onethe systems

systemsalgorithmic abstraction

given state

given time

statespartition rthe

sharpening occams razor

systemsjohannes kepler