![Page 1: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/1.jpg)
Ockham’s Razor: Ockham’s Razor: What it is, What it is,
What it isn’t, What it isn’t, How it works, and How it works, and
How it doesn’tHow it doesn’t
Kevin T. KellyKevin T. KellyDepartment of PhilosophyDepartment of Philosophy
Carnegie Mellon UniversityCarnegie Mellon Universitywww.cmu.eduwww.cmu.edu
![Page 2: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/2.jpg)
““Efficient Convergence Implies Ockham's Efficient Convergence Implies Ockham's Razor”,Razor”, Proceedings of Proceedings of the 2002 International the 2002 International Workshop on Computational Models of Workshop on Computational Models of Scientific Scientific Reasoning and ApplicationsReasoning and Applications, Las Vegas, , Las Vegas, USA, June USA, June 24-24- 27, 2002.27, 2002.(with C. Glymour) (with C. Glymour) “Why Probability Does Not “Why Probability Does Not Capture the Capture the Logic of Scientific Logic of Scientific Justification”,Justification”, C. Hitchcock, ed., C. Hitchcock, ed., Contemporary Contemporary Debates in the Philosophy of ScienceDebates in the Philosophy of Science, Oxford: , Oxford: Blackwell, 2004.Blackwell, 2004.““Justification as Truth-finding Efficiency: How Justification as Truth-finding Efficiency: How Ockham's Ockham's Razor Works”,Razor Works”, Minds and MachinesMinds and Machines 14: 14: 2004, pp. 485-505.2004, pp. 485-505.““Learning, Simplicity, Truth, and Learning, Simplicity, Truth, and Misinformation”,Misinformation”, The The Philosophy of Philosophy of InformationInformation, under review., under review.““Ockham's Razor, Efficiency, and the Infinite Ockham's Razor, Efficiency, and the Infinite Game ofGame of
Science”,Science”, proceedings, proceedings, Foundations of the Foundations of the Formal Sciences 2004: Formal Sciences 2004: Infinite Game TheoryInfinite Game Theory, , Springer, under review.Springer, under review.
Further ReadingFurther Reading
![Page 3: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/3.jpg)
Which Theory to Choose?Which Theory to Choose?
T1 T2 T3 T4 T5
???
Compatible with data
![Page 4: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/4.jpg)
Use Ockham’s RazorUse Ockham’s Razor
SimpleSimple ComplexComplexT1 T2 T3 T4 T5
![Page 5: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/5.jpg)
DilemmaDilemma
If you If you knowknow the truth is simple, the truth is simple,
then you then you don’t needdon’t need Ockham. Ockham.
T1 T2 T3 T4 T5SimpleSimple ComplexComplex
![Page 6: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/6.jpg)
DilemmaDilemma
If you If you don’t knowdon’t know the truth is simple, the truth is simple,
then how couldthen how could a a fixedfixed simplicity biassimplicity bias help you if the truth is complex?help you if the truth is complex?
T1 T2 T3 T4 T5SimpleSimple ComplexComplex
![Page 7: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/7.jpg)
PuzzlePuzzleA A fixedfixed bias is like a bias is like a brokenbroken thermometer. thermometer.
How could it possibly help you find How could it possibly help you find unknownunknown truth? truth?
Cold!
![Page 8: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/8.jpg)
I. Ockham I. Ockham Apologists Apologists
![Page 9: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/9.jpg)
Wishful ThinkingWishful Thinking
Simple theories are nice if true:Simple theories are nice if true: TestabilityTestability UnityUnity Best explanationBest explanation Aesthetic appealAesthetic appeal Compress dataCompress data
So is believing that you are the So is believing that you are the emperor.emperor.
![Page 10: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/10.jpg)
OverfittingOverfitting Maximum likelihood estimates based on Maximum likelihood estimates based on
overly complex theories can have greater overly complex theories can have greater predictive errorpredictive error (AIC, Cross-validation, etc.). (AIC, Cross-validation, etc.). Same is true even if you Same is true even if you know know the the true true model is model is
complexcomplex.. Doesn’t Doesn’t convergeconverge to true model. to true model. Depends on Depends on random datarandom data..
Thanks, but a simpler Thanks, but a simpler model still has lower model still has lower predictive error.predictive error.
The truth is The truth is complex.complex.-God--God- .
..
.
![Page 11: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/11.jpg)
Messy worlds are Messy worlds are legionlegion
Tidy worlds are Tidy worlds are fewfew..
That is why the tidy worldsThat is why the tidy worlds
Are those most likely Are those most likely truetrue. (Carnap). (Carnap)
unity
Ignorance = KnowledgeIgnorance = Knowledge
![Page 12: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/12.jpg)
Ignorance = KnowledgeIgnorance = Knowledge
1/3 1/3 1/3
Messy worlds are Messy worlds are legionlegion
Tidy worlds are Tidy worlds are fewfew..
That is why the tidy worldsThat is why the tidy worlds
Are those most likely Are those most likely truetrue. (Carnap). (Carnap)
![Page 13: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/13.jpg)
Ignorance = KnowledgeIgnorance = Knowledge
2/6
1/6
2/6
1/6
Messy worlds are Messy worlds are legionlegion
Tidy worlds are Tidy worlds are fewfew..
That is why the tidy worldsThat is why the tidy worlds
Are those most likely Are those most likely truetrue. (Carnap). (Carnap)
![Page 14: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/14.jpg)
Depends on NotationDepends on Notation
But mess depends onBut mess depends on coding coding,,
which Goodman noticed, too.which Goodman noticed, too.
The picture is The picture is invertedinverted if if
we we translatetranslate green to grue. green to grue.2/6
1/6
2/6
1/6 NotationIndicates truth?
![Page 15: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/15.jpg)
Same for Algorithmic Same for Algorithmic ComplexityComplexity
Goodman’s problemGoodman’s problem works against every works against every fixedfixed simplicity ranking (independent of the processes by simplicity ranking (independent of the processes by which data are generated and coded prior to learning).which data are generated and coded prior to learning).
Extra problem:Extra problem: any pair-wise ranking of theories can any pair-wise ranking of theories can be be reversedreversed by choosing an by choosing an alternative computer alternative computer languagelanguage..
So how could simplicity help us find the true theory? So how could simplicity help us find the true theory?
NotationIndicates truth?
![Page 16: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/16.jpg)
Just Beg the QuestionJust Beg the Question AssignAssign high prior probability to high prior probability to
simple theories.simple theories. Why Why shouldshould you? you? Preference for Preference for complexitycomplexity has the same has the same
“explanation”.“explanation”.
You presume simplicityTherefore you should presume simplicity!
![Page 17: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/17.jpg)
Miracle ArgumentMiracle Argument
Simple data would be a Simple data would be a miraclemiracle if a if a complex theory were true (Bayes, complex theory were true (Bayes, BIC, Putnam). BIC, Putnam).
![Page 18: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/18.jpg)
Begs the QuestionBegs the Question
S C
““Fairness” between theories Fairness” between theories
bias against complex worlds. bias against complex worlds.
![Page 19: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/19.jpg)
Two Can Play That GameTwo Can Play That Game
S C
““Fairness” between Fairness” between worlds worlds
bias against simple bias against simple theory.theory.
![Page 20: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/20.jpg)
ConvergenceConvergence
At least a simplicity bias At least a simplicity bias doesn’t doesn’t preventprevent convergenceconvergence to the truth to the truth (MDL, BIC, Bayes, SGS, etc.).(MDL, BIC, Bayes, SGS, etc.). Neither do Neither do otherother biases. biases. May as well recommend May as well recommend flat tiresflat tires since since
they can be fixed.they can be fixed.
OSI
MP L E
OCO
M P L EX
![Page 21: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/21.jpg)
Does Ockham Have No Does Ockham Have No Frock?Frock?
Philosopher’s stone,Philosopher’s stone,Perpetual motion,Perpetual motion,Free lunchFree lunch
Ockham’s Razor???Ockham’s Razor???
Ash Heap of Ash Heap of HistoryHistory
. . .
![Page 22: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/22.jpg)
II. How Ockham II. How Ockham Helps You Find Helps You Find
the Truth the Truth
![Page 23: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/23.jpg)
What is Guidance?What is Guidance? Indication or trackingIndication or tracking
Too Too strongstrong FixedFixed bias can’t bias can’t indicateindicate anything anything
Convergence Convergence Too Too weakweak True of other biasesTrue of other biases
““Straightest” convergenceStraightest” convergence Just right?Just right?
C
C
S
S
CS
![Page 24: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/24.jpg)
A True StoryA True StoryNiagara FallsNiagara Falls
PittsburghPittsburgh
ClarionClarion
![Page 25: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/25.jpg)
A True StoryA True StoryNiagara FallsNiagara Falls
PittsburghPittsburgh
ClarionClarion
![Page 26: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/26.jpg)
A True StoryA True StoryNiagara FallsNiagara Falls
PittsburghPittsburgh
ClarionClarion
![Page 27: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/27.jpg)
A True StoryA True StoryNiagara FallsNiagara Falls
PittsburghPittsburgh
ClarionClarion
!!
![Page 28: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/28.jpg)
ClarionClarion
A True StoryA True StoryNiagara FallsNiagara Falls
PittsburghPittsburgh
![Page 29: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/29.jpg)
A True StoryA True Story
??
![Page 30: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/30.jpg)
A True StoryA True Story
![Page 31: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/31.jpg)
A True StoryA True Story
Ask Ask directions!directions!
![Page 32: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/32.jpg)
A True StoryA True Story
Where’s …Where’s …
![Page 33: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/33.jpg)
What Does She Say?What Does She Say?Turn around. Turn around. The freeway The freeway ramp is on ramp is on the left.the left.
![Page 34: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/34.jpg)
You Have Better IdeasYou Have Better Ideas
Phooey!Phooey!The Sun was on the The Sun was on the right!right!
![Page 35: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/35.jpg)
You Have Better IdeasYou Have Better Ideas
!!!!
![Page 36: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/36.jpg)
You Have Better IdeasYou Have Better Ideas
![Page 37: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/37.jpg)
You Have Better IdeasYou Have Better Ideas
![Page 38: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/38.jpg)
You Have Better IdeasYou Have Better Ideas
![Page 39: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/39.jpg)
Stay the Course!Stay the Course!
AhemAhem……
![Page 40: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/40.jpg)
Stay the Course!Stay the Course!
![Page 41: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/41.jpg)
Stay the Course!Stay the Course!
![Page 42: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/42.jpg)
Don’t Flip-flop!Don’t Flip-flop!
![Page 43: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/43.jpg)
Don’t Flip-flop!Don’t Flip-flop!
![Page 44: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/44.jpg)
Don’t Flip-flop!Don’t Flip-flop!
![Page 45: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/45.jpg)
Then Again…Then Again…
![Page 46: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/46.jpg)
Then Again…Then Again…
![Page 47: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/47.jpg)
Then Again…Then Again…
@@$##!@@$##!
![Page 48: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/48.jpg)
One Good Flip One Good Flip Can Save a Lot of FlopCan Save a Lot of Flop
PittsburghPittsburgh
![Page 49: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/49.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 50: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/50.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 51: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/51.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 52: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/52.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 53: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/53.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 54: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/54.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 55: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/55.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
Told ya!Told ya!
![Page 56: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/56.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 57: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/57.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 58: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/58.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 59: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/59.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 60: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/60.jpg)
The U-TurnThe U-Turn
PittsburghPittsburgh
![Page 61: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/61.jpg)
Your RouteYour Route
PittsburghPittsburgh
Needless U-turnNeedless U-turn
![Page 62: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/62.jpg)
The Best RouteThe Best Route
PittsburghPittsburgh
Told ya!Told ya!
![Page 63: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/63.jpg)
The Best Route Anywhere The Best Route Anywhere from Therefrom There
PittsburghPittsburgh NY, DCNY, DC
Told ya!Told ya!
![Page 64: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/64.jpg)
The Freeway to the TruthThe Freeway to the Truth
ComplexComplex SimpleSimple
Fixed adviceFixed advice for all destinations for all destinations Disregarding it entails an Disregarding it entails an extra course extra course
reversalreversal……
Told ya!Told ya!
![Page 65: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/65.jpg)
The Freeway to the TruthThe Freeway to the Truth
ComplexComplex SimpleSimple
……even if the advice points even if the advice points away fromaway from the goal!the goal!
Told ya!Told ya!
![Page 66: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/66.jpg)
Counting MarblesCounting Marbles
![Page 67: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/67.jpg)
Counting MarblesCounting Marbles
![Page 68: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/68.jpg)
Counting MarblesCounting Marbles
May come at any time…May come at any time…
![Page 69: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/69.jpg)
Counting MarblesCounting Marbles
May come at any time…May come at any time…
![Page 70: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/70.jpg)
Counting MarblesCounting Marbles
May come at any time…May come at any time…
![Page 71: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/71.jpg)
Counting MarblesCounting Marbles
May come at any time…May come at any time…
![Page 72: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/72.jpg)
Counting MarblesCounting Marbles
May come at any time…May come at any time…
![Page 73: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/73.jpg)
Counting MarblesCounting Marbles
May come at any time…May come at any time…
![Page 74: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/74.jpg)
Counting MarblesCounting Marbles
May come at any time…May come at any time…
![Page 75: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/75.jpg)
Ockham’s RazorOckham’s Razor
3 ?
If you answerIf you answer, answer with the , answer with the current countcurrent count..
![Page 76: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/76.jpg)
AnalogyAnalogy MarblesMarbles = detectable “effects”. = detectable “effects”. Late appearanceLate appearance = difficulty of = difficulty of
detection.detection. CountCount = model (e.g., causal graph). = model (e.g., causal graph). Appearance timesAppearance times = free = free
parameters.parameters.
![Page 77: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/77.jpg)
AnalogyAnalogy
U-turnU-turn = model = model revisionrevision (with (with content loss)content loss)
HighwayHighway = revision-efficient = revision-efficient truth-finding method.truth-finding method.
TTTT
![Page 78: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/78.jpg)
The U-turn ArgumentThe U-turn Argument
Suppose you Suppose you converge to the truthconverge to the truth but but
violate Ockham’s razor violate Ockham’s razor along the way.along the way.
33
![Page 79: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/79.jpg)
The U-turn ArgumentThe U-turn Argument
Where is that extra marble, anyway?Where is that extra marble, anyway?
33
![Page 80: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/80.jpg)
The U-turn ArgumentThe U-turn Argument
It’s not coming, is it?It’s not coming, is it?
33
![Page 81: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/81.jpg)
The U-turn ArgumentThe U-turn Argument
If you never say 2 you’ll never If you never say 2 you’ll never converge to the truth….converge to the truth….
33
![Page 82: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/82.jpg)
The U-turn ArgumentThe U-turn Argument
That’s it. You should have listened to That’s it. You should have listened to Ockham.Ockham.
33222222
![Page 83: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/83.jpg)
The U-turn ArgumentThe U-turn Argument
Oops! Well, no method is infallible!Oops! Well, no method is infallible!
33222222
![Page 84: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/84.jpg)
The U-turn ArgumentThe U-turn Argument
If you never say 3, you’ll never If you never say 3, you’ll never converge to the truth….converge to the truth….
33222222
![Page 85: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/85.jpg)
The U-turn ArgumentThe U-turn Argument
Embarrassing to be back at that old Embarrassing to be back at that old theory, eh?theory, eh?
33222222 33
![Page 86: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/86.jpg)
The U-turn ArgumentThe U-turn Argument
And so forth…And so forth…
33222222 33 44
![Page 87: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/87.jpg)
The U-turn ArgumentThe U-turn Argument
And so forth…And so forth…
33222222 33 44 55
![Page 88: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/88.jpg)
The U-turn ArgumentThe U-turn Argument
And so forth…And so forth…
33222222 33 44 55 66
![Page 89: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/89.jpg)
The U-turn ArgumentThe U-turn Argument
And so forth…And so forth…
33222222 33 44 55 6677
![Page 90: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/90.jpg)
The ScoreThe Score
You: You:
33222222 33 44 55 6677
Subproblem
![Page 91: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/91.jpg)
The ScoreThe Score
Ockham:Ockham:
222222 33 44 55 6677
Subproblem
![Page 92: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/92.jpg)
Ockham is NecessaryOckham is Necessary
If you If you converge to the truthconverge to the truth, ,
and and you you violate Ockham’s razorviolate Ockham’s razor
then then some convergent method some convergent method beats beats
your worst-case revision boundyour worst-case revision bound in in each answer in the subproblem each answer in the subproblem entered at the time of the violation.entered at the time of the violation.
![Page 93: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/93.jpg)
Ockham is SufficientOckham is Sufficient
If you If you converge to the truthconverge to the truth, ,
and and you you never violate Ockham’s razornever violate Ockham’s razor
then then You You achieve the worst-case revision achieve the worst-case revision
boundbound of each convergent solution of each convergent solution in each answer in each in each answer in each subproblem.subproblem.
![Page 94: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/94.jpg)
EfficiencyEfficiency
EfficiencyEfficiency = achievement of the = achievement of the best worst-case revision bound in best worst-case revision bound in each answer in each subproblem. each answer in each subproblem.
![Page 95: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/95.jpg)
Ockham Efficiency Ockham Efficiency TheoremTheorem
Among the convergent methods…Among the convergent methods…
Ockham = Efficient!Ockham = Efficient!
Efficient
Inefficient
![Page 96: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/96.jpg)
““Mixed” StrategiesMixed” Strategies
mixed strategy mixed strategy = chance of output = chance of output depends only on actual experience.depends only on actual experience.
convergence in probabilityconvergence in probability = chance = chance of producing true answer of producing true answer approaches 1 in the limit.approaches 1 in the limit.
efficiencyefficiency = achievement of best = achievement of best worst-case worst-case expectedexpected revision bound revision bound in each answer in each subproblem. in each answer in each subproblem.
![Page 97: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/97.jpg)
Ockham Efficiency Ockham Efficiency TheoremTheorem Among the Among the mixedmixed methods that methods that
converge in probability…converge in probability…
Ockham = Efficient!Ockham = Efficient!
Efficient
Inefficient
![Page 98: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/98.jpg)
Dominance and Dominance and “Support”“Support”
EveryEvery convergent method is convergent method is weakly weakly dominateddominated in revisions by a clone in revisions by a clone who says “?” until stage who says “?” until stage nn..
1.1. Convergence Convergence Must leap Must leap eventually.eventually.
2.2. EfficiencyEfficiency Only leap to Only leap to simplest.simplest.
3.3. Dominance Dominance Could always Could always wait longer.wait longer. Can’t wait forever!
![Page 99: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/99.jpg)
III. Ockham on III. Ockham on Steroids Steroids
![Page 100: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/100.jpg)
Ockham Wish ListOckham Wish List General definitionGeneral definition of Ockham’s razor. of Ockham’s razor. Compare revisions even when Compare revisions even when not not
bounded within answersbounded within answers.. Prove theorem for Prove theorem for arbitrary empirical arbitrary empirical
problemsproblems..
![Page 101: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/101.jpg)
Empirical ProblemsEmpirical Problems Problem = Problem = partition of a topological partition of a topological
space.space. Potential answersPotential answers = partition cells. = partition cells. EvidenceEvidence = open (verifiable) = open (verifiable)
propositions.propositions.Example: SymmetryExample: Symmetry
![Page 102: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/102.jpg)
Example: Parameter Example: Parameter FreeingFreeing
Euclidean topology.Euclidean topology. Say which parameters are zero.Say which parameters are zero. Evidence = open neighborhood. Evidence = open neighborhood.
•Curve fitting
a2
a1
0
a1 = 0a2 = 0
a1 > 0a2 = 0
a1 > 0a2 > 0
a1 = 0a2 > 0
![Page 103: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/103.jpg)
The PlayersThe Players
Scientist:Scientist: Produces an answer in response to Produces an answer in response to
current evidence.current evidence. Demon:Demon:
Chooses evidence in response to Chooses evidence in response to scientist’s choicesscientist’s choices
![Page 104: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/104.jpg)
WinningWinning
Scientist wins…Scientist wins… by defaultby default if demon doesn’t present an if demon doesn’t present an
infinite nested sequence of basic open infinite nested sequence of basic open sets whose intersection is a singleton.sets whose intersection is a singleton.
else by meritelse by merit if scientist eventually if scientist eventually always produces the true answer for always produces the true answer for world selected by demon’s choices.world selected by demon’s choices.
![Page 105: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/105.jpg)
Comparing RevisionsComparing Revisions
One answer sequence One answer sequence maps into maps into another iff another iff there is an order and answer-preserving there is an order and answer-preserving
map from the first to the second (? is map from the first to the second (? is wild).wild).
Then the revisions of first are Then the revisions of first are as as good asgood as those of the second. those of the second.? ? ? ? ? . . .
. . .
![Page 106: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/106.jpg)
Comparing RevisionsComparing Revisions
The revisions of the first are The revisions of the first are strictly strictly betterbetter if, in addition, the latter if, in addition, the latter doesn’t map back into the former. doesn’t map back into the former.
? . . .
? ? ? ? ? . . .
![Page 107: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/107.jpg)
Comparing MethodsComparing Methods
F F is is as good asas good as G G iffiff
each output sequence of each output sequence of FF is as good is as good as some output sequence of as some output sequence of G.G.
F Gas good as
![Page 108: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/108.jpg)
Comparing MethodsComparing Methods
FF is is better thanbetter than GG iff iff
FF is as good as is as good as GG and and
GG is is not not as good as as good as FF
F Gnot as good as
![Page 109: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/109.jpg)
Comparing MethodsComparing Methods
FF is is strongly better thanstrongly better than GG iff each iff each output sequence of output sequence of FF is strictly is strictly better than an output sequence of better than an output sequence of GG but …but …
strictly better than
![Page 110: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/110.jpg)
Comparing MethodsComparing Methods
…… no output sequence of no output sequence of GG is as is as good as any of good as any of FF..
not as good as
![Page 111: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/111.jpg)
TerminologyTerminology
Efficient solution: Efficient solution: as good as any as good as any solution in any subproblem.solution in any subproblem.
![Page 112: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/112.jpg)
What Simplicity Isn’tWhat Simplicity Isn’t Syntactic length.Syntactic length. Data-compression (MDL).Data-compression (MDL). Computational ease.Computational ease. Social “entrenchment” (Goodman).Social “entrenchment” (Goodman). Number of free parameters (BIC, Number of free parameters (BIC,
AIC).AIC). Euclidean dimensionalityEuclidean dimensionality
Only by accident!!
Symptoms…
![Page 113: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/113.jpg)
What Simplicity IsWhat Simplicity Is SimplerSimpler theories are compatible theories are compatible
with with deeper problems of inductiondeeper problems of induction..
Worst demon
Smaller demon
![Page 114: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/114.jpg)
Problem of InductionProblem of Induction
No true information entails the true No true information entails the true answer.answer.
Happens in Happens in answer boundariesanswer boundaries..
![Page 115: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/115.jpg)
Demonic PathsDemonic PathsA A demonic pathdemonic path from from ww is a is a
sequence of alternating answers sequence of alternating answers that a demon can force an that a demon can force an arbitrary convergent method arbitrary convergent method through starting from through starting from ww. .
0…1…2…3…4
![Page 116: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/116.jpg)
Simplicity Simplicity DefinedDefinedThe The AA-sequences-sequences are the demonic are the demonic
sequences beginning with answer sequences beginning with answer AA..
AA is is as simpleas simple as as BB iff each iff each BB--sequence is as good as some sequence is as good as some AA--sequence. sequence. 2, 3
2, 3, 42, 3, 4, 5
<<<
33, 4
3, 4, 5.
. .
So 2 is simpler than 3!
![Page 117: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/117.jpg)
Ockham AnswerOckham Answer An answer as simple as any other An answer as simple as any other
answer.answer. = number of observed particles.= number of observed particles.
. .
.
So 2 is Ockham!
2, …, n2, …, n, n+12, …, n, n+1, n+2
<<<
nn, n+1
n, n+1, n+2
![Page 118: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/118.jpg)
Ockham LemmaOckham LemmaAA is Ockham iff is Ockham iff
for all demonic for all demonic p, p, ((AA**pp) ≤ some demonic sequence. ) ≤ some demonic sequence.
3
I can force you through
2but not
through 3,2.
So 3 isn’t Ockham
![Page 119: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/119.jpg)
Ockham AnswerOckham AnswerE.g.: Only simplest curve E.g.: Only simplest curve
compatible with data is Ockham.compatible with data is Ockham.
a2
a1
0
Demonic sequence:
Non-demonic sequences:
![Page 120: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/120.jpg)
General Efficiency General Efficiency TheoremTheorem
If the topology is metrizable and If the topology is metrizable and separable and the question is separable and the question is countable then: countable then:
Ockham = Efficient.Ockham = Efficient.
Proof: uses Martin’s Borel Proof: uses Martin’s Borel Determinacy theorem.Determinacy theorem.
![Page 121: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/121.jpg)
Stacked ProblemsStacked ProblemsThere is an Ockham answer at There is an Ockham answer at
every stage.every stage.
0
1
2
3
. . .
1
![Page 122: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/122.jpg)
Non-Ockham Non-Ockham Strongly Strongly WorseWorse
If the problem is a If the problem is a stackedstacked countable partition over a countable partition over a restricted Polish space:restricted Polish space:
Each Ockham solution is Each Ockham solution is stronglystrongly better than each non- better than each non-Ockham solution in the Ockham solution in the subproblem entered at the subproblem entered at the time of the violation.time of the violation.
![Page 123: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/123.jpg)
Simplicity Simplicity Low Low DimensionDimension
Suppose God says the true Suppose God says the true parameter value is rational.parameter value is rational.
![Page 124: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/124.jpg)
Simplicity Simplicity Low Low DimensionDimension
Topological dimension and Topological dimension and integration theory dissolve.integration theory dissolve.
Does Ockham?Does Ockham?
![Page 125: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/125.jpg)
Simplicity Simplicity Low Low DimensionDimension
The proposed account survives in The proposed account survives in the preserved limit point the preserved limit point structure.structure.
![Page 126: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/126.jpg)
IV. Ockham and IV. Ockham and Symmetry Symmetry
![Page 127: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/127.jpg)
Respect for SymmetryRespect for Symmetry If severalIf several simplest simplest alternatives alternatives
are available, are available, don’t break the don’t break the symmetry.symmetry.
•Count the marbles of each color.•You hear the first marble but don’t see it.•Why red rather than green?
?
![Page 128: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/128.jpg)
Respect for SymmetryRespect for Symmetry Before the noise, (Before the noise, (00, , 00) is ) is
Ockham.Ockham. After the noise, After the noise, no answer is no answer is
OckhamOckham::
((00, , 11))((11, , 00))
DemonicDemonic Non-demonicNon-demonic
((00, , 11) () (11, , 00))((11, , 00) () (00, , 11))
?
Right!
((00, , 00))
![Page 129: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/129.jpg)
Goodman’s RiddleGoodman’s Riddle Count Count oneiclesoneicles--- a oneicle is a particle at --- a oneicle is a particle at
any stage but one, when it is a non-any stage but one, when it is a non-particle. particle.
Oneicle tranlation is auto-Oneicle tranlation is auto-homeomorphism that homeomorphism that does not preserve does not preserve the problemthe problem. .
Unique Unique Ockham answer is current Ockham answer is current oneicle count.oneicle count.
Contradicts Contradicts unique Ockham answer in unique Ockham answer in particle counting. particle counting.
![Page 130: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/130.jpg)
SupersymmetrySupersymmetry Say Say whenwhen each particle appears. each particle appears. Refines Refines counting problem.counting problem. EveryEvery auto-homeomorphism auto-homeomorphism
preserves problem.preserves problem.
No No answer is Ockham.answer is Ockham. No No solution is Ockhamsolution is Ockham.. No No method is efficient.method is efficient.
![Page 131: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/131.jpg)
Dual SupersymmetryDual Supersymmetry Say only whether particle count is Say only whether particle count is even even
or oddor odd.. CoarsensCoarsens counting problem. counting problem. Particle/OneicleParticle/Oneicle auto-homeomorphism auto-homeomorphism
preserves problem.preserves problem.
Every Every answer is Ockham.answer is Ockham. Every Every solution is Ockhamsolution is Ockham.. Every Every solution is efficient.solution is efficient.
![Page 132: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/132.jpg)
Broken SymmetryBroken Symmetry CountCount the even or just report the even or just report oddodd.. CoarsensCoarsens counting problem. counting problem. RefinesRefines the even/odd problem. the even/odd problem. Unique Unique Ockham answer at each Ockham answer at each
stagestage.. Exactly Exactly Ockham solutions are Ockham solutions are
efficient.efficient.
![Page 133: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/133.jpg)
Simplicity Under Simplicity Under RefinementRefinement
Particle countingOneicle counting Twoicle counting
Particle countingor odd particles
Oneicle countingor odd oneicles
Twoicle countingor odd twoicles
Time of particle appearance
Even/odd
Broken symmetryUnique Ockham answer
SupersymmetryNo answer is Ockham
Dual supersymmetryBoth answers are Ockham
![Page 134: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/134.jpg)
Proposed Theory is RightProposed Theory is Right ObjectiveObjective efficiency is grounded in efficiency is grounded in
problems.problems. Symmetries in the preceding problems Symmetries in the preceding problems
would would wash outwash out stronger simplicity stronger simplicity distinctions. distinctions.
Hence, such distinctions would amount Hence, such distinctions would amount to mere to mere conventionsconventions (like coordinate (like coordinate axes) that couldn’t have anything to do axes) that couldn’t have anything to do with objective efficiency. with objective efficiency.
![Page 135: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/135.jpg)
Furthermore…Furthermore… If Ockham’s razor is forced to choose If Ockham’s razor is forced to choose
in the supersymmetrical problems in the supersymmetrical problems then either:then either:
following Ockham’s razor following Ockham’s razor increases increases revisions revisions in some counting problems in some counting problems OrOr
Ockham’s razor leads to Ockham’s razor leads to contradictionscontradictions as a problem is coarsened or refined. as a problem is coarsened or refined.
![Page 136: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/136.jpg)
V. Conclusion V. Conclusion
![Page 137: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/137.jpg)
What Ockham’s Razor IsWhat Ockham’s Razor Is
““Only output Ockham answers”Only output Ockham answers” Ockham answerOckham answer = a topological = a topological
invariant of the empirical problem invariant of the empirical problem addressed.addressed.
![Page 138: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/138.jpg)
What it Isn’tWhat it Isn’t
preference for preference for brevity, brevity, computational ease, computational ease, entrenchment, entrenchment, past success, past success, Kolmogorov complexity, Kolmogorov complexity, dimensionality, etc….dimensionality, etc….
![Page 139: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/139.jpg)
How it WorksHow it Works
Ockham’s razor is necessary for Ockham’s razor is necessary for mininizing revisionsmininizing revisions prior to prior to convergence to the truth.convergence to the truth.
![Page 140: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/140.jpg)
How it Doesn’tHow it Doesn’t
No possible methodNo possible method could: could: Point at the truth;Point at the truth; Indicate the truth;Indicate the truth; Bound the probability of error;Bound the probability of error; Bound the number of future revisions.Bound the number of future revisions.
![Page 141: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/141.jpg)
Spooky OckhamSpooky Ockham
Science Science without support or safety without support or safety netsnets..
![Page 142: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/142.jpg)
Spooky OckhamSpooky Ockham
Science Science without support or safety without support or safety netsnets..
![Page 143: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/143.jpg)
Spooky OckhamSpooky Ockham
Science Science without support or safety without support or safety netsnets..
![Page 144: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/144.jpg)
Spooky OckhamSpooky Ockham
Science Science without support or safety without support or safety netsnets..
![Page 145: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/145.jpg)
VI. Stochastic VI. Stochastic OckhamOckham
![Page 146: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/146.jpg)
““Mixed” StrategiesMixed” Strategies
mixed strategy mixed strategy = chance of = chance of output depends only on actual output depends only on actual experience.experience.
Pe(M = H at n) = Pe|n(M = H at n).
e
![Page 147: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/147.jpg)
Stochastic CaseStochastic Case
Ockham = Ockham =
at each stage, you produce a non-at each stage, you produce a non-Ockham answer with Ockham answer with prob = 0prob = 0. .
EfficiencyEfficiency = =
achievement of the best worst-case achievement of the best worst-case expected expected revision bound in each revision bound in each answer in each subproblem over all answer in each subproblem over all methods that converge to the truth methods that converge to the truth in probabilityin probability. .
![Page 148: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/148.jpg)
Stochastic Efficiency Stochastic Efficiency TheoremTheorem
Among the stochastic methods Among the stochastic methods that converge in probability, that converge in probability, Ockham = Efficient!Ockham = Efficient!
Efficient
Inefficient
![Page 149: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/149.jpg)
Stochastic MethodsStochastic Methods
Your chance of producing an answer is Your chance of producing an answer is a a function of observations made so farfunction of observations made so far. .
22
Urn selected in light of observations.
p
![Page 150: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/150.jpg)
Stochastic U-turn Stochastic U-turn ArgumentArgument
Suppose you Suppose you converge converge in probabilityin probability to the truthto the truth but produce a non-Ockham but produce a non-Ockham answer with prob > 0.answer with prob > 0.
33r >
![Page 151: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/151.jpg)
Choose small Choose small > 0. Consider answer > 0. Consider answer 4.4.
33
Stochastic U-turn Stochastic U-turn ArgumentArgument
r >
![Page 152: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/152.jpg)
Stochastic U-turn Stochastic U-turn ArgumentArgument
By By convergenceconvergence in probability in probability to the to the truth:truth:
3322p > 1 - /3
r >
![Page 153: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/153.jpg)
Stochastic U-turn Stochastic U-turn ArgumentArgument
Etc. Etc.
3322p> 1-/3
r >
33p > 1-/344p > 1-/3
![Page 154: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/154.jpg)
Stochastic U-turn Stochastic U-turn ArgumentArgument Since Since can be chosen arbitrarily small, can be chosen arbitrarily small,
sup prob of ≥ 3 revisions ≥ sup prob of ≥ 3 revisions ≥ rr.. sup prob of ≥ 2 revisions =1sup prob of ≥ 2 revisions =1
33r >
22p> 1-/333p > 1-/344p > 1-/3
![Page 155: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/155.jpg)
Stochastic U-turn Stochastic U-turn ArgumentArgument So sup Exp revisions is ≥ So sup Exp revisions is ≥ 2 + 32 + 3rr..
But for Ockham = But for Ockham = 22. .
33r >
Subproblem
22p> 1-/333p > 1-/344p > 1-/3
![Page 156: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/156.jpg)
VII. Statistical VII. Statistical InferenceInference
(Beta Version)
![Page 157: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/157.jpg)
The Statistical Puzzle of The Statistical Puzzle of SimplicitySimplicity
Assume:Assume: Normal distribution, Normal distribution, , , ≥ 0. ≥ 0. Question:Question: = 0 or = 0 or > 0 ?> 0 ? Intuition: Intuition: = 0 is simpler than = 0 is simpler than > 0 .> 0 .
= 0 mean
![Page 158: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/158.jpg)
AnalogyAnalogy
Marbles: Marbles: potentially small “effects”potentially small “effects” Time:Time: sample sizesample size Simplicity:Simplicity: fewer free parameters tied to fewer free parameters tied to
potential “effects”potential “effects” Counting:Counting: freeing parameters in a modelfreeing parameters in a model
![Page 159: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/159.jpg)
U-turn “in Probability”U-turn “in Probability” Convergence in probability:Convergence in probability: chance chance
of producing true model goes to unityof producing true model goes to unity
Retraction in probability:Retraction in probability: chance of producing chance of producing a model drops from above a model drops from above to below 1 – to below 1 –
Sample size
1
0
Chance of producingtrue model
Chance of producingalternative model
![Page 160: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/160.jpg)
= 0
sample mean
Suppose You (Probably) Suppose You (Probably) Choose a Model More Choose a Model More
Complex than the TruthComplex than the Truth
mean
zone for choosing > 0 > 0
Revision Counter: 0
> 0
![Page 161: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/161.jpg)
= 0
sample mean
Eventually You Retract to Eventually You Retract to the Truth (In Probability)the Truth (In Probability)
mean
zone for choosing = 0 = 0
Revision Counter: 1
![Page 162: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/162.jpg)
> 0
sample mean
So You (Probably) Output So You (Probably) Output an Overly Simple Model an Overly Simple Model
NearbyNearby
mean
zone for choosing = 0 = 0
Revision Counter: 1
![Page 163: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/163.jpg)
> 0
sample mean
mean
zone for choosing > 0 > 0
Eventually You Retract to Eventually You Retract to the Truth (In Probability)the Truth (In Probability)
Revision Counter: 2
![Page 164: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/164.jpg)
But Standard (Ockham) But Standard (Ockham) Testing Practice Requires Testing Practice Requires
Just One Retraction!Just One Retraction! = 0
sample mean
mean
zone for choosing = 0 = 0
Revision Counter: 0
![Page 165: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/165.jpg)
In The Simplest World, No In The Simplest World, No RetractionsRetractions
= 0
sample mean
mean
zone for choosing = 0 = 0
Revision Counter: 0
![Page 166: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/166.jpg)
= 0
sample mean
mean
zone for choosing = 0 = 0
Revision Counter: 0
In The Simplest World, No In The Simplest World, No RetractionsRetractions
![Page 167: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/167.jpg)
In Remaining Worlds, In Remaining Worlds, at Most One Retractionat Most One Retraction
> 0
mean
zone for choosing > 0 > 0
Revision Counter: 0
![Page 168: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/168.jpg)
> 0 mean
Revision Counter: 0
In Remaining Worlds, In Remaining Worlds, at Most One Retractionat Most One Retraction
zone for choosing > 0 > 0
![Page 169: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/169.jpg)
> 0 mean
Revision Counter: 1
In Remaining Worlds, In Remaining Worlds, at Most One Retractionat Most One Retraction
zone for choosing > 0 > 0
![Page 170: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/170.jpg)
So Ockham Beats All ViolatorsSo Ockham Beats All Violators Ockham:Ockham: at most one revision. at most one revision. Violator:Violator: at least two revisions in at least two revisions in
worst caseworst case
![Page 171: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/171.jpg)
SummarySummary Standard practice is to “test” the point Standard practice is to “test” the point
hypothesis rather than the composite hypothesis rather than the composite alternative.alternative.
This amounts to favoring the “simple” This amounts to favoring the “simple” hypothesis a priori.hypothesis a priori.
It also minimizes revisions in probability!It also minimizes revisions in probability!
![Page 172: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/172.jpg)
Two Dimensional Two Dimensional ExampleExample
Assume:Assume: independent bivariate normal independent bivariate normal distribution of unit variance.distribution of unit variance.
Question:Question: how manyhow many components of the components of the joint mean are zero?joint mean are zero?
Intuition: Intuition: more nonzeros = more more nonzeros = more complexcomplex
Puzzle:Puzzle: How does it help to favor How does it help to favor simplicity in less-than-simplest worlds?simplicity in less-than-simplest worlds?
![Page 173: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/173.jpg)
A Real Model Selection A Real Model Selection MethodMethod
Bayes Information Criterion (BIC)Bayes Information Criterion (BIC) BIC(BIC(MM, sample) = , sample) =
- log(max prob that - log(max prob that MM can assign to can assign to sample) + sample) +
+ log(sample size) + log(sample size) model complexity model complexity ½. ½.
BIC method: choose BIC method: choose MM with least with least BIC score.BIC score.
![Page 174: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/174.jpg)
Official BIC PropertyOfficial BIC Property
In the limit, minimizing BIC finds a In the limit, minimizing BIC finds a model with maximal conditional model with maximal conditional probability when the prior probability probability when the prior probability is flat over models and fairly flat over is flat over models and fairly flat over parameters within a model. parameters within a model.
But it is also revision-efficient.But it is also revision-efficient.
![Page 175: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/175.jpg)
AIC in Simplest WorldAIC in Simplest World
nn = 2 = 2 = (0, 0).= (0, 0).
Retractions Retractions = 0= 0
ComplexComplex
SimpleSimple
-2 -1 0 1 2 3
-2
-1
0
1
2
3
![Page 176: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/176.jpg)
AIC in Simplest WorldAIC in Simplest World
nn = 100 = 100 = (0, 0).= (0, 0).
Retractions Retractions = 0= 0
ComplexComplex
SimpleSimple
-0.4 -0.2 0 0.2 0.4
-0.4
-0.2
0
0.2
0.4
![Page 177: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/177.jpg)
AIC in Simplest WorldAIC in Simplest World
nn = 4,000,000 = 4,000,000 = (0, 0).= (0, 0).
Retractions Retractions = 0= 0
ComplexComplex
SimpleSimple
-0.004 -0.002 0 0.002 0.004
-0.004
-0.002
0
0.002
0.004
![Page 178: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/178.jpg)
BIC in Simplest WorldBIC in Simplest World
nn = 2 = 2 = (0, 0).= (0, 0).
Retractions Retractions = 0= 0
ComplexComplex
SimpleSimple
-2 -1 0 1 2 3
-2
-1
0
1
2
3
![Page 179: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/179.jpg)
BIC in Simplest WorldBIC in Simplest World
nn = 100 = 100 = (0, 0).= (0, 0).
Retractions Retractions = 0= 0
ComplexComplex
SimpleSimple
-0.75 -0.5 -0.25 0 0.25 0.5 0.75 1
-0.75
-0.5
-0.25
0
0.25
0.5
0.75
1
![Page 180: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/180.jpg)
BIC in Simplest WorldBIC in Simplest World
nn = 4,000,000 = 4,000,000 = (0, 0).= (0, 0).
Retractions Retractions = 0= 0
ComplexComplex
SimpleSimple
-0.0075-0.005-0.0025 0 0.0025 0.005 0.0075
-0.0075
-0.005
-0.0025
0
0.0025
0.005
0.0075
![Page 181: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/181.jpg)
BIC in Simplest WorldBIC in Simplest World
nn = 20,000,000 = 20,000,000 = (0, 0).= (0, 0).
Retractions Retractions = 0= 0
ComplexComplex
SimpleSimple
-0.006 -0.004 -0.002 0 0.002 0.004 0.006
-0.006
-0.004
-0.002
0
0.002
0.004
0.006
![Page 182: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/182.jpg)
Performance in Complex Performance in Complex WorldWorld
nn = 2 = 2 = (.05, .005).= (.05, .005).
Retractions Retractions = 0= 0
-2 -1 0 1 2 3
-2
-1
0
1
2
3
ComplexComplex
SimpleSimple
95%
![Page 183: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/183.jpg)
Performance in Complex Performance in Complex WorldWorld
nn = 100 = 100 = (.05, .005).= (.05, .005).
-0.75 -0.5 -0.25 0 0.25 0.5 0.75 1
-0.75
-0.5
-0.25
0
0.25
0.5
0.75
1
Retractions Retractions = 0= 0
ComplexComplex
SimpleSimple
![Page 184: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/184.jpg)
Performance in Complex Performance in Complex WorldWorld
nn = 30,000 = 30,000 = (.05, .005).= (.05, .005).
-0.1 -0.05 0 0.05 0.1
-0.1
-0.05
0
0.05
0.1 Retractions Retractions
= 1= 1
ComplexComplex
SimpleSimple
![Page 185: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/185.jpg)
Performance in Complex Performance in Complex WorldWorld
nn = 4,000,000 (!) = 4,000,000 (!) = (.05, .005).= (.05, .005).
-0.04 -0.02 0 0.02 0.04
-0.04
-0.02
0
0.02
0.04 Retractions Retractions = 2= 2
ComplexComplex
SimpleSimple
![Page 186: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/186.jpg)
QuestionQuestion Does the statistical retraction minimization Does the statistical retraction minimization
story extend to violations in less-than-simplest story extend to violations in less-than-simplest worlds?worlds?
Recall that the deterministic argument for Recall that the deterministic argument for higher retractions required the concept of higher retractions required the concept of minimizing retractions in each “subproblem”.minimizing retractions in each “subproblem”.
A “subproblem” is a proposition verified at a A “subproblem” is a proposition verified at a given time in a given world. given time in a given world.
Some analogue “in probability” is required. Some analogue “in probability” is required.
![Page 187: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/187.jpg)
Subproblem.Subproblem. HH is an is an -subroblem in -subroblem in ww at at nn: :
There is a likelihood ratio test of {There is a likelihood ratio test of {ww} at } at significance < significance < such that this test has power such that this test has power < 1 - < 1 - at each world in at each world in HH. .
w
sample size = n
H
reject
worlds
accept
reject
![Page 188: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/188.jpg)
Significance SchedulesSignificance Schedules A A significance schedulesignificance schedule ((..) is a monotone ) is a monotone
decreasing sequence of significance levels decreasing sequence of significance levels converging to zero that drop so slowly that converging to zero that drop so slowly that power can be increased monotonically with power can be increased monotonically with sample size.sample size.
aa((nn))aa((nn+1)+1)
nn+1+1
nn
![Page 189: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/189.jpg)
Ockham Violation Ockham Violation InefficientInefficient
(X, Y)Subproblem At sample size n
![Page 190: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/190.jpg)
Ockham Violation Ockham Violation InefficientInefficient
(X, Y)
Ockham violation:Probably say blue hypothesisat white world(p > )
Subproblem At sample size n
![Page 191: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/191.jpg)
Ockham Violation Ockham Violation InefficientInefficient
(X, Y)
Probably say blue
Probably say white
Subproblem at time of violation
![Page 192: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/192.jpg)
Ockham Violation Ockham Violation InefficientInefficient
(X, Y)
Probably say blue
Probably say white
Subproblem at time of violation
![Page 193: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/193.jpg)
Ockham Violation Ockham Violation InefficientInefficient
(X, Y)
Probably say blue
Probably say white
Probably say blue
Subproblem at time of violation
![Page 194: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/194.jpg)
Oops! Ockham Oops! Ockham InefficientInefficient
(X, Y) Subproblem
![Page 195: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/195.jpg)
Oops! Ockham Oops! Ockham InefficientInefficient
(X, Y) Subproblem
![Page 196: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/196.jpg)
Oops! Ockham Oops! Ockham InefficientInefficient
(X, Y) Subproblem
![Page 197: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/197.jpg)
Oops! Ockham Oops! Ockham InefficientInefficient
(X, Y) Subproblem
![Page 198: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/198.jpg)
Oops! Ockham Oops! Ockham InefficientInefficient
(X, Y)
Two retractions
Subproblem
![Page 199: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/199.jpg)
LocalLocal Retraction Retraction EfficiencyEfficiency
(X, Y) Subproblem
Ockham does as well as best subproblem Ockham does as well as best subproblem performance in performance in some neighborhoodsome neighborhood of of ww..
At most one retraction
Two retractions
![Page 200: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/200.jpg)
Ockham Violation Ockham Violation InefficientInefficient
(X, Y)Subproblem at time of violation
Note: Note: nono neighborhood around neighborhood around ww avoids extra retractions.avoids extra retractions.
w
![Page 201: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/201.jpg)
Gonzo Ockham Gonzo Ockham InefficientInefficient
(X, Y) Subproblem
Gonzo = probably saying simplest Gonzo = probably saying simplest answer in entire subproblem answer in entire subproblem entered in simplest world.entered in simplest world.
![Page 202: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/202.jpg)
BalanceBalance
Be Ockham (avoid complexity) Be Ockham (avoid complexity) Don’t be Gonzo Ockham (avoid Don’t be Gonzo Ockham (avoid
bad fit).bad fit).
Truth-directed:Truth-directed: sole aim is to find sole aim is to find true modeltrue model with minimal revisions! with minimal revisions!
No circles:No circles: totally worst-case; no totally worst-case; no prior bias toward simple worlds.prior bias toward simple worlds.
![Page 203: Ockham’s Razor: What it is, What it isn’t, How it works, and How it doesn’t](https://reader035.vdocuments.net/reader035/viewer/2022062517/568134b2550346895d9bceb2/html5/thumbnails/203.jpg)
THE ENDTHE END