![Page 1: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/1.jpg)
Ockham’s Razor Ockham’s Razor without without
Circles, Evasions, or MagicCircles, Evasions, or Magic
Kevin T. KellyKevin T. KellyDepartment of PhilosophyDepartment of Philosophy
Carnegie Mellon UniversityCarnegie Mellon Universitywww.cmu.eduwww.cmu.edu
![Page 2: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/2.jpg)
I. The Simplicity I. The Simplicity PuzzlePuzzle
![Page 3: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/3.jpg)
Which Explanation is Right?Which Explanation is Right?
???
![Page 4: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/4.jpg)
Ockham Says:Ockham Says:
Choose theSimplest!
![Page 5: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/5.jpg)
But Maybe…But Maybe…
Gotcha!
![Page 6: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/6.jpg)
The Simplicity PuzzleThe Simplicity Puzzle
An indicator must be An indicator must be sensitivesensitive to to what it indicates.what it indicates.
simple
![Page 7: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/7.jpg)
The Simplicity PuzzleThe Simplicity Puzzle
An indicator must be An indicator must be sensitivesensitive to to what it indicates.what it indicates.
complex
![Page 8: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/8.jpg)
The Simplicity PuzzleThe Simplicity Puzzle
But Ockham’s razor always points But Ockham’s razor always points at simplicity.at simplicity.
simple
![Page 9: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/9.jpg)
The Simplicity PuzzleThe Simplicity Puzzle
But Ockham’s razor always points But Ockham’s razor always points at simplicity.at simplicity.
complex
![Page 10: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/10.jpg)
A. Circular AccountsA. Circular Accounts
![Page 11: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/11.jpg)
Prior ProbabilityPrior Probability Assign high prior probability to Assign high prior probability to
simple theories.simple theories.
On the presumption of simplicity, presume simplicity.
![Page 12: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/12.jpg)
Phenomenon Phenomenon ee:: Venus appears to Venus appears to bob back and forth about the sun bob back and forth about the sun against the fixed stars.against the fixed stars.
Venus
Sun
Miracle ArgumentMiracle Argument
![Page 13: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/13.jpg)
Miracle ArgumentMiracle Argument
e e would would not be a miraclenot be a miracle given given S S;; ee would be a would be a miraclemiracle given given CC. .
C S
![Page 14: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/14.jpg)
Miracle ArgumentMiracle Argument
e e would would not be a miraclenot be a miracle given given S S;; ee would be a would be a miraclemiracle given given CC. .
C S
’
![Page 15: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/15.jpg)
However…However…
e e would would notnot be a miracle given be a miracle given CC((););
C S
Why not this?
![Page 16: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/16.jpg)
Bayesian ExplanationBayesian Explanation
Ignorance (over theories)Ignorance (over theories) + Ignorance (over parameter settings + Ignorance (over parameter settings
in each theory)in each theory) = Knowledge (against complex = Knowledge (against complex
parameter settings).parameter settings).Simpletheory
Complextheory
Bayesian
![Page 17: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/17.jpg)
= The Old Paradox of = The Old Paradox of IndifferenceIndifference
Bayesian
Ignorance (over blue, non-blue)Ignorance (over blue, non-blue) + Ignorance (over ways of being non-+ Ignorance (over ways of being non-
blue)blue) = Knowledge (that the truth is blue as = Knowledge (that the truth is blue as
opposed to any other color)opposed to any other color)Not-BlueBlue
![Page 18: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/18.jpg)
In Any Event…In Any Event…
The coherentist foundations of The coherentist foundations of Bayesianism have Bayesianism have nothing to do nothing to do with short-run truth-with short-run truth-conduciveness.conduciveness.
![Page 19: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/19.jpg)
B. Evasive AccountsB. Evasive Accounts
![Page 20: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/20.jpg)
Simple theories have virtues:Simple theories have virtues: TestableTestable UnifiedUnified ExplanatoryExplanatory SymmetricalSymmetrical BoldBold Compress dataCompress data
Theoretical VirtuesTheoretical Virtues
![Page 21: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/21.jpg)
Theoretical VirtuesTheoretical Virtues
Simple theories have virtues:Simple theories have virtues: TestableTestable UnifiedUnified ExplanatoryExplanatory SymmetricalSymmetrical BoldBold Compress dataCompress data
But to conclude that the virtuous But to conclude that the virtuous theory is theory is truetrue is is wishful thinking.wishful thinking.
Kelly Maximus
![Page 22: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/22.jpg)
Bayesian ConvergenceBayesian Convergence
Too-simple theories get shot down…Too-simple theories get shot down…
ComplexityTheories
Updated opinion
![Page 23: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/23.jpg)
Bayesian ConvergenceBayesian Convergence
Plausibility is transferred to the next-Plausibility is transferred to the next-simplest theory…simplest theory…
Blam! ComplexityTheories
Updated opinion
Plink!
![Page 24: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/24.jpg)
Bayesian ConvergenceBayesian Convergence
Plausibility is transferred to the next-Plausibility is transferred to the next-simplest theory…simplest theory…
Blam! ComplexityTheories
Updated opinion
Plink!
![Page 25: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/25.jpg)
Bayesian ConvergenceBayesian Convergence
Plausibility is transferred to the next-Plausibility is transferred to the next-simplest theory…simplest theory…
Blam! ComplexityTheories
Updated opinion
Plink!
![Page 26: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/26.jpg)
Bayesian ConvergenceBayesian Convergence
The true theory is nailed to the fence.The true theory is nailed to the fence.
Blam! ComplexityTheories
Updated opinion
Zing!
![Page 27: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/27.jpg)
Over-fittingOver-fitting Empirical estimatesEmpirical estimates based on based on
complex models have greater complex models have greater expected distance from the truth.expected distance from the truth.
Truth
Unconstrained aim at the true value
![Page 28: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/28.jpg)
Over-fittingOver-fitting Empirical estimatesEmpirical estimates based on based on
complexcomplex models have greater models have greater expected distance from the truth.expected distance from the truth.
Pop!
Unconstrained aim at the true value
![Page 29: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/29.jpg)
Clamped aim at the true value
OverfittingOverfitting Empirical estimatesEmpirical estimates based on based on
constrainedconstrained models can have lower models can have lower expected distance from the truth…expected distance from the truth…
![Page 30: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/30.jpg)
ConvergenceConvergence
But But alternative strategies alternative strategies also also converge: converge: Any finite variant of a convergent Any finite variant of a convergent
strategy convergesstrategy converges (Reichenbach, Salmon).(Reichenbach, Salmon).
![Page 31: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/31.jpg)
Over-fittingOver-fitting Empirical estimatesEmpirical estimates based on based on
constrainedconstrained models can have lower models can have lower expected distance from the truth…expected distance from the truth…
Pop!
Clamped aim at the true value
(Akaike, Vapnik, Sober and Forster, etc.)
![Page 32: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/32.jpg)
Over-fittingOver-fitting ...even if the simple theory is ...even if the simple theory is knownknown
to be falseto be false……
You missed!
Clamped aim at the true value
![Page 33: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/33.jpg)
Makes Sense…Makes Sense………when loss of an answer is similar in when loss of an answer is similar in
nearby distributions.nearby distributions.
Similarity ofProbability distributions
Closeis goodenough!
p
Loss
![Page 34: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/34.jpg)
But Truth Matters…But Truth Matters………when loss of an answer is when loss of an answer is
discontinuousdiscontinuous with similarity---e.g., in with similarity---e.g., in causalcausal prediction. prediction.
Not now!
pSimilarity ofProbability distributions
Loss
![Page 35: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/35.jpg)
C. Magical NaturalismC. Magical Naturalism
Truth
![Page 36: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/36.jpg)
MagicMagic SimplicitySimplicity indicates indicates truth via an truth via an unknown cause.unknown cause.
God
Simple B(Simple)
Simple B(Simple)
Simple B(Simple)
Leibniz
Kant
Ouija board
![Page 37: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/37.jpg)
MagicMagic Ockham says: Ockham says: explain Ockham’s razor explain Ockham’s razor
withoutwithout undetected causes. undetected causes.
Metaphysicians
for Ockham
![Page 38: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/38.jpg)
MagicMagic Ockham says:Ockham says: explain Ockham’s razor explain Ockham’s razor
withoutwithout undetected causes. undetected causes.
Metaphysicians
for Ockham
![Page 39: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/39.jpg)
II. A New DirectionII. A New Direction
![Page 40: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/40.jpg)
Ancient HintAncient Hint“Living in the midst of ignorance and considering themselves intelligent and enlightened, the senseless people go round and round, following crooked courses, just like the blind led by the blind.” Katha Upanishad, I. ii. 5, c. 600 BCE.
Truth
![Page 41: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/41.jpg)
Diagnosis of Standard Diagnosis of Standard AccountsAccounts
Truth
Truth
Short-run Reliability: Short-run Reliability: too strong.too strong.
Long-run ConvergenceLong-run Convergence: too weak.: too weak.
![Page 42: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/42.jpg)
Natural AlternativeNatural Alternative
Truth
Truth
Short-run Reliability: Short-run Reliability: too strong.too strong.
Long-run ConvergenceLong-run Convergence: too weak.: too weak.
Straightest Convergence: Straightest Convergence: just right?just right?
Truth
![Page 43: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/43.jpg)
III. Navigation by III. Navigation by Fixed DirectionsFixed Directions
complex
![Page 44: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/44.jpg)
Asking for DirectionsAsking for Directions
Where’s …Where’s …
![Page 45: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/45.jpg)
Asking for DirectionsAsking for Directions
Turn around. The freeway ramp is on the left.Turn around. The freeway ramp is on the left.
![Page 46: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/46.jpg)
Asking for DirectionsAsking for Directions
Goal
![Page 47: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/47.jpg)
Best RouteBest Route
Goal
![Page 48: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/48.jpg)
Best Route to Any GoalBest Route to Any Goal
![Page 49: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/49.jpg)
Disregarding Advice is Disregarding Advice is BadBad
Extra U-turn
![Page 50: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/50.jpg)
Best Route to Any GoalBest Route to Any Goal
…so fixed advice can help you reach a hidden goalwithout circles, evasions, or magic.
![Page 51: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/51.jpg)
Retraction = Epistemic Retraction = Epistemic U-turnU-turn
Choosing Choosing TT and then failing to and then failing to choose choose T T next.next.
T’T’
TT
??
![Page 52: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/52.jpg)
Curve FittingCurve Fitting Truth = some polynomial Truth = some polynomial YY = = ff((XX)) Data = increasingly precise open Data = increasingly precise open
intervals around intervals around YY at finitely many at finitely many specified values of specified values of X.X.
![Page 53: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/53.jpg)
Epistemic NestingEpistemic Nesting
An arbitrary amount of constant An arbitrary amount of constant data…data…
![Page 54: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/54.jpg)
Epistemic NestingEpistemic Nesting
An arbitrary amount of constant An arbitrary amount of constant data…data…
![Page 55: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/55.jpg)
Curve FittingCurve Fitting
Is compatible with a linear (non-Is compatible with a linear (non-constant) law. constant) law.
![Page 56: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/56.jpg)
Curve FittingCurve Fitting
An arbitrary amount of linear An arbitrary amount of linear data…data…
![Page 57: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/57.jpg)
Curve FittingCurve Fitting
……is compatible with a quadratic is compatible with a quadratic law, etc.law, etc.
![Page 58: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/58.jpg)
In Step with the DemonIn Step with the Demon
ConstantLinear
Quadratic
Cubic
There yet?Maybe.
![Page 59: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/59.jpg)
In Step with the DemonIn Step with the Demon
ConstantLinear
Quadratic
Cubic
There yet?Maybe.
![Page 60: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/60.jpg)
In Step with the DemonIn Step with the Demon
ConstantLinear
Quadratic
Cubic
There yet?Maybe.
![Page 61: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/61.jpg)
In Step with the DemonIn Step with the Demon
ConstantLinear
Quadratic
Cubic
There yet?Maybe.
![Page 62: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/62.jpg)
Ahead of the DemonAhead of the Demon
ConstantLinear
Quadratic
Cubic
There yet?Maybe.
![Page 63: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/63.jpg)
Ahead of the DemonAhead of the Demon
ConstantLinear
Quadratic
Cubic
I think you will lead me here!
![Page 64: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/64.jpg)
Pressure BuildsPressure Builds
ConstantLinear
Quadratic
Cubic
Maybe I will never move.
![Page 65: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/65.jpg)
RetractionRetraction
ConstantLinear
Quadratic
Cubic
You’re back!
![Page 66: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/66.jpg)
Another RetractionAnother Retraction
ConstantLinear
Quadratic
Cubic
I decided to move after all.
![Page 67: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/67.jpg)
Another RetractionAnother Retraction
ConstantLinear
Quadratic
Cubic
I decided to move again.
![Page 68: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/68.jpg)
Another RetractionAnother Retraction
ConstantLinear
Quadratic
Cubic
Déjà vu?
![Page 69: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/69.jpg)
Ockham PathOckham Path
ConstantLinear
Quadratic
Cubic
![Page 70: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/70.jpg)
Ockham Violator’s PathOckham Violator’s Path
ConstantLinear
Quadratic
Cubic
See, you shouldn’t have run ahead of meeven though you were right!
![Page 71: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/71.jpg)
Worst-case Retraction Worst-case Retraction Bounds for Ockham and Bounds for Ockham and
ViolatorViolator
ConstantLinearQuadratic Cubic
Extra first retraction
Worse performance Even in the complexityclass favored by theOckham violator
![Page 72: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/72.jpg)
Ockham Efficiency Ockham Efficiency Theorem:Theorem:
ConstantLinearQuadratic Cubic
Ockham’s razor is necessary for Ockham’s razor is necessary for retraction-efficiency.retraction-efficiency.
Best possible performance
Sub-optimal performance
![Page 73: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/73.jpg)
IV. Back-and-forth IV. Back-and-forth Ockham Efficiency Ockham Efficiency
TheoremTheorem
![Page 74: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/74.jpg)
Stalwartness PrincipleStalwartness Principle
After selecting an answer, never drop it while it After selecting an answer, never drop it while it remains uniquely simplest in light of the data.remains uniquely simplest in light of the data.
Violation yields extra retraction. Violation yields extra retraction.
![Page 75: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/75.jpg)
Overly-simple AnswersOverly-simple Answers An overly simple answer counts as an error in simplest worlds that Ockham does An overly simple answer counts as an error in simplest worlds that Ockham does
not commit.not commit. Consider total number of retractions and total number of errors as independent Consider total number of retractions and total number of errors as independent
costs.costs. Make only easy (Pareto) comparisons between vectors of form:Make only easy (Pareto) comparisons between vectors of form:
(total errors, total retractions).(total errors, total retractions).
![Page 76: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/76.jpg)
Intermediate ViolationsIntermediate Violations
When inputs When inputs ee have been received, compare have been received, compare the violator only to methods that have always the violator only to methods that have always done the same just prior to the end of done the same just prior to the end of ee. .
![Page 77: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/77.jpg)
Ockham Efficiency Ockham Efficiency RepresentationRepresentation
Theorem:Theorem: Among the convergent methods, the Among the convergent methods, the always stalwart and Ockham methods are always stalwart and Ockham methods are exactlyexactly the methods that are always jointly efficient in the methods that are always jointly efficient in terms of total errors and total retractions.terms of total errors and total retractions.
![Page 78: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/78.jpg)
IV. The Nature of IV. The Nature of Empirical SimplicityEmpirical Simplicity
![Page 79: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/79.jpg)
What Empirical What Empirical Simplicity Is NotSimplicity Is Not
Syntactic brevitySyntactic brevity Program lengthProgram length DimensionalityDimensionality Free parametersFree parameters Fewer causes or Fewer causes or
entitiesentities TestabilityTestability Prior probabilityPrior probability
![Page 80: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/80.jpg)
What Simplicity IsWhat Simplicity Is
ConstantLinear
Quadratic
Cubic
The The simplicitysimplicity of answer of answer TT given given ee in in question question QQ = =
the maximum number of retractions the maximum number of retractions nature can nature can forceforce from an arbitrary, from an arbitrary, convergent solution to a given convergent solution to a given problem, prior to convergence to problem, prior to convergence to TT after presenting after presenting ee. .
![Page 81: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/81.jpg)
MeritsMerits Depends on problem.Depends on problem. Grue-proof (topological Grue-proof (topological
invariant).invariant). Immediate epistemological Immediate epistemological
motive.motive. Non-trivial for discrete Non-trivial for discrete
parameter spaces.parameter spaces. Non-trivial in deterministic Non-trivial in deterministic
problems.problems. Usual accounts are special Usual accounts are special
cases under particular, cases under particular, materialmaterial assumptions. assumptions.
![Page 82: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/82.jpg)
Can be Done Much More Can be Done Much More GenerallyGenerally
Branching simplicity degrees.Branching simplicity degrees. Hypotheses that lose and Hypotheses that lose and
recover their status as uniquely recover their status as uniquely simplest.simplest.
Can be defined for sets of Can be defined for sets of sampling distributions.sampling distributions.
![Page 83: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/83.jpg)
Ockham Efficiency Ockham Efficiency RepresentationRepresentation
Theorem holds for a broad Theorem holds for a broad range of problems under the range of problems under the general simplicity definition.general simplicity definition.
![Page 84: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/84.jpg)
IV. Application: IV. Application: Non-experimental Non-experimental
Policy AnalysisPolicy Analysis
![Page 85: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/85.jpg)
Predictive LinksPredictive Links
Correlation or co-dependency allows Correlation or co-dependency allows one to one to predictpredict YY from from XX. .
Ash traysLu
ng
can
cer
Ash traysLinked toLung cancer!
scientistpolicy maker
![Page 86: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/86.jpg)
PolicyPolicy
Policy Policy manipulatesmanipulates XX to achieve a to achieve a change in change in YY..
Ash traysLu
ng
can
cer
Prohibit ash trays!
Ash traysLinked toLung cancer!
![Page 87: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/87.jpg)
PolicyPolicy
Policy Policy manipulatesmanipulates XX to achieve a to achieve a change in change in YY..
Ash traysLu
ng
can
cer
We failed!
![Page 88: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/88.jpg)
Correlation is not Correlation is not CausationCausation
Manipulation of Manipulation of XX can can destroydestroy the the correlation of correlation of XX with with YY..
Ash traysLu
ng
can
cer
We failed!
![Page 89: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/89.jpg)
Standard RemedyStandard Remedy
Randomized controlled studyRandomized controlled study
Ash traysLu
ng
can
cer
That’s what happensif you carry out thepolicy.
![Page 90: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/90.jpg)
Experimental InfeasibilityExperimental InfeasibilityExpenseExpenseMoralityMorality
Lead
IQ
Let me force a few thousand childrento eat lead.
![Page 91: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/91.jpg)
Experimental InfeasibilityExperimental InfeasibilityExpenseExpenseMoralityMorality
Lead
IQ
Just joking!
![Page 92: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/92.jpg)
Ironic AllianceIronic Alliance
Lead
IQ
Ha! You will never prove thatlead affects IQ…
![Page 93: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/93.jpg)
Ironic AllianceIronic Alliance
Lead
IQ
And you can’t eliminatejobs on a mere whim.
![Page 94: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/94.jpg)
Lead
IQ
So I will keep on polluting, which will never settle the matter because it is not a randomized trial.
Ironic AllianceIronic Alliance
![Page 95: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/95.jpg)
Remedy: Causes from Remedy: Causes from CorrelationsCorrelations
Protein A
Protein B
Protein C Cancer protein
PatternsPatterns of conditional correlation of conditional correlation cancan imply unambiguous causal imply unambiguous causal conclusions conclusions (Pearl, Spirtes, Glymour, Scheines, etc.)(Pearl, Spirtes, Glymour, Scheines, etc.)
Eliminate protein C!
![Page 96: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/96.jpg)
Basic IdeaBasic Idea
CausationCausation is a directed, acyclic is a directed, acyclic network over variables.network over variables.
What makes a network What makes a network causalcausal is a is a relation of relation of compatibilitycompatibility between between networks and joint probability networks and joint probability distributions.distributions.X
YZ
X
YZ
compatibility
![Page 97: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/97.jpg)
Joint distribution Joint distribution pp is is compatible compatible with with directed, acyclic network directed, acyclic network GG iff: iff:
Causal Markov Condition: Causal Markov Condition: each variable each variable XX is independent of its non-effects given is independent of its non-effects given its immediate causes.its immediate causes.
Faithfulness Condition: Faithfulness Condition: every conditional every conditional independence relation that holds in independence relation that holds in pp is a is a consequence of the Causal Markov Cond. consequence of the Causal Markov Cond.
CompatibilityCompatibility
Y Z
X
W
V
V
![Page 98: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/98.jpg)
B C
Common CauseCommon Cause
•B yields info about C (Faithfulness);•B yields no further info about C given A (Markov).
A
A
B C
![Page 99: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/99.jpg)
Causal ChainCausal Chain
•B yields info about C (Faithfulness);•B yields no further info about C given A (Markov).
B
A
C
A
B
C
![Page 100: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/100.jpg)
Common EffectCommon Effect
•B yields no info about C (Markov);•B yields extra info about C given A (Faithfulness).
A
B C
A
B C
![Page 101: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/101.jpg)
DistinguishabilityDistinguishability
A
B CA
B
C
A
C
B
A
B C
indistinguishable distinctive
![Page 102: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/102.jpg)
Causation from CorrelationCausation from Correlation
Protein A
Protein B
Protein C Cancer protein
The following network is causally The following network is causally unambiguous if all variables are unambiguous if all variables are observed.observed.
![Page 103: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/103.jpg)
Causation from Causation from CorrelationCorrelation
Protein A
Protein B
Protein C Cancer protein
The red arrow is also The red arrow is also immune to immune to confounding by unobserved common confounding by unobserved common causes.causes.
![Page 104: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/104.jpg)
Brave New World for Brave New World for PolicyPolicy
Protein A
Protein B
Protein C Cancer protein
Experimental Experimental (un-confounded) (un-confounded) conclusions from conclusions from non-experimental non-experimental data!data!
Eliminate protein C!
![Page 105: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/105.jpg)
V. The CatchV. The Catch
![Page 106: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/106.jpg)
Problem of InductionProblem of Induction Causal structure depends on Causal structure depends on
statistical dependency statistical dependency discriminations that might be discriminations that might be arbitrarily subtle.arbitrarily subtle.
independence
dependence
data
![Page 107: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/107.jpg)
Ockham’s Razor is Ockham’s Razor is CrucialCrucial
Ockham’s razorOckham’s razor: assume no more : assume no more causal connections than necessary.causal connections than necessary.
![Page 108: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/108.jpg)
Causal RetractionsCausal Retractions No guaranteeNo guarantee that small that small
dependencies will not be detected dependencies will not be detected later.later.
Can have Can have spectacular impactspectacular impact on on prior causal conclusions.prior causal conclusions.
![Page 109: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/109.jpg)
Current Policy AnalysisCurrent Policy Analysis
Protein A
Protein B
Protein C Cancer protein
Eliminate protein C!
Protein A
Protein B
Protein C Cancer protein
![Page 110: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/110.jpg)
As Sample Size As Sample Size Increases…Increases…
Rescind that order!
Protein A
Protein B
Protein C Cancer protein
Protein D
![Page 111: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/111.jpg)
As Sample Size Increases As Sample Size Increases Again…Again…
Protein A
Protein B
Protein C Cancer protein
Protein D
Protein E
Etc.
Eliminate protein C again!
![Page 112: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/112.jpg)
Causal Flipping TheoremCausal Flipping Theorem Unamabiguous causal conclusions in Unamabiguous causal conclusions in
a linear causal model can be forced a linear causal model can be forced by nature to flip any finite number of by nature to flip any finite number of times as the sample size increases.times as the sample size increases.
![Page 113: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/113.jpg)
Extremist ReactionExtremist Reaction Since causal discovery cannot lead Since causal discovery cannot lead
straight to the truth, it is straight to the truth, it is not not justified.justified.
I must remain silent.Therefore, I
win.
![Page 114: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/114.jpg)
Moderate ReactionModerate Reaction
““Many explanations have been offered Many explanations have been offered to make sense of the here-today-gone-to make sense of the here-today-gone-tomorrow nature of medical wisdom — tomorrow nature of medical wisdom — what we are advised with confidence what we are advised with confidence one year is reversed the nextone year is reversed the next — but the — but the simplest one is that it is simplest one is that it is the natural the natural rhythm of sciencerhythm of science.” .”
((Do We Really Know What Makes us HealthyDo We Really Know What Makes us Healthy, NY , NY
Times Magazine, Sept. 16, 2007).Times Magazine, Sept. 16, 2007).
![Page 115: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/115.jpg)
Skepticism InvertedSkepticism Inverted
Unavoidable Unavoidable retractions are justified retractions are justified because they are unavoidable.because they are unavoidable.
Avoidable retractions Avoidable retractions are not are not justified because they are avoidable.justified because they are avoidable.
So the So the best possible methodsbest possible methods for for causal discovery are those that causal discovery are those that minimize causal retractionsminimize causal retractions..
The best possible means for finding The best possible means for finding the truth are the truth are justifiedjustified..
![Page 116: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/116.jpg)
New DirectionsNew Directions Extension of unique efficiency theorem Extension of unique efficiency theorem
to mixed strategies, stochastic model to mixed strategies, stochastic model selection and numerical computations.selection and numerical computations.
Latent variables as Ockham Latent variables as Ockham conclusions.conclusions.
Degrees of retraction.Degrees of retraction. Ockham pooling of marginal Ockham Ockham pooling of marginal Ockham
conclusions.conclusions. Retraction efficiency assessment of Retraction efficiency assessment of
standard model selection methods.standard model selection methods.
![Page 117: Ockham’s Razor without Circles, Evasions, or Magic](https://reader035.vdocuments.net/reader035/viewer/2022062408/56813ff8550346895dab2620/html5/thumbnails/117.jpg)
Some ReadingSome Reading "Ockham’s Razor, Truth, and Information", in , in
Handbook of the Philosophy of InformationHandbook of the Philosophy of Information, , forthcoming, J. van Benthem and P. Adriaans, forthcoming, J. van Benthem and P. Adriaans, eds. eds.
"Ockham’s Razor, Empirical Complexity, and Truth-finding Efficiency", , Theoretical Computer ScienceTheoretical Computer Science, 383: 270-289, , 383: 270-289, 2007. 2007.
““A New Solution to the Puzzle of A New Solution to the Puzzle of Simplicity”,Simplicity”, forthcoming, forthcoming, Philosophy of SciencePhilosophy of Science..
Pre-prints at:Pre-prints at:
www.hss.cmu.edu/philosophy/faculty-kelly.phpwww.hss.cmu.edu/philosophy/faculty-kelly.php