george & foster (2000) “calibration and empirical … · seminar series in quantitative...
TRANSCRIPT
![Page 1: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/1.jpg)
Seminar Series in Quantitative Studies in Consumer Behavior
11 April 2006
George & Foster (2000)George & Foster (2000)
““Calibration and Empirical BayesCalibration and Empirical BayesVariable SelectionVariable Selection””
Discussion of
![Page 2: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/2.jpg)
11 April 2006 2
Overview
Calibration Empirical Bayes
Bayes
Cp / AIC
BIC
RIC
Data based methods for prior specification
![Page 3: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/3.jpg)
11 April 2006 3
Common Approaches
Choose model that maximizes:
dimension of model γ
“dimension penalty”
![Page 4: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/4.jpg)
11 April 2006 4
Common Approaches
F Criterion
2Cp
(AIC)Mallows (1973)Akaike (1973)
log n BIC Schwartz (1978)
2 log p RIC Foster & George (1994)
![Page 5: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/5.jpg)
11 April 2006 5
Bayes Approach
Model Space Posterior
Marginal Likelihood
Prior Distributions
Calibrate a particular class of priors to F
![Page 6: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/6.jpg)
11 April 2006 6
Priors
Bernoulli / g-priorHyperparameters
Note:
c controls the expected size of nonzero coefficients
w controls the number of nonzero coefficients
![Page 7: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/7.jpg)
11 April 2006 7
g-priors are…
…conditionally compatible
…a limiting case of a latent factor regression model (West, 2001)
…computationally tractable
...can be problematic for model selection (Jeffreys, 1961; Berger & Pericchi, 2001):
![Page 8: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/8.jpg)
11 April 2006 8
Calibration
![Page 9: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/9.jpg)
11 April 2006 9
F(c,w)
0
-5
-10
5
10
very negative
very positive
![Page 10: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/10.jpg)
11 April 2006 10
Calibration
F(c,w) Criterion c w
2Cp
(AIC)3.92 0.5
0.5
0.5
log n BIC n
2 log p RIC p2
![Page 11: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/11.jpg)
11 April 2006 11
Empirical Bayes Estimation of c and w
• “Default” choices can bias posterior away from “true” underlying model
• Fully Bayes approach: prior p(c,w)
• Stochastic search methods only visit a small area of the model space
![Page 12: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/12.jpg)
11 April 2006 12
Note:
Maximize (numerically) to get
Not computationally feasible for large p
Empirical Bayes Estimation of c and w
Marginal Likelihood
![Page 13: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/13.jpg)
11 April 2006 13
Note:
Only p distinct terms - maximization is (computationally) feasible
MAP model maximizes
Empirical Bayes Estimation of c and w
Orthogonal Design
![Page 14: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/14.jpg)
11 April 2006 14
Conditional EBayes Approach
conditional on a model γ
maximize rather than marginalize:
Note:
Under the “usual” estimate
![Page 15: George & Foster (2000) “Calibration and Empirical … · Seminar Series in Quantitative Studies in Consumer Behavior 11 April 2006 George & Foster (2000) “Calibration and Empirical](https://reader031.vdocuments.net/reader031/viewer/2022022617/5ba92e1609d3f2fb228c2a11/html5/thumbnails/15.jpg)
11 April 2006 15
Conditional EBayes Approach
Notes:
Computation easy, but still have to search model space
When p(y , γ | c, w) dominated by single model, CMML and CCML are similar