learning from the experience of others - stanford...

29

1 Learning from the Experience of Others Bradley Efron Stanford University

Upload: others

Post on 23-May-2020

12 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

1

Learning from theExperience of Others

Bradley EfronStanford University

Page 2: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

2

What is “Statistics”?

Page 3: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

3

Page 4: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

4

The Puzzled Physicist

Page 5: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

5

Bayes’ Rule (1763)

Page 6: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

6

Learning from Experience

Page 7: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

7

Page 8: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

8

Eighteen Baseball Players

0.265.265.265Grand Average0.239.200.1567/4518. Alvis0.244.316.1788/4517. Munson0.249.286.2009/4516. Campaneris0.254.226.22210/4515. E Rodriguez0.254.264.22210/4514. Petrocelli

:::::0.277.222.33315/454. Johnstone0.281.276.35616/453. F Howard0.286.298.37817/452. F Robinson0.290.346.40018/451. Clemente

James-Stein“TRUTH”Avehits/ABNameObserved

Page 9: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

9

Page 10: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

10

Regression to the Mean (Galton 1886)

Page 11: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

11

James-Stein Estimation (1956)

18

Page 12: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

12

Stein’s Paradox

Page 13: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

13

Corbet’s Butterflies(Malaysia 1943)

Page 14: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

14

Learning from Other Butterflies

Page 15: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

15

Proving the Magic Formula

Page 16: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

16

Shakespeare’s Word Counts

837 …104314632292434314376?

6 …543210

Page 17: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

17

Shakespeare’s Missing Words

Page 18: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

18

Gene 4124: Prostate Cancer Study

Page 19: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

19

Page 20: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

20

Hypothesis Test

Page 21: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

21

Page 22: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

22

Prostate Cancer Study

Page 23: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

23

Doing 6033 Hypothesis Tests at Once

Page 24: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

24

Page 25: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

25

False Discovery Rates(Benjamini and Hochberg 1995)

Page 26: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

26

Page 27: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

27

False Discovery Control Algorithm

Page 28: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

28

Learning from the Other z-Values

Page 29: Learning from the Experience of Others - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2008Learning.pdf · Learning from the Experience of Others Bradley Efron Stanford

29

References

STANFORD GEOTHERMAL PROGRAM STANFORD UNIVERSITY

statweb.stanford.edustatweb.stanford.edu/~ckirby/brad/talks/2007SimInference.pdf · Guo et al. (2006) “Rat toxicogenomic study reveals analytical consistency across microarray platforms”

Stanford Universitystatweb.stanford.edu/~ckirby/fileshare/Mannshardt_WWEParticle.pdf · Delhi, who said, 'You have an interesting view, you understand India.' He offered me a job,

Stanford BaSketBall - Stanford University

Conversations with Carl Morris - Stanford Universityfinmath.stanford.edu/~ckirby/brad/talks/2015Three2ndThoughts.pdfage sex bmi map tc ldl hdl tch ltg glu prog 59 1 32.1 101 157 93.2

Generalized Linear Models - Stanford Universitystatweb.stanford.edu/~ckirby/brad/STATS305B_Part-3_corrected-2.pdf · Normal theory linear regression, including the analysis of variance,

Stanford Universitychadj/HallJonesQJE.pdf · Title: Why Do Some Countries Produce So Much More Output Per Worker Than Others? Created Date: 20171010172130Z

Model Selection, Estimation, and Bootstrap Smoothingstatweb.stanford.edu/~ckirby/brad/talks/2012ModelSelection.pdf · Model Selection, Estimation, and Bootstrap Smoothing Bradley

Helping Others - Stanford House HK · for others makes people feel good. And helping others can make their lives better. You are about to meet five fantastic kids who feel good a

Log Forwarder for Windows - Stanford Universityweb.stanford.edu/group/storage/others/LogForwarder.pdf · 2013-07-16 · Forwarder for Windows Program group accessible from the Windows

Pragmatics and Discourse - Stanford Universityweb.stanford.edu/~clark/1990s/Clark, H.H. _ Bly, B. _Pragmatics and... · Pragmatics and Discourse ... and others are worked out only

Introduction - Stanford Universitystatweb.stanford.edu/~ckirby/brad/STATS305B_Part-1_corrected-2.pdfIntroduction The inner circle in Figure 1 represents normal theory, the preferred

Model selection, estimation, and bootstrap smoothing - …statweb.stanford.edu/~ckirby/brad/papers/2012ModelSelection.pdf · Model selection, estimation, and bootstrap smoothing Bradley

Stanford Sustainability Year in Reviewsustainable.stanford.edu/.../files/documents/Stanford_year_in_review.pdf · are big initiatives , others are small. Some programs are for long

STANFORD GEOTHERMAL PROGRAM STANFORD ......STANFORD GEOTHERMAL PROGRAM STANFORD UNIVERSITY STANFORD, CALIFORNIA 94305 SGP-TR-35 SECOND ANNUAL REPORT TO U.S. DEPARTMENT …

Stanford Health Library - Stanford Medicine | Stanford

FindLaw | Indictment of Stanford Financial Group Executives, Others

Home | Stanford HAIHome | Stanford HAI

Reconcilable differences. - Stanford Universitystatweb.stanford.edu/~ckirby/olkin/Reconcilable Differences_The... · Gleaning Insight from Conflicting Scientific Studies by INGRAM

Peer-to-peer archival data trading Brian Cooper Joint work with Hector Garcia-Molina (and others) Stanford University

CURVATURE AND INFERENCE FOR MAXIMUM LIKELIHOOD ESTIMATESstatweb.stanford.edu/~ckirby/brad/papers/2016CurvatureInferenceMLEs.pdf · CURVATURE AND INFERENCE FOR MAXIMUM LIKELIHOOD ESTIMATES

DesignandAnalysisofClinicalTrials inDrugDevelopmentstatweb.stanford.edu/~ckirby/lai/pubs/2010_ClinicalTrials.pdf · PD&PK Pharmacodynamics(PD):thestudyofdrugeﬀectsandtheir mechanismsofaction

Large-Scale Inference - Stanford Universitystatweb.stanford.edu/~ckirby/brad/LSI/monograph_CUP.pdf · 2010. 3. 29. · 1 Empirical Bayes and the James–Stein Estimator 1 1.1 Bayes

Brad Efron - Stanford Universitystatweb.stanford.edu/~ckirby/brad/talks/2007BayesEBFreq.pdf · Foundations of Statistics Brad Efron h y p e r p a r a m e t e r d e n s i t y d e n

THE ECOLOGY OF DAM REMOVAL - Amazon S3s3.amazonaws.com/american-rivers-website/wp... · 3 Junk and others 1989, Heiler and others 1995, Stanford and others 1996 4 Poff and others

Bayesian Inference and the Parametric Bootstrapstatweb.stanford.edu/~ckirby/brad/papers/2011BootstrapMCMC.pdfBayesian Inference and the Parametric Bootstrap Bradley Efron Stanford

A Conversation with Tze Leung Lai - Stanford Universitystatweb.stanford.edu/~ckirby/lai/pubs/StatistSci...2019/12/04 · A CONVERSATION WITH TZE LEUNG LAI 3 Fig 1. Tze at age 1.5

Stanford Universityjurafsky/labphon.pdf · Stanford University

Bayesian inference and the parametric bootstrapstatweb.stanford.edu/~ckirby/brad/papers/2012Bayesian...Bayesian inference and the parametric bootstrap Bradley Efron Stanford University

Decision Dynamics and Decision States in the Leaky Competing Accumulator Model Jay McClelland Stanford University With Juan Gao, Marius Usher and others

Large-Scale Inference - Stanford Universitystatweb.stanford.edu/~ckirby/brad/LSI/monograph_CUP_edited.pdf · 2 Large-Scale Hypothesis Testing 15 2.1 A Microarray Example 15 2.2 Bayesian

The Automatic Construction of Bootstrap Confidence Intervalsstatweb.stanford.edu/~ckirby/brad/talks/2018Automatic-Constructio… · Bootstrap Confidence Intervals bca algorithm makes

Voting to Tell Others - Econometrics Laboratory, UC Berkeleysdellavi/wp/turnout16-05-23.pdf · Voting to Tell Others ... Stanford, Toulouse, UCSD, and at the 2011 ESA World Congress

CS345A: Data Mining on the Web - Stanford Universityinfolab.stanford.edu/~ullman/mining/2009/introduction.pdfE.g., Netflix Challenge . Others have dealt with engineering solutions

Statistical Modeling: Applications and Recent Advancesstatweb.stanford.edu/~ckirby/lai/pubs/2009_StatModeling.pdf · Statistical Modeling: Applications ... design and analysis Statistical