a bayesian hierarchical model of pitch framing in major...
TRANSCRIPT
A Bayesian Hierarchical Model of Pitch Framing inMajor League Baseball
Sameer K. Deshpande and Abraham J. WynerStatistics Department, The Wharton School
University of Pennsylvania
1 August 2016
Introduction
• Framing: ability of a catcher to affect likelihood a taken pitch iscalled a strike
• Most estimates: top framers can save ∼ 20 – 25 runs per season ... 2- 2.5 wins above replacement!
• ESPN The Magazine: “If you have confidence in the additional 2WAR that framing would have given Lucroy, his 2014 season wouldhave been worth about $56M dollars on free agent market thisoffseason”
Deshpande & Wyner Catcher Framing JSM 2016 2 / 16
Overview
• Hierarchical Bayesian logistic regression model of called strikeprobability
• Estimate of each catcher’s effect on each umpire over and abovefactors like pitch location, count, and other pitch participants
• Translate these effects to estimates of the impact framing (i.e. runssaved) has on the game, along with natural uncertainty quantification
We use PITCHf/x data from 2011 – 2015
• Horizontal and vertical coordinates of pitch as it crosses home plate
• Main focus is on 2014 season: 300k called pitches
Deshpande & Wyner Catcher Framing JSM 2016 3 / 16
Model
Let y = 1 for called strike, y = 0 for ball. For umpire u:
log
(P(y = 1)
P(y = 0)
)= θu0 + θub + θuc + θup + θucount + f u(x , z)
where
• θub , θuc , θ
up : partial effect of batter b, catcher c and pitcher p on
umpire u’s log-odds of calling strike
• θucount : partial effect of count on umpire u
• f u(x , z): function of pitch location
• θu0 : intercept
Deshpande & Wyner Catcher Framing JSM 2016 4 / 16
Incorporating Pitch Location
Direct Parametrization:
• Each coordinate as linear predictor: f u(x , z) = θux x + θuz z
• Polar coordinates: f u(x , z) = θur r(x , z) + θuφφ
Indirect Parametrization:
1. Fit a Generalized Additive Model of called strike probability assmooth function of location:
I Uses data from 2011 – 2013I Separate GAM for each combination of batter and pitcher handedness
2. Use forecasted log-odds of called strike a linear predictor in model:
f u(x , z) = θulpˆlog-odds
Deshpande & Wyner Catcher Framing JSM 2016 5 / 16
Hierarchical Model
For each of the 93 umpires u1, . . . , u93
log
(P(yui = 1)
P(yui = 0)
)= xu>i Θu
Θu1 , . . . ,Θu93 |µ ∼ N(µ, σ2I
)µ ∼ N
(0, τ2I
)• σ = 1: less than 0.3% chance that one umpire would call the same
pitch strike 99% of time and other umpire calls strike 1% of time
• τ = 0.5: replacing player by baseline player unlikely to change calledstrike probability from 75% to 25%
• Model fit using Stan
Deshpande & Wyner Catcher Framing JSM 2016 6 / 16
Posterior Densities of Player Effects
(a) Hank Conger (b) Jonathan Lucroy
Deshpande & Wyner Catcher Framing JSM 2016 7 / 16
Impact of Framing
For each catcher, look at all of the called pitches he received:
• p̂: fitted probability of strike
• p̂0: fitted probability of strike with catcher replaced by baselinecatcher
• p̂ − p̂0: catcher’s “framing effect”
• Sum ρ× (p̂ − p̂0) over all called pitches received
Value of a called strike, ρ, depends on the count:
Deshpande & Wyner Catcher Framing JSM 2016 8 / 16
Value of a called strike on an 0 – 1 pitch
Between 2011 and 2014:
• 182,405 0 – 1 pitches taken: 140,667 balls, 41,738 called strikes
• Avg. # runs allowed in rest of inning after called ball: 0.322
• Avg. # runs allowed in rest of inning after called strike: 0.265
Conditional on an 0 – 1 pitch being taken:called strike saves ρ = 0.057 runs, on average
Deshpande & Wyner Catcher Framing JSM 2016 9 / 16
Estimated Runs Saved, On Average
Catcher Runs Saved (SD) 95% Interval P(> 0) N BP
Miguel Montero 25.1 (7.1) [11.3, 38.8] 0.999 8086 11.2 (8172)
Mike Zunino 19.9 (7.3) [5.4, 34.1] 0.997 7615 20.4 (7457)
Jonathan Lucroy 19.5 (8.1) [3.8, 35.3] 0.991 8398 16.4 (8241)
Rene Rivera 18.9 (5.3) [8.6, 29.2] 1.000 5091 22.5 (5182)
Hank Conger 17.6 (4.5) [8.8, 26.4] 1.000 4743 23.8 (4768)
Russell Martin 15.4 (5.9) [3.6, 27.2] 0.994 6388 14.9 (6502)
Buster Posey 15.0 (6.1) [3.1, 26.9] 0.992 6385 23.6 (6190)
Travis d’Arnaud 13.5 (6.1) [1.8, 25.7] 0.986 6573 8.8 (6276)
Brian McCann 12.9 (5.4) [2.2, 23.2] 0.992 6335 9.7 (6471)
Christian Vazquez 12.4 (3.4) [5.9, 18.9] 1.000 3198 13.7 (3370)
Deshpande & Wyner Catcher Framing JSM 2016 10 / 16
Spatially Aggregate Framing Effect
Is Montero really a better pitch framer than Vazquez?
• 8086 pitches vs 3198
• Results further confounded by other pitch participants, location,counts
We can integrate ρ× (p̂ − p̂0) over all batter, pitcher, umpire, count,location combinations.
• Framing analog of Spatial Aggregate Fielding Evaluation of Jensen,Shirley, and Wyner (2008)
• SAFE2: Estimate how many runs catcher saves through framing on4000 “average” pitches
Deshpande & Wyner Catcher Framing JSM 2016 11 / 16
SAFE2
Rank Player Mean (SD) 95% Interval P(> 0)
1. Rene Rivera 15.1 (4.4) [6.5, 23.6] 1.000
2. Hank Conger 14.7 (4.4) [6.1, 23.3] 1.000
3. Christian Vazquez 14.6 (4.9) [5.0, 24.3] 0.999
4. Miguel Montero 12.8 (3.7) [5.5, 19.9] 0.999
5. Yasmani Grandal 12.5 (4.5) [3.8, 21.4] 0.998
6. Mike Zunino 11.5 (4.1) [3.6, 19.5] 0.997
7. Martin Maldonado 11.4 (5.9) [0.1, 23.3] 0.975
8. Chris Stewart 11.1 (5.6) [0.2, 22.2] 0.977
9. Russell Martin 10.3 (4.0) [2.4, 18.0] 0.995
10. Drew Butera 10.1 (5.2) [0.1, 20.3] 0.976
Deshpande & Wyner Catcher Framing JSM 2016 12 / 16
Conclusions
• SAFE2 year-to-year correlation encouraging: 0.5 – 0.75
• There are some players with statistically distinguishable effects onsome umpires
• Even with these effects, out-of-sample performance similar to that ofunderlying GAM’s: non-stationarity between seasons
• Our estimate of framing’s impact similar to others, but considerableuncertainty in our estimates!
Deshpande & Wyner Catcher Framing JSM 2016 13 / 16
Thanks!
Fitted Called Strike Probabilities
0 – 1 pitch, Yasiel Puig, Madison Bumgarner, Buster Posey
(a) Angel Hernandez (b) Average Umpire (c) Scott Barry
Deshpande & Wyner Catcher Framing JSM 2016 15 / 16
Average # Runs Given Up and Value of Strike
Count Ball Strike Value of strike, ρ
0-0 0.367 (0.002) 0.305 (0.002) 0.062 (0.002)
0-1 0.322 (0.002) 0.265 (0.004) 0.057 (0.004)
0-2 0.276 (0.003) 0.178 (0.007) 0.098 (0.008)
1-0 0.427 (0.003) 0.324 (0.003) 0.103 (0.005)
1-1 0.364 (0.003) 0.280 (0.004) 0.084 (0.005)
1-2 0.302 (0.003) 0.162 (0.006) 0.140 (0.006)
2-0 0.571 (0.007) 0.370 (0.006) 0.201 (0.009)
2-1 0.468 (0.005) 0.309 (0.006) 0.159 (0.008)
2-2 0.383 (0.004) 0.165 (0.006) 0.218 (0.007)
3-0 0.786 (0.013) 0.481 (0.008) 0.305 (0.015)
3-1 0.730 (0.010) 0.403 (0.009) 0.327 (0.014)
3-2 0.706 (0.008) 0.166 (0.008) 0.540 (0.011)
Table : Standard errors in parentheses
Deshpande & Wyner Catcher Framing JSM 2016 16 / 16