fin285a:computer simulations and risk assessment section 9...
TRANSCRIPT
Fin285a:Computer Simulations andRisk Assessment
Section 9Backtesting and Stress Testing
Danı́elson, 8.1-8.3.1, 8.5, 8.6
Overview
Fall 2017: Fin285: 9 2 / 25
What is backtesting?
Regulatory issues
Backtesting details
Backtest examples
Stress testing
What is backtesting?
What is backtesting?
Regulatory issues
Backtesting details
Backtest examples
Stress testing
Fall 2017: Fin285: 9 3 / 25
Backtest basics
Fall 2017: Fin285: 9 4 / 25
•Test VaR and ES estimates
•Use past data
•Forecast into the future (windows)
• Important for regulation (Basel II and III)
•Out of sample forecasting
⇒Clean future forecasts
⇒VaR estimates at time t use only data before time t
Regulatory issues
What is backtesting?
Regulatory issues
Backtesting details
Backtest examples
Stress testing
Fall 2017: Fin285: 9 5 / 25
Market risk capital (reserves)
Fall 2017: Fin285: 9 6 / 25
•Capital (liquid) held in reserves by banks and other financials
⇒Required by regulation
⇒ Important during crisis
•Previously: Set by fraction of portfolio in different asset classes
⇒Sovereign debt
⇒Corporate debt
⇒Home mortgages
⇒Equity
•Now (> 2005): Also, involves some aspect of internal risk (VaR)measures (Internal models)
•Needs to be carefully monitored (backtesting)
Example
Fall 2017: Fin285: 9 7 / 25
Capitalt = Ct = Atmax(VaRt(1%),VaRt(1%)) + Constant (9.1)
VaRt(1%) = Average over previous 60 days (9.2)
At =
3 if vt ≤ 4
3 + 0.2(vt − 4) if 5 ≤ vt ≤ 9
4 if 10 ≤ vt
(9.3)
vt = VaR(1%) violations over past 250 trading days
Think about the dynamics of this:
•Volatility goes up
•vt rises
•Need to shift portfolio to cash
Backtesting details
What is backtesting?
Regulatory issues
Backtesting details
Backtest examples
Stress testing
Fall 2017: Fin285: 9 8 / 25
Estimation windows (length = 500)
Fall 2017: Fin285: 9 9 / 25
[1, 500],VaR501
[2, 501],VaR502
Full Sample
[3, 502],VaR503
Estimation and Testing
Fall 2017: Fin285: 9 10 / 25
Estimation
Start End Test
1 500 5012 501 502. . . . . . . . .
Recursive windows
Fall 2017: Fin285: 9 11 / 25
Windows can extend all the way back to the start of thedata:
Estimation
Start End Test
1 500 5011 501 5021 502 5031 503 504. . . . . . . . .
VaR violations (exceptions)
Fall 2017: Fin285: 9 12 / 25
ηt =
{
1 if yt ≤ −VaRt
0 if yt > −VaRt
(9.4)
v1 =∑
ηt (9.5)
V R =v1
pWT
=Observed violations
Expected violations=
v1
WT
p(9.6)
•WT testing length
• If working, V R ≈ 1
Backtest examples
What is backtesting?
Regulatory issues
Backtesting details
Backtest examples
Stress testing
Fall 2017: Fin285: 9 13 / 25
Null hypothesis
Fall 2017: Fin285: 9 14 / 25
•VaR(p)
•vt is binomial (like coin)
•Two features to test:
•VaR probability:
⇒Pr(ηt = 0) = 1− p
⇒Pr(ηt = 1) = p
⇒Variability of p estimates in windows (see our old examples)
• Independence:
⇒ηt independent over time
⇒ηt are unpredictable
Backtest code
Fall 2017: Fin285: 9 15 / 25
•Python: backTest.py
•Rolling historical VaR
•Estimate VaR using past 500 days
•Report violations and violation ratio
•Simpler than code in textbook
•Statistical test: like coin, and election polls
Independence
Fall 2017: Fin285: 9 16 / 25
•VaR violations (exceptions) should be independent
•Find conditional probability that violation (ηt) follows violation(ηt−1)
•For independence this is equal to the VaR probability level
•Can also look at correlations of violations
•ηt should be unpredictable using any past data
Testing independence
Fall 2017: Fin285: 9 17 / 25
Pr(ηt = 1|ηt−1 = 1) =Pr((ηt = 1)and(ηt−1 = 1))
Pr(ηt = 1)(9.7)
•Python: backTest.m
•Find conditional probability (above)
•For independence this is equal to the VaR probability level
•A violation today contains no information on tomorrow
Testing expected shortfall
Fall 2017: Fin285: 9 18 / 25
•This can be done
•A little trickier than VaR
•Requires more data
•See textbook
Testing expected shortfall
Fall 2017: Fin285: 9 19 / 25
•First, assume rolling expected tail return:
ESt(p) = R̃t (9.8)
•Now, key ratio:E(Rt|Rt ≤ −VaR(p))
ESt(p)= 1 (9.9)
•For correct ES estimate this ratio should be 1
•Python: backTestES.py
•Difficult to do analytic statistical tests
•Bootstrap confidence bands
•Generally requires more data than testing VaR
Problems with backtesting
Fall 2017: Fin285: 9 20 / 25
•Window/time selection
⇒Enough extreme events?
⇒Regime changes
•True “out of sample” (data snooping)
⇒Need clean tests
⇒This rarely happens
⇒Multiple model problems
•Critical for policy and reserves
Stress testing
What is backtesting?
Regulatory issues
Backtesting details
Backtest examples
Stress testing
Fall 2017: Fin285: 9 21 / 25
Scenario analysis
Fall 2017: Fin285: 9 22 / 25
•Test important scenarios
•Find portfolio sensitivity to key risk factors
Vt =K∑
j=1
βjft,j (9.10)
•Extreme moves in key risk factors
•Types of tests
⇒Replay actual events (crashes of 1987/2008)
⇒Generate new extreme events (Greece leaves Euro, Brexit)
⇒Monte-carlo with extra weight on extremes
⇒Structural breaks in the times series (changing correlations)
•Scenarios often dictated by regulation
Smooth incorporation into risk models
Fall 2017: Fin285: 9 23 / 25
(f, fstress) simulations for normal and stressed situations.
fnew =
{
f with probability (1− ǫ)
fstress with probability ǫ(9.11)
Stress test problems
Fall 2017: Fin285: 9 24 / 25
•Finding scenarios
•Probabilities for worst case scenario??
⇒Preparing for a 10,000 year flood
•What if institutions “plan around” given scenarios, but not readyfor other events?This is like our “manipulating VaR” examples.
•Endogenous feedback:
⇒What happens when everyone is hit with this shock
⇒Fire sales across markets
Overview
Fall 2017: Fin285: 9 25 / 25
What is backtesting?
Regulatory issues
Backtesting details
Backtest examples
Stress testing