Practical Portfolio Optimization
Victor DeMiguelProfessor of Management Science and Operations
London Business School
Based on joint research with
Lorenzo Garlappi Alberto Martin-Utrera Xiaoling MeiU. of British Columbia U. Carlos III de Madrid U. Carlos III de Madrid
Francisco J. Nogales Yuliya Plyakha Raman UppalU. Carlos III de Madrid Goethe University Frankfurt EDHEC Business School
Grigory VilkovGoethe University Frankfurt
International Conference on Continuous OptimizationJuly 30, 2013
“The motivation behind my dissertation was to applymathematics to the stock market” Harry Markowitz
“This is not a dissertation in economics, and we cannotgive you a PhD in economics for this” Milton Friedman
Mean-variance efficient frontier
I Efficient frontier: Investor concerned only about mean andvariance of returns chooses portfolio on efficient frontier.
-
6Expected
Return
Variance
..........................
........................
........................
........................
........................
.......................................................
.................................................................
...........................................................................
..................................................................................... ..............................................
3 / 70
Mean Variance portfolio optimization
minw
w>Σw︸ ︷︷ ︸variance
− w>µ︸︷︷︸mean
/γ
s.t. w ∈ C︸ ︷︷ ︸constraints
I w portfolio weight vector
I Σ covariance matrix of asset returns
I µ mean asset returns
I γ risk aversion parameter
4 / 70
Implementation: estimation and rebalancing
-
-�Estimation window
?
Next month
Portfolio for January 2009
TimeJan 98 Dec 08
5 / 70
-
-�Estimation window
?
Next month
Portfolio for January 2009
TimeJan 98 Dec 08
-
-�Estimation window
?
Next month
TimeFeb 98 Jan 09
Portfolio for February 2009
Will the portfolios for January 2009 and February 2009 be similar?
6 / 70
Problem: unstable portfolios
I The portfolios for consecutive months usually differgreatly:
I Unstable portfolios: Extreme weights that fluctuate a lot aswe rebalance the portfolio
I Why?
I Estimation error!!!
-
6Weight onrisky asset
Time
�����������CCCCCCCC��������BBBBBBB����������BBBBB@@
7 / 70
Problem: unstable portfolios
I The portfolios for consecutive months usually differgreatly:
I Unstable portfolios: Extreme weights that fluctuate a lot aswe rebalance the portfolio
I Why?
I Estimation error!!!
-
6Weight onrisky asset
Time
�����������CCCCCCCC��������BBBBBBB����������BBBBB@@
8 / 70
The 1/N paper
The 1/N paperDeMiguel, Garlappi, Uppal (RFS, 2009)
Objective: : Quantify impact of estimation error by comparingmean-variance and equal-weighted portfolio (1/N).
Why use the 1/N portfolio as a benchmark?
I Estimation-error free,I Simple but not simplistic:
I Does have some diversification, though not “optimally”diversified;
I With rebalancing =⇒ contrarian;I Without rebalancing =⇒ momentum.
I Ancient wisdom
I Rabbi Issac bar Aha (Talmud, 4th Century): Equal allocationA third in land, a third in merchandise, a third in cash.
I Investors use it, even nowadays.11 / 70
Thomson Reuters equal weighted commodity index
What we do
I Empirically: Compare fourteen portfolio rules to 1/Nacross seven datasets
I Sharpe ratio
SR =mean
std .dev .
I Analytically: Derive critical estimation windowlength for mean-variance strategy to outperform 1/N .
I Simulations: Extend analytical results to modelsdesigned to handle estimation error.
13 / 70
What we find
I Empirically: None of the fourteen portfolio modelsconsistently dominates 1/N across seven separatedatasets (SR and turnover).
I Analytically: Based on U.S. stock market data, criticalestimation window for sample-based mean variance(MV) to outperform 1/N is
I Approximately 3,000 months for 25-asset portfolio
I Approximately 6,000 months for 50-asset portfolio
I Simulations: Even models designed to handle estimationerror need unreasonably large estimation windows tooutperform 1/N .
14 / 70
Portfolios tested
I Bayesian portfolios
15 / 70
Historical return data
Sample mean
Historical return data
Sample mean
Mean prior distribution
Historical return data
Sample mean
Mean prior distribution Mean posterior
distribution
Bayes rule
Portfolios tested
I Bayesian portfolios
I Diffuse prior [Barry, 1974], [Klein and Bawa, 1976], [Brown, 1979],
I Informative empirical prior [Jorion, 1986]
I Prior belief on asset pricing model [Pastor, 2000] and
[Pastor and Stambaugh, 2000]
I Portfolios with moment restrictions
I Portfolios subject to shortsale constraints
I Optimal combinations of portfolios
19 / 70
Portfolios tested
I Bayesian portfolios
I Portfolios with moment restrictions
I Minimum-variance portfolio often outperforms mean-varianceportfolios; [Jagannathan and Ma, 2003],
I Value-weighted market portfolio optimal in a CAPM world,
I Missing-factor portfolio. [MacKinlay and Pastor, 2000] impose
Σ = νµµ> + σ2IN .
I Portfolios subject to shortsale constraints
I Optimal combinations of portfolios
20 / 70
Portfolios tested
I Bayesian portfolios
I Portfolios with moment restrictions
I Portfolios subject to shortsale constraints[Jagannathan and Ma, 2003] show they perform well inpractice.
I Optimal combinations of portfolios
21 / 70
Portfolios tested
I Bayesian portfolios
I Portfolios with moment restrictions
I Portfolios subject to shortsale constraints
I Optimal combinations of portfolios[Kan and Zhou, 2007]
wKZ = a wmean-variance + b wminimum-variance,
where a and b minimize portfolio loss.
22 / 70
Empirical results: Out-of-sample Sharpe ratios - I
Sector Indust. Inter’l Mkt/ FF FFStrategy Portf. Portf. Portf. SB/HL 1-fact. 4-fact.
1/N 0.19 0.14 0.13 0.22 0.16 0.18
mean var (in sample) 0.38 0.21 0.21 0.29 0.51 0.54
Classical approach that ignores estimation errormean variance 0.08 -0.04 -0.07 0.22 -0.07 0.00
Bayesian approach to estimation errorBayes Stein 0.08 -0.03 -0.05 0.25 -0.06 0.00data and model 0.14 -0.05 0.10 0.02 0.07 0.24
Moment restrictionsminimun variance 0.08 0.16 0.15 0.25 0.28 -0.02market portfolio 0.14 0.11 0.12 0.11 0.11 0.11missing factor 0.19 0.13 0.12 0.06 0.15 0.15
23 / 70
Why does minimum-variance portfolio outperformmean-variance portfolio?
I In sample, min-var portfolio has smallest expected return.
-
6Expected
Return
Variance
.........................
........................
.......................
.......................
........................
........................................................
..................................................................
.............................................................................
.......................................... .............................................in-sample frontier
24 / 70
Why does minimum-variance portfolio outperformmean-variance portfolio?
I In sample, min-var portfolio smallest expected return.
I Out of sample, minimum-variance portfolio has typicallyhighest Sharpe ratio; Jorion (1985, 1986, 1991).
I Estimation error in mean larger than in variance; Merton (1980).
I Jagannathan and Ma (2003): “estimation error in the sample mean is solarge that nothing much is lost in ignoring the mean altogether”.
-
6Expected
Return
Variance
.........................
........................
.......................
.......................
........................
........................................................
..................................................................
.............................................................................
.......................................... .............................................
........ .......... .............. ................. .................... ....................... .......................... ............................. ................................ ................................... ...................................... ......................................... ............................................
in-sample frontier
out-of-sample frontier
25 / 70
Mean variance portfolio returns are extreme
Jul−84 Jul−88 Sep−92 Nov−96 Jan−01 Apr−05 Jun−09−1.5
−1
−0.5
0
0.5
1
1.5
Dates
Por
tfolio
Ret
urns
Mean−VarianceEqually Weighted
26 / 70
Empirical results: Out-of-sample Sharpe ratios - II
Sector Indust. Inter’l Mkt/ FF FFStrategy Portf. Portf. Portf. SB/HL 1-fact. 4-fact.
1/N 0.19 0.14 0.13 0.22 0.16 0.18
mean var (in sample) 0.38 0.21 0.21 0.29 0.51 0.54
Shortsale constraintsmean variance 0.09 0.07 0.08 0.11 0.20 0.20Bayes Stein 0.11 0.08 0.08 0.15 0.20 0.21minimum variance 0.08 0.14 0.15 0.25 0.15 0.36
Optimal combinations of portfoliosmean & min-var 0.07 -0.03 0.06 0.25 -0.01 0.001/N & min-var 0.12 0.16 0.14 0.25 0.26 -0.02
27 / 70
Why do shortselling constraints help?
I Intuitively, they prevent extreme (large) positive and negativeweights in the portfolio.
I Theoretically, imposing shortselling constraints is likereducing the covariances assets that we would short;Jagannathan and Ma (2003).
Σ = Σ− λe> − eλ>
-
6Weight onrisky assetconstrained
Time
�����������CCCCCCCC��������BBBBBBB����������BBBBB@@
28 / 70
Results: Turnover relative to 1/N
Sector Indust. Inter’l Mkt/ FF FFStrategy Portf. Portf. Portf. SB/HL 1-fact. 4-fact.
1/N 3.05% 2.16% 2.93% 2.37% 1.62% 1.98%
Mean variance and Bayesian portfoliosmean variance 39 607479 4236 2.83 9408 2672Bayes Stein 22.41 10092 1760 1.85 10510 2783data and model 1.72 20468 45.24 76.85 762.83 32.46
Moment restrictionsminimum variance 6.54 21.65 7.30 1.11 45.47 6.83market portfolio 0 0 0 0 0 0missing factor 1.10 1.05 1.10 4.41 1.09 0.98
Shortsale constraintsmean variance 4.53 7.17 7.23 4.12 17.53 13.82Bayes Stein 3.64 7.22 6.10 3.65 17.32 13.07minimum variance 2.47 2.58 2.27 1.11 3.93 1.76
Optimal combinations of portfoliosmean & min-var 19.82 9943 819.30 2.61 3762 42871/N & min-var 4.82 15.66 4.24 1.11 34.10 6.80
29 / 70
Empirical results: Summary
I In-sample, mean-var strategy has highest Sharpe ratio.
I Out-of-sample
I 1/N does quite well in terms of Sharpe ratio and turnover.
I Sample-based mean-var strategy has worst Sharpe ratio.
I Bayesian policies typically do not out-perform 1/N.
I Constrained policies do better than unconstrained,but constraints alone do not help (need extra momentrestrictions).
I Min-var-constrained does well, but:I Sharpe Ratios and CEQ statistically indistinguishable from
1/N.
I Better than 1/N in only one dataset (20-size/bm portfolios).
I Turnover is 2-3 times higher than 1/N.
30 / 70
Beating 1/N
Beating 1/N
minw
w>Σw︸ ︷︷ ︸variance
− w>µ︸︷︷︸mean
/γ
s.t. w ∈ C︸ ︷︷ ︸constraints
Improve portfolio performance using
I. better covariance matrix,
II. better mean,
III. better constraints.
33 / 70
I. Better covariance matrix
To estimate covariance matrix better:
1. Use higher-frequency data
I more accurate estimates of covariance matrix; [Merton, 1980].
2. Use factor models
3. Use shrinkage estimators
34 / 70
I. Better covariance matrix
To estimate covariance matrix better:
1. Use higher-frequency data
2. Use factor models
I r︸︷︷︸returns
= a︸︷︷︸intercept
+ B︸︷︷︸slopes
f︸︷︷︸factors
+ε.
I more parsimonious estimates; [Chan et al., 1999].
3. Use shrinkage estimators
35 / 70
I. Better covariance matrix
To estimate covariance matrix better:
1. Use higher-frequency data
2. Use factor models
3. Use shrinkage estimators
I [Ledoit and Wolf, 2004] combine sample covariance and
identity
ΣLW = α Σ + (1− α)I .
36 / 70
Shrinkage estimators
ΣI
Distribution of sample covariance matrix estimator
Distribution of shrunkcovariance matrix estimator
Distribution of deterministiccovariance matrix estimator Bias
Variance Σ
Variance Σ
ΣLW
37 / 70
I. Better covariance matrix
To obtain better estimates of covariance matrix:
1. Use higher-frequency data
2. Use factor models
3. Use shrinkage estimators
4. Use robust optimization
I Goldfarb and Iyengar (MOR, 2003) and many others
minw
maxΣ∈U
w>Σw− w>µ/γ
s.t. w ∈ C
5. Use robust estimation
38 / 70
I. Better covariance matrix
To obtain better estimates of covariance matrix:
1. Use higher-frequency data
2. Use factor models
3. Use shrinkage estimators
4. Use robust optimization
5. Use robust estimation
39 / 70
Portfolio Selection with Robust EstimationDeMiguel and Nogales (Operations Research, 2009)
Deviation
.
..................................................
...............................................
...........................................
........................................
.....................................
..................................
..............................
...........................
........................
.....................
.................... ................... .................. ................. ................. ..............................................................................
........................
...........................
..............................
..................................
.....................................
........................................
...........................................
...............................................
..................................................
Variance
I Square function amplifies the impact of outliers
40 / 70
Portfolio Selection with Robust Estimation
Deviation
.
..................................................
...............................................
...........................................
........................................
.....................................
..................................
..............................
...........................
........................
.....................
.................... ................... .................. ................. ................. ..............................................................................
........................
...........................
..............................
..................................
.....................................
........................................
...........................................
...............................................
..................................................
�����������
@@
@@
@@@
@@
@@
Mad
I Absolute value function reduces impact of outliers
41 / 70
Portfolio Selection with Robust Estimation
Deviation
.
..................................................
...............................................
...........................................
........................................
.....................................
..................................
..............................
...........................
........................
.....................
....................
...................
.................. ................. ................. .....................................
....................
.....................
........................
...........................
..............................
..................................
.....................................
........................................
...........................................
...............................................
..................................................
�����������
@@
@@
@@@
@@
@@
...............
......
...................
.................. ................. ................. .....................................
....................@@
@@@
@@@
@@@
�����������
M-estimator
I M-estimators use smooth error function
42 / 70
Portfolio Selection with Robust Estimation
Deviation
.
..................................................
...............................................
...........................................
........................................
.....................................
..................................
..............................
...........................
........................
.....................
.................... ................... .................. ................. ................. ..............................................................................
........................
...........................
..............................
..................................
.....................................
........................................
...........................................
...............................................
..................................................
�����������
@@
@@
@@@
@@
@@
. .................... ................... .................. ................. ................. .........................................................
@@@
@@@
@@
@@@
�����������
.
....................................
.................................
..............................
...........................
.......................
.....................
.................... ................... .................. ................. ................. ..............................................................................
.......................
...........................
..............................
.................................
....................................
S-estimator
I S-estimators: the impact of outliers is bounded
43 / 70
II. Better mean
To obtain better estimates of mean:
1. Ignore means
2. Use Bayesian estimates
3. Use robust optimization
4. Use option-implied information
44 / 70
II. Better mean
To obtain better estimates of mean:
1. Ignore means
2. Use Bayesian estimates
3. Use robust optimization
4. Use option-implied information
45 / 70
II. Better mean
To obtain better estimates of mean:
1. Ignore means
2. Use Bayesian estimates
3. Use robust optimization
minw
w>Σw−minµ∈U
w>µ/γ
s.t. w ∈ C
4. Use option-implied information
46 / 70
II. Better mean
To obtain better estimates of mean:
1. Ignore means
2. Use Bayesian estimates
3. Use robust optimization
4. Use option-implied information
47 / 70
Improving Portfolio Selection UsingOption-Implied Volatility and Skewness
DeMiguel, Plyakha, Uppal, and Vilkov, forthcoming in JFQA
I Implied volatilities improvevolatility by 10-20%.
I Implied correlations do notimprove performance.
I Implied skewness andvolatility risk premiumproxy mean returns.
I Improve Sharpe ratioeven with moderatetransactions costs forweekly and monthlyrebalancing.
48 / 70
II. Better mean
To obtain better estimates of mean:
1. Ignore means
2. Use Bayesian estimates
3. Use robust optimization
4. Use option-implied information
5. Use stock return serial dependence
49 / 70
Stock Return Serial Dependence and Out Of SamplePerformance, DeMiguel, Nogales, Uppal (2013)
I Stock return serialdependence
I Contrarian: “buy losersand sell winners”.
I Momentum: “buywinners and sell losers”.
I Can this be exploitedsystematically with manyassets?
I Yes, for transaction costsbelow 10 basis points.
50 / 70
II. Better mean
To obtain better estimates of mean:
1. Ignore means
2. Use Bayesian estimates
3. Use robust optimization
4. Use option-implied information
5. Use stock return serial dependence
6. Exploit anomalies: size, momentum, value
I “Expected Returns” by Antti Ilmanen.
51 / 70
III. Better constraints
To improve performance impose:
1. Shortsale constraints
2. Parametric portfolios
3. Norm constraints
52 / 70
III. Better constraints
To improve performance impose:
1. Shortsale constraints
2. Parametric portfolios[Brandt et al., 2009] impose constraint
w1
w2
...wN
=
wM
1wM
2...
wMN
+
me1 btm1 mom1
me2 btm2 mom2
......
...meN btmN momN
θme
θbtmθmom
,
me = market equity,btm = book-to-market ratio,
mom = momentum or average return over past 12 months.
3. Norm constraints
53 / 70
III. Better constraints
To improve performance impose:
1. Shortsale constraints
2. Parametric portfolios
3. Norm constraints
54 / 70
A Generalized Approach to Portfolio Optimization:Improving Performance by Constraining Portfolio Norms
DGNU (MS, 2009)
minw
w>Σw
s.t. w>e = 1
‖w‖ ≤ δ
I Nests 1/N, shrinkage covariance matrix of[Ledoit and Wolf, 2004], and shortsale constraints,
I Diversification: 2-norm,
I Leverage constraint: 1-norm.55 / 70
Help wanted
I Optimization can make a difference in portfolio selection.
I Research opportunities
I Integer programming:
I fixed costs and cardinality constraints,
I value at risk.
I Approximate dynamic programming:
I dynamic portfolio choice,
I transaction costs.
I Calibration, calibration, calibration:
57 / 70
Help wanted
I Optimization can make a difference in portfolio selection.
I Research opportunities
I Integer programming:
I fixed costs and cardinality constraints,
I value at risk.
I Approximate dynamic programming:
I dynamic portfolio choice,
I transaction costs.
I Calibration, calibration, calibration:
58 / 70
Help wanted
I Optimization can make a difference in portfolio selection.
I Research opportunities
I Integer programming:
I fixed costs and cardinality constraints,
I value at risk.
I Approximate dynamic programming:
I dynamic portfolio choice,
I transaction costs.
I Calibration, calibration, calibration:
59 / 70
Help wanted
I Optimization can make a difference in portfolio selection.
I Research opportunities
I Integer programming:
I fixed costs and cardinality constraints,
I value at risk.
I Approximate dynamic programming:
I dynamic portfolio choice,
I transaction costs.
I Calibration, calibration, calibration:
60 / 70
Help wanted
I Optimization can make a difference in portfolio selection.
I Research opportunities
I Integer programming:
I fixed costs and cardinality constraints,
I value at risk.
I Approximate dynamic programming:
I dynamic portfolio choice,
I transaction costs.
I Calibration, calibration, calibration:
61 / 70
Same dog, different collar?Devil Is in One Detail: Calibration
Constraints Robust
optimization
Bayesian
portfolios
Shrinkage
estimators
Gotoh &
Takeda CMS 2011
DGNU MS 2009
Goldfarb
Iyengar MOR 2003
Caramanis
Mannor
Xu (2011)
Jorion JFQA 1986
Jaganathan
Ma JoF, 2003
DGNU MS 2009
62 / 70
Help wanted
I Optimization can make a difference in portfolio selection.
I Research opportunities
I Integer programming:I fixed costs and cardinality constraints,I value at Risk.
I Approximate dynamic programming:I dynamic portfolio choice,I transaction costs.
I Calibration, calibration, calibration:I “Size Matters: Optimal Calibration of Shrinkage Estimators
for Portfolio Selection”, D., Martin-Utrera, Nogales, (JB&F,
2012):I Choose criterion carefully, get nonparametric.I “Parameter Uncertainty in Multiperiod Portfolio Optimization
with Transaction Costs”, D., Martin-Utrera, Nogales.
I Shrink both target portfolio and trading rate.
I Statistics/big data/real-time estimation and optimization:I high-frequency trading.
63 / 70
Help wanted
I Optimization can make a difference in portfolio selection.
I Research opportunities
I Integer programming:I fixed costs and cardinality constraints,I value at Risk.
I Approximate dynamic programming:I dynamic portfolio choice,I transaction costs.
I Calibration, calibration, calibration:I “Size Matters: Optimal Calibration of Shrinkage Estimators
for Portfolio Selection”, D., Martin-Utrera, Nogales, (JB&F,
2012):I Choose criterion carefully, get nonparametric.I “Parameter Uncertainty in Multiperiod Portfolio Optimization
with Transaction Costs”, D., Martin-Utrera, Nogales.
I Shrink both target portfolio and trading rate.
I Statistics/big data/real-time estimation and optimization:I high-frequency trading.
64 / 70
High frequency trading
65 / 70
High frequency trading
66 / 70
Thank you
Victor DeMiguelhttp://www.london.edu/avmiguel/
67 / 70