1 noisy portfolios imre kondor collegium budapest and eötvös university sphinx econophysics...
TRANSCRIPT
1
Noisy Portfolios
Imre Kondor
Collegium Budapest and Eötvös University
SPHINX Econophysics Workshop, Oxford, 27-29 September, 2004
2
Collegium Budapest
Contents
1. Background and motivation2. The model/simulation approach3. Filtering and results4. Beyond the Gaussian case: non-stationarity5. Beyond the variance as risk measure:
absolute deviation, CVaR (expected shortfall), and maximal loss
6. The minimax problem
3
Collegium Budapest
Coworkers
• Szilárd Pafka and Gábor Nagy (CIB Bank, Budapest)
• Marc Potters (Science & Finance)• Richárd Karádi (Institute of Physics, Budapest
University of Technology)
4
Collegium Budapest
Background
• Correlations of returns play central role in financial theory and applications
• The covariance matrix is determined from empirical data – it contains a lot of noise
• Markowitz’ portfolio theory suffered from the curse of dimensions from the very outset
• Economists have developed a number of dimension reduction techniques
• Recent contribution from random matrix theory (RMT)
5
Collegium Budapest
Our purpose
To develop a model/simulation-based approach to test and compare previous methods
6
Collegium Budapest
Initial motivation: a paradox
• According to L.Laloux, P. Cizeau, J.-P. Bouchaud, M. Potters, PRL 83 1467 (1999) and Risk 12 No.3, 69 (1999)
and to
V. Plerou, P. Gopikrishnan, B. Rosenow, L.A.N. Amaral, H.E. Stanley, PRL 83 1471 (1999)
there is a huge amount of noise in empirical covariance matrices, enough to make them useless
• Yet they are in widespread use and banks still survive
7
Collegium Budapest
Some key points
• Laloux et al. and Plerou et al. demonstrate the effect of noise on the spectrum of the correlation matrix C. This is not directly relevant for the risk in the portfolio. We wanted to study the effect of noise on a measure of risk. The whole covariance philosophy corresponds to a Gaussian world, so our first risk measure will be the variance.
8
Collegium Budapest
Optimization vs. risk management
• There is a fundamental difference between the two kinds of uses of the covariance matrix σ for optimization resp. risk measurement.
• Where do people use σ for portfolio selection at all?- Goldman&Sachs technical document- tracking portfolios, benchmarking, shrinkage- capital allocation (EWRM)- hidden in softwares
9
Collegium Budapest
Optimization
• When σ is used for optimization, we need a lot more information, because we are comparing different portfolios.
• To get optimal portfolio, we need to invert σ, and as it has small eigenvalues, error gets amplified.
10
Collegium Budapest
Risk measurement – management - regulatory capital calculation
Assessing risk in a given portfolio – no need to invert σ – the problem of measurement error is much less serious
11
Collegium Budapest
Dimensional reduction techniques in finance
• Impose some structure on σ. This introduces bias, but beneficial effect of noise reduction may compensate for this.
• Examples:- single-index models (β’s) All these help.- multi-index models Studies are based- grouping by sectors on empirical data- principal component analysis- Baysian shrinkage estimators, etc.
12
Collegium Budapest
Contribution from econophysics
• Random matrices first appeared in a finance context in G. Galluccio, J.-P. Bouchaud, M. Potters, Physica A 259 449 (1998)
• Then came the two PRL’s with the shocking result that most of the eigenvalues of σ were just noise
• How come σ is used in the industry at all ?
13
Collegium Budapest
Market data are noisy themselves –
non-stationary processIf we want to assess noise reduction techniques we’d better use well-controlled data, such as those generated by a known stochastic process
Expected returns are hard to estimate from time series
We wanted to separate this part of the problem, too.
14
Collegium Budapest
Main source of error
Lack of sufficient information
input data: N ×T ( N - size of portfolio,
required info: N × N T - length of time series)
Quality of estimate is measured by Q = T/N
Theoretically, we need Q >> 1.
Practically, T is bounded by 500-1000 (2-4 yrs),
whereas N can be several hundreds or thousands.
Dimension (effective portfolio size) must be reduced
15
Collegium Budapest
Our approach
• Choose model correlation matrix Cº• Generate finite time series with Cº• Apply various filtering methods and compare
their efficiency• Models:
1. Unit matrix2. Single-index model3. Market + sectors model4. Semi-empirical (bootstrap) model
16
Collegium Budapest
Simplified portfolio optimization
• Go for the minimal risk portfolio (apart from the riskless asset)
(constraint on return omitted)
ij
jijip ww min2
i
iw 1
jkjk
jij
iw)(
)(
1
1
*
17
Collegium Budapest
Measure of the effect of noise
where w* are the optimal weights corresponding to
and , resp.
ijjiji
ijjiji
ww
ww
q)*0()0()*0(
)*1()0()*1(
20
)0(σ )1(σ
18
Collegium Budapest
Numerical results before filtering
19
Collegium Budapest
Analytical result
can be shown easily for Model 1. It is valid within O(1/N) corrections also for more general models.
TN
q
1
10
20
Collegium Budapest
Results for the market + sectors model
21
Collegium Budapest
Comments on the efficiency of filtering techniques
• Results depend on the model used for Cº.• Market model: still scales with T/N, singular at
T/N=1 much improved (filtering
technique matches structure), can go even below T=N.
• Market + sectors: strong dependence on parametersRMT filtering outperforms
the other two• Semi-empirical: data are scattered, RMT wins in most
cases
histq0
marketq0
22
Collegium Budapest
• Filtering is very powerful in supressing noise, particularly when it matches the underlying structure.
23
Collegium Budapest
One step towards reality: Non-stationary case
• Volatility clustering →ARCH, GARCH, integrated GARCH→EWMA in RiskMetrics (finite memory)
t – actual time
T – window
α – attenuation factor ( Teff ~ -1/log α)
1
0,,1
1 T
kktjkti
kTij rr
24
Collegium Budapest
• RiskMetrics: αoptimal = 0.94
memory of a few months, total weight of data preceding the last 75 days is < 1%.
• Filtering is useful also here. Carol Alexander applied standard principal component analysis. RMT helps choosing the number of principal components in an objective manner.
• We need upper edge of RMT spectrum for exponentially weighted random matrices
25
Collegium Budapest
Exponentially weighted Wishart matrices
0
)1(k
jkikk
ij xx
1,0, 2 ikikik xxiidx
constNthatsoN )1(,01,
26
Collegium Budapest
Density of eigenvalues:
where v is the solution to:
)1()(
N
0)1()sin(loglog)tan(
N
27
Collegium Budapest
Spectra of exponentially weighted and standard Wishart matrices
28
Collegium Budapest
• The RMT filtering wins again – better than plain EWMA and better than plain MA.
• There is an optimal α (too long memory will include nonstationary effects, too short memory looses data).
The optimal α (for N= 100) is 0.996 >>RiskMetrics α.
29
Collegium Budapest
Absolute deviation as a risk measure
• Some methodologies (e.g. Algorithmics) choose the absolute deviation rather than the standard deviation to characterize the fluctuation of portfolios. The objective function to minimize is then:
instead of
ij ij t iiitj
ttjitijiji wx
Twxx
Twww
211
t i
iitabs wxT
1
30
Collegium Budapest
We generate artificial time series again (say iid normal), determine the true abs. deviation and compare it to the
„measured” one:
We get:
t i
iitmeasured wxTiw
1min
N
wq i
i
abs 1
2'
31
Collegium Budapest
• The result scales in T/N again. The optimal portfolio is more risky than in the variance-based optimization.
• Geometrical interpretation: in the original Markowitz case the optimal portfolio is found as the point where the ellipsoid corresponding to a fixed variance first touches the plane corresponding to the budget constraint. In the abs. deviation case the ellipsoid is replaced by a polyhedron, and the solution occurs at one of its corners. A small error in the specification of the polyhedron makes the solution jump to another corner, thereby increasing the fluctuation in the portfolio.
32
Collegium Budapest
The abs. deviation-based portfolios can be filtered again, by associating a covariance matrix with the time series, then filtering this matrix, and generating a new time series via this reduced matrix. This procedure significantly reduces the noise in the abs. deviation.
Note that this risk measure can be used in the case of non-Gaussian portfolios as well.
33
Collegium Budapest
CVaR optimization
CVaR is the conditional expectation beyond the VaR quantile. For continuous pdf’s it is a coherent risk measure and as such it is strongly promoted by a group of academics. In addition, Uryasev showed that its optimizaton can be reduced to linear programming for which extremely fast algorithms exist.
CVaR-optimized portfolios tend to be much noisier than any of the previous ones. One reason is the instability related to the linear risk measure, the other is that a high quantile sacrifices most of the data.
34
Collegium Budapest
A pessimistic risk measure: maximal loss
• Select the worst return in time and minimize this over the weights:
• This risk measure is subadditive and homogeneous, hence
convex.• Budget constraint:
• For T < N there is no solution• The existence of a solution for T > N is a probabilistic issue,
depending on the time series sample
iiti xw
twi
maxmin
1i
iw
35
Collegium Budapest
Why is the existence of an optimum a random event?
• To get a feeling, consider N=T=2.• The two planes
intersect the plane of the budget constraint in two straight lines. If one of these is decreasing, the other is increasing with , then there is a solution, if both increase or decrease, there is not. It is easy to see that for elliptically distributed ’s the probability of there being a solution is ½.
xwf
xwf
ii
i
ii
i
2
2
12
1
2
11
w1
x
36
Collegium Budapest
Conjectured probability distribution for the existence of an optimum
• For T>N the probability of a solution is conjectured to be
- cumulative binomial distribution• For T→infinity, p →1.
1
11
11
2
T
NkT k
Tp
37
Collegium Budapest
Probability of the existence of a solution under maximum loss.F is the standard normal distribution.
38
Collegium Budapest
Probability of the existence of an optimum under CVaR.F is the standard normal distribution.
The optimization of CVaR behaves similarly
39
Collegium Budapest
Some references
• Physica A 299, 305-310 (2001) • European Physical Journal B 27, 277-280
(2002)• Physica A 319, 487-494 (2003)• To appear in Physica A, e-print: cond-
mat/0305475• submitted to Quantitative Finance, e-
print: cond-mat/0402573
40
Collegium Budapest
Model 1
Spectrum
λ = 1,
N-fold degenerate
Noise will split this
into band
1
0
C =
41
Collegium Budapest
The economic content of the single-index model
return market return with standard deviation σ
The covariance matrix implied by the above:
The assumed structure reduces # of parameters to N.If nothing depends on i then this is just the caricature Model 2.
iMiii rr
0
0
,0
i
Mi
ji
r
ji
22
iijMjiij
42
Collegium Budapest
Model 2: single-index
Singlet: λ1=1+ρ(N-1) ~ O(N)
eigenvector: (1,1,1,…)
λ2 = 1- ρ ~ O(1)
(N-1) – fold degenerate
ρ
1
43
Collegium Budapest
Model 4: Semi-empirical
Very long time series (T’) for many assets (N’).
Choose N < N’ time series randomly and derive Cº from these data. Generate time series of length T << T’ from Cº.
The error due to T is much larger than that due to T’.
44
Collegium Budapest
How to generate time series?
• Given independent standard normal• Given • Define L (real, lower triangular) matrix such that
(Cholesky)
Get:
„Empirical” covariance matrix will be different from .
For fixed N, and T → ,
itx
TLL)0(σ
)0(σ
j
jtijit xLy
)0()1( σσ
)0(σ
45
Collegium Budapest
Model 3: market + sectors
This structure has also been studied by economists
1
1
0)(~)()1(1 10111 NONNN
)(~)1(1 110112 NONN singlet
1
1N
N - fold degenerate
)1(~1 13 O
1N
NN - fold degenerate
46
Collegium Budapest
Risk measurement
Given fixed wi’s. Choose to generate data. Measure from finite T time series.
Calculate
It can be shown
, for .
)0(σ)1(σ
)0(
)1(
0
q
TOq
110 T
47
Collegium Budapest
Filtering
Single-index filter:
Spectral decomposition of correlation matrix:
to be chosen so as to preserve trace
N
k
kj
kiji
marketij
k
kj
kikij
vvvvC
vvC
2
111
NNTrC 11
48
Collegium Budapest
Random matrix filter
to be chosen to preserve trace again
where
and - the upper edge of the random band.
K
k
N
Kk
kj
ki
kj
kik
randij vvvvC
1 1
1max KK 2
max 1
T
N
49
Collegium Budapest
Covariance estimates
after filtering we get
and
We compare these on the following figure
T
tjtit
histij yy
T 1
1
marketijji
marketij C )( rand
ijjirand
ij C
histiii
ji
histij
ijC
)(
50
Collegium Budapest
Results for the single-index (market) model
51
Collegium Budapest
Results for the semi-empirical model