Benchmarking money manager performance:

Issues & evidence

Louis K. C. ChanUniversity of Illinois Urbana-

Champaign

March 2006

Objectives

• The evaluation and attribution of investment performance is crucial for investment research and practice– Money manager performance

– Results of investment strategies & trading rules

– Effects of managerial decisions on shareholder wealth

• Academic and practitioner research has produced a large array of methods for evaluating and attributing investment performance

Objectives

• Question: are conclusions sensitive to the choice of evaluation and attribution methods? why?

• We compare the results from various methods applied to common samples– Set of active institutional money managers– Passive indexes

Evaluating method performance• Many widely-used methods draw on evidence

from asset pricing studies that size, value/growth describe much of the variation in returns (notably Fama and French (1992), Fama and French (1993))

• We concentrate on benchmarking methods based on size, value/growth– Characteristic-matched control portfolios– Time-series factor model regressions– Effective asset mix regressions– Cross-sectional regressions on characteristics

• 1998 – 2000 market boom as stress test of benchmarking methods

Evaluating manager performance

• Much previous work on evaluating performance of mutual and closed-end funds (e.g. Jensen (1968), Elton et al. (1993), Malkiel (1995), Gruber (1995), Carhart (1997), Daniel et al. (1997), Kothari and Warner (2001), etc.)

• Managers of pension plan equity assets are just as important, but much less previous research (see LSV 1992, Coggin et al. 1993)

A first look: characteristic-matched portfolios

vs. 3 factor model

Benchmark details

• Benchmarks vary according to– Characteristics or loadings– Measuring size, value/growth style– Treating size, value/growth effects

separately– Portfolio weighting scheme– Frequency of benchmark reconstitution

Benchmark details

• Characteristics versus loadings– Predict benchmark return using portfolio’s

attributes (size, book-to-market …) or predict benchmark return using portfolio’s loadings on factors

– Some evidence that attributes predict returns better than loadings (Daniel and Titman 1997)

– Data on holdings not generally accessible

Building performance benchmarks

• Measuring size, value/growth style– Size: market capitalization (float?)– Value/growth orientation usually measured by

book-to-market ratio (book value of equity divided by market value of equity)

– Book value of equity does not record value of intangible assets; includes goodwill from acquisitions

Building performance benchmarks

• Treating size, value/growth effects separately– E.g. independent 2-way sorts by size, BM– In one-way sorts by book-to-market equity

large stocks typically are classified as growth – Under an independent size/BM sort procedure

large-cap managers, regardless of large value/large growth style, will tend to be compared against a growth benchmark

Building performance benchmarks

• Weighting scheme for stocks in benchmark– Equal-weighting– Value-weighting

• Benchmark reconstitution frequency– Over time benchmark becomes more

heterogeneous and may no longer correspond to managed portfolio’s features

Data

• Holdings and returns every quarter for 199 portfolios offered by money managers to clients, 1989Q1 - 2001Q4

• Domestic U.S. equity portfolios only• Different styles (large/mid/small,

value/blend/growth)• Some selection bias

Results outline• Performance relative to benchmarks based

on characteristics– Overall active manager sample– Classified by investment style– Diagnostics

• Performance relative to benchmarks based on loadings– Overall active manager sample– Classified by investment style– Diagnostics

Performance measures

• Abnormal return = portfolio’s return minus return on benchmark portfolio

• Tracking error volatility = standard deviation of quarterly difference between portfolio’s return and benchmark’s return

Benchmark performance

Benchmark performance

Benchmark comparisons

Performance based on regression benchmarks

• Three factor model excess return is ( rpt – rft ) – benchmark return

• benchmark return is from fitted regression β(rmt – rft ) + s SMBt + h HMLt

Regression-based benchmark details

• Exposures estimated– over full period (including the quarter when

we measure performance)– or leaving out the quarter when we measure

performance

• Measuring size, value/growth factors– High versus low book-to-market– Other indicators of value/growth orientation

Building regression-based benchmarks

• 3 factor model accounts for size, value/growth separately

• E.g. benchmark return for small value manager = return for market exposure

plus return for smallness plus return for value

• Benchmark credits manager for smallness even though small stocks’ performance is because small growth does better than small value

Regression-based benchmarks• Alternative: compare manager to a

selection of passive benchmarks (effective asset mix regressions)

rpt = α + w1*LGt + w2*LVt

+ w3*MCGt + w4*MCVt

+ w5*SGt + w6*SVt + υpt

w1, … ,w6 portfolio weights (between 0 and 1, add up to 1)

Building regression-based benchmarks

• Another widely-used alternative: each stock’s predicted return is from a cross-sectional regression using stock characteristics, industry dummy variables rit = α + β1*X1i + β2*X2i + …

Regression-based benchmark comparisons

Regression-based benchmark comparisons

Conclusions

• Benchmarking methods that appear similar on the surface can lead to very different conclusions about investment performance

• Popular methods (characteristic-matched reference portfolios, 3 factor time series regression models, cross-sectional regression) have disappointing ability to track managed active portfolios and passive benchmarks

Conclusions

• Methods based on within-size classifications, use multiple measures of value-growth orientation, improve ability to track managed and passive portfolios

• Given the fragility in reliably separating skill from style, detailed decomposition and attribution of performance should be treated with caution