Chapter 22

Performance evaluation and prediction

Measuring past performance and predicting future performance Predictions based on past performance are

generally unreliable (Discuss readings) If you cannot predict whether you can trade

profitably, you should not trade. If you cannot predict which managers will be

successful, you should not employ active investment managers.

Passive investment managements use buy and hold strategies, e.g., Index replicators

Luck or skill?

Good performance evaluations must discriminate between luck and skill.

SkilledUnskilled

Lucky Blessed Insufferable

Unlucky Cursed Doomed

Performance evaluation methods

Absolute performance measurement• Total returns• IRR• Holding period return

Relative performance measurement• Difference between a portfolio return and benchmark

return (see Table 22-1)• Market-adjusted returns• Risk-adjusted returns

Market timing

Raw return

= (Raw return – Beta x Market return)

+ (Beta x Market return – Market return)

+ Market return

= Excess return (selection)

+ Market timing return

+ Market return

Past performance can be used to predict future performance if

Past performance reflects skills The manager’s skill will continue to

generate good future returns The manager still has the skills

In general, correlation between past and future performance is low!!

Statistical test

Null Hypothesis (H0)

Not True True

Test Reject Power α (Type I error)

Result Accept Type II error Confidence

Level (CL)

Power = 1 – Type II error CL = 1 – Type I error

Testing manager’s skill

Manager is (H0)

SkilledNot Skilled

Test Skilled Power α (Type I error)

Result Not Type II error Confidence

Skilled Level (CL)

Student’s t-test and confidence level t-ratio = Adjusted return/SE of Adjusted return = (Rp – Rm)/SE of (Rp - Rm)

If t-ratio is greater than critical value, adjusted return cannot be due to luck.

The critical value is determined by confidence level.

Confidence level = 1 – significance level (alpha)

If confidence level is 95% (i.e., significance level is 5%), critical value is 1.64. It means that there is 5% chance that t-ratio will be greater than 1.64 even if the manager has no skills.

Null hypothesis Ho: No skills

There is 5% chance that null hypothesis is rejected even if Ho is true. This is also called Type I error.

Power of test – want to maximize The power of test is the probability of

rejecting Ho when it is false. The probability that Ho is rejected when the

manager is skilled, i.e., the probability that the t-ratio is greater than the critical value, given that the manager is skilled.

Recall that Type II error is the probability of accepting Ho when it is false.

Power of test = 1 – Type II error

Power of test Increases when confidence level

decreases (i.e., when alpha increases) Increases with the manager’s skills Decreases with the importance of luck Increases with the number of

observations (years) Discuss Table 22-2 and Table 22-3

Determining the optimal confidence and power levels Suppose we choose active manager if the

test indicates that he is skilled and index fund otherwise.

Suppose we choose CL and PL to maximize the expected market-adjusted return.

Other assumptions and Table 22-5Annual expected net excess return

True manager status

Test result Skilled Not skilled

Skilled (Active mgr.) 1.00% -2.00%

Not skilled (Index) -0.15% -0.15%

Probability

True manager status

Test result Skilled Not skilled

Skilled 1/3xPL 2/3 x (1 – CL)

Not skilled 1/3x(1-PL) 2/3 x CL

Discuss Table 22-6 and Table 22-7

Important problems with statistical performance evaluation Distributional shape

• Normality• Peso problem

Fraudulent returns• Return smoothing• Pyramid schemes

Sample selection bias

Sample selection in the mutual fund industry

Avoiding the sample selection bias