optimal revision of uncertain estimates in project portfolio selection
DESCRIPTION
Optimal revision of uncertain estimates in project portfolio selection. Eeva Vilkkumaa, Juuso Liesiö, Ahti Salo Department of Mathematics and Systems Analysis, Aalto University School of Science and Technology. Contents. Project portfolio selection Optimizer’s curse Revised estimates - PowerPoint PPT PresentationTRANSCRIPT
Optimal revision of uncertain estimates in project portfolio selection
Eeva Vilkkumaa, Juuso Liesiö, Ahti Salo
Department of Mathematics and Systems Analysis,
Aalto University School of Science and Technology
Contents
• Project portfolio selection• Optimizer’s curse• Revised estimates• Discussion
Project portfolio selection
• Select a subset of projects within a budget, e.g., k out of n projects with the aim of maximizing the sum of the projects’ values μi, i=1,...,n
• The values μi are generally unknown, whereby decisions about which projects to select are made based on estimates Vi about μi.
Estimates Portfolio selection Values
t
Optimizer’s curse in portfolio selection
• Assume that the estimates are unbiased
• Portfolio maximization selects, on average, overestimated projects → the value of the portfolio is less than expected based on the estimation information (optimizer’s curse; cf. Smith and Winkler, 2006):
where is the index set of the selected projects.
],[][ ˆˆ
Ki iKi i VEE
].[][ ii EVE
K̂
Optimizer’s curse in portfolio selection
• Choosing 10 projects out of 100
• Values i.i.d with
• Unbiased estimates
• The larger the estimation error variance, the harder it is to identify the best projects, and the larger the difference between the estimated and realized portfolio value
µi ~ N(0,12)
Vi = µi + εi, εi ~ N(0,σ2)
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 112
14
16
18
20
22
24
26
Standard deviation of estimation error
Por
tfol
io v
alue
Estimated
True
Standard deviation of estimation error
Optimal revision of the estimates
• Estimates do not account for the uncertainties
• Use Bayesian revised estimates instead as a basis for project selection
• For instance, with µi ~ N(mi,σi2), Vi ~ N(µi,τi
2):
• The estimate V and the prior information m are weighted according to their uncertainty.
]|[ˆ VEv ii
iiiii mVv )1(ˆ 22
2
ii
ii
where
iv̂
Optimal revision of the estimates
• With revised estimates the optimizers’ curse is eliminated, that is
where is the index set of the projects selected using revised estimates
• Previous example – Choosing 10 projects out of 100
– True values i.i.d. with
– Unbiased estimates
],ˆ[][ ~~
Ki iKi i vEE
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 112
14
16
18
20
22
24
26
Standard deviation of estimation error
Por
tfolio
val
ue
Estimated
True
µi ~ N(0,12)
Vi = µi + εi, εi ~ N(0,σ2)
K~
Standard deviation of estimation error
Revised estimates and portfolio composition• In the previous example, the projects’ values were
identically distributed, and the estimation errors had equal variances
• Then, prioritization among the projects remains unchanged when the estimates are revised, because
• In general, using revised estimates may result in a different project prioritization than estimates
jijiji VVVVvv )1()1(ˆˆ
Revised estimates and portfolio composition
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Pro
ject
val
ue
Same error variances
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Pro
ject
val
ue
Different error variances
Estimate Revisedestimate
Revisedestimate
EstimateEstimate Revised estimate
Estimate Revised estimate
Same error variances Different error variances• Choosing 3 projects out of 8
• True values i.i.d. With µi ~ N(0,12)
• On the left, estimates with equal error variance for all projects
• On the right, four projects (dashed) more difficult to estimate
Vi = µi + εi, εi ~ N(0,0.52)
Vi = µi + εi, εi ~ N(0,12)
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Pro
ject
val
ue
Same error variances
-3
-2.5
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Pro
ject
val
ue
Different error variances
Estimate Revisedestimate
Revisedestimate
Estimate
Revised estimates and portfolio composition
• On the left, equal error variances → estimates are shifted towards the common prior mean (zero) in the same proportion
• On the right: the revised estimates of the ”dashed” projects are more drawn towards zero, because the estimation information is less reliable
• Selection of 3 projects leads to different portfolios depending on whether the estimates are revised or not Estimate Revised
estimateEstimate Revised
estimate
Same error variances Different error variances
Revised estimates and portfolio value• The use of revised estimates yields at
least as high overall portfolio value as the use of initial estimates, i.e.
• Example:– Selection of 10 out of 100 projects with
values µi ~ N(3,12)
– Population contains two types of projects:
– Revised estimates yield higher portfolio value for any non-trivial division between projects with small and large estimation error variances
0 10 20 30 40 50 60 70 80 90 10042
43
44
45
46
47
48
Share of projects with large error variance [%]P
ortfo
lio v
alue
][][ ˆ~
Ki iKi i EE
1) εi ~ N(0,0.12) - small error variance2) εi ~ N(0,12) - large error variance
Share of projects with large error variance [%]
Optimal
Estimates
Revised estimates
Revised estimates and correct choices
• The share of correctly selected projects increases with revised estimates in the normally distributed case, i.e.,
where K is the index set of the projects in the optimal portfolio
• In the previous example, the difference between the two portfolios is statistically significant (α=0.05), when the share of projects with large error variance is between 25-55%
}ˆ|{#}~
|{# KKiiKKii
Share of projects with large error variance [%]
Discussion
• Selection based on project prioritization resulting from estimates– The value of the portfolio will, on average, be lower than
expected– If there are differences in the projects’ estimation error
variances, too many projects with large error variance will be selected
• Suggestions for improving the selection process– Accounting for the uncertainties by using revised estimates– Sorting the projects in terms of estimation error variances by,
e.g., budget division