mat-2.108 independent research project in applied...

Helsinki University of Technology Department of Engineering Physics and Mathematics Systems Analysis Laboratory

Mat-2.108 Independent Research Project in Applied Mathematics A Simulation Study on the Computation of Non–Dominated Portfolios

Harri Räsänen, 61847C 08/03/2006

Contents Page 1. Introduction 3 2. Robust Portfolio Modeling 5 2.1. Additive Portfolio Value 5 2.2. Feasibility of Portfolio and Constraints 6 2.3. Incompleteness of Information 7 2.4. Non-Dominated Portfolios 8 3. Computation of Non-Dominated Portfolios 10 4. Simulation Design for the Algorithm 13 4.1. Experiment Design 13 4.2. Randomized Generation of Project Realizations 14 5. Performance of the Algorithm 16 5.1. General Overview of the Results 16 5.2. Results with Point Estimate Scores 17 5.3. Results with Score Intervals 19 6. Conclusions 22

References 23 Appendices 26 Appendix 1 26 Appendix 2 27

2

1. INTRODUCTION

Selection problems to form a portfolio of projects with multiple, conflicting and incomparable objectives against scarce resources arise commonly in industrial organizations. For example, a company maximizing value of its R&D projects or strategic product portfolio against a budget faces this situation (Strummer & Heidenberger, 2003; Lindstedt et al., 2005). The problem thus combines multi-criteria decision analysis (MCDA) (e.g. Keeney & Raiffa, 1976; Clemen, 1996) and capital budgeting (CB) (e.g. Lorie & Savage, 1955; Luenberger, 1998). In the simplest case of CB, independent projects are defined with a performance measure (e.g. net present value) and a cost. Total value of a portfolio is then maximized against a budget. This problem can be formulated as binary linear programming (binary LP) problem. It is also assumed that there are no well-established markets for projects (see e.g. Brealey & Myers, 1996; Luenberger, 1998). Most interesting situations, however, arise when decision maker (DM) has to evaluate projects with regard to multiple criteria. Complete information, however, about the parameters of a multi-criteria selection problem may not be available to a DM. For example, DM could not be able to state her/his preference over evaluation criteria or instead of exact estimates for project’s performance measures regard to different criteria, only interval data could be available. Liesiö et al., (2005) develop Robust Portfolio Modeling (RPM) extending principles of preference programming methods into multi-criteria portfolio problems. In RPM context, a subset of project proposals are funded in view of multiple evaluation criteria. Computation of Non-dominated portfolios to form a subset (of all portfolios) is done with regard to sets of feasible parameters that are considered against DM’s preference statements. Values of individual projects as well as portfolios are modeled with an additive weighting model. Non-dominated portfolios are desirable to DM since they have greater or equal overall value compared to dominated portfolios. CB is part of investment process and strategic planning of corporations and is thus an important function (see e.g. Brealey & Myers, 1996; Luenberger, 1998). Reported experiences suggest that it is valuable for firms to use a systematic approach to analyze their project portfolios (Cooper et al., 1999) and use of incomplete information add value to practitioners (Hämäläinen, 2004; Keefer et al., 2004).

3

Liesiö (2004) presents an algorithm for computing non-dominated portfolios subject to incomplete information about criterion weights (linear inequalities) and project specific performance levels (intervals). Allowing incompleteness of information can increase number of non-dominated portfolios to very large with number of project proposals. This makes the portfolio selection problem more complex and challenging in the sense of computational efficiency. As the number of project proposals and/or number of selection criteria increases, computation of feasible portfolios becomes prohibitive (see e.g. Strummer & Heidenberger, 2003). The main objective of the study is to simulate an algorithm for solving non-dominated portfolios with incomplete information. To fulfil the main objective, this study also intends to

i) introduce portfolio selection in the RPM context, ii) design a simulation setup and simulate the algorithm with random data, and iii) analyze statistically performance of the algorithm.

This study is constructed with five sections and proceeds as follows. This first section defines problem, framework, and objectives of the study. Section 2 focuses on the theoretical presentation of the RPM framework. Section 3 views computational aspects of non-dominated portfolios. Section 4 presents the simulation design, and simulation results are discussed in Section 5. Conclusions are given in Section 6.

4

2. ROBUST PORTFOLIO MODELING The framework of the study is robust because the algorithm analyzed uses parameters which are defined with incomplete information (see Liesiö et al., 2005). To form understanding to support performance analysis, basic framework for robust portfolio modeling is given. 2.1. Additive Portfolio Value A set of m projects proposals that are evaluated against criteria, is denoted with n

{ }mx,KxX ,1= . For project , score of -th selection criterion is denoted with . Thus vector

jx i0≥j

iv [ ]jn

jj vvv ,,1 L= describes score of j -th project and forms j -th row of score matrix , where nmv ×

+ℜ∈ [ ] jiji vv = . Overall measure for value in MAVT is

commonly represented with additive value function (see e.g. Keeney & Raiffa, 1976). Then overall value of a project j is

, (1) ( ) ∑=

=n

i

jii

j vwxV1

where is the weight i.e. relative importance of i -th criterion. Criteria weights

correspond to increase in projects value when a criterion is changed from worst to the best level. Vector of weights is defined by

iw

( )Tnwww ,,1 L= and are scaled as

⎭⎬⎫

⎩⎨⎧

=≥ℜ∈=∈ ∑=

1,01

0n

iii

nw wwwSw . (2)

A project portfolio p is a subset of all portfolios and the set of all possible

portfolios is given by the power set

Xp ⊆XP 2= . Each portfolio’s overall value is

. (3) ( ) ( ) ∑ ∑∑∑∑= ∈∈ =∈

===n

i px

jii

px

n

i

jii

px

j

jjj

vwvwxVvwpV11

,,

Thus the overall value of a portfolio is weighted sum of all project values included in portfolio.

5

2.2. Feasibility of Portfolio and Constraints Resources are commonly scarce and only some of the project proposals can be funded. Availability of -th ( ) resource is denoted by . Total budget vector is

defined by .

k qk ,,1L= kB

[ ] qTqBBB +ℜ∈= ,,1 L

If project is started (its proposal is funded), it consumes units of -th

resource. Vector

jx 0≥jkc k

( ) [ Tjq

jj ccxC ,,1 L= ] describes cost of j-th project. Cost of a portfolio is

sum of costs of projects included in portfolio as ( ) ( )∑

∈

=px

j

j

xCpC and ( ) 0=∅C . (4)

A portfolio is feasible when p ( ) BpC ≤ i.e. costs do not exceed budget. A set of feasible portfolios is formed with ( ){ }BpCPpPF ≤∈= | .

RPM problem can include other constraints in addition to scarce resources. Project interdependencies (e.g. mutually exclusive projects) and non-linearity in objective function are examples of possible additional constraints (see e.g. Strummer & Heidenberger, 2003). In Liesiö (2004), three types of additional constraint were formulated: logical (e.g. different versions of same project), allocation (to control allocation of resources), and synergies modeling constraints (e.g. two chosen projects yield better outcome than sum of their individual outcomes). Now, with scores and costs of projects and budget known, the optimal portfolio maximizes the overall value of portfolio (3) against resource constraints by solving a binary LP problem

( ) { } ,1,0,maxmax1 11,,1 1 ⎪⎭

⎪⎬⎫

⎪⎩

⎪⎨⎧

∈≤⇔ ∑ ∑∑∑∑= ==∈ =∈

m

j

m

jj

jj

n

i

jiijzz

px

n

i

jiiPp

zBxCzvwzvwmjF L

(5)

where equals 1 if and only if (i.e. project is included to portfolio ) and

0 if not. jz px j ∈ jx p

6

2.3. Incompleteness of Information The incompleteness of information in context of RPM means that parameter values are modeled by set inclusion. Instead of exactly known parameter values for project scores and DM’s preferences (i.e. weights), e.g. intervals, distributions, or rank orders are used. Methods for the elicitation of incomplete and complete weight information have been studied in the field of preference programming (e.g. Weber, 1987; Salo & Hämäläinen, 1992, 2001; Salo & Punkka, 2003; Salo & Hämäläinen, 2004; Salo & Punkka, 2005). In RPM (Liesiö et al.,2005), principles of preference programming are extended to multi-criteria portfolio problems. In Liesiö et al. (2005) and the algorithm analyzed, incompleteness of information is modeled by two approaches. First criterion weights are modeled by the set of feasible weights where is given by (2). Weight region i.e. set of weight vectors

is constrained by DM’s preference statements which are defined by liner inequalities. Total absence of weight information is defined by and a point estimate in

corresponds to complete weight information. If for example criterion 1 is at least as important as criterion 2, then

0ww SS ⊆ 0

wS wS

0ww SS = 0

wS

{ }210 | wwSwS ww ≥∈= .

Second, project scores are modeled with value intervals . The true value lies

between lower and upper bounds i.e. such that for all

and

⎥⎦

⎤⎢⎣

⎡

−

_

, ji

ji vv j

iv

⎥⎦

⎤⎢⎣

⎡∈

−

_

, ji

ji

ji vvv

_j

ij

ij

i vvv ≤≤−

mj ,,1L= ni ,,1L= . Therefore the set of feasible scores is given by

. There is no assumptions of probability distributions of

’s except that prob .

⎭⎬⎫

⎩⎨⎧

⎥⎦

⎤⎢⎣

⎡∈ℜ∈=

−

×+

_

,| ji

ji

ji

nmv vvvvS

jiv 1,

_

=⎟⎟⎠

⎞⎜⎜⎝

⎛⎥⎦

⎤⎢⎣

⎡∈

−

ji

ji

ji vvv

For each portfolio p , overall lower and upper bounds of the interval are defined through feasible scores and weights vSv∈ wSw∈ such that

( ) ( ) ∑∑∑∈ = −∈

==px

n

i

jii

px

j

jj

vwxVwpV1

, and (6)

( ) ( ) ∑∑∑∈ =∈

==px

n

i

jii

px

j

jj

vwxVwpV1

_

, , (7)

7

where V and V are linear in w . The overall value of for portfolio is thus given by p

( ) ( ) ( )⎥⎦⎤

⎢⎣⎡∈

∈∈wpVwpVvwpV

ww SwSw,max,,min,, . (8)

Non-empty set called information set is formed as a product of feasible weight and score parameters, i.e.

S wS

vS vw SSS ×≡ and denotation ( ) Svws ∈= , implies and .

wSw∈

vSv∈

While analysis is based on set of feasible parameters i.e. when incomplete information is used, there is no optimal portfolio as a unique solution. Instead the analysis leads to a set of non-dominated portfolios which form a basis for decision recommendations. 2.4. Non-Dominated Portfolios With concept of dominance number of competing portfolios can be reduced to feasible ones. Because of incomplete information there is not one optimal portfolio but set of feasible portfolios. If a portfolio is dominated it can not be optimal within the feasible region i.e. in the view of the information set and is therefore discarded from further analysis.

S

Portfolio is said to dominate portfolio p p′ if the overall value of is greater than or equal to the overall value

( )vwpV ,, p( )vwpV ,,′ of p′ for all feasible parameters

(scores and weights), and greater for at least one feasible combination of the parameters. With project score intervals, if the least (6) possible value ( )wpV , of greater than the highest (7) possible value

p( )wpV ,′ of p′ , dominates p p′ . This dominance is denoted

as and formally stated. pp ′f

Definition 1: Let . Ppp ∈′, pp ′f if

( ) ( )vwpVvwpV ,,,, ′≥ ∀ ( ) Svw ∈, and (9)

( ) ( )vwpVvwpV ,,,, ′> ∃ ( ) Svw ∈, .

Computationally, dominance checks can be modeled by (10).

8

Theorem 1: For and any information set , ∀ Ppp ∈′, S pp ′f if and only if

( ) ( )[ ] 0,\,\min ≥′−′

∈wppVwppV

wSw and (10)

( ) ( )[ ] 0,\,\max >′−′∈

wppVwppVwSw

Naturally there is no rational for a DM to consider p′ if and dominated

portfolios can therefore be discarded from further analysis. If a portfolio is not dominated by any other portfolio, it’s called non-dominated portfolio.

pp ′f

Definition 2: The set of non-dominated portfolios is defined NP

{ ppPpP FN f′∈= | ∀ }FPp ∈′ . (11)

Determination of dominance (pairwise) to acquire therefore requires pairwise

comparisons of portfolios. NP

9

3. COMPUTATION OF NON-DOMINATED PORTFOLIOS Non-dominated portfolios are computed to narrow the scope of potential portfolios for a DM. As number of project proposals, for example, in a corporation’s division can be large there is a definite need for an algorithm to tackle the portfolio selection problem. A single objective CB problem can be formed as binary LP-problem (Luenberger, 1998) and several algorithms have been developed to this context (e.g. Ignizio & Cavalier, 1994). This simple CB problem can be also modeled to context of combinatorial optimization as a knapsack problem for which several algorithms have been developed. The single criterion knapsack searches a subset, from set of projects, that maximizes a linear function of the chosen projects subject to one inequality constraint (e.g. Mathello & Toth, 1990; Captivo et al., 2003). There is no room for additional or logical

constraints and complete information about project scores i.e. _

ji

ji vv =−

∀ ji, is

required. Therefore it can not serve as efficient algorithm for computation of non-dominated portfolios as such. Problems with several resource constraints and with multiple benefits can be modeled with multi-dimensional and multi-objective knapsack problems, respectively (see e.g. Erlebach et al., 2002). These methods have, however, focused on the computation of Pareto-optimal solutions. A portfolio is considered Pareto-optimal when there is no feasible portfolio which has as least as high scores in terms of all criteria and higher in terms at least one criterion. On the other hand, multi-dimensional knapsack algorithms would be suitable to search non-dominated portfolios if complete information concerning project scores would be available. Knapsack algorithms which are based on dynamic programming have been introduced e.g. in Mathello & Toth, 1990 and Erlebach et al., 2002. Dynamic programming algorithms are often applied to optimization problems and basically proceed by

i) characterizing the structure of an optimal solution, ii) defining recursively the value of an optimal solution, iii) computing the value of an optimal solution (bottom-up), and iv) constructing an optimal solution from computed information.

10

Steps i)–iii) form the basis of a dynamic programming solution to a problem where iv) can be omitted if the value of an optimal solution is required (see e.g. Cormen et al., 1997). Knapsack algorithms also consider the search-task as a binary tree search where fathomed nodes of the tree are harvested. However, there are no reported algorithms for multi-objective knapsack problems with incomplete score information. Strummer & Heidenberger (2003) presented a three phase brute force algorithm for determining Pareto-optimal portfolios in MCCB context. Because Pareto-optimality of all portfolios is calculated, the algorithm is relatively inefficient limiting number of projects to be included to analysis to about 30. Enumeration of all portfolios mP 2=

(i.e. brute force –method) is not attractive computationally since calculation of portfolios with e.g. 40 projects would take days although calculation with 20 projects is done in couple seconds. This can be seen as a major drawback since number of interdependent projects can be hundreds and therefore make analysis practically almost impossible. The presented method assumed no DM’s prior preference information over criterion, but on the other hand, criterion specific scores were modeled with point estimates. A dynamic programming algorithm for non-dominated portfolios is introduced in Liesiö et al. (2005). The algorithm build on the work of Villarreal and Karwan (1981) and it corresponds to multi-objective and multi-dimensional knapsack algorithm. Project score intervals and weight information are handled with dynamic programming algorithm. In the algorithm, projects are treated sequentially and only portfolios that use resources efficiently are stored for subsequent rounds. Set of non-dominated portfolios is obtained by structuring auxiliary sets of non-dominated portfolios. Even simple knapsack problems belong to the group of NP-hard problems. Robustness i.e. incomplete information makes calculation efficiency more challenging as the number of project proposals and/or number of selection criteria is increased. Therefore the performance of the algorithm needs to be analyzed. The analysis is done by simulating the algorithm with randomized data to solve non-dominated portfolios. By simulating the algorithm, information about its applicability can be given. The algorithm introduced in Liesiö (2004) and analyzed in this study is based on knapsack type binary-tree search. The algorithm has three phases (expand, update and backtrack) and it moves in binary-tree determining dominated branches using dynamic programming before starting the binary-tree search.

11

In the expand phase, affordable project proposals are included to portfolio in the order of project indexes. The algorithm is made more efficient by sorting projects in the binary tree according to their value in the center of the feasible weight space relative to the average resource consumption. Projects are included to portfolios with LIFO-principle (last in, first out). When the portfolio is full, the algorithm moves to the update phase. Portfolio is discarded from set of non-dominated portfolios in the update phase if it does not satisfy additional constraints. In this case the algorithm will move to the next phase. If this is not the case, however, all non-dominated portfolios which are dominated by the current portfolio are discarded and current portfolio is added to the set of the non-dominated portfolios. In the backtrack phase project proposals are excluded from portfolio using LIFO-principle.

12

4. SIMULATION DESIGN FOR THE ALGORITHM To gain insight about performance of the algorithm, its computational efficiency needed to be simulated. We examine algorithms calculation time and number of non-dominated portfolios found by varying parameters (number of projects and selection criteria and project score intervals) describing projects instances. Impact of weight information

to the computational efficiency is discarded from the analysis. Impact of weights to computational efficiency is examined in Salminen (2005).

0wS

4.1. Experiment Design The simulation was done by generating realizations of projects by varying different parameters which were thought to have most significant impact to calculation time and number of non-dominated portfolios found. These parameters were number of projects, number of selection criteria, and intervals for project scores as in Table 1 below. Table 1. Simulation Design Parameters.

20 2 0

30 3 0.1

40 4 0.2

No. of Projects

No. of Criteria

Score Interval

Therefore total number of different simulation cases was 27 ( 333 ⋅⋅= ). The increase in number of projects and/or selection criteria will clearly result in increased computational effort needed. Lower and upper bounds of score intervals are calculated with parameter (see section 2.3 and (9)). Inclusion of score intervals (i.e. ), which is the main focus of the analysis, is also expected to increase computational effort needed. Compared to point estimates, intervals should increase the number of solutions i.e. non-dominated portfolios. This should also apply to increasing width of the score intervals (i.e. d from 0 to 0.1 or from 0.1 to 0.2).

m n

d 0>d

On the other hand, use of preference statements decrease computation effort since feasible weight space is narrowed and therefore the number of non-dominated portfolios is decreased. The relative decrease in computational effort needed resulting from preference statements is thought to be more significant in problems where is high n

13

(Liesiö, 2004). Observations and computational tests made in Liesiö (2004) indicate that the algorithm is somewhat efficient at least for problems where 5≤n and . 50≤m 4.2. Randomized Generation of Project Realizations To enable statistical analysis of the algorithm’s performance, each simulation case was run 50 times with different data sets making total number of runs 1350 ( ). Randomized projects point estimate scores were first generated for case of 50 project proposals for each data set. Thus the total number of generated project proposals was 2500 ( ).

50333 ⋅⋅⋅=

5050 ⋅= The simulations were needed to be done with same data to ensure comparability. Thus the same project score data was used by adjusting the algorithm for number of project proposals ( ) and number of criteria (40,30,20=m 4,3,2=n ). Intervals for criterion specific scores were formed by adjusting the algorithm ( 2.0,1.0,0=d ) for generated

point estimates. The algorithm calculates then lower and upper bounds for the intervals as

and (9) ( ) ji

ji vdv ⋅−=−

1 ( ) ji

ji vdv ⋅+= 1_

where all criterion specific scores are assumed to be uniformly distributed real

numbers

jiv

for ( 1,0~ Uniformv j

i ) 4,3,2=i and 50,,1K=j . (10)

The project proposals are also needed to be assigned a cost. It was assumed that projects with greater scores cost more on the average. Thus cost could be assigned to a project proposal as of sum criterion specific scores and a randomized component drawn from lognormal distribution. Cost of project jC j is given by

∑=

=n

i

ji

yj veCj

1

(11)

where (see Salminen, 2005). Standard deviation ( σ,0~ Normy j ) σ of 0,95 was used

to ensure sufficient variation for ’s (see table 2). jC

14

Table 2. Frequencies of ’s and ’s. jye jC

Count % Count %165 7 % 455 6 %359 14 % 801 11 %681 27 % 1,488 20 %695 28 % 1,937 26 %427 17 % 1,649 22 %173 7 % 1,170 16 %

Total 2500 100 % 7500 100 %

Category

FrequenciesjCjye

41,0 <≤ jy Ce j

21,41 <≤ jy Ce j

1,21 <≤ jy Ce j

2,1 <≤ jy Ce j

4,2 <≤ jy Ce j

4, >jy Ce j

In table 2, the total count of ’s is three times of ’s. This results from usage of identical ’s for ’s according to (11) (2,3 and 4 criteria problems). Frequencies are presented graphically in figures 5 and 6 in appendix 1.

jC jyejye jC

Resource constraint i.e. budget was also needed for problem. The budget was set to cover 20% of the costs of project proposals in each simulation case.

15

5. PERFORMANCE OF THE ALGORITHM The simulation was conducted with desktop workstation which configuration is presented in table 3. Simulation results would be affected to some extent by calculation and memory capacity of the simulation computer. Table 3. Configuration of the Simulation Desktop.

Processor 2.4 GHz Intel™

Physical Memory 1024 MB

Operating System Windows XP™

Some of the smallest simulation cases were solved in less than one millisecond and therefore slightly biased results of these cases. This was either due the fact that the algorithm and its supporting resources were written in Java™ or because of the operating system of the workstation. Before the actual simulations it also came apparent that the project proposal data has impact on the difficulty of calculation task and therefore the results too. The algorithm was tested with various data sets and the mean calculation time and mean number of non-dominated portfolios varied significantly between simulation cases of same size but different data. 5.1. General Overview of the Results All simulations with point estimate scores (i.e. 0=d ) were completed. However when score intervals were employed, some of the larger simulation cases proved to be too time consuming to calculate in reasonable time (see table 4). These cases had to be discarded from the analysis as calculation of 50 data sets would have taken several hours to finish. Table 4. Discarded Simulation Cases.

30 4 0.2

40 3 0.1

40 3 0.2

40 4 0.1

40 4 0.2

No. of Projects

No. of Criteria

Score Interval

16

In all simulation cases completed, increase in had more significant impact on mean and variation of calculation time than increase in . When score intervals were em-ployed, estimation of calculation time became worthless as size of simulation case was increased to with 0.1 score intervals and

mn

4,30 == nm 3,30 == nm with 0.2 score

intervals. 5.2. Results with Point Estimate Scores Mean calculation time of the algorithm with point estimates for all simulation cases is presented in figure 1. Table 7 in appendix 2 present summarized results of the calculation time. Clearly, the mean calculation time increases significantly with both,

and n , but relatively more with m . Similar increase can be examined in confidence intervals. For the small and medium size cases (m

30,20=m and 3,2=n ), however, mean

and variations can be somewhat biased due 0 millisecond calculation time of some data sets.

Figure 1. Mean Calculation Time of the Algorithm with Po

otal calculation time of 50 data sets for all simulation cases is presented in table 5 be-

m ) to

int Estimates. Tlow. With 3≤n and 40≤m calculation of 50 data sets was somewhat reasonable as worst case ,40 n ok about 4 minutes and 45 seconds to complete. On the other hand, the calculation time of the largest case ( 4,40

( 3==== nm ) was about 42 minutes

although case with 4,30 == nm was completed in l ute.

ess than min

17

Table 5. Total Calculation Time of 50 Data Sets.

fferent simulation cases is presented in igure 2. As calculation time of the algorithm, the mean number of non-dominated port-

Mean number of non-dominated portfolios of diFfolios increases as well with m and n . However, as the mean calculation time increased relatively more with m in all cases, the mean number of non-dominated portfolios increased relatively more wit n in smallest cases (from 2,20h == nm to 3,20 == nm compared to 2,30= nm ) and with m in other cases. The number of non-dominated portfolios stays reasonable (under 200) when 3,40

=≤≤ nm 30≤

or when 4, ≤nm .

timate Scores. The mean number of non-dominated portfolios is already

Figure 2. Number of Non-Dominated Portfolios with Point Es

difficult to calculate when 4,40 == nm . Case of 5,50 == nm would result in too large amount of non-

ominated portfolios. However, a simulation case of 50 data sets could be extended

in Milliseconds [in Minutes]

20 220 [0.00]

30 1,527 [0.03]

40 12,604 [0.21]

20 478 [0.01]

30 8,622 [0.14]

40 286,122 [4.77]

20 1,188 [0.02]

30 57,675 [0.96]

40 2,538,180 [42.30]

No. of Criteria

No. of Projects

2

3

4

d

18

either direction alone. For example cases with 5,30 == nm or 3,50 == nm would be

if the mea of non-dominated portfolios would increase accordingly. . 5.3. Results with Score Intervals

manageable n number

hen score intervals were used mean calculation tim sig-estimates (see figure 3). Tables 8 & 9 in appendix 2

resent summarized results of the calculation time. The mean calculation time and its

W e of the algorithm increasednificantly compared when point pvariation again increased relatively more with m (than n ). Increases were larger com-pared ones with point estimates. Increases in the mean calculation time relative to m were also larger when 0.2 score intervals were used compared to 0.1 score intervals. Estimation of the calculation time becomes unreasonable with 0.1 score intervals when

4,30 == nm and with 0.2 score intervals when 3,30 == nm . This is due significant

lculation times of 50 data se ple with 0.2 score intervals and variation in ca ts. For examcase with 3,30 == nm , quickest data set was calculated in 0.58 seconds and the worst

ost 15 minutes to complete (see table 10 in appendix 2). With 0.1 score intervals and case with 4,30case took alm

== nm , the worst case data set took about 118 minutes to

finish as the quickest was solved in 0.13 seconds (see table 10 in appendix 2).

Figure 3. Calculation Time of the Algorithm with 0.2 (Left) and 0.1 (Right) Score Intervals.

19

Wide confidence intervals in figure 3 above do not relate to any specific data set that s equal for all projects but cost data was could bias the results. The score data used wa

calculated in terms of number of criteria according to (10) and (11). For example, the fastest data set of case 3,30 == nm with 0.2 score intervals does not relate to the fastest data set of case 4,30 == nm with 0.1 score intervals. However, the fastest or

worst case data sets tend t een cases with same number of criteria used. Table 6 presents total calculation time of 50 data sets for all simulation cases with s

o same betw

core tervals. Most time consuming completed case was in 4,30 == nm with 0.1 score inter-

vals, lasting over 2 hours. With 0.2 score intervals, case with 4,20 == nm last over 47

minutes to calculate 50 times. Simulations of discard e table 4) were can-celled after 2 hours as only couple (at most) data sets were calc

ed cases (seulated.

able 6. Total Calculation Time of 50 Data Sets with Score Intervals.

rent simulation ases is when score intervals were used. As expected, when score intervals are used the umber of non-dominated portfolios is significantly higher compared to when point es-mates are used.

re used the mean numb of non-dominated portfolios increased rela-vely more with in all cases.

in Milliseconds [in Minutes] in Milliseconds [in Minutes]

20 298 [0.00] 766 [0.01]

30 13,604 [0.23] 126,689 [2.11]

40 902,730 [15.05] 2,867,427 [47.79]

20 1,222 [0.02] 3,410 [0.06]

30 140,338 [2.34] 1,818,165 [30.30]

40 N.A. N.A. N.A. N.A.

20 3,632 [0.06] 10,344 [0.17]

30 7,871,901 [131.20] N.A. N.A.

40 N.A. N.A. N.A. N.A.

2

3

4

0.1 Interval 0.2 IntervalNo. of Criteria

No. of Projects

T

Figure 4 presents mean number of non-dominated portfolios of diffecnti Results with 0.1 score intervals are more similar (than ones with 0.2 score intervals), compared to ones with point estimates, in sense that the mean number of non-dominated portfolios increased relatively more with m excluding the smallest cases. When 0.2 score intervals we erti m

20

Figure 4. Number of Non-Dominated Portfolios with 0.1 (Left) and 0.2 (Right) Score Intervals. When 0.1 score intervals were employed, the number of non-dominated portfolios was reasonable when or 3,30 ≤≤ nm2,40 =≤ nm . With 0.2 score intervals even case of

produced mean of over 500 non-dominated portfolios. Clearly case of ld d with both score intervals. With 0.1 score interval,

could produce a manageable amount of non-dominated portfolios

3,30 == nm5,20 == nm cou be calculate

case of 2,50 == nm

assuming that number of non-dominated portfolios would increase accordingly.

21

6. CONCLUSIONS

ith point estimate scores the algorithm completed all simulations with 50 data sets fficiently as worst case was calculated under 5 minutes. When point estimate scores are sed a simulation case of 50 data sets could be extended from by (from 4 to 5) if

or by (from 40 to 50) if

We

n30=

um 3=nm . Larger cases in terms of and could not

e calculated in reasonable time.

non-dominated portfolios with point estimates increased with nd as the calculation time of the algorithm (excluding smallest cases). The number

n mb

m The mean number of na

of non-dominated portfolios was under 200 when 3,40 ≤≤ nm or 4,30 ≤≤ nm but

cases could be extended either direction alone.

core int rvals were used the r simulation cases could no be co pleted. As expected, employment of score iWhen s e large t m

ntervals increased significantly the mean calculation me and the mean number of non-dominated portfolios. This applied also to widening

n . C

tiof the score intervals from 0.1 to 0.2. With 0.1 score intervals, the algorithm calculated efficiently when ≤m ases with 2,40 =≤ nm and 3,20

4,30 ≤== nm were efficiently calculated with 0.2 score

tervals. Due to variations in calculation time of different data sets, estimation of the

he number of non-dominated portfolios was significantly higher when score intervals

er of non-do

inalgorithm is unvaluable with larger cases when score intervals are used. The variation of the results in the calculation time of the algorithm are to some extent dependable of the variation of data in the data sets. Twere used and increased with m and n as well. With 0.2 score intervals the mean numb minated portfolios was significantly higher than with 0.1 score intervals. However, with either interval the smallest simulation cases could be extended by n and with 0.1 score intervals also by m .

22

REFERENCES 1. Brealey, R. A., Myers S. C. (1996). Principles of Corporate Finance. Fifth Edition.

McGraw-Hill Companies, Inc, New York.

. Captivo, Eugénia M., Clímaco, João, Figueira José, Martins, Ernesto & José Luis Santos (2003). Solving Bicriteria 0–1 Knapsack Problems Using a Labelling Algorithm. Computers & Operations Research, Vol. 30, pp. 1865–1886.

, (1996). Making Hard Decisions – An Introduction to Decision Analysis. Second Edition. Duxbury Press, Belmont.

4. Coop inschmidt (1999). New Product Portfolio Management: Practises and Performance. Journal of Product

5. Corm ction

to Algorithms. The MIT Press, Cambridge.

6. Erleb ting Multiobjective Knapsack Problems. Management Science, Vol. 48, pp. 1603–1612.

7. Gustaarch, Vol. 53, pp. 946–958.

. 26–31.

2.

ood

ridge.

2

3. Clemen, R. T.

er, Robert G., Edgett, Scott J. & Elko J. Kle

Innovation Management, Vol. 16, pp. 333–351.

en, Thomas C., Leiserson, Charles E. & Ronald L. Rivest (1997). Introdu

ach, T., Kellerer, H., Pferschy, U. (2002). Approxima

fsson, J. & Salo, A. (2005). Contingent Portfolio Programming for the Management of Risky Projects. Operations Rese

8. Hämäläinen, R.P., (2004). Reversing the Perspective on the Applications of Deci-

sion Analysis. Decision Analysis, Vol. 1, pp 9. Keefer, D.L., Kirkwood, C.W., Corner, J.L., (2004). Perspective on Decision

Analysis Applications, 1990–2001. Decision Analysis, Vol. 1, pp. 4–2 10. Ignizio, J. & Cavalier, M. (1994). Linear Programming. Prentice Hall, Englew

Cliffs. 11. Keeney, R. L., Raiffa, H. (1976). Decision with Multiple Objectives: Preferences

and Value Trade-offs. John Wiley & Sons, Inc, Camb

23

12. Liesiö, J. (2004). Robust Multicriteria Optimization for Project Portfolios. Master’s .

Available from WWW: http://www.sal.hut.fi/Publications/pdf-files/tlie04.pdf

13. Liesiö, J., Mild, P. & Salo, A. (2005). Preference Programming for Robust Portfolio Modeling and Project Selection. European Journal of Operational Research (to

4. Lindstedt, M., Liesiö, J., Salo, A., (2005). Participatory Development of a Strategic

5. Lorie, J. & Savage, L. (1955). Three Problems in Capital Rationing. Journal of

16. Luenberger, D. G., (1998). Investment Science, Oxford University Press, New

York.

17. MathChichester.

Independent Research Project in Applied Mathematics, Helsinki University of Technology, Systems Analysis

df-files/esal05.pdf

–1061.

n ion.

33–545. 21. Salo, ). Preference Programming. Helsinki University of

Technology, Systems Analysis Laboratory, Manuscript.

Thesis, Helsinki University of Technology, Systems Analysis Laboratory

appear).

1Product Portfolio in a Telecommunication Company, International Journal of Technology Management (to appear).

1Business, Vol. 28, pp. 229–239.

ello, S., Toth, P (1990). Knapsack Problems: Algorithms and Computer Implementations. John Wiley & Sons,

18. Salminen, Lauri (2005). A Simulation Study on the Computation of Potentially

Optimal Multicriteria Portfolios.

Laboratory. Available from WWW: http://www.sal.tkk.fi/Opinnot/Mat-2.108/p

19. Salo, A. & Hämäläinen, R. (1992). Preference Assesment by Imprecise Ratio

Statements. Operations Research, Vol. 40, pp. 1053 20. Salo, A. & Hämäläinen, R. (2001). Preference Ratios in Multiattribute Evaluatio

(PRIME) – Elicitation and Decision Procedures under Incomplete InformatIEEE Transactions on Systems, Man, and Cybernetics. Vol. 31, pp. 5

A. & Hämäläinen, R. (2004

24

25

n

athematical Programming, Vol. 21, pp. 204–223.

25. Webe on. European Journal of Operational Research. Vol. 28, pp. 44–57.

22. Salo, A. & Punkka, A. (2005). Rank Inclusion in Criteria Hierarchies. EuropeaJournal of Operational Research, Vol. 163, pp. 338–356.

23. Srummer, C., Heidenberger, K. (2003). Interactive R&D Portfolio Analysis with

Project Interdepencies and Time Profiles of Multiple Objectives. IEEE Transactions on Engineering Management, vol. 50, pp. 175–183.

24. Villarreal, B. & Karwan, M. H. (1981). Multicriteria Integer Programming: A

(Hybrid) Dynamic Programming Recursive Algorithm. M

r, M. (1987). Decision Making with Incomplete Informati

Figure 5. Frequencies of ’s. Figure 6. Frequencies of ’s. jye jC

455 801 1488 1937 1649 1170

410 <≤ jC 2141 <≤ jC 121 <≤ jC 21 <≤ jC 42 <≤ jC 4>jC

455 801 1488 1937 1649 1170455 801 1488 1937 1649 1170

410 <≤ jC 2141 <≤ jC 121 <≤ jC 21 <≤ jC 42 <≤ jC 4>jC

26

165 359 681 695 427 173

410 <≤ jye 2141 <≤ jye 121 <≤ jye 21 <≤ jye 42 <≤ jye 4>jye

165 359 681 695 427 173165 359 681 695 427 173

410 <≤ jye 2141 <≤ jye 121 <≤ jye 21 <≤ jye 42 <≤ jye 4>jye

APPENDICES

Appendix 1

Appendix 2

Table 7. Calculation Time of the Algorithm with Point Estimate Scores.

8. Calculation Time of the Algorithm with 0.1 Score Intervals.

27

No. of No. ofCriteria Projects

20 4 [0.00] 10 [0.01] 0 [0.00] 0 [0.00] 0 [0.00] 0 [0.00] 62 [0.06]

30 31 [0.03] 36 [0.04] 0 [0.00] 15 [0.02] 16 [0.02] 31 [0.03] 203 [0.20]

40 252 [0.25] 268 [0.27] 46 [0.05] 78 [0.08] 140 [0.14] 339 [0.34] 1,265 [1.27]

20 10 [0.01] 11 [0.01] 0 [0.00] 0 [0.00] 15 [0.02] 16 [0.02] 46 [0.05]

30 172 [0.17] 157 [0.16] 31 [0.03] 47 [0.05] 118 [0.12] 231 [0.23] 687 [0.69]

40 5,722 [5.72] 10,704 [10.70] 235 [0.24] 973 [0.97] 2,469 [2.47] 5,410 [5.41] 67,289 [67.29]

20 24 [0.02] 34 [0.03] 0 [0.00] 15 [0.02] 16 [0.02] 27 [0.03] 188 [0.19]

30 1,154 [1.15] 1,389 [1.39] 31 [0.03] 301 [0.30] 555 [0.55] 1,425 [1.43] 6,656 [6.66]

40 50,764 [50.76] 78,302 [78.30] 2,890 [2.89] 14,512 [14.51] 35,030 [35.03] 59,634 [59.63] 551,467 [551.47]

3

4

2

Mean StdPercentile

in Milliseconds [in Seconds]

Min 25 MaxPercentile

Median 75

Table


20 6 [0.01] 11 [0.01] 0 [0.00] 0 [0.00] 0 [0.00] 15 [0.02] 47 [0.05]

30 272 [0.27] 570 [0.57] 31 [0.03] 67 [0.07] 102 [0.10] 223 [0.22] 3,186 [3.19]

40 18,055 [18.05] 43,130 [43.13] 390 [0.39] 1,340 [1.34] 4,633 [4.63] 10,837 [10.84] 216,147 [216.15]

20 24 [0.02] 33 [0.03] 0 [0.00] 15 [0.02] 16 [0.02] 16 [0.02] 141 [0.14]

30 2,807 [2.81] 6,630 [6.63] 94 [0.09] 414 [0.41] 1,047 [1.05] 3,043 [3.04] 46,250 [46.25]

40 N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A. N.A.

20 73 [0.07] 135 [0.14] 0 [0.00] 31 [0.03] 32 [0.03] 47 [0.05] 688 [0.69]

30 157,438 [157.44] 1,005,231 [1005.23] 125 [0.13] 2,434 [2.43] 5,227 [5.23] 17,630 [17.63] 7,121,609 [7121.61]


3

4

2

Mean Std Min 25 Median MaxPercentilePercentile

75


Table 9. Calculation Time of the Algorithm with 0.2 Score Intervals.


20 15 [0.02] 35 [0.04] 0 [0.00] 0 [0.00] 8 [0.01] 16 [0.02] 235 [0.24]

30 2,534 [2.53] 7,238 [7.24] 68 [0.07] 348 [0.35] 609 [0.61] 1,323 [1.32] 42,924 [42.92]

40 57,349 [57.35] 99,874 [99.87] 986 [0.99] 10,026 [10.03] 24,977 [24.98] 61,117 [61.12] 488,419 [488.42]

20 68 [0.07] 112 [0.11] 0 [0.00] 16 [0.02] 31 [0.03] 47 [0.05] 454 [0.45]

30 36,363 [36.36] 128,181 [128.18] 578 [0.58] 4,090 [4.09] 7,914 [7.91] 26,168 [26.17] 894,797 [894.80]


20 207 [0.21] 464 [0.46] 0 [0.00] 47 [0.05] 78 [0.08] 133 [0.13] 2,313 [2.31]




Max

4

2

3

PercentilePercentile75Mean 25 MedianMinStd

mat-2.108 independent research project in applied...

Documents