stochastic linear programming by series of monte-carlo estimators

Stochastic Linear Programming by Series of Monte-Carlo Estimators

Leonidas SAKALAUSKASInstitute of Mathematics&Informatics

Vilnius, LithuaniaE-mail: <sakal;@ktl.mii.lt>

CONTENT Introduction Monte-Carlo estimators Stochastic differentiation

Dual solution approach (DS) Finite difference approach (FD) Simulated perturbation stochastic approximation (SPSA) Likelihood ratio approach (LR)

Numerical study of stochastic gradient estimators Stochastic optimization by series of Monte-Carlo

estimators Numerical study of stochastic optimization

algorithm Conclusions

Introduction

We consider the stochastic approach for stochastic linear problems which distinguishes by adaptive regulation of the Monte-Carlo estimators statistical termination procedure stochastic ε–feasible direction approach to avoid

“jamming” or “zigzagging” in solving a constraint problem

Two-stage stochastic programming problem with recourse

],|[min),( my RyhxTyWyqxQ

subject to the feasible set

nRxbxAxD ,

where

( ) ( , ) minnx D

F x c x E Q x

W, T, h are random in general and defined by absolutely continuous probability density )(p

Monte-Carlo estimators of objective function

)(py j

Let the certain number N of scenarios for some is provided :

),,...,,( 21 NyyyY

and the sampling estimator of the objective function

as well as the sampling variance are computed

nRx

N

j

j xQyxQN

xD1

22 ))(~),((1

1)(

)(~)(~ xQxcxF

N

j

jyxQN

xQ1

),(1)(~

The gradient1

1( ) ( , ),N

j

j

g x g x yN

Monte-Carlo estimators of stochastic gradient

1

1( ) , ,N Tj j

j

A x g x y g x g x y g xN n

as well as the sampling covariance matrix:

are evaluated using the same random sample, where

)(),( xFxEg

Statistical testing of optimality hypothesis under asymptotic normality

),,(/))(~())(())(~()( 12

nNnFishnxgxAxgnNT T

Optimality hypothesis is rejected if

1) the statistical hypothesis of equality of gradient to zero is rejected

NxD /)(~2

2) or confidence interval of the objective function exceeds the admissible value

Stochastic differentiation

We examine several estimators for stochastic gradient:Dual solution approach (DS);Finite difference approach (FD);Simulated perturbation stochastic approach

(SPSA);Likelihood ratio approach (LR).

Dual solution approach (DS)

The stochastic gradient is expressed as

using the set of solutions of the dual problem

1 *,g x c T u

],0|)[(max)( * mTTu

T uqWuuxThuxTh

Finite difference (FD) approach

In this approach the each ith component of the stochastic gradient is computed as:

is the vector with zero components except ith one, equal to 1, is certain small value

2 ,g x

2 ( , ) ( , )( , ) i

if x y f x y

g x y

i

Simulated perturbation stochastic approximation (SPSA)

where is the random vector, which components obtain values 1 or -1 with probabilities p=0.5, is some small value (Spall (2003))

2,,,3 yxfyxfyxg

Likelihood ratio (LR) approach

4 ( , ) ( ) ( ) ln ( )yg x y f x y f x p y

Rubinstein, Shapiro (1993), Sakalauskas (2002)

Methods for stochastic differentiation have been explored with testing functions

here

min)()( 0 xEfxF

20 1( ) ( (1 cos( )));n

i i i i iif y a y b c y

.i i iy x

Numerical study of stochastic gradient estimators (1)

),0( 2dNi

Numerical study of stochastic gradient estimators (2)Stochastic gradient estimators from samples of size (number

of scenarios) N was computed at the known optimum point X (i.e. ) for test functions, depending on n parameters.

This repeated 400 times and the corresponding sample of Hotelling statistics was analyzed according to and criteria

0)( xF

2T 22

criteria on variable number n and Monte Carlo sample size N (critical value 0,46)

2

Nn

50 100 200 500 1000

2 0.30 0.24 0.10 0.08 0.043 0.37 0.12 0.09 0.06 0.044 0.19 0.19 0.13 0.08 0.045 0.75 0.13 0.12 0.08 0.066 1.53 0.34 0.10 0.10 0.087 1.56 0.39 0.13 0.08 0.098 1.81 0.42 0.27 0.18 0.109 4.18 0.46 0.26 0.20 0.1210 8.12 0.56 0.53 0.25 0.17

criteria on variable number n and Monte Carlo sample size N (critical value 2,49)

2

N n

50 100 200 500 1000

2 2.57 1.14 0.66 0.65 0.423 2.78 0.82 0.65 0.60 0.274 3.75 1.17 0.79 0.53 0.315 4.34 1.46 0.85 0.64 0.366 8.31 2.34 0.79 0.79 0.767 8.14 2.72 1.04 0.52 0.458 10.22 2.55 1.87 0.89 0.529 20.86 2.59 1.57 1.41 0.7810 40.57 3.69 3.51 1.56 0.98

Sample size, N

1000 0,92 5,611500 0,76 4,152000 0,55 3,632100 0,68 2,842200 0,23 1,282500 0,19 1,143000 0,12 0,66

Statistical criteria on Monte Carlo sample size N for number of variable n=40 (critical values 0,46 ir 2,49)

2 2

Imties tūris, N

1000 4,42 23,112000 1,31 6,463000 1,17 6,053300 0,46 2,423500 0,22 1,254000 0,09 0,56

Statistical criteria on Monte Carlo sample size N for number of variable n=60

(critical values 0,46 ir 2,49)

2 2

Imties tūris, N

1000 15,53 83,262000 5,39 27,675000 0,79 3,976000 0,27 1,487000 0,13 0,6810000 0,07 0,39

Statistical criteria on Monte Carlo sample size N for number of variable n=80(critical values 0,46 ir 2,49)

2 2

Conclusion: T2-statistics distribution may be approximated by Fisher law, when number of scenarios:

Variable number, n

Number of scenarios , Nmin

(Monte Carlo sample size)20 100040 220060 3300100 6000

Numerical study of stochastic gradient estimators (8)

Frequency of optimality hypothesis on the distance to optimum (n=2)

Dual Solution

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3

N=100 N=200 N=500 N=1000 N=5000

Finite Difference

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3

N=100 N=200 N=500 N=1000 N=5000

SPSA

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3 0.4 0.5

N=100 N=200 N=500 N=1000 N=5000

Likelihood Ratio

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3, 0.4 0.5 0.6 0.7 0.8

N=100 N=200 N=500 N=1000 N=5000

Frequency of optimality hypothesis on the distance to optimum (n=10)

Dual Solution

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Serija1 Serija2 Serija3 Serija4 Serija5

Finite Difference

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3 0.4 0.5

N=100 N=200 N=500 N=1000 N-5000

SPSA

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3 0.4 0.5

N=100 N=200 N=500 N=1000 N=5000

Likelihood Ratio

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3, 0.4 0.5 0.6 0.7 0.8

N=100 N=200 N=500 N=1000 N=5000

Frequency of optimality hypothesis on the distance to optimum (n=20)

Dual Solution

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7

Serija1 Serija2 Serija3 Serija4 Serija5

Finite Difference

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3 0.4 0.5

N=100 N=200 N=500 N=1000 N-5000

SPSA

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3 0.4 0.5

N=100 N=200 N=500 N=1000 N=5000

Likelihood Ratio

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

100%

0 0.1 0.2 0.3, 0.4 0.5 0.6 0.7 0.8

N=100 N=200 N=500 N=1000 N=5000

Frequency of optimality hypothesis on the distance to optimum (n=50)

Frequency of optimality hypothesis on the distance to optimum (n=100)

Conclusion: stochastic differentiation by Dual Solution and Finite Difference approaches enables us to reliably estimate the stochastic gradient, when: .

SPSA and Likelihood Ratio works when

2 100n

2 20n

Numerical study of stochastic gradient estimators (14)

Gradient search procedure

1( ) 0, 0, 0 ( )ni n j j xV x g Ag g if x g

)(~ txg

Let some initial point be chosen, the random sample of a certain initial size N0 be generated at this point, and Monte-Carlo estimators be computed. The iterative stochastic procedure of gradient search is:

x D R n0

)(~1 ttt xgxx

where the projection of to ε - feasible set:)(~ txg

The rule to choose number of scenarios

We propose a following rule to regulate number of scenarios:

maxmin11 ,,

)(~())(()(~(),,(maxmin NNn

xgxAxgnNnFishnN ttTt

tt

Thus, the iterative stochastic search is performed until statistical criteria don’t contradict to optimality conditions

Linear convergenceUnder some conditions on finiteness and smooth

differentiability of the objective function the proposed algorithm converges a.s. to the stationary point:

0)(lim2

tV

t

txF

with linear rate

t

tt

CLKl

NKxx

NKxxE

)(12

0

202

where K, L, C, l are some constants (Sakalauskas (2002), (2004))

Linear ConvergenceSince the Monte-Carlo sample size increases with

geometric progression rate it follows:

10

QNN tt

ii

Conclusion:

the approach proposed enables us to solve SP problems by computing a finite number times of expected objective function

Numerical study of stochastic optimization algorithm Test problems have been solved from the Data Base of

two-stage stochastic linear optimisation problems:

http://www.math.bme.hu/~deak/twostage/ l1/20x20.1/ .

Dimensionality of the tasks from n=20 to n=80 (30 to 120 at the second stage)

All solutions given in data base are achieved and in a number of that we succeeded to improve the known decisions, especially for large number of variables

Two stage stochastic programming problem (n=20)

The estimate of the optimal value of the objective function given in the database is 182.94234 0.066

(improved to 182.59248 0.033 )

N0=Nmin=100, Nmax=10000 Maximal number of iterations , generation of

trials was broken when the estimated confidence interval of the objective function exceeds admissible value .

Initial data as follows:

Solution repeated 500 times

= =0.95; 0.99, 0.1; 0.2; 0.5; 1.0.

max 100t

Frequency of stopping under number of iterations and admissible confidence interval

0

20

40

60

80

100

1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96

1

0,5

0,2

0,1

Change of the objective function under number of iterations and admissible interval

182

182,5

183

183,5

184

184,5

1 12 23 34 45 56 67 78 89 100

0,1

0,2

0,5

1

Change of confidence interval under number of iterations and admissible interval

01234567

1 8 15 22 29 36 43 50 57 64 71 78 85 92 99

0,1

0,2

0,5

1

Change of the Hotelling statistics under admissible interval

0123456789

10

1 11 21 31 41 51 61 71 81 91

0,1

0,2

0,5

1

Change of the Monte-Carlo sample size under number of iterations and admissible interval

0

200000

400000

600000

800000

1000000

1200000

1400000

1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76 81 86 91 96

0,1

0,2

0,5

1

1

jt

tj

NN

0

5

10

15

20

25

30

8 10 12 14 16 18 20 22 24 26 28 30 32 34 36

0,1

0,2

0,5

1

Ratio under admissible interval (1)

Accuracy Objective Function

0.1 182.6101 20.14

0.2 182.6248 19.73

0.5 182.7186 19.46

1 182.9475 19.43

1

jt

tj

NN

1

jt

tj

NN

Ratio under admissible interval (2)

Two-stage SP problem first stage: 80 variables, 40 constraints second stage: 80 variables, 120

constraints

DB given solution 649.604 0.053. Solution by developed algorithm: 646.444 0.999.

Solving DB Test Problems (1)

Two-stage SP problem first stage: 80 variables, 40 constraints second stage: 80 variables, 120

constraints

DB given solution 6656.637 0.814. Solution by developed algorithm: 6648.548 0.999.

Solving DB Test Problems (2)

Two-stage SP problem first stage: 80 variables, 40 constraints second stage: 80 variables, 120

constraints

DB given solution 586.329 0.327. Solution by developed algorithm: 475.012 0.999.

Solving DB Test Problems (3)

Metodų palyginimas

0

0,5

1

1,5

2

2,5

0 50 100 150 200

Skaičiavimų laikas (s)

Pasi

klia

utin

as in

terv

alas

Dekompozicijos

Monte-Karlo

Comparison with Benders decomposition

0

0.5

1

1.5

2

2.5

0 20 40 60 80 100 120 140 160 180

Computing time (s)

Adm

issi

ble

inte

rval

Dekompozicijos Monte-Karlo

Comparison with Benders decomposition

Conclusions The stochastic iterative method has been developed

to solve the SLP problems by a finite sequence of Monte-Carlo sampling estimators

The approach presented is reasoned by the statistical termination procedure and the adaptive regulation of size of Monte-Carlo samples

The computation results show the approach developed provides estimators for a reliable solving and testing of optimality hypothesis in a wide range of dimensionality of SLP problems (2<n<100).

The approach developed enables us generate almost unbounded number of scenarios and solve SLP problems with admissible accuracy

Total volume of computations solving SLP exceeds only several times the volume of scenarios needed to evaluate one value of the expected objective function

References Rubinstein, R, and Shapiro, A. (1993). Discrete events systems:

sensitivity analysis and stochastic optimization by the score function method. Wiley & Sons, N.Y.

Shapiro, A., and Homem-de-Mello, T. (1998). A simulation-based approach to two-stage stochastic programming with recourse. Mathematical Programming, 81, pp. 301-325.

Sakalauskas, L. (2002). Nonlinear stochastic programming by Monte-Carlo estimators. European Journal on Operational Research, 137, 558-573.

Spall G. (2003) Simultaneous Perturbation Stochastic Approximation. J.Wiley&Sons

Sakalauskas, L. (2004). Application of the Monte-Carlo method to nonlinear stochastic optimization with linear constraints. Informatica, 15(2), 271-282.

Sakalauskas L. (2006) Towards implementable nonlinear stochastic programming. In Eds K.Marti et al. Coping with uncertainty, Springer Verlag

Announcements

Welcome to the EURO Mini Conference

“Continuous Optimization and Knowledge Based Technologies (EUROPT-2008)”

May 20-23, 2008, Neringa, Lithuania

http://www.mii.lt/europt-2008