OR705: Large Scale Linear Programming

Project Report

Solving Two-Stage Stochastic Programs: A Benders Decomposition

Based Approach

Binghui Li

1 Introduction

1.1 Motivation of project

Large engineering projects such as energy systems are subject to strong and long-live path depen-dencies, owing to technological, infrastructural, institutional and behavioral lock-ins [1]. In view ofaddressing climate change, policy makers need to make long-term plans before locking into carbon-intensive pathways, where redirecting would be difficult due to additional social and technologicalexpenses [1, 2]. Therefore, rather than a set of totally different plans from multiple scenarios, weshould consider a hedging strategy where future uncertainties are accounted for.

Stochastic programming belongs to the class of methods for decision making under uncertain-ties. Compared with deterministic optimization model, a stochastic programming model optimizesstage-wise decision variables such that the probability-weighted objective function is minimized/-maximized. In general, a stochastic programming model includes multiple stages separated by timepoints associated with a specific type of uncertainty and it is assumed that the uncertainty is notresolved until that time point. Therefore, the solution from a two-stage stochastic programmingmodel consists of two parts, the first-stage solution where future uncertainty is not resolved andthe second-stage solution, which include the same set of decision variables but with different values,representing decisions made after the uncertainty is resolved.

In the class of OR705, we have studied the method of Danzig-Wolfe decomposition and its dual form,Benders decomposition. Due to the structure of linear stochastic programs, Benders decompositionis an effective method to solve large scale linear stochastic program [3], where the second stagedecision variables are decoupled from the first-stage decision variables. In this project, I will tryto apply Benders decomposition into the field of energy capacity expansion model by a simple casestudy.

1.2 Problem to be studied

This study considers an energy infrastructure expansion problem. A small island, the electric powersystem of which can be viewed as an isolated system, has existing energy infrastructure providingelectricity to the residents on this island. Due to the growing population, the island is planning to

1

expand its electricity generation capacity to meet the increasing demand.

The island can select the types of the new installed capacity from multiple different technologies:thermal power plants, hydroelectricity and solar PV. However, due to the limited area of the island,the resource potential is limited, i.e., there are upper bounds for the available capacities of differenttechnologies. For example, the available hydro power resource is limited because the number ofrivers is finite.

The island is also facing future uncertainties associated with the implementation of an emissionregulation. Specifically, the emission regulation set a threshold on the annual total emission formultiple air pollutants. To avoid incurring penalty due to violating the emission limit, the outputof thermal power plants must be reduced to a certain level below their rated capacities to reducethe emission. However, whether or not the emission regulation will be implemented is still underdebate in the legislature of the island.

In addition to the uncertainties associated with the policy, the availability of renewables such ashydro and solar power are affected by the climate. For example, the availability of hydro power isaffected by precipitation, while the output of solar PV panel is determined by the weather condition.

Therefore, a linear stochastic programming model is developed to aid in the planning and decisionmaking of the future energy infrastructure expansion under the aforementioned uncertainties. Mul-tiple scenarios with their probability distribution specified will be given. The optimal solution tothe stochastic program will provide insights in terms of the future path of the energy infrastructureexpansion.

1.3 Background Knowledge

In the context of energy system modeling, stochastic modeling has been extensively employed. Athree stage stochastic model has been proposed by Kanudia and Loulou [4] based on extendedMARKAL. The model is used to analyze the energy system of Quebec and Ontario with emphasison greenhouse gas abatement. Hu and Hobbs [5] developed a two-stage stochastic programmingmodel based on MARKAL and applied it to the electric power sector of the US. They conducted adecision analysis and simulated the market response to multiple future uncertainties, which includeelectricity demand growths, natural gas prices and the power sector greenhouse gas regulations.

Benders decomposition is a way to split large scale mathematical programming problems into mul-tiple smaller ones, and thereby simplifying the solution by solving one master problem and multiplesubproblems [6]. Different from Danzig-Wolfe decomposition where the restricted master problemis solved by delayed column generation, the restricted master problem of Benders decomposition issolved by delayed constraint generation, a cutting plane method. It is commonly used for stochas-tic two-stage programs with recourse where the problem can be split in the first and second stageproblem [3], and also applied to integer or mixed integer programming problems [7].

2

2 Proposed work

2.1 Mathematical model

Let t = 1, 2, 3, · · · , T be the time periods considered and i = 1, 2, 3, · · · , I indicate different tech-nologies: thermal, hydro and solar PV, etc. We assume that the existing capacities for technologyi is gi at the beginning. Since the natural resource is limited on the studied island, we use Pi torepresent the resource potential for technology i on this island. To account for the total cost of thecapacity expansion process, we use ci as the investment cost ($/kW) and vi as the variable cost($/kWh) for technology i. The investment cost represent the ammortized one-time up-front costand the variable cost include the cost of operation & maintenance, consumables and replacementparts and fuel costs for thermal units.

In addition, to account for the uncertainty of the future, a random variable αξit is defined to repre-sent the availability factor of technology i during time t under random scenario ξ,∀ξ ∈ {1, · · · ,K},which is the time percentage that a power plant can be effectively operated during time t. Weassume that to achieve the goal of CO2 emission, the thermal units will have reduced availabilityfactors.

We further define the following decision variables:

• xi: Newly installed capacity of technology i, they represent the first stage decision variables.

• yξit: Electricity generation from technology i during time period t. They represent the secondstage decision variables.

To satisfy the demand dt,∀t = 1, · · · , T , we have the following two-stage stochastic model [8, 3] :

minI∑i=1

cixi + Eξ

[I∑i=1

(T∑t=1

viyξit

)](1a)

s.t. xi ≤ Pi,∀i, (1b)

yξit ≤ αξit(xi + gi),∀i, t, (1c)

I∑i=1

yξit = dt, ∀t, (1d)

xi, yξit ≥ 0. (1e)

In the above model formulation, the objective function (Equation 1a) represent the total cost, whichconsists of the investment costs and variable costs. The first constraint (Equation 1b) indicates thatthe installed capacity for each technology i cannot exceed its technological potential. In addition,the electricity generation during each time period cannot exceed the available capacity, as shownin the second constraint (Equation 1c). Last, the electricity demand must be met by generationduring each time period (Equation 1d).

2.2 Algorithm

Benders decomposition method is mentioned in class as applying Danzig-Wolfe decomposition tothe dual. Suppose we have a primal problem:

3

min z = cTx

s.t. A0x = b0

A1x = b1

x ≥ 0.

The dual of the it can be written as:

max z = bT0 w + bT1 u

s.t. AT0 w + AT

1 u ≤ c

w,u: unrestricted.

The dual problem is then transformed into the following by applying Benders’ Decomposition:

maxw: unrestricted

{bT0 w + max

u: unrestrictedbT1 u

s.t. AT0 w + AT

1 u ≤ c

}

= maxw: unrestricted

bT0 w + min (c−AT

0 w)Txs.t. A1x = b1

x ≥ 0

= max

w: unrestricted

{bT0 w + min

x∈X(c−AT

0 w)Tx

}.

Therefore, in the class notes Benders Decomposition is applied to “maximize a lower bound”. How-ever, in literature[6, 9], it is more common to minimize an upper bound. In addition, the “minimizea upper bound” implementation is more appropriate for the model studied in this project. We willbreifly introduce this technique. It must be pointed out that in principle, these two implementa-tion of Benders Decomposition are in essence the same and both of the can be viewed as applyingDanzig-Wolfe Decomposition into the dual and just as we have seen in class that the “maximizea lower bound” Benders decomposition is applied Danzig-Wolfe decomposition to the dual of aminimization problem, we can show that the “minimize an upper bound” Benders decompositionis equivalent to applying Danzig-Wolfe decomposition to the dual of a maximization problem.

Suppose we have a linear program as shown in Equation 2

min z = cTx + fTy

s.t. Ax = b

Bx + Dy = d

x,y ≥ 0.

(2)

Follow the same decomposition technique introduced in class, the above problem can be furtherwritten as:

minAx=b,x≥0

cTx + min fTy

s.t. Dy = d−Bxy ≥ 0

= min

Ax=b,x≥0

{cTx + max (d−Bx)Tπ

s.t. DTπ ≤ f

}

4

= minAx=b,x≥0

{cTx + max

vj∈V(d−Bx)Tvj

}This can be viewed as minimize the upper bound. Notice that even with different x, the feasibleregion {π|DTπ ≤ f} of the the inside maximization problem does not change. In addition, accordingto the Fundamental Theory of Linear Programming, we know that the optimal solution of the insidemaximization problem, if exists, must be on a vertex. Therefore, by defining the set of all verticesof {π|DTπ ≤ f} as V, we can decompose the problem into a master problem (MP) and an auxiliaryproblem (AP).

minx,θ

z = cTx + θ

s.t. Ax = b

θ ≥ (b−Ax)Tvj , j ∈ {1, 2, 3 · · · |V|}x ≥ 0, θ unrestricted

and

maxπ

(d−Bx)Tπ

s.t. DTπ ≤ f

π unrestricted.

3 Mathematical derivations

The model in Equation 1 is not in the standard form of linear program, we will transform it into thestandard form. For simplification, we assume only 3 technologies exist (I = 3). First, we expressthe decision variables as the following vector form:

wT =[xT yT1 yT2 · · · yTK

],

where x is the first stage decision variable:

xT =[x1 x2 x3

],

and yξ is a 3T × 1 vector representing the second stage decision variables under scenario ξ:

yTξ =[yξ11 yξ12 · · · yξ1T yξ21 yξ22 · · · yξ2T yξ31 yξ32 · · · yξ3T

].

With the decision variables defined as above, we can rewrite the objective function as the followingform:

cTx + Pr1fTy1 + Pr2f

Ty2 + . . .+ PrKfTyK ,

where c = [c1, c2, c3]T and f = [v1, . . . , v1, v2, . . . , v2, v3, . . . , v3]

T . The constraints in Equation 1bcan also be expressed as follows: [

I3 0 0 0]·w ≤ p, (3)

5

where I is an identity matrix and p is the vector of technical potentials. Similarly, the constraintsrepresented by Equation 1c can be expressed as the following matrix form: B1 I3T

B2 I3TB3 I3T

·w ≤ g1

g2g3

, (4)

where gξ is the vector representing the available power under scenario ξ and likewise, Bξ representsthe coefficient matrix of x under scenario ξ:

Bξ = −

αξ11αξ12

...

αξ1Tαξ21αξ22

...

αξ2Tαξ31αξ32

...

αξ3T

.

In addition, the set of constraints in Equation 1d can be expressed as: 0 D0 D0 D

·w = l, (5)

where l is the load vector andD =

[IT IT IT

].

Therefore, by combining Equations 3, 4, 5 and adding slack variables to the inequality constraints,the original problem can then be transformed into the following form:

min cTx + Pr1fTy1 + Pr2f

Ty2 . . . + PrKfTyK

s.t. Ax = bB1x + Dy1 = d1

B2x + Dy2 = d2...

. . ....

BKx + DyK = dK

(6)

x,y1, · · · ,yK ≥ 0,

which will be solved by applying Benders decomposition. In the new objective function, Prξrepresents the probability of scenario ξ. The constraint Ax = b represents the first-stage constraintand Bξx + Dyξ = dξ represents second-state constraint for each specific scenario.

6

4 Implementation

By applying Benders decomposition technique in Section 2.2, we can decouple the problem inEquation 6 into a first-stage problem (MSP):

minx,θ

z = cTx +∑ξ

θξ

s.t. Ax = b

θξ ≥ (b−Ax)Tvj , j ∈ {1, 2, 3 · · · |V|}x ≥ 0, θξ unrestricted

and multiple second-stage problems (APs):

maxπ

θξ(x) = (dξ −Bξx)Tπξ

s.t. DTπξ ≤ f

πξ unrestricted.

In reality, we do not solve the complete MSP because there are too many constraints. Instead, wesolve a restricted MSP by only considering part of those constraints, suppose we have a solution(x, θξ) for the restricted MSP. Then we solve the APs and obtain the second stage objective functionsθξ(x), if all APs have optimal solutions and their objective function values θξ ≥ θξ(x) then we haveoptimal solution for the original problem. Otherwise, we may have two possible outcomes:

• The objective function θξ(x) ≥ θξ with optimal solution π∗ξ , we have identified a violatedconstraint and we add the following optimality cut to the MSP and solve the MSP again:

θξ ≥ (dξ −Bξx)Tπ∗ξ .

• The AP may be unbounded, since we are solving the dual form it indicates that the primalAP is infeasible. In this case, suppose the extreme direction of the dual AP is π∗ξ , we add thefollowing feasibility cut to the MSP and solve it:

0 ≥ (dξ −Bξx)Tπ∗ξ .

Note that the following problem will be solved to obtain the extreme direction of AP:

max (dξ −Bξx)Tπ

s.t. DTπ ≤ 0−eT ≤ π ≤ eT

(7)

Therefore, we have the following step-by-step implementation of Benders decomposition:

1. Initialization. Solve the following problem to obtain the intial x:

min z = cTx

s.t. Ax = b

x ≥ 0.

If the problem is infeasible or unbounded, then stop, the problem is infeasible or unbounded.Substitute x into the APs to obtain either feasibility cuts or optimality cuts.

7

2. Solve the restricted MSP. Set the cost coefficient for θξ to 1 only if there is at least onefeasibility cut for θξ. If the restricted MSP is infeasible then stop, otherwise we proceed tonext step with a solution (x, θξ). We set lower bound (LB) as the objective value of the MSP:

LB = cTx +∑ξ

θξ

3. Solve the APs. If all APs are feasible, then calculate the upper bound (UB) and go to thenext step:

UB = cTx +∑ξ

θξ(x).

If at least one AP is unbounded above, identify the extreme direction by solving Equation 7.Add the feasibility cut to the MSP, go to step 2.

4. If UB − LB ≤ ε, where ε is a predefined tolerance, stop the problem is optimal. Otherwise,identify the violating constraints and add the feasibility cut to the MSP, go to step 2.

5 List of codes

In this project, the method of Benders decomposition is implemented in MATLAB:

function [x, fval , exitflag , output] = benders(prob , c, f, A, B, D, b, d)

% Apply Benders decomposition to solve a two stage stochastic program.

% Note that f, B and d are cell arrays.

% Some auxiliary parameters

nx = size(c, 1); % Dimension of the first stage decision variable x.

ny = size(D, 2); % Dimen of the second stage decision variable y^{\xi}.

npi = size(D, 1); % Dimension of the dual variable for AP.

nxi= length(prob); % Number of scenarios.

niter = 100; % Limit of number of iteratios.

% Start Benders decomposition

LBs = nan(niter , 1); UBs = nan(niter , 1); LB = -inf; UB = inf;

eps = 1E-3; % Tolerance for convergence.

opts = optimoptions(@linprog , ’Algorithm ’, ’dual -simplex ’ ,...

’display ’, ’off’);

% Initialization

xk = linprog(c, [], [], A, b, zeros(nx , 1), [], [], opts);

counter = 0;

% The additional cost vector for \thetas , note that if any optimality cut

% has been added , then the cost coefficient for that \theta should be 1,

% otherwise (no cut or only feasibility cut has been added) it is 0.

c_theta = zeros(nxi , 1);

A_theta = zeros(size(A, 1), nxi); % The additional columns for \thetas

% Initial cut: x >= 0, \theta ^{\xi} unrestricted

cut_left = [eye(nx) zeros(nx, nxi);];

8

cut_right = zeros(nx, 1);

% Result containers

pi = nan(npi , nxi); z = nan(nxi , 1); flag = nan(nxi , 1);

tic;

while( (UB -LB) >= eps && counter < niter )

counter = counter + 1;

% AP

for i = 1: nxi

[this_x , this_fval , this_flag] = linprog( -1.*(d{i}-B{i}*xk) ,...

D’, prob(i).*f{i}, [], [], [], [], [], opts);

if this_flag == 1 % Optimal solution found , add optimality cut

c_theta(i) = 1;

pi(:, i) = this_x;

z(i) = this_fval;

flag(i) = this_flag;

elseif this_flag == -3 % Unboundness detected.

% Determine the extreme direction.

[nDt , mDt] = size(D);

this_x = linprog( -1.*(d{i}-B{i}*xk), D’, zeros(mDt , 1) ,...

[],[], -1.*ones(nDt , 1), ones(nDt , 1), [], opts);

pi(:, i) = this_x;

z(i) = nan; flag(i) = this_flag;

else

error(’Error!’);

end

end

% Negative because a "-" is added for maximization.

if flag(i) == 1

UB = min([c’*xk - sum(z), UB]);

else

UB = inf; % The dual AP is unbounded above.

end

% MSP

G = nan(nxi , nx); alpha = zeros(nxi , nxi); g = nan(nxi , 1);

for i = 1: nxi

G(i, :) = pi(:, i)’*B{i};

g(i) = d{i}’*pi(:, i);

if flag(i) == 1

% AP has an optimal solution , add an optimality cut , where

% alpha = 1.

alpha(i, i) = 1;

else

% If AP is unbounded , we should obtain the extreme direction

% and add a feasibility cut (alpha = 0).

alpha(i, i) = 0;

end

end

% Add new cuts: cut_left *[x; theta] >= cut_right

9

old_cut_l = cut_left;

cut_left = [old_cut_l; G alpha ;];

old_cut_r = cut_right;

cut_right = [old_cut_r; g;];

[x_theta , LB] = linprog ([c;c_theta ],...

-cut_left , -cut_right , [A A_theta], b, [], [], [], opts);

xk = x_theta (1: size(c, 1));

LBs(counter) = LB;

UBs(counter) = UB;

toc;

end

x = nan(nx + ny*nxi , 1); x(1: nx) = xk;

% Solve for the second stage decision variables y^{\xi}.

for i = 1: nxi

y = linprog(prob(i).*f{i}, [], [], D, d{i} - B{i}*xk ,...

zeros(ny, 1), [], [], opts);

x(nx + (i - 1)*ny + 1: nx + i*ny) = y;

end

fval = (LB + UB)/2;

exitflag = 1;

output = struct ();

output.LB = LBs; output.UB = UBs;

output.iterations = counter;

end

6 Computational experiment

6.1 Test problems

Two computational experiments have been conducted in this project, the first one is an EuropeanFarmer problem from Section 1.1 of the textbook by Birge and Louveaux [8]. This problem has 3scenarios. In addition, it has 3 first-stage decision variables, and 6 second-stage decision variables,which totals 21 decision variables. The total number of constraints is 13, including 1 first-stageconstraint and 4 second-stage constraints.

The second problem is the problem proposed in Section 2.1 with 2 scenarios (K = 2), 3 technologies(I = 3) and 24 hours (T = 24).

The data of both problems are inputted directly in the MATLAB script. It must be pointed outthat although the test problems are small in scale, the methods introduced here can be applied tolarger scale problem with millions of decision variables and constraints.

6.1.1 An European Farmer problem

The linear programming solver of MATLAB is also used to solve this problem as a reference. Thefull problem definition is:

10

min 150x1 + 230x2 + 260x3−1

3(170w11 − 238y11 + 150w21 − 210y21 + 36w31 + 10w41)−1

3(170w12 − 238y12 + 150w22 − 210y22 + 36w32 + 10w42)−1

3(170w13 − 238y13 + 150w23 − 210y23 + 36w33 + 10w43)s.t. x1 + x2 + x3 ≤ 500

3x1 + y11 − w11 ≥ 2003.6x2 + y21 − w21 ≥ 240w31 + w41 ≤ 24x3w31 ≤ 60002.5x1 + y12 − w12 ≥ 2003x2 + y22 − w22 ≥ 240w32 + w42 ≤ 20x3w32 ≤ 60002x1 + y13 − w13 ≥ 2002.4x2 + y23 − w23 ≥ 240w33 + w43 ≤ 16x3w33 ≤ 6000x, y, w ≥ 0

where x1, x2, x3 are the first-stage decision variables and w1ξ, y1ξ, w2ξ, y2ξ, w3ξ, w4ξ, ξ = 1, 2, 3 arethe second-stage decision variables. The code of this problem is listed in the appendix.

6.1.2 The Energy Infrastructure Expansion problem

See the previous section for problem description. The data is directly embedded in the code, seeAppendix 3.

6.2 Running results

6.2.1 An European Farmer problem

The objective function values from both methods are the same, both are -108390, as indicated bythe red horizontal line on Figure 1. The following tables show the stage-wise decision variablescomparison. Both MATLAB’s linear program solver linprog and Benders decomposition give thesame results, which are identical to the solution provided by [8], indicating good accuracy. Inaddition, Figure 1 shows that Benders decomposition converges after 6 scenarios.

Table 1: First-stage decision variables.

x1 x2 x3linprog 170 80 250benders 170 80 250

11

Table 2: Second-stage decision variables.

w1ξ y1ξ w2ξ y2ξ w3ξ w4ξ

linprog, ξ = 1 310 0 48 0 6000 0benders, ξ = 1 310 0 48 0 6000 0

linprog, ξ = 2 225 0 0 0 5000 0benders, ξ = 2 225 0 0 0 5000 0

linprog, ξ = 3 140 0 0 48 4000 0benders, ξ = 3 140 0 0 48 4000 0

1 2 3 4 5 6# of iterations

150

100

50

0

50

100

Net

cost

(th

ousa

nd $

)

LBUBObj

Figure 1: The European Farmer problem convergence.

6.2.2 The Energy Infrastructure Expansion problem

The code is attached in Appendix 3 due to space limit. In this problem, Benders decompositionconverges after 7 iterations, as shown in Figure 2, note that the red horizontal line indicates theoptimal objective function value. The upper bounds at the first six iterations are infinity, sinceat least one AP in these iterations are unbounded above and the last iteration the upper boundconverges with the lower bound, indicating the optimality condition.

12

1 2 3 4 5 6 7# of iterations

2100

2200

2300

2400

2500

2600

2700

Tota

l co

st (

mill

ion $

)

LBUBObj

Figure 2: The energy problem convergence.

The first stage decision variables are compared with the solutions given by MATLAB’s linprog, asshown in Table 3. Both Benders decomposition and the MATLAB LP solver give the same resultsfor the second stage solutions, as plotted in Figure 3. Note that unlike in the European Farmer casewhere the second-sage decision variables are different across scenarios, the second-stage decisionvariables in two different scenarios in this case are identical. In addition, Table 3 shows that onlynatural gas power plant is developed, indicating that natural gas is the most economical technologyunder current assumptions, regardless of the reduced availability factors.

Table 3: First-stage decision variables.

Installed capacity (MW) NG Hydro Solar

linprog 2930 0 0benders 2930 0 0

13

5 10 15 20Time (h)

0

500

1000

1500

2000

2500

3000

3500

Pow

er

genera

tion (

MW

)

NGHydroSolar

Figure 3: Hourly generation in both scenarios.

7 Performance analysis

The time consumption break-down for both test cases are shown in Figure 4. Each case is repeated10 times to reduce random errors. The time consumption breakdown of the European Farmerproblem is presented in Figure 4a. Since Benders decomposition converges after 6 iterations, thetime consumption each iteration is listed separately.

It shows that the total time consumption of Benders decomposition is 16 times longer than MAT-LAB’s solver, and even the single iteration is longer. One possible reason is that the problem sizeis way too small and the time consumption of linprog function is not dominating. More time isspent in value assignment and matrix operations such as multiplication and inversion. In addition,each Benders iteration solves two smaller linear program problems, which could be longer thansolving one large linear programs when the sizes are close.

The performance of the Energy Infrastructure Expansion problem is shown in Figure 4b. Similarto the Farmer problem, the time consumption in each Benders iteration is approximately 3 timeslonger than using MATLAB’s LP solver. By summing up time consumption in all 7 iterations,Benders decomposition consumes 20 times more time than MATLAB’s LP solver consumes. Theadditional solving for extreme directions of the APs in the Energy problem is the main cause forthe time consumption increase.

14

linprog Iter 1 Iter 2 Iter 3 Iter 4 Iter 5 Iter 60.00

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08Tim

e (

s)

(a) The European Farmer problem

linprog Iter 1 Iter 2 Iter 3 Iter 4 Iter 5 Iter 6 Iter 70.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

Tim

e (

s)

(b) The Energy Capacity Expansion Problem

Figure 4: Time consumption break-down. The white bar represents the time consumption ofMATLAB’s linprog function and grey bars represent benders function.

8 Conclusion and discussion

Large-scale optimization models are frequently formulated and analyzed by researchers to addressproblems in the field of energy system modeling. To solve their models, researchers will employefficient solution techniques such as Benders decomposition. This course project provides a step-by-step derivation and implementation of Benders decomposition for two-stage linear stochasticprogram with multiple scenarios. The method has been applied to a textbook problem and showsgood accuracy and performance. In addition, the method has been applied to a small scale energyinfrastructure expansion problem.

It shows that Benders decomposition can decompose large scale linear programming problem intosmaller problems with lower structure complexity. The complexity associated with structure ofthe problem has been converted into time complexity. Note that the two problems studied in thisproject are small in size, the method can be easily modified and applied to large-scale problems.Overall, this study contributes to the large-scale optimization in energy system modeling com-munity by enhancing our understanding of the characteristics of the stochastic energy model. Inaddition, the proposed method can be modified and applied to other applications, such as mixed-integer programming problem. It can also be applied to different areas other than energy systemmodeling, such as health care industry.

Several caveats exist: First, in solving the dual AP, the feasible region remains constant. Therefore,it does not need to actual solve dual AP each time from the very beginning, instead, the extremepoints that do not satisfy the optimal condition can be discarded to reduce the solution time.Second, since there are multiple APs in multiple scenarios problem, parallelism can be utilized toaccelerate the solving process by processing with multiple cases concurrently.

15

References

[1] Roger Fouquet. Path dependence in energy systems and economic development. Nature Energy,1:16098, 2016.

[2] M Granger Morgan. Opinion: Climate policy needs more than muddling. Proceedings of theNational Academy of Sciences, 113(9):2322–2324, 2016.

[3] Dimitris Bertsimas and John N Tsitsiklis. Introduction to linear optimization, volume 6. AthenaScientific Belmont, MA, 1997.

[4] Amit Kanudia and Richard Loulou. Advanced bottom-up modelling for national and regionalenergy planning in response to climate change. International Journal of Environment andPollution, 12(2-3):191–216, 1999.

[5] Ming-Che Hu and Benjamin F Hobbs. Analysis of multi-pollutant policies for the us powersector under technology and policy uncertainty using markal. Energy, 35(12):5430–5442, 2010.

[6] Jacques F Benders. Partitioning procedures for solving mixed-variables programming problems.Numerische mathematik, 4(1):238–252, 1962.

[7] Silvio Binato, Mario Veiga F Pereira, and Sergio Granville. A new benders decompositionapproach to solve power transmission network design problems. IEEE Transactions on PowerSystems, 16(2):235–240, 2001.

[8] John R Birge and Francois Louveaux. Introduction to stochastic programming. Springer Science& Business Media, 2011.

[9] G. Infanger. Planning Under Uncertainty: Solving Large-scale Stochastic Linear Programs.Professional Practices Pamphlet. Boyd & Fraser, 1994.

16

Appendix 1: The code using MATLAB’s linear programming solver linprog

function [x,fval ,exitflag ,output ,lambda] =...

check(prob , c, f, A, B, D, b, d)

tic;

nx = size(c, 1); ny = size(D, 2); nxi= length(prob);

OA = zeros(size(A, 1), ny); % Zero matrix for row Ax = b

Aeq = nan(size(A, 1) + nxi*size(D, 1), nx + nxi*ny);

Aeq (1: size(A, 1), :) = [A repmat(OA , 1, nxi)];

beq = nan(size(A, 1) + nxi*size(D, 1), 1);

beq (1: size(A, 1)) = b;

obj = nan(nx + nxi*ny , 1);

obj (1: nx) = c;

row_s = size(A, 1);

for i = 1: nxi

O = zeros(size(B{i}, 1), ny);

Aeq(row_s + 1: row_s + size(B{i}), :) =...

[B{i} repmat(O, 1, i-1) D repmat(O, 1, nxi -i)];

beq(row_s + 1: row_s + size(B{i})) = d{i};

obj(nx + (i-1)*ny + 1: nx + i*ny) = prob(i).*f{i};

row_s = row_s + size(B{i});

end

% Use MATLAB ’s linprog function.

opts = optimoptions(@linprog , ’Algorithm ’,’dual -simplex ’);

[x,fval ,exitflag ,output ,lambda] = ...

linprog(obj ,[],[],Aeq ,beq ,zeros(size(obj , 1), 1) ,[],[],opts);

toc;

end

17

Appendix 2: The Code for the European Farmer problem

function farmer ()

% The European farmer problem from Section 1.1 of Birge & Louveaux , 2011.

% First stage decision variables: (x1, x2, x3, s0),

% Second stage decision variables :(w1, y1, w2, y2, w3, w4, s1, s2, s3, s4)

% where (s0, s1, s2, s3, s4) are all slack variables.

prob = [1/3; 1/3; 1/3];

nxi= length(prob); % Number of scenarios.

% First stage parameters.

c = [150 230 260 0]’; A = [1 1 1 1]; b = 500;

% Second stage parameters.

f = cell(nxi , 1);

f{1} = -1.*[170 -238 150 -210 36 10 0 0 0 0]’;

f{2} = -1.*[170 -238 150 -210 36 10 0 0 0 0]’;

f{3} = -1.*[170 -238 150 -210 36 10 0 0 0 0]’;

B = cell(nxi , 1);

B{1} = diag ([3 3.6 24 0]);

B{2} = diag ([2.5 3 20 0]);

B{3} = diag ([2 2.4 16 0]);

D = [

-1 1 0 0 0 0 -1 0 0 0;

0 0 -1 1 0 0 0 -1 0 0;

0 0 0 0 -1 -1 0 0 -1 0;

0 0 0 0 -1 0 0 0 0 -1;

];

d = cell(nxi , 1);

d{1} = [200 240 0 -6000]’;

d{2} = [200 240 0 -6000]’;

d{3} = [200 240 0 -6000]’;

[x, obj , exitflag , output , lambda] = check(prob , c, f, A, B, D, b, d);

[x_benders , obj_benders , exitflag_benders , output_benders] =...

benders(prob , c, f, A, B, D, b, d);

counter = output_benders.iterations;

LBs = output_benders.LB; UBs = output_benders.UB;

plot (1: counter , LBs(~ isnan(LBs)), ’-ko’);

hold on;

plot (1: counter , UBs(~ isnan(UBs)), ’-k.’);

hold off;

legend(’LB’, ’UB’);

xlabel(’# of iterations ’);

xticks (1:1: counter);

ylabel(’objective function ’);

end

18

Appendix 3: The Code for the Energy Infrastructure Expansion problem

function energy ()

% The energy infrastructure expansion problem. NG, Hydro , solar.

nxi = 2; % 2 scenarios

prob = [0.3; 0.7]; % Good , bad scenarios.

c_cap = 1E3 .*[912; 2651; 3279]; % Annualized capital cost , $/MW.

c_var = [25.78; 5.76; 0]; % Variable O&M cost , $/MWh. Fuel cost included.

g = [2000; 1000; 700]; % Existing capacities , MW.

P = [5000; 3000; 3000]; % Technical potentials , MW.

alpha = cell(nxi , 1);

alpha {1} = [

0.87 0.2 0;

0.88 0.2 0;

0.89 0.21 0;

0.9 0.21 0;

0.91 0.22 0;

0.92 0.22 0.0019;

0.93 0.23 0.0518;

0.94 0.23 0.19;

0.95 0.24 0.339;

0.96 0.24 0.446;

0.97 0.25 0.509;

0.98 0.25 0.541;

0.99 0.24 0.544;

0.98 0.24 0.519;

0.97 0.23 0.466;

0.96 0.23 0.386;

0.95 0.22 0.277;

0.94 0.22 0.151;

0.93 0.21 0.034;

0.92 0.21 0.0011;

0.91 0.2 0;

0.9 0.2 0;

0.89 0.19 0;

0.88 0.19 0;

]; % Availability factors for good scenario , NG , hydro , solar.

alpha {2} = [

0.52 0.2 0;

0.53 0.2 0;

0.53 0.21 0;

0.54 0.21 0;

0.55 0.22 0;

0.55 0.22 0.0019;

0.56 0.23 0.0518;

0.56 0.23 0.19;

0.57 0.24 0.339;

0.58 0.24 0.446;

0.58 0.25 0.509;

0.59 0.25 0.541;

0.59 0.24 0.544;

19

0.59 0.24 0.519;

0.58 0.23 0.466;

0.58 0.23 0.386;

0.57 0.22 0.277;

0.56 0.22 0.151;

0.56 0.21 0.034;

0.55 0.21 0.0011;

0.55 0.2 0;

0.54 0.2 0;

0.53 0.19 0;

0.53 0.19 0;

]; % Availability factors for bad scenario , NG , hydro , solar.

demand = [

2714

2703

2696

2712

2756

2824

2918

2979

3030

2995

2896

2783

2660

2562

2506

2502

2586

2849

2937

2922

2882

2798

2670

2525

]; % Demand , MW.

% Transform the raw problem into the standard form.

ntech = size(c_cap , 1); % Number of technologies considered.

nhour = size(demand , 1); % Number of hours studied.

c = [c_cap; zeros(ntech , 1);];

temp = repmat(c_var ’, nhour , 1); temp = [temp (:); zeros(ntech*nhour , 1)];

f = cell(nxi , 1);

for i = 1: nxi

f{i} = temp;

end

A = [eye(ntech) eye(ntech)];

B = cell(nxi , 1);

for i = 1: nxi

20

B{i} = zeros(ntech*nhour+nhour , 2*ntech);

for j = 1: ntech

row_s = 1+(j-1)*nhour;

row_e = j*nhour;

B{i}(row_s: row_e , j) = -1.*alpha{i}(:, j);

end

end

D = [

eye(nhour*ntech) eye(nhour*ntech);

repmat(eye(nhour), 1, ntech) zeros(nhour , ntech*nhour);

];

b = P;

d = cell(nxi , 1);

for i = 1: nxi

temp = repmat(g’, nhour , 1);

d{i} = [alpha{i}(:).*temp (:); demand ];

end

% Use MATLAB ’s linprog to solve it.

[x,fval ,exitflag ,output ,lambda] = check(prob , c, f, A, B, D, b, d);

% Use Benders Decomposition to solve it.

[x_b , fval_b , exitflag_b , output_b] =...

benders(prob , c, f, A, B, D, b, d);

counter = output_b.iterations;

LBs = output_b.LB; UBs = output_b.UB;

plot (1: counter , LBs(~ isnan(LBs)), ’-ko’);

hold on;

plot (1: counter , UBs(~ isnan(UBs)), ’-k.’);

hold off;

legend(’LB’, ’UB’);

xlabel(’# of iterations ’);

xticks (1:1: counter);

ylabel(’objective function ’);

end

21