epubwu institutional repositoryepub.wu.ac.at/5421/1/5863.pdf · epubwu institutional repository...

ePubWU Institutional Repository

Nils Löhndorf and David Wozabal

Indifference pricing of natural gas storage contracts.

Paper

Original Citation:

Löhndorf, Nils and Wozabal, David

(2017)

Indifference pricing of natural gas storage contracts.

WU Vienna University of Economics and Business, Vienna.

This version is available at: https://epub.wu.ac.at/5421/Available in ePubWU: February 2017

ePubWU, the institutional repository of the WU Vienna University of Economics and Business, isprovided by the University Library and the IT-Services. The aim is to enable open access to thescholarly output of the WU.

http://epub.wu.ac.at/

https://epub.wu.ac.at/5421/

http://epub.wu.ac.at/

Indifference pricing of natural gas storage contracts

Nils Lohndorf1, David Wozabal21Vienna University of Economics and Business, Vienna, Austria

2Technische Universitat Munchen, Munich, Germany

[email protected], [email protected]

Natural gas markets are incomplete due to physical limitations and low liquidity, but most valuation

approaches for natural gas storage contracts assume a complete market. We propose an alternative approach

based on indifference pricing which does not require this assumption but entails the solution of a high-

dimensional stochastic-dynamic optimization problem under a risk measure. To solve this problem, we

develop a method combining stochastic dual dynamic programming with a novel quantization method that

approximates the continuous process of natural gas prices by a discrete scenario lattice. In a computational

experiment, we demonstrate that our solution method can handle the high dimensionality of the optimization

problem and that solutions are near-optimal. We then compare our approach with rolling intrinsic valuation,

which is widely used in the industry, and show that the rolling intrinsic value is sub-optimal under market

incompleteness, unless the decision-maker is perfectly risk-averse. We strengthen this result by conducting a

backtest using historical data that compares both trading strategies. The results show that up to 40% more

profit can be made by using our indifference pricing approach.

Key words : stochastic-dynamic programming, asset pricing, natural gas industry, Markov processes

History :

1. Introduction

Over the past decade natural gas has increasingly replaced other fossil fuels in the United States.

The U.S. Energy Information Administration projects that this trend is likely to continue and

that natural gas consumption will make up 35% of total energy consumption by 2040 (EIA

2016). This will not only increase trading activities in natural gas markets but also lead to a

growing importance of storage facilities which are used to buffer variability in demand. Such

storage facilities are scarce, as they require natural geological formations like depleted gas fields,

salt domes, or aquifers. Due to its scarcity, storage capacity is an important asset in energy

trading. Often, the ownership of gas and storage is separated, and a contract is used to transfer

usage rights from storage owners to third parties. Pricing such storage contracts is therefore a

2 Indifference pricing of natural gas storage contracts

problem of practical importance in energy trading, and numerous commercial software products

offer valuation tools (e.g., MSCI 2015, KYOS 2015, Lacima 2017).

The value of storage derives from buying and injecting gas at times of low prices and with-

drawing and selling when prices are high. To do so, owners of storage trade on commodity

exchanges, such as the New York Mercantile Exchange (NYMEX) in the United States, which

hosts spot and futures markets for natural gas.

Existing approaches for gas storage valuation typically assume that gas markets are complete

and that storage can be priced using replication. The idea of replication is that the value of

a derivative equals the initial capital required to replicate its cash flow by an optimal trading

strategy in the underlying (Karatzas and Shreve 1998). Such a strategy involves buying, holding,

and selling the underlying as well as operating a cash account. The fundamental theorem of

asset pricing states that, in a complete market, the optimum of the replication problem equals

the expected discounted cash flow under a uniquely defined equivalent martingale measure.

This approach is commonly referred to as risk-free pricing, as all market participants agree on

one price, irrespective of their risk preferences.

The complete market assumption, however, is not tenable for gas storage valuation, because

gas markets are rarely as liquid as financial markets, and the storage and transport of gas is

subject to various physical limitations. In particular, the storage of gas is expensive, which

contrasts the situation in financial markets, where storing the underlying is costless. Since the

replication of any derivative requires the ownership of storage, the value of storage cannot

be replicated, unless one owns it in the first place, which contradicts the idea of replication.

Carmona and Ludkovski (2010) point out that pricing a storage contract should take market

incompleteness into account and should be based on joint optimization of the portfolio of futures

contracts and storage operation.

In incomplete markets, there exists no unique equivalent martingale measure that can be used

for risk-free valuation. Consequently, the market price of an asset and individual valuations of

agents with different risk preferences may vary. To model the market price of storage, an actu-

arial value can be calculated as the expected cash flow under a martingale measure calibrated

Indifference pricing of natural gas storage contracts 3

to observed market prices. However, to determine the value an individual agent ascribes to stor-

age, one has to determine the price at which the agent is indifferent between holding and selling

the storage (Karatzas and Shreve 1998). This valuation is dependent on risk preferences and

endowment and is referred to as indifference pricing (see Carmona 2009, for a comprehensive

exposition of the subject).

We propose the first indifference pricing approach for natural gas storage valuation. The

indifference value of storage depends on the discounted rewards that result from an optimal

operational and trading strategy. Finding this strategy requires solving a high-dimensional

stochastic-dynamic decision problem that takes into account the full dynamics of futures prices,

the dynamic nature of storage and future contract balances, as well as physical constraints of

the storage contract.

To position our contribution, we begin by discussing the extant literature on gas storage

valuation. Two heuristic approaches popular among practitioners are basket-of-spread options

valuation and rolling intrinsic valuation. Both approaches make simplifying assumptions to

approximate the extrinsic value of storage. While the intrinsic value can be obtained by finding

the optimal schedule of injections and withdrawals based on current futures prices, its extrinsic

value captures the additional value of flexibility to adapt this schedule to price changes. The

basket-of-spread options approach approximates the extrinsic value by the value of a portfolio

of (time) spread options. The rolling intrinsic approach, by contrast, simulates the possible

evolution of the futures curve over time and approximates the extrinsic value as the expected

gains that result from changes in the schedule of injections and withdrawals. To simulate the

dynamics of the futures curve, multifactor models, such as the multivariate Black model, are a

popular choice (for further reference, see Eydeland and Wolyniec 2003, Gray and Khandelwal

2004b,a).

Further approaches from the literature are based on risk-free pricing. These approaches use

spot-based models and ignore the dynamics of the futures curve (e.g. Boogert and de Jong

2008, Chen and Forsyth 2007, Thompson et al. 2009, Carmona and Ludkovski 2010, Secomandi

2010, Bjerksund et al. 2011, Boogert and de Jong 2012, Felix and Weber 2012). While spot


price models keep the underlying optimization problem low-dimensional, traders are suspicious

of these approaches, as they are usually not consistent with observed futures prices (Lai et al.

2010).

Lai et al. (2010) and Nadarajah et al. (2015) account for futures price dynamics in a complete

market setting and use approximate dynamic programming techniques (Powell 2011) that are

based on low-dimensional relaxations of the otherwise high-dimensional stochastic-dynamic

optimization problem. Both approaches yield no significant benefit over the rolling intrinsic

solution.

Our indifference pricing approach avoids making simplifying assumptions and tackles the

pricing problem in its full complexity. The contribution of our approach is three-fold: first,

we propose a high-dimensional, joint optimization model for futures trading as well as stor-

age operation that takes risk preferences into account. Second, as the optimization problem

is subject to the curse of dimensionality, we propose an approximate dynamic programming

approach to solve the problem. Third, to test whether accounting for the added complexity is

worthwhile, we report the results of a comprehensive experiment based on historical data as

well as simulations.

Our first contribution is a model for the indifference value of storage. In Section 2.2, we

formulate the joint problem of futures trading and storage operation as a Markov decision

process (MDP), whereby risk preferences are modeled using the nested conditional value-at-

risk. In Section 2.3, we show that the optimal solution to this problem reduces to the intrinsic

solution under perfect risk aversion and that, for all other cases, we have to find the optimal

policy of the MDP to compute the indifference value.

Our second contribution addresses the curse of dimensionality. In Section 3, we introduce

approximate dual dynamic programming (ADDP) as a unifying framework that integrates sce-

nario lattices with Pereira and Pinto (1991)’s stochastic dual dynamic programming (SDDP)

algorithm to solve a general class of discrete-time, continuous-state, risk-averse MDPs. The

SDDP algorithm has proven to be successful for tackling stochastic-dynamic optimization prob-

lems with many decision epochs. However, conventional SDDP is limited to stagewise indepen-

dent randomness. It does not seem to be widely known that this limitation can be overcome by


using scenario lattices. Discretizing a continuous Markov process to a lattice has been first pro-

posed in Bally and Pages (2003) and applied to SDDP independently in Bonnans et al. (2012)

and Lohndorf et al. (2013). Bally and Pages (2003) propose an approach to construct scenario

lattices for Brownian motions using optimal quantization. We extend their work and propose an

alternative construction method that combines quantization with ideas from moment matching

(Høyland et al. 2003). In Section 4.3, we demonstrate that this combination is crucial to deter-

mine the indifference value of storage. In Section 3.3, we show how the nested CVaR can be

integrated into SDDP by adapting the approach of Philpott et al. (2013) to scenario lattices.

Our third contribution concerns a comparison of the rolling intrinsic solution and indifference

pricing. Lai et al. (2010) derive a dual upper bound for the value of storage in a complete market

setting and demonstrate that the rolling intrinsic value is near-optimal. Secomandi (2015)

provides additional theoretical support for this finding. In Section 4.4, we demonstrate that the

result of Lai et al. (2010) does not hold in incomplete markets, since the optimal risk-neutral

policy yields higher profits than the rolling intrinsic policy, which we show to be the optimal

policy under perfect risk aversion. In Section 4.5, we compare both approaches in a rolling

horizon, out-of-sample backtest using historical market data of natural gas futures prices from

NYMEX Henry Hub. We find that indifference pricing leads to up to 40% higher profits than the

rolling intrinsic solution when managing a storage contract over time, which provides additional

evidence for the importance of considering market incompleteness in practical applications.

Our results provide energy traders with a new approach to jointly optimize futures trading

and storage management, which is an attractive alternative to rolling intrinsic valuation. In

particular, the model allows traders to choose a statistical model that matches the empirical

realities of gas markets and to choose the level of risk exposure that matches their preferences.

Furthermore, as the method is quite general, we believe that the proposed solution approach is

of interest to researchers working on similar high-dimensional stochastic-dynamic optimization

problems.


2. Model Formulation

2.1. Assumptions

We consider the problem of a price-taking energy trader who manages a gas storage contract

over time, which grants the buyer the right to inject, withdraw, and store gas over a finite time

horizon. The injection, withdrawal, and storage capacity is limited and defined by the contract.

We assume that injection and withdrawal limits are independent of storage level, i.e., ratchets

are ignored as is common with storage contracts.

To decide about buying and selling storage capacity, a trader has to price the contract at the

time of inception. The utility of the storage is derived from the distribution of future discounted

rewards (cash flow) arising from buying and selling futures contracts while fulfilling matured

contracts. In line with the literature on gas storage valuation, we assume that rewards are

generated in monthly increments (Eydeland and Wolyniec 2003, Lai et al. 2010).

In contrast to the extant literature, we use an indifference pricing approach to deal with the

storage valuation problem in incomplete gas markets. Let us therefore introduce the concept of

indifference pricing and how it differs from the more common risk-free pricing.

To avoid notational clutter, we define [I] = 1, . . . , I for I ∈N. Denote Ft, t ∈ [T ], as the

random futures prices defined on the measure space (Ω,F ,P), with T as the number of discrete

time periods. The fundamental theorem of asset pricing states that, in a complete market,

there exists a unique equivalent (risk-neutral) martingale measure Q on (Ω,F) that can be

used to price derivatives. More specifically, let Ct(Ft, πt) be the discounted immediate reward

in t dependent on the policy π= (π1, . . . , πT ), where πt maps the history F t = (F1, . . . ,Ft) to a

decision in t. Then, the risk-free value of storage is given by

maxπEQ

(T∑t=1

Ct(Ft, πt(Ft))

), (1)

i.e., the maximal expected discounted rewards under the risk-neutral measure Q. In an incom-

plete market, no unique risk-neutral measure Q exists and we use the physical measure, P,

which represents the actual real-world probabilities of the evolution of the futures curve. In

this setting, we can no longer assume that pricing is independent of risk preferences and, hence,


have to replace the expectation in (1) by a more general probability functional, R, to model a

decision-maker’s risk-aversion. Consequently, the indifference value of storage is given by

maxπRP(C1(F1, π1(F 1)), . . . ,CT (FT , πT (F T ))

), (2)

where RP is a dynamic risk measure dependent on the physical measure P (Ruszczynski 2010).

For what follows, we disregard the measure theoretic subtleties above and directly work with

the random variables Ft and their distributions under P.

The physical measure, P, is generally not a martingale and therefore might lead to positive

expected gains from trading in the futures market. To ensure that pricing is not distorted by

excessive trading, we enforce that all futures contracts yield a physically implementable policy

under the storage’s operational constraints at all times. Following Lai et al. (2010), we assume

that each physical injection and/or withdrawal incurs a marginal cost in addition to an in-kind

fuel loss and that storing natural gas incurs no holding cost.

As is common in the literature, e.g., Clewlow and Strickland (2000), Eydeland and Wolyniec

(2003), Lai et al. (2010), the dynamics of the futures curve is assumed to follow a multivariate

geometric Brownian motion (MGBM). At time t ∈ [T ], let Ft = (Ftt, . . . ,FtT ) be the vector of

futures prices, with Ftt being the spot price and Ftj being the futures price with maturity in

period j = t+ 1, . . . , T . Then, the price process is given by the following stochastic differential

equations,

dFtjFtj

= µjdt+σjdZj, dZjdZk = ρjk, 1≤ j, k≤ T, (3)

where dZj with j ∈ [T ] are correlated increments of standard Brownian motions. Note that, in

contrast to Lai et al. (2010), we allow for non-zero drifts µj 6= 0, resulting in processes that are

not martingales. In our problem, we discretize this process to monthly time increments.

We model the risk preferences in (2) by the nested conditional value-at-risk (CVaR) (e.g.,

see Ruszczynski and Shapiro 2006). For a random variable X with distribution function FX , let

CVaRα(X) = α−1∫ α

0F−1X (t)dt and define ρα,λ(X) = λCVaRα(X) + (1− λ)E(X) for 0 ≤ λ ≤ 1

and 0<α≤ 1. For a sequence of random variables X1, . . . ,XT the nested CVaR is defined as

X1 + ρα,λ (X2 + ρα,λ (X3 + · · · )) , (4)


i.e., by a convex combination of expectation and CVaR that recursively includes other convex

combinations of expectation and CVaR. Unlike the terminal CVaR, which measures the risk of

the distribution of total discounted rewards, the nested CVaR is time-consistent, which means

that past losses and gains do not affect the policy (Shapiro 2009).

2.2. Markov Decision Process

We model the decision problem outlined above as a finite horizon, continuous-state, discrete-

time Markov decision process (MDP). The state of the MDP comprises the resource state, Rt,

which includes state variables that the decision-maker can influence, and the environmental

state, Ft, which evolves independently from the decisions. The resource state, Rt, includes

all futures contracts positions, ftj, with j ≥ t, as well as the gas storage level, st, while the

environmental state consists of the futures prices.

The decision policy, π= π1, . . . , πT, includes all trading and operational decisions subject to

the state-dependent feasible set Πt(Rt). Starting with a deterministic state (R1,F1), we define

the dynamic programing equations of the problem as

Vt(Ft,Rt) = maxπt∈Πt(Rt)

Ct(Ft, πt) + ρα,λ

[Vt+1(Ft+1,Rt+1(πt))

∣∣Ft] , t∈ [T ], (5)

with VT+1 ≡ 0. Therefore, an optimal policy, π∗, maximizes the sum of immediate reward and

ρα,λ of the value that results from future storage operations and trading.

To specify the feasible set, Πt(Rt), denote itj and wtj as the injection and withdrawal decisions

in period t needed to close futures contract position ftj that matures in periods j. Physical

injection and withdrawal incurs marginal costs of ci and cw, respectively, as well as in-kind

losses di and dw. Futures buying and selling decisions are denoted by xtj and ytj, respectively.

Accordingly, the immediate reward is given by

Ct(Ft, πt) =T∑j=t

γjFtj(ytj −xtj)− γt(ciitt + cwwtt), (6)

where γt is the discount factor for period t. Note that in order to avoid transaction costs being

charged more than once, we only account for injection and withdrawal costs incurred in period

t but not for planned future injections and withdrawals.


When a futures contract matures, it must be either fulfilled physically through storage oper-

ation or cleared in the spot market, i.e.,

ftt +xtt− ytt = itt−wtt, t∈ [T ]. (7)

The physical storage balance is given by

st+1,t+1 = stt + diitt− dwwtt, t∈ [T ]. (8)

To enforce the assumption that futures contract positions yield an implementable schedule, all

tradable contracts that mature in some future period must be balanced with future injections

and withdrawals at all times,

ftj = itj −wtj, t∈ [T − 1], j = t+ 1, . . . , T. (9)

Moreover, all contracts are subject to balance constraints

ft+1,j = ftj +xtj − ytj, t∈ [T − 1], j = t+ 1, . . . , T, (10)

which track changes in futures contract positions resulting from purchases or sales in t. Injec-

tions and withdrawals for j > t are subject to storage balance

st,j+1 = stj + diitj − dwwtj, t∈ [T − 1], j = t, . . . , T, (11)

where stj is the storage level in period j as projected in period t if all trades are physically

balanced by storage operations.

Storage operational limits are given by the following set of constraints,

i≤ itj ≤ i, t∈ [T ], j = t, . . . , T, (12)

w≤wtj ≤w, t∈ [T ], j = t, . . . , T, (13)

s≤ stj ≤ s, t∈ [T ], j = t, . . . , T. (14)

At the end of period t, the physical storage state as well as the futures contract position yields

the final resource state

Rt+1 = st+1,t+1×ft+1,jTj=t+1, t∈ [T ]. (15)


Note that the resource state’s dimensionality decreases in t because one futures contract matures

in each period.

Remark 1. The above constraints define the feasible sets Πt(Rt), t ∈ [T ], for the policy π.

Since all constraints as well as the objective function are linear, the maximization problem at

stage T is linear. The resource state RT appears only on the right-hand side of the constraints,

such that the objective value and therefore the value function VT is a concave function in RT .

As concavity is preserved by ρα,λ, the value function in T −1 is also concave in RT . By backward

induction, it follows that all value functions Vt are concave functions in Rt and all problems in

(5) are convex optimization problems.

In line with Powell (2011), let us define the post decision value in t+ 1 as

Vt(Ft,Rt+1) = ρα,λ[Vt+1(Ft+1,Rt+1)

∣∣Ft] . (16)

Since R 7→ Vt(Ft,R) is concave, it can be approximated from above by the minimum of a set

of L supporting hyperplanes, ˆVt, which is defined by scalars al(Ft, Rlt+1) and slope vectors

bl(Ft, Rlt+1), whereby Rl

t+1 denotes a feasible post-decision resource state, i.e.,

ˆVt(Ft,Rt+1) = minl

al(Ft, R

lt+1) + bl(Ft, R

lt+1)>(Rt+1− Rl

t+1), l= 1, . . . ,L, t∈ [T − 1]. (17)

Combining (6) with (17), (5) can be approximated as

Vt(Ft,Rt)≈ max∑T

j=t γjFtj(ytj −xtj)− γt(ciitt + cwwtt) + vts.t. vt ≤ al(Ft, Rl

t+1) + bl(Ft, Rlt+1)>(Rt+1− Rl

t+1), l ∈ [L], t∈ [T − 1],(7), (8), (9), (10), (11), (12), (13), (14),xtj, ytj.itj,wtj, stj ≥ 0, t∈ [T − 1], j = t, . . . , T,ft+1,j ∈R, t∈ [T − 1], j = t, . . . , T − 1,vt ∈R, t∈ [T − 1].

(18)

2.3. Special Cases

Depending on parameters α and λ, the optimal policy has the optimal expectation maximizing

policy as well as the rolling intrinsic solution as special cases. The first case is trivial. By setting

λ = 0, ρα,λ reduces to the expected value and we obtain a policy that maximizes expected

discounted rewards.


To relate our policy to the rolling intrinsic solution, we consider the case of λ= 1, i.e., the

decision-maker maximizes only the CVaR. By additionally setting α= 0, we obtain

CVaR0(X) = ess inf(X).

Under the nested CVaR objective with α= 0 and λ= 1, the optimization problem becomes

maxπ

ρα,λ(C(F1, π1), . . . ,C(FT , πT )) = maxπ

T∑t=1

ess infC(Ft, πt). (19)

Proposition 1. 1. The optimal policy for (19) is the rolling intrinsic policy. In particular,

the optimal first-stage objective value equals the intrinsic value of storage.

2. The optimal policy for ρα,1 converges to the rolling intrinsic policy as α→ 0.

Proof. In t = 1, the rolling intrinsic policy maximizes the first-stage reward, C1(F1, π1),

subject to storage constraints, irrespective of possible gains that might arise because of random

price changes in the future. Let these decisions be denoted by π∗1 .

Any policy with lower first-stage reward must earn a positive amount through price changes

in later periods almost surely to compensate for the difference if it is to be optimal for (19).

Therefore, it is enough to show that for every decision π1, there is a positive probability that

the value of associated futures contracts decreases in subsequent periods.

In particular, the set of prices in t= 2, that do not cause the value of the futures positions,

f2,t, to increase, contains

H=F2 = (F2,2, . . . ,F2,T ) : f2,t(F2,t−F1,t)≤ 0, ∀ t= 2, . . . , T

. (20)

H is an intersection of half-spaces in RT−1 that intersect the positive orthant. As prices under

(3) are log-normally distributed, the probability of H is positive. This concludes the proof of 1.

To prove 2, note that CVaRα converges to CVaR0 as α→ 0. Therefore, it follows that ρα,1α→0−→

ρ0,1 and consequently from Shapiro et al. (2009), Theorem 7.27, that ρα,1epi−→ ρ0,1, as α→ 0.

Considering the above and the fact that the decisions are bounded, it follows that the decisions

as well as objective values converge (see Shapiro et al. 2009, Proposition 7.26).


Remark 2. The indifference value is the optimal objective value of the MDP, which is not the

same as the expected discounted rewards from following the optimal policy, unless the agent

is risk-neutral. By Proposition 1, this implies that the indifference value is the intrinsic value

for any agent for whom the rolling intrinsic policy is optimal. This implication contrasts the

classical interpretation of rolling intrinsic valuation which defines the value of storage as the

expected discounted reward from following the rolling intrinsic policy. However, this presumes

that the decision-maker is risk-neutral which contradicts Proposition 1. Therefore, the rolling

intrinsic value cannot be used for indifference pricing in our setting.

3. Method

The problem defined in the last section is a discrete-time, continuous-state, risk-averse MDP.

As it is generally not possible to solve such problems exactly, we use approximate dynamic

programming (Powell 2011) to approximate the optimal policy of the continuous-state problem,

which is equivalent to approximating the value functions of the MDP.

The problem from Section 2.2 divides the state space of the MDP into an environmental

state, Ft, which is exogenously given and random, and a resource state, Rt, which is determined

by the decision process. This separation enables us to approximate the value functions in two

steps: first, we search for an optimal set of representative discrete states for the environmental

state, which we organize in a scenario lattice. To accomplish this goal, we propose a novel

method that combines optimal quantization with moment matching.

Second, we use a version of SDDP that approximates the value function at each node of

the lattice by a concave, piecewise-linear function of the resource state. This contrasts conven-

tional SDDP, as it allows the data process to be Markovian, rather than requiring stagewise

independence.

We refer to this two-stage procedure as approximate dual dynamic programming (ADDP) to

emphasize that we construct an approximate solution for a continuous-state MDP. A policy

for the continuous-state MDP can be obtained by using the piecewise-linear value function

associated with the discrete state closest to a given (continuous) environmental state to make


a decision. The result is an approximate value function that is piecewise-constant in the envi-

ronmental state and piecewise-linear in the resource state.

This procedure has several advantages. A scenario lattice can be constructed by a compu-

tationally inexpensive stochastic gradient algorithm. Furthermore, it eliminates the need to

discretize the high-dimensional resource state. Instead, the procedure exploits the fact that

for a given environmental state of the MDP, i.e., a node on the lattice, the value function is

piecewise-linear and concave in the resource state.

3.1. Scenario Lattices

Since the stochastic process defined in (3) is Markovian, the conditional distribution of future

prices in t+ 1 at stage t does not depend on the entire price history but only on prices in t. If

we discretize (3) to a scenario tree, many branches of the tree would have identical sub trees

which can be combined without loss of information. We refer to such a recombining scenario

tree as scenario lattice, in line with the terminology often used in mathematical finance.

More formally, a lattice is a graph organized in a finite number of layers. Each layer is

associated with a discrete point in time and contains a finite number of nodes. Successive layers

are connected by arcs. A node represents a possible state of the stochastic process, and an arc

represents the possibility of a state transition from one node on a given layer to a successor

node on the next layer. Each arc is associated with a probability weight, and weights of all

outgoing arcs of a node add up to one. Note that for a scenario tree, we would have to add the

requirement that every node in stage t has only one predecessor in stage (t− 1).

Denote Nt as the number of nodes in t, and Ftn, n∈ [Nt], as the state of the stochastic process

at node n in t. Further, denote Ft =Ftn : n∈ [Nt]

as the set of all possible states in the lattice

layer corresponding to time t. Assuming that all state transition between nodes on consecutive

stages have positive probabilities, the set of possible scenarios on the lattice is given by

F1× F2× · · ·× FT−1× FT . (21)

As the number of stages grows, the additional nodes needed to construct a lattice are those

of the newly added stages, while the number of nodes in a scenario tree (with non-trivial


Figure 1 A tree with 31 nodes representing 16 scenarios on the left and lattice with 15 nodes representing

120 scenarios on the right.

conditional distributions) grows exponentially in the number of stages. See Figure 1 for an

illustrative comparison.

Our objective is to construct a scenario lattice such that the optimal policy for the lattice

process yields a close to optimal policy for the true process. In this context, the discretization

error is the difference between the optimal objective value of the stochastic optimization problem

defined with the true process (3) and the value of the optimal policy for the lattice process

implemented on the true process. From stability theory for stochastic programs, it is known

that minimizing this error is closely linked to the Wasserstein distance between the the discrete

and the continuous process (Dupacova et al. 2003, Bally and Pages 2003, Pflug and Pichler

2012).

While there exists an extensive literature on scenario tree generation (see Kaut and Wallace

2007, for a survey) there exists virtually no literature on scenario lattice generation for general

MDPs.

3.2. Lattice Quantization

In this section, we describe a two-step method that discretizes a continuous process to a lattice

using optimal quantization. In the first step, the method searches for the optimal nodes of the

lattice by minimizing the Wasserstein distance between the nodes and the unconditional dis-

tributions at each stage. In the second step, the method estimates the transition probabilities

between nodes on consecutive stages, for which we propose a new technique that performs this

estimation in a recursive manner. Following ideas from moment matching, the new technique


ensures that the conditional expectations on the lattice coincide with the conditional expec-

tations of the true process. As we will see in Section 4.3, this step is crucial to accurately

approximate the indifference value of storage.

To minimize the distance between the price process and the lattice, we resort to optimal

quantization. We can infer from Bally and Pages (2003) that the discretization error depends on

the stagewise Wasserstein distance between the true process F and its discrete approximation

F . More specifically, at each stage t, we find quantizers Ftn, n∈ [Nt], such that

∑n∈Nt

∫Γtn

||Ft− Ftn||22 P(dFt) (22)

is minimal for t= 2, . . . , T , where

Γtn =

x : n= arg min

m

||x− Ftm||22, m∈ [Nt]

(23)

is the Voronoi partition associated with the quantizers Ftn, which serve as the nodes of the

lattice in t.

Unfortunately, finding the optimal quantizers is an NP-hard problem (Aloise et al. 2009,

Lohndorf 2016). Bally and Pages (2003), faced with a similar problem, propose a method based

on stochastic gradient descent which we will adopt here.

Denote (βk)Kk=1 as a sequence of stepsizes with 0≤ βk ≤ 1, k ∈ [K]. To solve the quantization

problem using stochastic gradient descent, we draw random sequences (F kt )Tt=1 from the price

process, for k ∈ [K], and define

F ktn =

F k−1tn +βk

(F kt − F k−1

tn

)if n= arg minm

||F k

t − F k−1tm ||22, m∈ [Nt]

,

F k−1tn otherwise,

(24)

with F 0tn ≡ 0, for n ∈ [Nt], t = 2, . . . , T , k ∈ [K]. Pages and Printems (2003) show that if the

sequence (βk)Kk=1 satisfies

∑∞k=1 βk =∞ and

∑∞k=1 β

2k <∞, then the resulting nodes are local

minimizers of (22).

To estimate the transition probabilities between lattice nodes at subsequent stages, let us fix

the nodes of the lattice as Ftn ≡ FKtn and denote pt, t ∈ [T − 1], as the |Nt| × |Nt+1| transition

matrix between layers t and t+1 with elements ptnm, where each ptnm defines the (conditional)

probability of a state transition from Ftn to Ft+1,m.


Bally and Pages (2003) propose to estimate the transition probabilities by

ptnm =

∑K

k=1 IΓtn(F kt )IΓtm(F k

t+1)∑K

k=1 IΓtn(F kt )

, n∈ [Nt], m∈ [Nt+1], t∈ [T − 1], (25)

with IA as the indicator function of the set A. We refer to this method as forwards estimation

since probabilities are estimated from forward simulations of the price process.

A problem with forwards estimation is that, for any given node in t < T , the (conditional)

expected successor state on the lattice does not exactly match the conditional expected successor

state for the continuous process but usually slightly deviates from this value. This, in turn,

introduces a bias into the optimal policy on the lattice, which leads to a value function that

overestimates the value of storage, as we will demonstrate in Section 4.3.

We propose to estimate the transition probabilities in a different manner. Instead of esti-

mating the transition probabilities during forward simulations, we adjust the predecessor nodes

layer by layer while going backwards in time. We refer to this method as backwards estimation.

In contrast to forwards estimation, adjusting the nodes of the lattice going backwards ensures

that the conditional means of the lattice equal those of the true process, thereby eliminating

the aforementioned bias.

Assume that nodes Ft+1,m, m ∈ [Nt+1], are fixed and have already been corrected. Since we

have to correct the nodes in t+ 1, before we can compute the conditional means of the nodes

in t, we begin in T − 1. Before correcting node n in t, we estimate the vector of transition

probabilities, ptn, based on a sequence of sample state transitions, (F kt+1)Kk=1 from (3) conditional

on the values stored in FKtn , by setting

ptnm =K−1

K∑k=1

IΓtm(F kt+1), m∈ [Nt+1], n∈ [Nt], t= T − 1, . . . ,1. (26)

Using these probabilities, the conditional expectation on the lattice for a node, FKtn , is given by

E[Ft+1|FKtn ] =

Nt+1∑m=1

ptnmFt+1,m, n∈ [Nt], t∈ [T − 1], (27)

whereas its conditional expectation for the multivariate GBM is

E[Ft+1|FKtn ] = FK

tn (eµt , . . . , eµT ), (28)


with as point-wise product. To ensure that these two values match for a given node, we correct

the relevant elements of the state vector, Ftn = (Ftn1, . . . , FtnT ),

Ftnj =

Nt+1∑m=1

ptnmFt+1,me−µj , j = t+ 1, . . . , T, n∈ [Nt], t= T − 1, . . . ,1. (29)

Note that, as long as we can explicitly calculate a correction step as in (29), this algorithm can

be adapted to any other Markov process.

The following proposition shows that matching conditional expectations also ensures that

the conditional expectations of states at later stages matches the conditional expectations of

the true process.

Proposition 2. Let (Ft)t∈[T ] be the MGBM process defined in (3), and let (Ft)t∈[T ] be the

corresponding lattice process. Assume that nodes and probabilities of the lattice are such that

E(Ft+r|Ft+r−1 = Ft+r−1,n) =E(Ft+r|Ft+r−1 = Ft+r−1,n), ∀ n∈ [Nt+r−1], ∀ r ∈ [s],

then

E(Ft+s|Ft = Ftk) =E(Ft+s|Ft = Ftn), n∈ [Nt].

Proof. W.l.o.g. (Ft)t∈[T ] takes values in R. Note that because the process is GBM, we have

E(Ft+s|Ft = x) = esµx,

where µ is the drift of the process. We show the result for s = 2, the general case follows by

induction.

E(Ft+2|Ft = Ftk) =∑

j∈Nt+1

ptkj∑

m∈Nt+2

ptjmFt+2,m =∑

j∈Nt+1

ptkjE(Ft+2|Ft+1 = Ft+1,j)

=∑

j∈Nt+1

ptkjeµFt+1,j = eµ

∑j∈Nt+1

ptkjFt+1,j = eµE(Ft+1|Ft = Ftk)

=eµeµFtk =E(Ft+2|Ft = Ftk)

3.3. Stochastic Dual Dynamic Programming on Lattices

In this section, we show how SDDP can be used to approximate the optimal policy for the

discrete price process represented by the lattice and how this policy can be implemented under

the true continuous price process.


SDDP iteratively alternates between simulating the approximate optimal policy and updat-

ing the approximate post-decision value functions, ˆVt, which, in turn, define the approximate

optimal policy of the next iteration.

At each iteration, l ∈ [L], SDDP performs a forward pass, during which a sequence of state

transitions, ( ˆF ltn)Tt=2, is drawn from the lattice process to generate a sequence of sample resource

states, (Rlt)Tt=2. Denote ˆV l

t as the approximate post-decision value function after the l-th iter-

ation. Then, sample resource states can be obtained by following the incumbent approximate

optimal policy, i.e.,

Rlt+1 ∈ arg max

πt∈Πt(Rlt)

Ct(F

ltn, πt) + ˆV l−1

t ( ˆF ltn,Rt+1(πt))

, t∈ [T − 1], l ∈ [L], (30)

with Rl1 =R1 and ˆV 0

t ≡ 0 (i.e., for l= 1, vt = 0 in (18)).

After each forward pass, SDDP performs a backward pass adding new hyperplanes to the sets

of supporting hyperplanes that define the approximate post-decision value functions.

To obtain the post-decision value function at any node of the lattice, we have to evaluate

the nested CVaR of the value functions at all possible successor nodes. To accomplish this, we

adapt the method proposed by Philpott et al. (2013) for lattices. In contrast to other methods

for handling the nested CVaR, this approach requires no reformulation of the model and does

not augment the state space of the dynamic program.

The method of Philpott et al. (2013) is based on the idea of reweighing transition probabil-

ities. In general, for a discrete random variable X with possible realizations X1, . . . ,XM and

corresponding probabilities p1, . . . , pM , the risk measure ρα,λ has the dual representation

ρα,λ(X) = minξ∈U

∑i∈[M ]

piξiXi, (31)

where

U =

ξ ∈RM :∑i∈[M ]

ξipi = 1, ξi = (1−λ) +ληi,0≤ ηi ≤1

α, ∀ i∈ [M ]

. (32)

Using (16), this enables us to write

Vt(Ftn,Rt+1) = minξ∈U

∑m∈[Nt+1]

ptnmξmVt+1(Ft+1,m,Rt+1). (33)


Hence, ρα,λ can be viewed as an expectation with a changed probability measure. The super-

gradients can be constructed from this representation by noting that

∂Rt+1Vt(Ftn,Rt+1) =

∑m∈[Nt+1]

ptnmξ∗m∂Rt+1

Vt+1(Ft+1,m,Rt+1), (34)

where the weights, ξ∗m, m∈ [Nt+1], are optimal for Rt+1 in (33).

As the optimal weights are easily identified by sorting the values Vt+1(Ft+1,m,Rt+1), t∈ [T−1],

the above method can be used to construct value function approximations during the backward

pass similar to the expectation case.

We adapt this idea to the piecewise-linear approximate post-decision value functions. In this

setting, the values corresponding to (33) at sample resource states, Rlt, are given by

al(Ftn, Rlt+1) =

Nt+1∑m=1

ptnmξ∗m maxπt+1∈Πt+1(Rl

t+1)

Ct+1(Ft+1,m, πt+1) + ˆV l

t+1(Ft+1,m,Rt+2(πt+1)),

(35)

n∈ [Nt], t= T − 1, . . . ,1, l ∈ [L].

Accordingly, as in (34), the slope vectors of the supporting hyperplanes at Rlt+1, are given by

bl(Ftn, Rlt+1) = ∂Rl

t+1al(Ftn, R

lt+1), n∈ [Nt], t= T − 1, . . . ,1, l ∈ [L], (36)

which can be obtained from the dual solution of (35).

The convergence properties of the algorithm are summarized in the following proposition.

Proposition 3. 1. The approximate value functions are upper bounds of the problem’s true

value functions.

2. The approximate optimal policy converges to the optimal policy in finitely many iterations.

Proof. Note that the function RT 7→ VT−1(FT−1,n,RT ) is concave, since the initial resource

state only appears in the right-hand side of the optimization problem in T . The hyperplanes are

supergradients and hence upper bounds of the function. The minimum of these upper bounds

remains an upper bound. An inductive argument over the stages establishes 1.

To prove 2, note that as all maximization problems are linear, the functions RT 7→

VT−1(FT−1,n,RT ) are actually piecewise-linear, and therefore equal to the minimum of a


finite number of supergradients (see Philpott and Guan 2008, Lemma 1, and Remark 1

above). Assume that the algorithm stops adding hyperplanes to the approximate post-decision

value functions after iteration n ∈ N, and there is a sampling path (F1,n1 , . . . , FT−1,nT−1)

such that FT−1,nT−1= FT−1,n leading to a resource state RT at the end of period T − 1

with VT−1(FT−1,n,RT )< ˆVT−1(FT−1,n,RT ). Then, by the Borel-Cantelli Lemma, the sequence

(F1,n1 , . . . ,FT−1,nT−1) will be sampled in some iteration n′ >n. Accordingly, a new supporting

hyperplane will be added at RT , which contradicts the choice of n.

To speed up the algorithm, we use the ε-approximation of Lohndorf et al. (2013), which

rejects hyperplanes during the backward pass that do not improve the approximation by more

than ε > 0. If ε is set to zero each post-decision value function node will consist of exactly l

hyperplanes after iteration l, as in conventional SDDP. Discarding hyperplanes leads to a looser

approximation but also decreases problem size and hence computational effort as we will see in

Section 4.3.

Since the approximate value function obtained by SDDP are upper bounds of the true value

functions on the lattice, the optimal objective value of the optimization problem in t= 1 is an

upper bound for the true value on the lattice. Let us refer to this value as lattice upper bound,

LUB = maxπ1∈Π1(R1)

C(F1, π1) + ˆV L

1 (F1,R2).

While we cannot derive an upper bound of the problem under the true process, we can easily

obtain a lower bound for the optimal expectation (λ= 0), which we refer to as the process lower

bound (PLB). Denote π∗ as the optimal policy obtained under the lattice process. To compute

the PLB, we draw K random sequences (F kt )Tt=1 from the true price process, and then use the

following policy to make decisions,

π′

t ≡ arg maxπt∈Πt(Rt)

C(F k

t , πt) + ˆVt(Ftn,Rt+1(πt))

(37)

whereby n= arg minm

||F k

t − Ftm||22, m∈ [Nt]

. The PLB is the sample average of the imme-

diate rewards, which are accumulated over time following the policy π′,

PLB =C1(F1, π∗1) +K−1

K∑k=1

T∑t=2

Ct(Fkt , π

′

t). (38)


Although the LUB is not an upper bound for the true problem, it is reasonable to conclude

that if, for the risk-neutral case (λ = 0), the gap between LUB and PLB is small, the value

function approximations are sufficiently accurate and the lattice is a good approximation of the

true process.

4. Numerical Results

4.1. Instances

For our numerical analysis, we use the exact same parameters as reported in Lai et al. (2010).

Their setup combines real price data with storage contract characteristics from the literature.

Lai et al. (2010) calibrate four driftless versions of the MGBM price process, using the

information available at the closing of NYMEX Henry Hub on 2006-01-03 (Spring), 2006-06-01

(Summer), 2006-08-31 (Fall), and 2006-12-01 (Winter). The interest rates as reported by the

Dept of Treasury on the four selected days define the monthly discount factor of the model.

The initial state of the MGBM, F1, is given by the spot price and the futures prices of the

first 23 maturities on each of the four days. Implied volatilities of the 23 futures prices are

obtained from prices of NYMEX call options on natural gas futures. Storage instances vary in

their injection (withdrawal) limits ranging from 0.15 mmBtu/month (0.3 mmBtu/month) for

instance A to 0.45 mmBt/month (0.9 mmBt/month) in instance C. If not stated otherwise,

model instance A with Spring prices will serve as our base case. The resulting problems have

24 decision stages. All constant model parameters are summarized in Table 5 in the Appendix.

Since we assume the market to be incomplete, we estimated the drift parameters using the

same data as Lai et al. (2010) to facilitate a comparison of the results (see Table 4 in the

Appendix).

In this setup, there are 23 prices, 23 contracts, and 1 storage, so that we end up with a 24-

dimensional resource state and a 23-dimensional environmental state at the time of valuation.

Classic stochastic-dynamic programming with a look-up table representation and ten possible

states in each dimension would end up with a total of 1047 states. Similarly, if we were to

replace a 100-node-per-stage lattice with the equivalent scenario tree, we would end up with

with ≈ 10023 = 1046 terminal nodes. Both alternatives are computationally intractable.


4.2. Implementation

Unless stated otherwise, for our numerical experiments, we ran SDDP for 1000 iterations and

constructed a lattice with 1000 nodes per stage and backwards estimation as base case. The

algorithm rejects supporting hyperplanes that improve the current approximation by less than

ε= 0.001.

All lattices were built on the basis ofK = 106 price scenarios for the quantization and forwards

estimation step as well as K = 103 state transitions sampled from each node for backwards

estimation. As stepsize rule for the stochastic gradient algorithm we used

βk =100

k+ 1000, k ∈ [K]. (39)

See Figure 2 for a visual comparison of the evolution of the spot price over time for different

lattice sizes. Inspection of the graphs does not reveal any glaring inconsistencies between the

true process and lattices with varying number of nodes per stage. Quite the contrary, important

process characteristics seem to remain intact after quantization, namely mean, variance, and

skewness of the unconditional distribution as well as serial dependence.

The rolling intrinsic policy can be obtained by performing forward passes without a value

function, i.e., Vt ≡ 0, t∈ [T ]. In contrast to Lai et al. (2010), we do not consider future injection

and withdrawal cost (see Section 2.2). Nevertheless, we found that the resulting discounted

rewards deviate by less than 2% from the values reported in Lai et al. (2010), and we therefore

decided to ignore this detail in favor of model parsimony.

The implementation of the algorithms is available as a Java library called QUASAR.1 Com-

putations were conducted remotely on a shared virtual machine (Dual-Xeon E5-2650) with 16

available CPU threads and 64 GB memory.

4.3. Approximation Quality

Lai et al. (2010) show that the rolling intrinsic policy is near-optimal under risk-free pricing.

We can therefore use the rolling intrinsic policy as benchmark for this case. We are particularly

1 QUASAR is available from http://www.quantego.com.

http://www.quantego.com


Mar2006

Jun Oct Dez Mar2007

Jun Oct Dez0

10

20

30

40

50

60

70

$/m

mB

tu

(a) Multivariate GBM

Mar2006

Jun Oct Dez Mar2007

Jun Oct Dez0

10

20

30

40

50

60

70

$/m

mB

tu

(b) 10-node lattice

Mar2006

Jun Oct Dez Mar2007

Jun Oct Dez0

10

20

30

40

50

60

70

$/m

mB

tu

(c) 100-node lattice

Mar2006

Jun Oct Dez Mar2007

Jun Oct Dez0

10

20

30

40

50

60

70

$/m

mB

tu

(d) 1000-node lattice

Figure 2 Time series simulation of the spot price showing 1000 sample paths drawn from the continuous

process as well as three different lattices with 10, 100, and 1000 nodes per stage. Darker areas of

the graph represent nodes and transitions that are more frequently sampled, while lighter regions

are sampled less frequently. The unconditional stagewise means are indicated by a black solid line.

interested in the difference between the rolling intrinsic value and LUB/PLB to study the

influence of lattice quantization on solution quality.

We varied parameters of the price process, the quantization method, number of nodes per

stage, and the rejection threshold, ceteris paribus vis-a-vis the base case. Table 1 reports the

total number of hyperplanes (Hyps), the number of linear programming problems solved until

termination (LPs solved), the total computation time (Time), the lattice upper bound (LUB),

the process lower bound (PLB), and the rolling intrinsic value (RI). All policies are executed

for 105 sample paths generated with the true process. Each run was repeated five times to study

the variability of LUB and PLB under different random seeds.

Under default algorithm settings, the difference between the rolling intrinsic value and the

PLBs is below 1.4 percent, irrespective of transition probability estimation method and season.


Season ε |Nt| LQL Hyps LPs solved Time LUB (SE) PLB (SE) RI

Spring 10−3 1000 B 490k 22.6m 1041s 4.208 (.037) 4.164 (.004) 4.175F 597k 23m 1345s 6.749 (.005) 4.153 (.002)

Summer B 502k 22.6m 1032s 4.632 (0.026) 4.598 (.002) 4.662F 630k 23m 1380s 7.319 (.003) 4.585 (.002)

Fall B 563k 22.6m 1105s 4.048 (.020) 4.039 (.002) 4.094F 667k 23m 1433s 6.520 (.002) 4.037 (.001)

Winter B 696k 22.6m 1147s 1.694 (.011) 1.701 (.002) 1.698F 864k 23m 1492s 3.769 (.002) 1.690 (.002)

Spring 10−1 1000 B 111k 22.6m 899s 4.662 (.025) 4.089 (.007) 4.17510−1 F 107k 23m 1144s 7.282 (.043) 4.070 (.005)10−5 B 1549k 22.6m 1428s 4.173 (.017) 4.160 (.003)10−5 F 2262k 23m 2028s 6.737 (.006) 4.149 (.003)

10−3 100 B 44k 2.3m 137s 4.069 (.012) 4.132 (.003)100 F 54k 2.3m 174s 6.562 (.008) 4.125 (.003)10000 B 5105k 221m 11590s 4.225 (.016) 4.174 (.002)10000 F 6622k 230m 12579s 7.536 (.002) 4.161 (.005)

ε = cut rejection threshold, |Nt| = nodes per stage, B = backwards estimation, F = forwards estimation, LUB =lattice upper bound, PLB = process lower bound, RI = rolling intrinsic value, SE = standard error for 5 repetitions

Table 1 Influence of different algorithmic settings on solution quality

Although the difference is small, backwards estimation yields consistently higher rewards than

forwards estimation and even marginally higher rewards than the rolling intrinsic policy for

the Winter prices. Nevertheless, the difference in rewards between both policies is small when

executed under the true process. We conclude that the approximate optimal policies are near-

optimal.

The most notable result of the analysis concerns the difference between forwards estimation

and backwards estimation with respect to the objective value of the MDP as given by the

LUB. With forwards estimation, the LUB exhibits a substantial upwards bias, which implies

that the optimal value of storage cannot be accurately calculated. With backwards estimation,

by contrast, the gap between LUB and PLB vanishes and the bias is effectively eliminated.

We conclude that, by achieving consistency between approximate value (LUB) and true value

(PLB), the approximated optimal objective value of the MDP reflects the correct value of

storage.

Changing the cut rejection threshold has a smaller influence on the PLB than on the LUB,

where the latter increases as the threshold increases and decreases as the threshold decreases.

Decreasing the threshold from ε = 10−3 to ε = 10−5 increases the number of hyperplanes by

a factor of 3 (4), thereby increasing computation time by 37 (51) percent when backwards


estimation (forwards estimation) was used.

Changing number of nodes has an even smaller effect than the threshold. Computation times

and the number of hyperplanes scale fairly linearly in the number of nodes. With 10000 nodes

per stage and backwards estimation, the difference between process lower bound and rolling

intrinsic value is not significant, which indicates that the optimal discrete policy eventually

converges towards a near-optimal policy if the number of nodes is sufficiently large. Never-

theless, even with only 100 nodes per stage, the LUB (PLB) deviates less than 3 (2) percent

from the rolling intrinsic value, which indicates that quality solutions can be obtained at low

computational cost.

Although solving the optimization problem entails solution of several million LPs and pro-

duces a large number of supporting hyperplanes, an efficient parallel implementation keeps run

times well below 30 minutes (with default settings), which highlights the practical applicability

of our approach.

4.4. Indifference Pricing

For our next analysis, we turn towards studying the performance of our approach using indif-

ference pricing. We use the MGBM with the estimated drifts as physical measure. Following

Lai et al. (2010), we created 12 instances of the model that combine one out of the four variants

of the price process with one out of the three possible storage capacity limits. We evaluate four

policies: the first policy (EV) maximizes the expected value; the second policy (Low) maximizes

a weighted combination of expected value and CVaR with λ= 0.1 and α= 0.05; the third policy

(High) maximizes only the CVaR with α= 0.05 and λ= 1; and the fourth policy is the rolling

intrinsic policy (RI).

Table 2 reports the computational results, including the indifference value of storage under

the corresponding risk measure (LUB) as well as the expected discounted rewards from following

the optimized policy (PLB) and its standard deviation for 105 sample paths generated with

the true process. Only in the risk-neutral case, LUB is equal to PLB. In all other cases, LUB

is smaller than PLB, as it includes the tail loss which is lower than the expected value. This


Instance Storage Value (LUB) Exp Dsc Rewards (PLB) Standard Deviation

Low High I EV Low High RI EV Low High RI

Spring A 4.25 3.88 3.76 5.90 4.88 4.26 4.23 5.98 3.27 0.48 0.27Spring B 5.22 4.65 4.31 7.18 6.14 5.39 5.32 6.97 4.07 0.69 0.43Spring C 5.60 4.86 4.40 7.77 6.75 5.90 5.82 7.20 4.58 0.86 0.58

Summer A 4.81 4.40 4.26 6.34 5.50 4.77 4.70 6.26 3.66 0.50 0.30Summer B 6.20 5.62 5.29 8.17 7.31 6.42 6.31 7.14 4.48 0.73 0.43Summer C 6.62 5.91 5.48 8.81 8.02 7.00 6.85 7.04 4.94 0.92 0.58

Fall A 4.18 3.74 3.53 5.63 4.88 4.19 4.13 5.74 3.33 0.45 0.34Fall B 6.28 5.56 5.10 8.08 7.36 6.51 6.43 6.05 4.23 0.68 0.53Fall C 7.45 6.57 5.94 9.40 8.62 7.69 7.59 7.50 4.67 0.87 0.71

Winter A 1.80 1.26 0.93 3.10 2.34 1.84 1.75 3.92 2.23 0.43 0.35Winter B 2.46 1.73 1.17 4.07 3.35 2.56 2.49 4.71 3.10 0.62 0.53Winter C 2.73 1.94 1.27 4.58 3.78 2.96 2.88 4.95 3.29 0.74 0.65

EV = maximize expected discounted rewards, Low = maximize nested CVaR with α = 0.05, λ = 0.1, High =maximize nested CVaR with α= 0.05, I = intrinsic value, RI = rolling intrinsic value

Table 2 Indifference value of storage and discounted rewards (in $/mmBtu) with different risk measures

supports the theoretical finding that the indifference value cannot be obtained from the expected

discounted rewards, as it is done in rolling intrinsic valuation.

Moreover, the table shows that there exists a clear trade-off between high expectation and

low standard deviation of the discounted rewards. A risk-neutral decision-maker who maximizes

the expectation achieves the highest rewards but is also exposed to a significant amount of risk.

Perfect risk aversion can be achieved by following the rolling intrinsic policy, which minimizes

risk exposure but also leads to lower rewards. These findings are consistent across all problem

instances and confirm our theoretical investigation from Proposition 1.

Let us now take a look at the distribution of the discounted rewards under different policies

as they evolve over time. Figure 3 shows four fan charts of the cumulative discounted reward

distribution for different values of α and λ.

The plots indicate that maximizing the expected value invites speculation with a high down-

side risk but also a high upside potential. While the corresponding policy invests in the begin-

ning and postpones realization of rewards to the end of the time horizon, the risk-averse policy

with λ= 1.0 realizes the largest fraction of the reward upfront with only marginal gains over

time.

The results show that the rolling intrinsic approach is not optimal for decision-makers who

are not perfectly risk averse. This result contributes to the literature which currently views the


Mar2006

Jun Oct Dez Mar2007

Jun Oct Dez10

5

0

5

10

15

20

$/m

mB

tu

6.00

(a) Optimal expectation (λ= 0.0)

Mar2006

Jun Oct Dez Mar2007

Jun Oct Dez10

5

0

5

10

15

20

$/m

mB

tu

5.25

(b) Nested CVaR with α= 0.05, λ= 0.1

Mar2006

Jun Oct Dez Mar2007

Jun Oct Dez10

5

0

5

10

15

20

$/m

mB

tu

4.32

(c) Nested CVaR with α= 0.05, λ= 1.0

Mar2006

Jun Oct Dez Mar2007

Jun Oct Dez10

5

0

5

10

15

20

$/m

mB

tu4.23

(d) Rolling intrinsic

Figure 3 Fan charts of accumulated rewards for different risk preferences which visualize the unconditional

distributions of rewards with dark colors around the median and lighter colors for more extreme

quantiles of the distribution. Additionally, the unconditional mean is indicated by a black solid line

with the average terminal profit on the right-hand side of each graph.

rolling intrinsic value as near-optimal. While our results confirm this view for risk-free pricing,

the results for indifference pricing show that the rolling intrinsic policy is clearly suboptimal

for a risk-neutral decision-maker.

4.5. Backtest with Historical Data

The previous section presented a comparison of different policies for simulated price paths.

In this section, we report how these policies perform in a real trading situation when evalu-

ated against actual prices instead of simulations. In particular, we are interested in a mutual

comparison of indifference pricing and rolling intrinsic valuation based on historical futures

prices.

We collected data of historical 12-month price futures curves of NYMEX Henry Hub natural

gas futures from 1991 to 2015. We used instance A and computed rewards that accumulate


Nodes/stage Iterations α λ Avg Reward Std Dev Difference to RI p-value

100 200 0.05 0.50 0.851 0.843 0.075 0.011

100 200 0 1.084 1.579 0.308 0.184100 200 0.01 0.50 0.836 0.874 0.060 0.043100 200 0.10 0.50 0.884 0.922 0.109 0.046100 200 0.05 1.00 0.808 0.821 0.033 0.062100 200 0.05 0.10 1.023 1.170 0.247 0.13610 100 0.05 0.50 0.844 0.866 0.069 0.0811000 1000 0.05 0.50 0.846 0.852 0.071 0.007

Table 3 Out-of-sample rewards (in $/mmBtu) based on historical Henry Hub data.

over a 12-month time horizon with zero interest rate from March to February. To estimate the

parameters of the MGBM price process, we used data from the respective previous 5 years for

each of the problems.

We investigated the decisions of a hypothetical gas trader who purchases a storage contract

each year and then manages the contract over the course of the year. This requires to solve

the optimization problem over a receding horizon and update model parameters accordingly.

Starting in 1996, each year began with a clean slate and a 12-month planning horizon. After

one problem was solved, the immediate reward from implementing the first-stage decision was

recorded and the final resource state was passed on as initial state to the subsequent 11-month

problem. Before solving the subsequent problem, MGBM parameters were reestimated and a

new lattice was built. This process was repeated over a receding horizon until only 1 month was

left. Then, a new contract was purchased. The resulting sample of records contains 20 annual

and 240 monthly reward observations.

The results of the backtest for different policy parameters are summarized in Table 3. The

rolling intrinsic policy which serves as benchmark lead to an average annual reward of 0.775

($/mmBtu) with a standard deviation of of 0.815. The table reports the difference in average

rewards between the rolling intrinsic policy and the approximate optimal policies for different

parameters, along with the p-values of a two-sided t-test for equal means.

The results show that all indifference pricing policies lead to higher rewards than the rolling

intrinsic policy. In line with the findings from the simulation study, we observe a clear trade-off

between reward and risk. The difference to the rolling intrinsic value is in the range of 8 to 40

percent and is more sensitive towards the risk preferences than to algorithmic parameters.


Although the difference is greatest for the risk-neutral case, the result is not significant at

the 90% confidence level, due to the high variability of the reward distribution. Setting λ ≥

0.5 decreases risk as expected, and p-values decrease accordingly, so that for these instances

differences become statistically significant.

Varying the number of nodes per stage had almost no effect. This confirms the result of the

simulation study which showed that coarse discretizations already provide quality solutions.

The results demonstrate that the rolling intrinsic policy is not optimal for a risk-neutral

decision maker and more profit can be made by solving the underlying stochastic-dynamic

optimization problem. We conclude that gas storage valuation benefits from using indifference

pricing instead of risk-free pricing, which provides additional evidence for the incompleteness

of gas markets.

5. Conclusion

We present a novel approach for valuation of gas storage contracts using indifference pricing

that fully embraces the incompleteness of gas markets. The underlying decision problem is

modeled as a high-dimensional stochastic-dynamic optimization problem that jointly considers

futures trading and storage operation and explicitly models a trader’s risk preferences.

We introduce ADDP as a generic framework for solving discrete-time, continuous-state, risk-

averse MDPs and demonstrate that the method is capable of handling the high dimensional

optimization problem which has been considered intractable so far, (e.g. Lai et al. 2010, Nadara-

jah et al. 2015). The algorithm relies on a discretization of the continuous-state futures price

process to a scenario lattice. We find that the approximate optimal policy is highly sensitive

to how well the lattice matches the conditional first moment of the price process. To avoid a

severe upward bias in the valuation, we propose a novel backwards estimation method which

ensures that the conditional means of the lattice coincide with the means of the true process.

Lai et al. (2010) and Secomandi (2015) provide theoretical and empirical support for the near-

optimality of the rolling intrinsic value under risk-free pricing. We extend this result by showing

that, in an incomplete market, the rolling intrinsic solution is only optimal for decision-makers

who are perfectly risk-averse. In all other cases, we demonstrate in a simulation study that


the rolling intrinsic solution leads to sub-optimal results. We further strengthen this finding by

conducting a comprehensive backtest using historical data and show that more profit can be

made by following an indifference pricing policy.

Future work should look at the effect of different futures price models, for example, the model

proposed in Clewlow and Strickland (2000), as well as the interplay of spot and futures trading.

References

Aloise D, Deshpande A, Hansen P, Popat P (2009) Np-hardness of euclidean sum-of-squares clustering.

Machine Learning 75(2):245–248, ISSN 1573-0565.

Bally V, Pages G (2003) A quantization algorithm for solving multidimensional discrete-time optimal

stopping problems. Bernoulli 9(6):1003–1049.

Bjerksund P, Stensland G, Vagstad F (2011) Gas storage valuation: Price modelling v. optimization

methods. The Energy Journal 32(1):203–228.

Bonnans J, Cen Z, Christel T (2012) Energy contracts management by stochastic programming tech-

niques. Annals of Operations Research 200(1):199–222.

Boogert A, de Jong C (2008) Gas storage valuation using a monte carlo method. The Journal of Deriva-

tives 15(3):81–98.

Boogert A, de Jong C (2012) Gas storage valuation using a multifactor price process. Journal of Energy

Markets 12(1).

Carmona R (2009) Indifference Pricing: Theory and Applications. Princeton Series in Financial Engi-

neering (Princeton University Press).

Carmona R, Ludkovski M (2010) Valuation of energy storage: an optimal switching approach. Quanti-

tative Finance 10(4):359–374.

Chen Z, Forsyth P (2007) A semi-lagrangian approach for natural gas storage valuation and optimal

operation. SIAM Journal on Scientific Computing 30(1):339–368.

Clewlow L, Strickland C (2000) Energy Derivatives: Pricing and Risk Management (Lacima Publica-

tions).

Dupacova J, Growe-Kuska N, Romisch W (2003) Scenario reduction in stochastic programming an

approach using probability metrics. Mathematical Programming, Series B 95(3):493–511.


EIA (2016) Annual Energy Outlook 2016 (U.S. Energy Information Administration), URL http://www.

eia.gov/outlooks/archive/aeo16.

Eydeland A, Wolyniec K (2003) Energy and Power Risk Management: New Developments in Modeling,

Pricing, and Hedging. Wiley Finance (Wiley).

Felix B, Weber C (2012) Gas storage valuation applying numerically constructed recombining trees.

European Journal of Operational Research 216:178–187.

Gray J, Khandelwal P (2004a) Realistic gas storge models ii: Trading strategies. Commodities Now

September:1–5.

Gray J, Khandelwal P (2004b) Towards a realistic gas storage model. Commodities Now June:1–4.

Høyland K, Kaut M, Wallace S (2003) A heuristic for moment-matching scenario generation. Computa-

tional Optimization and Applications 24(2):169–185.

Karatzas I, Shreve SE (1998) Methods of mathematical finance, volume 39 (Springer Science & Business

Media).

Kaut M, Wallace S (2007) Evaluation of scenario generation methods for stochastic programming. Pacific

Journal of Optimization 3(2):257–271.

KYOS (2015) KyStore: Gas storage. URL http://www.kyos.com/wp-content/uploads/2016/10/

KYOS-KyStore.pdf.

Lacima (2017) Valuation & Optimization Suite: Storage. URL http://lacimagroup.com/SiteFiles/

lacimagroupcomau/Storage_Fact_Sheet.pdf.

Lai G, Margot F, Secomandi N (2010) An approximate dynamic programming approach to benchmark

practice-based heuristics for natural gas storage valuation. Operations Research 58(3):564–582.

Lohndorf N (2016) An empirical analysis of scenario generation methods for stochastic optimization.

European Journal of Operational Research 255(1):121 – 132.

Lohndorf N, Wozabal D, Minner S (2013) Optimizing trading decisions for hydro storage systems using

approximate dual dynamic programming. Operations Research 61(4):810–823.

MSCI (2015) FEA @Energy Suite. URL https://www.msci.com/documents/1296102/1636401/FEA_

Factsheet.pdf.

Nadarajah S, Margot F, Secomandi N (2015) Relaxations of approximate linear programs for the real

option management of commodity storage. Management Science 61(12):3054–3076.

http://www.eia.gov/outlooks/archive/aeo16

http://www.eia.gov/outlooks/archive/aeo16

http://www.kyos.com/wp-content/uploads/2016/10/KYOS-KyStore.pdf

http://www.kyos.com/wp-content/uploads/2016/10/KYOS-KyStore.pdf

http://lacimagroup.com/SiteFiles/lacimagroupcomau/Storage_Fact_Sheet.pdf

http://lacimagroup.com/SiteFiles/lacimagroupcomau/Storage_Fact_Sheet.pdf

https://www.msci.com/documents/1296102/1636401/FEA_Factsheet.pdf

https://www.msci.com/documents/1296102/1636401/FEA_Factsheet.pdf


Pages G, Printems J (2003) Optimal quadratic quantization for numerics: the gaussian case. Monte Carlo

Methods and Applications 9:135–166.

Pereira M, Pinto L (1991) Multi-stage stochastic optimization applied to energy planning. Mathematical

Programming 52(2):359–375.

Pflug G, Pichler A (2012) A distance for multistage stochastic optimization models. SIAM Journal on

Optimization 22(1):1–23.

Philpott A, de Matos V, Finardi E (2013) On solving multistage stochastic programs with coherent risk

measures. Operations Research 61:957 – 970.

Philpott A, Guan Z (2008) On the convergence of stochastic dual dynamic programming and related

methods. Operations Research Letters 36(4):450–455.

Powell W (2011) Approximate dynamic programming. Solving the curses of dimensionality (Wiley).

Ruszczynski A (2010) Risk-averse dynamic programming for markov decision processes. Mathematical

Programming 125(2):235–261.

Ruszczynski A, Shapiro A (2006) Conditional risk mappings. Mathematics of Operations Research

31(3):544–561.

Secomandi N (2010) Optimal commodity trading with a capacitated storage asset. Management Science

56(3):449–467.

Secomandi N (2015) Merchant commodity storage practice revisited. Operations Research 63(5):1131–

1143.

Shapiro A (2009) On a time consistency concept in risk averse multistage stochastic programming.

Operations Research Letters 37(3):143 – 147.

Shapiro A, Dentcheva D, Ruszczyski A (2009) Lectures on Stochastic Programming: Modeling and Theory.

MOS-SIAM series on optimization (Siam).

Thompson M, Davison M, Rasmussen H (2009) Natural gas storage valuation and optimization: A real

options application. Naval Research Logistics 56(3):226–238.


Appendix A: Problem Parameters

t 1 2 3 4 5 6 7 8 9 10 11 12µt 0.0092 0.0095 0.0101 0.0101 0.01 0.0097 0.0098 0.01 0.01 0.0099 0.0101

t 13 14 15 16 17 18 19 20 21 22 23 24µt 0.0102 0.0108 0.0113 0.0114 0.0103 0.0107 0.0111 0.0113 0.0116 0.0111 0.0112 0.0109

Table 4 Drift parameters of the physical measure.

Parameter Notation Instance Value(s) Unit

Storage limit s 1.00 mmBtuInitial resource state R1 0Injection (withdrawal) limit i,w A 0.15 (0.30) mmBtu

B 0.30 (0.60) mmBtuC 0.45 (0.90) mmBtu

Injection cost ci 0.02 US$/mmBtuWithdrawal cost cw 0.01 US$/mmBtuInjection loss di 0.99 mmBtu/mmBtuWithdrawal loss dw 1.01 mmBtu/mmBtuInterest rates δ Spring 4.74 % p.a.

Summer 5.05 % p.a.Fall 5.01 % p.a.Winter 4.80 % p.a.

Time horizon T 24 month

Table 5 Parameters for the numerical study.

epubwu institutional repositoryepub.wu.ac.at/5421/1/5863.pdf · epubwu institutional repository...

Documents