hanoi, january 28 th 2015 rodolfo soncini-sessa dei – politecnico di milano imrr project 8 –...

Hanoi, January 28th 2015

Rodolfo Soncini-SessaDEI – Politecnico di Milano

IMRR Project

8 – Design algorithms 

INTEGRATED AND SUSTAINABLE WATER MANAGEMENT OF RED-THAI BINH RIVER SYSTEM

IN A CHANGING CLIMATE

IMRR phases

econnaissance

odeling the system

ndicators identification

cenarios definition

lternative design

valuation

RMISAE

Soncini Sessa, 2007

omparison … C

The Design Problem

(It)

scenario

Design algorithm

SDPStochastic Dynamic Programming

The Design Problem

(It)

scenario

Assumptions

The objectives are separable

Compensation is acceptable

then

The Design Problem for SDPIf et+1 is a white process

(It)

scenario

If et+1 is a white process

and

we do not consider exogenous information …

thenStochastic Dynamic Programming (SDP)

SDP algorithm

xt+1= ft (xt,ut,et+1)

et+1 ~ Ft (• )

utUt (xt)

accordingly

p= {mt(•); t= 0,1,…,h}

step costOptimal expected

Cost-to-go

SDP algorithm

Pros:1. It guarantees the best solution

(provided assumptions are satisfied)

Cons:2. Only one solution per run!

J1

J2

Only one solution !

Gestione delle Risorse Naturali, Politecnico di Milano

Stochastic Dynamic ProgrammingStochastic Dynamic Programming (SDP) suffers from a dual curse:

1) computational cost grows exponentially with state, control and disturbance dimension (curse of dimensionality [Bellman, 1967]);

Look-up tableH-function

unknown H-function

computations are numerically performed on a discretized variable domain

2) a dynamic model of any variable considered among the operating rule’s arguments has to be embedded in the algorithm (curse of modelling [Bertsekas and Tsitsiklis, 1996]).

timet t+1

models are use in a multiple one-step-ahead-simulation mode

Number of iterations for 1 reservoir:

101 x 801 x 52 x (365) x 3 = 22 x 106

x 3

Time per evaluation: 9 x 10-6 sec.

Total time: 3 minutes

Number of iterations for RTBR system:

104 x 804 x 55 x (365) x 3 = 1.4 x 1018

x 3

Time per evaluation: 3.7 x 10-5 sec.

Total time: 1,650,000 years!

Gestione delle Risorse Naturali, Politecnico di Milano

Stochastic Dynamic ProgrammingStochastic Dynamic Programming (SDP) suffers from a dual curse:

1) computational cost grows exponentially with state, control and disturbance dimension (curse of dimensionality [Bellman, 1967]);

Look-up tableH-function

unknown H-function

computations are numerically performed on a discretized variable domain

2) a dynamic model of any variable considered among the operating rule’s arguments has to be embedded in the algorithm (curse of modelling [Bertsekas and Tsitsiklis, 1996]).

timet t+1

models are use in a multiple one-step-ahead-simulation mode

Design algorithm

Genetic Algorithm

4th December 2013

GA are search methods based on two principles inspired by nature:

WHAT ARE GENETIC ALGORITHMS?

Genetics = recombination of structuresNatural Selection = survival of the fittest

The Design Problem

(It)

scenario

(It, θ)

(It, θ)

scenario

4th December 2013Gestione delle Risorse Naturali, Politecnico di Milano

Universal function approximators

Artificial Neural Networks with some particular features can be used as universal function approximators, i.e. as policies.

Multi-layer Perceptron

u1,t

uq,t

θ = [γ11,1, …., γ1

m,n, … , βL1, …, βL

q]

4th December 2013

SOLVING APPROACH: ANN to describe the control law ; GA to find the optimal ANN parameterization .

ALGORITHM:

Gestione delle Risorse Naturali, Politecnico di Milano

Run a system simulation for each individual

Selection, crossover and mutation

new population

initial population

time series of historical inflow

objectives

J1

J2

Initial (random) population

J1

J2

selection of the “best” solutions according to the Pareto dominance criterion

J1

J2

survival of the fittest

J1

J2

generation of a new population

J1

J2

selection of the “best” solutions according to the Pareto dominance criterion

J1

J2

survival of the fittest

J1

J2

iterating….

J1

J2

iterating….

J1

J2

iterating….

J1

J2

final approximation of the Pareto front

GA algorithm

Pros:1. The whole Pareto boundary is generated in one run

Cons:2. It does not guarantees the best solution, neither an

asymptotic convergence

Time per policy evaluation over 39 years for the RTBR system: 0.53 sec.

Dimθ = (2 x Ninput + Noutput) x Nneur Nneur ≥ Ninput + Noutput

Ninput= 4+2 Noutput = 4 Nneur = 10 Dimθ = 160 Num policies = 10160

4 reservoirs

Ninput= 3+2 Noutput = 3 Nneur = 9 Dimθ = 117 Num policies = 10117

3 reservoirs

Ninput= 1+2 Noutput = 1 Nneur = 5 Dimθ = 35 Num policies = 1035

1 reservoir

Too large!

Might be feasibleRunning time: 29 days

Numevaluations about 5.5 106

SDP 250 seconds = 470 policy evaluations

SDP is surely faster

How to reduce

the number of reservoirs

to 3 only?

We will see tomorrow.

GA with extreme events

Design scenario

20 normal years

10 extreme years

regular indicators

extreme indicators

JF , JS , JH ….. JeF , JeS

↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓

Extreme events

Pareto boundary (qualitative)

Flo

od

IHP1: Hydrop. Production

Extreme floods

Irrigation

Extreme vs regular floods

Regular Flood

Ext

rem

e fl

oods

Irri

gati

on

Trade off between extreme and standard floods

Hoa Binh and Ha Noi flooding

r

t

r,a

A

A

It is feasible only when A <C

A flood of volume A is coming.How to minimize flooding in Ha Noi?

Catch.

C

r

a

r

inflowa

C Capacity

releaser

r flooding threshould

HN

HB

Hoa Binh and Ha Noi flooding

t

r,a

C

A flood of volume A is coming.How to minimize flooding in Ha Noi?

C

r

a

r

inflowa

C Capacity

releaser

r flooding threshould

HN

HB

If A> C the spillway starts

acting

r Flooding!What can we do?

Hoa Binh and Ha Noi flooding

t

r,a

A flood of volume A is coming.How to minimize flooding in Ha Noi?

C

r

a

r

inflowa

C Capacity

releaser

r flooding threshould

HN

HB

*r

Intentionally produce a small flood!

What if the big

flood doesn’t

arrive?

r

C

We have flooded

for nothing!

Thanks for your attention

XIN CẢM ƠN

A51 A87 A20

H 22 22 20

I 330 316 124

F 90 88 106

extF 3973 2844 1490

F>13.4 814 230 0

A51 A87 A20

H 22 22 20

I 330 316 124

F 90 88 106

extF 3973 2844 1490

F>13.4 814 230 0

A51 A87 A20

H 22 22 20

I 330 316 124

F 90 88 106

extF 3973 2844 1490

F>13.4 814 230 0

4th December 2013Gestione delle Risorse Naturali, Politecnico di Milano

The evaluation scheme

a(m3/s)

r (m3/s)

s (m3)

q_YB(m3/s)

q_HY (m3/s)

h_HN(m)

q_ST(m3/s)

g_hyd(kwh)

g_flo(cm)

Hydropowerplant

(conceptual)

Flow routing

(data-driven)

Flow routing

(data-driven)

flooding cost deficit cost

g_sup(m3/s)2 2

Reservoirs model

(conceptual)

hydropower cost

P(kwh)

u (m3/s)

Gestione delle Risorse Naturali, Politecnico di Milano

Universal Approximation Theorem (Cybenko 1989, Funahashi 1989, Hornik et al. 1989)

Every continuous function defined on a closed and bounded set can be approximated arbitrarily closely by a Multi-Layer Perceptron, provided that the number n of neurons in the hidden layers is sufficiently high and that their activation function belongs to a restricted class of functions with particular properties. Precisely,

must be differentiable and monotonically increasing;

the input to the j-th neuron (denoted with ) must enjoy the following property:

Universal function approximators

Sigmoidal functions meet both the requirements.

e.g., the hyperbolic tangent is a sigmoidal function:

Gestione delle Risorse Naturali, Politecnico di Milano

Universal Approximation Theorem (Cybenko 1989, Funahashi 1989, Hornik et al. 1989)

Every continuous function defined on a closed and bounded set can be approximated arbitrarily closely by a Multi-Layer Perceptron, provided that the number n of neurons in the hidden layers is sufficiently high and that their activation function belongs to a restricted class of functions with particular properties.

Universal function approximators

In practice, a 2-layer perceptron is enough

output

parameters