hanoi, january 28 th 2015 rodolfo soncini-sessa dei – politecnico di milano imrr project 8 –...
TRANSCRIPT
Hanoi, January 28th 2015
Rodolfo Soncini-SessaDEI – Politecnico di Milano
IMRR Project
8 – Design algorithms
INTEGRATED AND SUSTAINABLE WATER MANAGEMENT OF RED-THAI BINH RIVER SYSTEM
IN A CHANGING CLIMATE
IMRR phases
econnaissance
odeling the system
ndicators identification
cenarios definition
lternative design
valuation
RMISAE
Soncini Sessa, 2007
omparison … C
The Design Problem
(It)
scenario
Design algorithm
SDPStochastic Dynamic Programming
The Design Problem
(It)
scenario
Assumptions
The objectives are separable
Compensation is acceptable
then
The Design Problem for SDPIf et+1 is a white process
(It)
scenario
If et+1 is a white process
and
we do not consider exogenous information …
thenStochastic Dynamic Programming (SDP)
SDP algorithm
xt+1= ft (xt,ut,et+1)
et+1 ~ Ft (• )
utUt (xt)
accordingly
p= {mt(•); t= 0,1,…,h}
step costOptimal expected
Cost-to-go
SDP algorithm
Pros:1. It guarantees the best solution
(provided assumptions are satisfied)
Cons:2. Only one solution per run!
J1
J2
Only one solution !
Gestione delle Risorse Naturali, Politecnico di Milano
Stochastic Dynamic ProgrammingStochastic Dynamic Programming (SDP) suffers from a dual curse:
1) computational cost grows exponentially with state, control and disturbance dimension (curse of dimensionality [Bellman, 1967]);
Look-up tableH-function
unknown H-function
computations are numerically performed on a discretized variable domain
2) a dynamic model of any variable considered among the operating rule’s arguments has to be embedded in the algorithm (curse of modelling [Bertsekas and Tsitsiklis, 1996]).
timet t+1
models are use in a multiple one-step-ahead-simulation mode
Number of iterations for 1 reservoir:
101 x 801 x 52 x (365) x 3 = 22 x 106
x 3
Time per evaluation: 9 x 10-6 sec.
Total time: 3 minutes
Number of iterations for RTBR system:
104 x 804 x 55 x (365) x 3 = 1.4 x 1018
x 3
Time per evaluation: 3.7 x 10-5 sec.
Total time: 1,650,000 years!
Gestione delle Risorse Naturali, Politecnico di Milano
Stochastic Dynamic ProgrammingStochastic Dynamic Programming (SDP) suffers from a dual curse:
1) computational cost grows exponentially with state, control and disturbance dimension (curse of dimensionality [Bellman, 1967]);
Look-up tableH-function
unknown H-function
computations are numerically performed on a discretized variable domain
2) a dynamic model of any variable considered among the operating rule’s arguments has to be embedded in the algorithm (curse of modelling [Bertsekas and Tsitsiklis, 1996]).
timet t+1
models are use in a multiple one-step-ahead-simulation mode
Design algorithm
Genetic Algorithm
4th December 2013
GA are search methods based on two principles inspired by nature:
WHAT ARE GENETIC ALGORITHMS?
Genetics = recombination of structuresNatural Selection = survival of the fittest
The Design Problem
(It)
scenario
(It, θ)
(It, θ)
scenario
4th December 2013Gestione delle Risorse Naturali, Politecnico di Milano
Universal function approximators
Artificial Neural Networks with some particular features can be used as universal function approximators, i.e. as policies.
Multi-layer Perceptron
u1,t
uq,t
θ = [γ11,1, …., γ1
m,n, … , βL1, …, βL
q]
4th December 2013
SOLVING APPROACH: ANN to describe the control law ; GA to find the optimal ANN parameterization .
ALGORITHM:
Gestione delle Risorse Naturali, Politecnico di Milano
Run a system simulation for each individual
Selection, crossover and mutation
new population
initial population
time series of historical inflow
objectives
J1
J2
Initial (random) population
J1
J2
selection of the “best” solutions according to the Pareto dominance criterion
J1
J2
survival of the fittest
J1
J2
generation of a new population
J1
J2
selection of the “best” solutions according to the Pareto dominance criterion
J1
J2
survival of the fittest
J1
J2
iterating….
J1
J2
iterating….
J1
J2
iterating….
J1
J2
final approximation of the Pareto front
GA algorithm
Pros:1. The whole Pareto boundary is generated in one run
Cons:2. It does not guarantees the best solution, neither an
asymptotic convergence
Time per policy evaluation over 39 years for the RTBR system: 0.53 sec.
Dimθ = (2 x Ninput + Noutput) x Nneur Nneur ≥ Ninput + Noutput
Ninput= 4+2 Noutput = 4 Nneur = 10 Dimθ = 160 Num policies = 10160
4 reservoirs
Ninput= 3+2 Noutput = 3 Nneur = 9 Dimθ = 117 Num policies = 10117
3 reservoirs
Ninput= 1+2 Noutput = 1 Nneur = 5 Dimθ = 35 Num policies = 1035
1 reservoir
Too large!
Might be feasibleRunning time: 29 days
Numevaluations about 5.5 106
SDP 250 seconds = 470 policy evaluations
SDP is surely faster
How to reduce
the number of reservoirs
to 3 only?
We will see tomorrow.
GA with extreme events
Design scenario
20 normal years
10 extreme years
regular indicators
extreme indicators
JF , JS , JH ….. JeF , JeS
↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓ ↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓↓
Extreme events
Pareto boundary (qualitative)
Flo
od
IHP1: Hydrop. Production
Extreme floods
Irrigation
Extreme vs regular floods
Regular Flood
Ext
rem
e fl
oods
Irri
gati
on
Trade off between extreme and standard floods
Hoa Binh and Ha Noi flooding
r
t
r,a
A
A
It is feasible only when A <C
A flood of volume A is coming.How to minimize flooding in Ha Noi?
Catch.
C
r
a
r
inflowa
C Capacity
releaser
r flooding threshould
HN
HB
Hoa Binh and Ha Noi flooding
t
r,a
C
A flood of volume A is coming.How to minimize flooding in Ha Noi?
C
r
a
r
inflowa
C Capacity
releaser
r flooding threshould
HN
HB
If A> C the spillway starts
acting
r Flooding!What can we do?
Hoa Binh and Ha Noi flooding
t
r,a
A flood of volume A is coming.How to minimize flooding in Ha Noi?
C
r
a
r
inflowa
C Capacity
releaser
r flooding threshould
HN
HB
*r
Intentionally produce a small flood!
What if the big
flood doesn’t
arrive?
r
C
We have flooded
for nothing!
Thanks for your attention
XIN CẢM ƠN
A51 A87 A20
H 22 22 20
I 330 316 124
F 90 88 106
extF 3973 2844 1490
F>13.4 814 230 0
A51 A87 A20
H 22 22 20
I 330 316 124
F 90 88 106
extF 3973 2844 1490
F>13.4 814 230 0
A51 A87 A20
H 22 22 20
I 330 316 124
F 90 88 106
extF 3973 2844 1490
F>13.4 814 230 0
4th December 2013Gestione delle Risorse Naturali, Politecnico di Milano
The evaluation scheme
a(m3/s)
r (m3/s)
s (m3)
q_YB(m3/s)
q_HY (m3/s)
h_HN(m)
q_ST(m3/s)
g_hyd(kwh)
g_flo(cm)
Hydropowerplant
(conceptual)
Flow routing
(data-driven)
Flow routing
(data-driven)
flooding cost deficit cost
g_sup(m3/s)2 2
Reservoirs model
(conceptual)
hydropower cost
P(kwh)
u (m3/s)
Gestione delle Risorse Naturali, Politecnico di Milano
Universal Approximation Theorem (Cybenko 1989, Funahashi 1989, Hornik et al. 1989)
Every continuous function defined on a closed and bounded set can be approximated arbitrarily closely by a Multi-Layer Perceptron, provided that the number n of neurons in the hidden layers is sufficiently high and that their activation function belongs to a restricted class of functions with particular properties. Precisely,
must be differentiable and monotonically increasing;
the input to the j-th neuron (denoted with ) must enjoy the following property:
Universal function approximators
Sigmoidal functions meet both the requirements.
e.g., the hyperbolic tangent is a sigmoidal function:
Gestione delle Risorse Naturali, Politecnico di Milano
Universal Approximation Theorem (Cybenko 1989, Funahashi 1989, Hornik et al. 1989)
Every continuous function defined on a closed and bounded set can be approximated arbitrarily closely by a Multi-Layer Perceptron, provided that the number n of neurons in the hidden layers is sufficiently high and that their activation function belongs to a restricted class of functions with particular properties.
Universal function approximators
In practice, a 2-layer perceptron is enough
output
parameters