direct policy conditioning for reservoir operation

Matteo Giuliani1, Andrea Castelletti1,2, Patrick M. Reed3

1 Dipartimento di Elettronica, Informazione, e Bioingegneria, Politecnico di Milano, Milano, Italy 2 Institute of Environmental Engineering ETH-Z, Zurich 3 Department of Civil and Environmental Engineering, Cornell University, Ithaca, NY

IFAC 2014 CAPE TOWN -‐ZA

Modelling and Control of Water Systems

Improving the protection of aquatic ecosystems by dynamically constraining reservoir operation via Direct Policy Conditioning

The fall of the social planner myth?

Stakeholder 1’s utility

Sta

keh

old

er 2

’s u

tility

utopia

REALITY

SOCIAL PLANNER’S PARETO OPTIMAL

dominated

unfeasible

A real world example

Hydropower reservoir

Power plant

Como city

Penstock

River Adda

River Adda

LegendLario

Lario catchment

River

Irrigated area

0 10 20 30 40 505Kilometers

Anghileri, D. et al. Journal of Water Resources Planning and Management, 139(5), 492–500, 2013

LakeComo

LakeComo

r

s 1

s 2

s 3

u 1

u 2

u 3

R2

R1

R2

R1

hydropower plant

irrigated area

H2

H1

H3

H2

H1

H3

q 3

q 2

q 1

q 3

q 2

q 1

s 1

s 2

s 3

u 2

u 3

u 1 m 1

m 2

m 3

(•)

m (•) (•)

(•)

UNCOORDINATED CENTRALIZED

(a) (b)

r

FIG. 3. The model scheme under uncoordinated (left) and centralized (right) man-agement.

23

800 900 1000 1100 1200 1300 1400 1500 1600

460’000

470’000

480’000

490’000

Irrigation deficit [m3/s]2

Hyd

ropo

wer

reve

nue

[eur

o/da

y]

H

ab

C6C5

C4

C3

C2

C1

CO2 CO1 UCC6 C5

C4

C3

C2

C1

UC

LakeComo

LakeComo

r

s 1

s 2

s 3

u 1

u 2

u 3

R2

R1

R2

R1

hydropower plant

irrigated area

H2

H1

H3

H2

H1

H3

q 3

q 2

q 1

q 3

q 2

q 1

s 1

s 2

s 3

u 2

u 3

u 1 m 1

m 2

m 3

(•)

m (•) (•)

(•)

UNCOORDINATED CENTRALIZED

(a) (b)

r

FIG. 3. The model scheme under uncoordinated (left) and centralized (right) man-agement.

23

UNCOORDINATED

CENTRALIZED (SOCIAL PLANNER)

Efficiency vs acceptability: how to trade-off?

acceptability

eff

icie

nc

y

utopia SOCIAL

PLANNER

INDIVIDUALISM

acceptability of the

social planner

efficiency of individualism

Giuliani M. et al., Journal of Water Resources Planning and Management, 2014

coordination mechanism design

Direct Policy Conditioning

an approach to condition the individualistic control policy and push it towards a social welfare equilibrium


an approach to condition the individualistic control policy and push it towards a social welfare equilibrium

PRIMARY obj.

SEC

ON

DA

RY o

bj.

utopia SECONDARY’s

OPTIMUM

PRIMARY’s OPTIMUM

COMPUTE THE SOCIAL PLANNER POLICIES


1

PRIMARY obj.

SEC

ON

DA

RY o

bj.


OPTIMUM

PRIMARY’s OPTIMUM



GET INSIGHTS ON HOW TO CONDITION THE PRIMARY’S POLICY

1 2

PRIMARY obj.

SEC

ON

DA

RY o

bj.


OPTIMUM

PRIMARY’s OPTIMUM

COMPUTE THE CONSTRAINED

PRIMARY POLICY




1 2

3 PRIMARY obj.

SEC

ON

DA

RY o

bj.


OPTIMUM

PRIMARY’s OPTIMUM


PRIMARY POLICY


Multi-objective optimization using the Direct Policy Search approach


Policy parameters vectors Objectives values

1 2

3 PRIMARY obj.

SEC

ON

DA

RY o

bj.


OPTIMUM

PRIMARY’s OPTIMUM


PRIMARY POLICY



Input Variable Selection of the most relevant parameters in explaining the secondary objectives


Subset of policy parameters to be conditioned

1 2

3 PRIMARY obj.

SEC

ON

DA

RY o

bj.


OPTIMUM

PRIMARY’s OPTIMUM

Single-objective optimization of the primary objective with restricted constraints on the sensitive policy parameters



Input Variable Selection of the most relevant parameters in explaining the secondary objectives


Subset of policy parameters to be conditioned

1 2

3 PRIMARY obj.

SEC

ON

DA

RY o

bj.


OPTIMUM

PRIMARY’s OPTIMUM

CASE STUDY

The Susquehanna river system

(b)

(a)

atomicpower plant

Baltimore

ChesterFishery andboating

FERC environmentalrequirements

Conowingohydropower plant

Muddy RunfacilityMarietta

station

PennsylvaniaMaryland

lateral inflow

Susquehanna RiverMuddy Run inflow

Lower Susquehanna

River

Maryland

New York

Pennsylvania

Conowingo pondChester

Baltimore

(b)

(a)

atomicpower plant

Baltimore

ChesterFishery andboating

FERC environmentalrequirements

Conowingohydropower plant

Muddy RunfacilityMarietta

station

PennsylvaniaMaryland

lateral inflow

Susquehanna RiverMuddy Run inflow

Lower Susquehanna

River

Maryland

New York

Pennsylvania

Conowingo pondChester

Baltimore

DPC experimental setting

1

SOCIAL PLANNER POLICIES •  POLICY: Gaussian Radial Basis function with 2 inputs (level & time) + 4

output (release outputs) + 4 basis functions: 32 parameters •  OPTIMIZATION: Borg MOEA parameterized as in Hadka and Reed [2013] •  NFE = 1,000,000 per replica •  30 replications to avoid dependence on randomness

1

Social Planner policies Giuliani. M. et al. Water Resources Research, 2014




INPUT VARIABLE SELECTION •  Tree based iterative input selection [Galelli and Castelletti, 2013]

1

2

Input Variable Selection

75

50

25

0

expla

ined

varia

nce

b t1 bt

2w42 c t

3 b t3w4

3bt4 w4

4

(a) Selected features and corresponding contribution in explaining the Environment objective

b t1 bt

2w42 c t

3 b t3w4

3bt4 w4

4−1

−0.5

0

0.5

1

lower bound policyreference policyPareto-optimal set

(b) Decision variables selected on the Pareto-optimal set

decis

ion va

riable

x1

x2

x3

u1

Gaussian Radial Basis Function [Giuliani et al. 2014]

b = Basis radius

c = Basis centre

w = Network weights

60% explained variance

Input Variable Selection

Reference p.: the best for the environment

Lower bound p. : current situation

para

met

er v

alue




INPUT VARIABLE SELECTION •  Tree based iterative input selection [Galelli and Castelletti, 2013]

CONSTRAINED PRIMARY POLICY •  Baseline policy with constraints on 8 policy parameters •  Default Borg MOEA parameterization [Hadka and Reed 2013] •  NFE = 100,000 per replication •  30 replications to avoid dependence on randomness •  Historical horizon 1999 (drought)

2

3

1

DPC policies’ performance

DPC policies’ performance

+18.6 x 106

US$/year

+ 36%

+46%

- 30% but ..

Conclusions

§  Direct Policy Conditioning as a coordination mechanism design

§  Preliminary results seem to be interesting in terms of improved

perfomance of current operation in the Susquehanna rb

§  Weakness in the physical interpretation of the parameters: how to

communicate the conditioning to the dam operator?

§  Sensitivity to the conditioning setting

THANKS

Programmed event supported by the TC

EGU General Assembly, Vienna 12 April—17 April 2015

EGU General Assembly

The EGU General Assembly 2015 will bring togethergeoscientists from all over the world into one meet-ing covering all disciplines of the Earth, Planetary andSpace Sciences. Especially for young scientists theEGU aims to provide a forum to present their workand discuss their ideas with experts in all fields ofgeosciences.In the divisions Energy, Resources and the Environ-ment (ERE) and Hydrological Sciences (HS) the fol-lowing sessions are proposed:

• Design and Operation of Combined Hydro/Wind/SolarPower Generation Systems: Computer Based Controland Optimization;

• Design and Operation of Water Resource Systems:Computer Based Control and Optimization.

Motivation

Many environmental systems have been modified andare still being modified by human intervention. Thisintervention usually takes the form of the constructionof additions to the system intended to change systembehaviour to better serve the needs of society.This implies that these systems and their behaviourare being designed. They are no longer governed bynatural processes alone. Therefore models of both thenatural and the artificial part of the system will beneeded. As the demands placed on water systems bysociety increase and are increasingly in conflict witheach other, it will become harder to define goals forthe modification of these systems and their behaviour.It will also become harder to design systems and oper-ating rules to satisfy these goals.The aim of these sessions is to bring together expertsin the fields of water management, hydro-, solar-, andwind-power, control theory and operations research to

discuss novel methods or novel ways of using tradi-tional methods to define and implement desired beha-viour for environmental systems.

Design and Operation of Water ResourceSystems: Computer Based Control andOptimization

For control theory water systems pose some uniquechallenges because of the presence of large delays andvery limited means of control. In fact for some sys-tems the limits on the size of the change that can beeffected in a given time period necessitate the use offorecasts to anticipate on system behaviour. For oper-ations research the special challenge is the presence ofincommensurable and conflicting optimization targets,the complex network of relations between stakeholdersand the lack of one clear shared motivation amongststakeholders. Moreover, a new awareness of more vari-ability in the climate on longer time scales and rapidsocial changes both pose new challenges for the decisionmaking process. This implies a need for more frequentreconsideration of decisions and a shorter time scale forthe decision process. This process will therefore needfaster models, for instance simplified dynamic modelsof hydrological systems, statistical process emulators,surrogate models (e.g. linear or nonlinear regression)based on data to feed faster optimization algorithms.Currently the following people and institutions are in-volved in the preparation of this session:

• Niels Schütze, Dresden University of Technology,Germany;

• Andrea Castelletti, Politecnico di Milano, Italy;• Francesca Pianosi, University of Bristol, United

Kingdom;• Renata Romanowicz, Institute of Geophysics,

Polish Academy of Sciences, Warszawa;• Ronald van Nooijen and Alla Kolechkina, Delft

University of Technology, Netherlands.

Design and Operation of Combined Hy-dro/Wind/Solar Power Generation Sys-tems: Computer Based Control and Op-timization

In most locations the yield of wind power or solar poweris uncertain. Hydropower seems an attractive means ofproviding backup power and storage of energy for fu-ture use. Combined schemes seem attractive, but willneed automatic control to optimize their yield. Un-certainty about yield and future supply and demandis a key issue for the management of these combinedschemes. They may also need special facilities for in-tegration in the current energy distribution infrastruc-ture.

Currently the following people and institutions are in-volved in the preparation of this session:

• Demetris Koutsoyiannis and Andreas Efstra-tiadis, National Technical University of Athens,Greece;

• Andrea Castelletti, Politecnico di Milano, Italy;

• Burlando Paolo, ETH Zürich, Zwitzerland;

• Patrick Michael Reed, Cornell University, USA;

• Alla Kolechkina and Ronald van Nooijen, DelftUniversity of Technology, Netherlands.

Key dates

• Call for papers for EGU 2015: 15 October 2014

• Deadline for receipt of abstracts: 7 January 2015

• Letter of acceptance to key authors: 23 January2015

• Conference: 12 April to 17 April 2015 in Vienna,Austria

Programmed event supported by the TC

26

thIUGG General Assembly 2015, Earth and Environmental Sciences for Future Generations

Prague, Czech Republic June 22 - July 2, 2015

IAHS Workshop Hw07

Announcement

At the 26th IUGG General Assembly inPrague in 2015 there will be an IAHS work-shop on Control of Water Resource SystemsHw07. The workshop is being organized underthe auspices of the International Commissionon Water Resources Systems (ICWRS).

Motivation

Today it is rare to find a water resource sys-tem where the interaction with society can beignored. Most systems consist of both nat-ural and manmade components and are gov-erned by both natural processes and processeswithin society. The interaction between soci-ety and the natural system is complex. Animportant part of this interaction consists ofour attempts as humans to alter the systembehaviour through the construction and ma-nipulation of structures such as wells, dams,pumps, weirs, gates, sluices and locks. Ina changing world it can no longer be takenfor granted that the operational rules for themanipulation of the manmade components ofthe water resource system will be appropriateover the whole life time of the infrastructure.

This workshop is intended for presentations onthe formulation and adaptation of operationalrules for the automated manipulation of man-made components of water resource systemswith changing boundary conditions, or, lessformally, for presentations on computer con-trol of water resource systems in a world influx.

Convener team

Currently the following people and institu-tions are involved in the preparation of thissession.

• Alla Kolechkina, Delft University ofTechnology, Netherlands

• Ronald van Nooijen, Delft University ofTechnology, Netherlands

• Andrea Castelletti, Politecnico di Mil-ano, Italy;

26

thGeneral Assembly of the Inter-

national Union of Geodesy and Geo-

physics (IUGG)

A better understanding of the way in whichour planet functions and of the effects of our

actions on its behaviour is needed to providefor the needs of future generations.This Scientific Assembly to be held in Praguefrom 22 June to 2 July 2015 will provide anopportunity for scientists from all geophysicaldisciplines and from all countries to meet andexchange knowledge and ideas. The Assemblyalso will also give the participants the oppor-tunity to inform the general public and policymakers.

Key dates for this workshop

• Abstract submission open: September2014

• Deadline for receipt of workshop ab-stracts: 31 January 2015

• Early bird registration deadline : 10April 2015

• Standard fee registration deadline : 15June 2015

• Conference: 22 June to 2 July in Prague,Czech Republic

TC 8.3 meeting – Wed 27 12:00 Dassen Room (Westin)