d03.2 implementation report: basic optimization model

INCO-CT-2004-509091

OPTIMA

Optimisation for Sustainable Water Resources Management Instrument type: Specific targeted research or innovation project Priority name: SP1-10

D03.2 Implementation Report: Basic Optimization Model

Due date: 17/03/2006 Actual submission date: 14/02/2006 Start date of project: 01/07/2004 Duration: 36 months Lead contractor of deliverable: ESS Revision: vs 1

Project co-funded by the European Commission within the Sixth Framework Programme (2002-2006) Dissemination level PU Public X PP Restricted to other programme participants (including the Commission Services) RE Restricted to a group specified by the consortium (including the Commission

Services)

CO Confidential, only for members of the consortium (including the Commission Services)

OPTIMA, D03.2 ESS GmbH

Table of Contents Table of Contents .......................................................................................................................... 2 EXECUTIVE SUMMARY............................................................................................................... 3

Project technical objectives ....................................................................................................... 4 Work Package 3: Objectives ......................................................................................................... 5 Problem solving by optimization.................................................................................................... 6

Mathematical formalism............................................................................................................. 7 The problem statement.............................................................................................................. 8 The decision process............................................................................................................... 10

The criteria ........................................................................................................................... 10 The control options............................................................................................................... 10 The scenario assumptions ................................................................................................... 10 Preference structure: objectives and constraints ................................................................. 11 Pareto Optimality and non-dominated alternatives .............................................................. 11 Participatory decision making processes ............................................................................. 12

Water Resources System Simulation ...................................................................................... 14 Basic economics .................................................................................................................. 15

Hyperspace and Gestalt .......................................................................................................... 16 Using Gestalt for Optimization ............................................................................................. 16 Specifying the constraints .................................................................................................... 18 Heuristics and genetic programming.................................................................................... 19

Application to the Water Resources System Optimization ...................................................... 19 The search space................................................................................................................. 20 Water Technologies ............................................................................................................. 21 Search strategies ................................................................................................................. 22 Constructing gradients ......................................................................................................... 22 Evolutionary Algorithms ....................................................................................................... 23

References and selected Bibliography .................................................................................... 25

2


EXECUTIVE SUMMARY The overall aim of OPTIMA is to develop, implement, test, critically evaluate, and exploit an innovative, scientifically rigorous yet practical approach to water resources management, in close cooperation with local and regional stakeholders, intended to increase efficiencies and to reconcile conflicting demands. Based on the European Water Framework Directive (2000/60/EC), the approach equally considers economic efficiency, environmental compatibility, and social equity as the pillars of sustainable development. The project realises both the importance of the socio-political and economic aspects, but also the importance of a reliable, consistent, and shared information basis for the policy and decision making process. The central analytical paradigm in OPTIMA is optimization, based on a dynamic water resources model system describing a river basin in terms of water demand and supply, technical and hydrological efficiency, access and reliability, costs and benefits at various levels of sectoral and spatial or temporal resolution and aggregation. The optimization approach operates on two closely integrated levels:

1. Conceptually, by providing a framework for expressing and subsequently reconciling multiple and conflicting demands for water (a preference structure), by computing what is feasible and achievable vis a vis the stakeholders expectations and aspirations.

2. Technically, by identifying (computing) optimal solutions (note the plural !) within the physical and administrative constraints of the water resources systems under study and the preference structure defined by the stakeholders as the (set of) objective functions.

OPTIMA uses a multi-stage procedure with a two-stage optimization approach specifically designed to facilitate a participatory approach and continuing stakeholder involvement:

1. A first phase builds a detailed model representation of the water resources system under study and establishes a baseline scenario for each case study;

2. A second phase based on evolutionary algorithms for complex optimization identifies feasible solutions that meet all or as many as possible of the user expectations expressed in terms of constraints on performance criteria,

3. A subsequent discrete multi-criteria phase is oriented toward conflict resolution: it defines the trade-offs between the conflicting objectives using a reference point methodology and the concept of Pareto-efficiency to arrive at a generally acceptable solution as global optimum.

This Deliverable addresses the second, basic optimization phase of the approach.

3


Project technical objectives The primary general objectives of the Programme include: Developing comprehensive Decision Support Systems (DSS) in support of policy analysis and enforcement as appropriate. Within this more generic framework, OPTIMA addresses a number of specific scientific objectives:

• To build, and test in a number of parallel comparative case studies, a consistent and well integrated set of advanced but practical DSS tools for efficient, "optimal" water management strategies and policies of use, designed for a participatory public decision making process.

• To extend the classical techno-economic approach by explicit consideration and inclusion in the two-phase optimisation methodology of acceptance and implementation criteria, where the method not only helps to generate optimal solutions, but facilitates the process of agreeing on what exactly optimal means in any particular case, as a shared community vision, including gender sensitive issues where feasible.

• To develop a generic approach to combine engineering analysis and formal optimisation with socio-economic considerations in a unifying and consistent multi-criteria multi-objective framework.

• To integrate expert systems technology and heuristics with complex simulation and optimisation models to improve their usability in data poor and data constrained application situations.

• To develop appropriate tools and methods for the communication of complex technical information to a broad range of participants and stakeholders in the policy making process, based on classical workshops and Internet technology, and in particular the easy and efficient elicitation of preferences and trade-offs in an interactive, reference point approach.

• To adapt and further develop formal methods of optimisation for highly complex, non-linear, dynamic, and spatially distributed systems that are non-differentiable by applying heuristics, genetic programming combined with local stochastic gradient methods and post-optimal analysis for large scale discrete multi-criteria problems.

4


Work Package 3: Objectives Work package 3 is dedicated to the development and implementation of the modeling tools, that are shared between the individual case studies. The main objective is to adapt and implement, based on the findings of WP 01, a set of quantitative analysis tools (simulation and optimisation models) that are easy to use with the available data, computationally efficient, and that can provide a realistic, comprehensive problem representation. The objectives include:

• Adaptation and implementation of the water resources management simulation model; • Adaptation and implementation of the optimisation framework (heuristics and genetic

programming) • Linkage of the simulation model system with the optimisation framework; • Adaptation of the interactive discrete multi-criteria optimisation module. • Adaptation implementation of a rule-based assessment model.

The tasks within the work package include adaptations of the software (the basic components are available as operational prototypes), installation with the basic data sets from the case studies (resulting from the parallel compilation of generic techno-economic data in WP 04 and the individual case study data), documentation and user training, and preparation for the optimisation scenarios for WP07- WP18. A major task is the building of the necessary interfaces between the components (simulation model, expert system, and the optimisation framework), the integration of the economic assessment (NPV calculations) with the basic simulation model, and the rule-based expert system for the assessment of environmental, and social criteria for the optimisation. Specific tasks include:

• Adaptation of the model system according to the user requirements of WP 01; • Fine tuning of the genetic programming algorithms to improve computational

performance and add the rule-based assessment components to the objective function and constraints;

• Preparation of linkages between the components based on a standard interface and protocol, using a three-tier client server architecture to facilitate web access;

• Integration of economic assessment (NPV of water supply and use, costs and benefits); • Integration of the basic simulation/optimisation model with the discrete multi-criteria

optimisation tool. • Adaptation of the interactive user interface for the discrete optimisation approach to

support web access with standard, ubiquitous and familiar web browser software.

5


Problem solving by optimization Our problems are man-made;

therefore, they may be solved by man. J.F. Kennedy

The reasonable man adapts to the world; the unreasonable one persists in

trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.

G.B. Shaw

OPTIMA starts with the thesis that there is a range of problems with the water resources management at the river basin scale around the Mediterranean than can be improved upon or even solved. The proposed method is to formulate a decision or optimization problem in a well structured way and search for a solution using an algorithmic approach, implemented as a Decision Support System (DSS), embedded in a participatory process involving local stakeholders to duly represent socio-economic criteria and perceptions and aspirations of the people involved. There are numerous ways to structure and describe an optimization or more general, decision problem, which is at the root of the OPTIMA approach. The starting point is a mismatch of expectations and the observed or projected state of a system: this is a problem. An observed, expected or predicted problem suggests that some actions need to be takes to correct the systems evolution to bring expectations and observed state into balance again. The elements of this approach are:

• The decision problem (the observed or expected deviation of the system in terms of the criteria, see below, from an accepted norm or more desirable state; as a simple example: water scarcity due to excessive upstream use for inefficient irrigation, water pollution that limits downstream use);

• The decision process: to build a decision support system requires a detailed understanding of the actual decision making processes, actors and stakeholders (and thus the intended audience and users), their responsibilities, interests, legal and regulatory framework, as well as the institutional structure and political and socio-economic framework within which possible decision can be taken. Environmental planning and management is increasingly participatory: a well planned involvement of actors and stakeholders is therefore essential for any successful (that is, applied) decision support system.

• The criteria used to describe the behavior and performance of the water resources system (availability and quality of water, overall and sectoral water budgets, supply/demand ratio, reliability of supply, costs and benefits),

• The control options or instruments evaluated (possible measures to affect the performance criteria such as land use change, change of crops, different allocation rules, alternative water technologies such as lining of canals and alternative irrigation technologies, pollution control, waste water treatment, pricing, education, etc.)

• The objectives and constraints or the preference structure for the ranking and selection of the alternatives (different expressions of minimizing costs and maximizing benefits including possibly non monetized environmental or social criteria).

6


Mathematical formalism Optimization is a basic method of operations research and engineering (Gillet, 1976). It is also common in water resources systems analysis (Loucks, Stedinger and Haith 1981) where in fact many methods and approaches have been developed and tested. Classical optimization assumes that we can formulate an objective function that can be maximized or minimized by varying the control variables of a system (model). An analytical solution would thus involve differentiating this objective function: where the first derivative reaches zero, we have found a maximum or minimum. This precise method, however, presupposes that the underlying system is differentiable, can be represented by a system of linear equations. A dynamic, distributed, non-linear model of a water resources management system that has any claim on a realistic process representation including water quality and economic assessment is certainly not. A general formulation of the mathematical programming or optimization problem looks like this:

Maximize [Z1(x), Z2(x), ….., Zp(X)]

Subject to gi(X) = b1 i = 1,2, …,m

In natural language the plain vector formulation of a multi-objective problem (we assume al objectives should be maximized) would read: Maximize the set of objective functions Zj(X), j= 1,2, …, p of the vector of decision variables X X = [ X1, X2, …, X1] subject to the constraints gi which could impose technical feasibility. While precise and concise, this representation would be of little use when dealing with a group of non technical stakeholders that wish to see their individual and common problems solved. Therefore, an important aspect of the project involves a representation and implementation of the method that is easy to communicate and understand and thus facilitates stakeholder involvement. We thus try to apply the language of the previous section and forego any mathematical formulation as much as possible. The terminology is based on the criteria that are on the one hand derived from the stakeholder perceptions of problems, and on the other hand can be represented in the modelling tools. These are at a first level of aggregation, basin wide, sectoral or local, node specific criteria such as:

• Overall water budget and quality • Groundwater sustainability • Technical efficiency • Supply/demand ratio • Reliability of supply, access to water • Development potential • Costs and benefits

7


The problem statement The first step in building a DSS is to define the decision problem (in the framework of OPTIMA, in general, water scarcity and low quality which affects regional development or in general quality and thus usefulness of the system, which in turn primarily derives from socio-economic development and resulting land-use change and thus modified water demands), that requires decisions to correct observed or expected deviations from a desirable system state: management options to maintain water resources and environmental quality and thus the potential and basis for regional socio-economic development. However, it is important to realize that what the scientists (and thus the usual DSS developers) sees as the main problem (e.g., low overall efficiency) may not at all be the main problem as seen by major stakeholders, for whom usually direct monetary or socio-economic aspects (loss of income for irrigated agriculture) are most relevant. While a good DSS must be solidly based in the physical and engineering sciences, it is important that it takes the extra step to link to more directly policy and decision relevant concepts, language and terminology. To ensure that, direct, early, and continuing involvement of key actors and representative stakeholders in any DSS project, and in particular in the design and testing phase, is mandatory. In OPTIMA, the initial problem statement reads: An important feature of the Mediterranean region and the immediate coastal zone in the South and East in particular is that water is a key limiting resource for sustainable development, increased quality of life and ultimately peace. Integrated water management as described under the EU Water Framework Directive 2000/60/EC and in particular in the Mediterranean coastal zone requires a balanced consideration of numerous aspects including both the socio-economic and the physical, environmental domain. While this principle is well understood and much discussed, it is rarely implemented in practice. Water is a key limiting resource for continuing economic development in the Mediterranean region, for increased quality of life and ultimately a peaceful and sustainable development. Given the severity of the situation and the importance of water, every contribution to the development of appropriate tools and solutions that considers a sufficiently broad range of criteria and decision variables likely to reduce pressure on already overextended water resources and to avoid irreversible damage, will have a positive impact. OPTIMA addresses integrated management of limited water resources in the region, with emphasis on water technologies for increased efficiency including recycling and reuse, alternative allocation strategies to improve overall efficiency, and the whole range of environmental impacts. This includes a broad range of considerations such as distribution and efficiency of competing uses (irrigation, industrial and municipal uses including tourism development, water quality, effluent control, etc.); complementary sources of supply (conjunctive use of surface and groundwater, desalination); technological considerations (waste water treatment and reuse, desalination, water technologies for efficient water use, etc.) and socio-demographic conditions (such as population growth, rapid urbanisation, industrialisation, stakeholders demand, the institutional framework, market forces, etc.) Together, these define a classical multi-objective multi-criteria decision problem. While there is a general understanding and appreciation of the issues, OPTIMA aims at a more direct involvement of local opinion and knowledge through representative stakeholders.

8


Through a questionnaire approach that was used with individual stakeholder institutions, the project has tried (and keeps updating the data set with every stakeholder workshop) to obtain a definition of the problems from the stakeholders, directly or through representative proxies.

Stakeholder data base: representation of 225 stakeholder institutions

Issues questionnaires (currently 80): selection and basic interface

The stakeholder inputs define the structure of the decision problem in terms of the

• Problem situations as the overall framework and starting point;

• parameters of the control mechanisms and strategies (and their associated costs from the socio-economic analysis) that lead to the best (given the preferences identified for the decision making process)

• set of criteria describing the system performance (and the associated benefits and

possibly also cost) in terms of the DM objectives (to be minimized or maximized, explicitly or implicitly).

9


The decision process A DSS, by definition, is built to support decisions. In the real world, these are the result of often complex and involved decision making processes characterized by multiple actors, conflicting objectives, vested interests and hidden agenda, and plural rationalities. A successful DSS approach must take these into account. To build a decision support system requires a detailed understanding of the actual decision making processes, actors and stakeholders (and thus the intended audience and users), their responsibilities, interests, legal and regulatory framework, as well as the institutional structure and political and socio-economic framework within which possible decision can be taken. Environmental planning and management is increasingly participatory: a well planned involvement of actors and stakeholders is therefore essential for any successful (that is, applied) decision support system.

The criteria The criteria that are used to describe the system and evaluate its behavior or performance under different sets of possible, future conditions (the most likely scenario on which we base the optimization approach is termed the baseline scenario) which combine both scenario assumptions (uncontrollable external variables such as climate change, possibly associated with some probability ranking) control options, policies, or strategies (measures taken to affect the system behavior such as land use control, waste water treatment, water allocation and pricing;; these usually imply monetary or social costs); The criteria must also be related to the models used, or rather vice versa, the models must address these criteria as their state and output variables, so that every model run (alternative scenario) can be evaluated in terms of the decision relevant criteria.

The control options The possible control options or strategies (expressed in terms of the control or decision variable available (e.g., waste water treatment plants, zoning and land use management); the definition and design of the control options is a central importance, as this is what ultimately constitutes the decisions to be supported by the DSS: choosing an effective (meeting the constraints) and efficient (maximizing or minimizing the objectives) control strategy.

The scenario assumptions The control options the set of instruments that can be applied and thus must be decided upon) should clearly be separated from the uncontrollable scenario assumptions that are not or not directly controllable (climate change, population development); since these assumptions are beyond the scope of the possible direct control, they can not be a meaningful part of the decision problem. Their role in OPTIMA can only be in terms of sensitivity or robustness of the eventual decision (choice of a preferred scenario).

10


Preference structure: objectives and constraints The objectives and constraints are expressed in terms of criteria or performance variables (water quality, aquaculture production) including the economic criteria (costs and benefits) that define the decision problem: this is what the decision maker intends to achieve with the measure or set of measures chosen.

• Constraints stipulate a minimum or maximum value for any one of the criteria that must be achieved in order to make the alternative acceptable of feasible.

• Objectives relate to criteria for which the decision makers want to achieve the largest or smallest possible value, respectively.

Multiple criteria and multiple (conflicting) objectives require the introduction of direct or indirect weights or trade-offs to compare different criteria together consistently. In OPTIMA, we use a discrete multi-criteria methodology and a reference point approach (See also: D03.3) Here it is essential to clearly discriminate between performance criteria that indeed should be maximized (like revenues from irrigated agriculture) versus intermediate or auxiliary variables (such as runoff and groundwater recharge) that do affect the systems performance but are not themselves performance variables or criteria. For any multi-criteria approach, it is also important to keep the criteria used as independent of each other as possible. If several criteria measure, in essence, the same or at least closely related attributes (supply demand, water availability, reliability of supply) this may introduce a bias in a multi-criteria approach that uses implicit weighting based on normalized (between nadir and utopia) distances in the model behavior hyperspace between any one alternative and the reference point (by default, utopia). While the tradeoff between criteria and objectives expressed in terms of criteria is performed in the second step using interactive reference point approach, there are two structural aspects of the performance space that lend themselves to automatic classification:

1. Feasibility of alternatives: an alternative is considered infeasible if it fail to meet any hard constraints.

2. Pareto optimality: any dominated alternative can be eliminated from further consideration automatically. A dominated alternative is one that is worth in every dimension (criteria) than any other alternative in the choice set. A non-dominated or pareto optimal alternative is one that is better in at least one of the criteria than any other alternative.

Pareto Optimality and non-dominated alternatives Alternatives generated by the evolutionary algorithm are not best, but good enough. One automatic classification that does not yet require a preference structure is the half order imposed by separating dominated and non-dominated solution: A solution is considered non-dominated (and thus a candidate for selection and trade-off) if it is better in at least one or the criteria than any other alternative. Assuming a high enough weight for that criterion, it would therefore be a potential candidate for the overall optimum. Pareto optimality, however, depends on the set of criteria considered. This is one of the aspects of the preference structure and thus subject to user choice. Therefore, this pre-filtering should be part of the final stage of the discrete multi-criteria analysis (see D03.3).

11


Participatory decision making processes Over the last 20 years, the socio-political climate with regard to environmental planning and policy making processes have changed dramatically, and in several respects. Environmental legislation has changed, more or less world wide, from a purely reactive system to attribute responsibility for damages to a pro-active concept of pre-empting damages through appropriate planning and mitigation. At the same time, the concept of an acceptable plaintiff in a citizen’s suit with claims under the categories of nuisance, trespass, strict liability, and negligence, has widened considerably. Central elements of environmental legislation such as EIA procedures provide fore explicit public participation and hearings. An important parallel development is the free access to environmental information. Basic principles of the regulatory and legal framework for environmental management and thus DSS design include: Principle of precise preparation: Governmental authority should identify and consider all relevant factors and circumstances when preparing the decision. Principle of trust: Citizens must trust governmental authorities in such a way that the expectations of citizens are not shamed. Principle of fair play: Citizens may not be denied any possibility for protection of their own interests and that partiality must be out of question. Principle of motivation: Every decision of governmental authority should be accomplished by a correct motivation. Specific principles pertaining to environmental law are: Stand-still principle: The quality of the environment in cleaner areas must not deteriorate and the pollution of the more polluted areas must not increase, but decrease. Principle of rectification at source: Environmental law and actions should be primarily centred on prevention of damage. Best practical means principle: A company should use the most effective by present technological standards and also economically viable for the company means of environmental protection. BATNEEC principle (Best Available Techniques Not Entailing Excessive Costs): States that it is mandatory to make the best use of available technology as long as the costs are not excessive. Organisations can also lodge claims before the judge, such as: compensation of damages; declaration of justice; and restoration into the previous state, and the prohibition of the continuance of a certain activity, to which an administrative penalty can be attached (www.eu.milieukontakt.nl/index.php?page_id=18). For a DSS, this means that it has to address this regulatory framework and the associated expectations, including those of usually well informed and active environmental NGOS. Elements of stakeholder participation in DSS projects include the identification of major actors and stakeholders, address and contact data bases that includes description and profiles of all stakeholder institutions identified and contacted including their mission statements and basic

12


descriptions, the maintenance of records of contacts and activities (see, for example, web accessible data bases like: http://aca.ess.co.at/INSTITUTIONS), formal information gathering activities using, for example, questionnaires to identify key problem issues (see, for example on-line tools like: http://aca.ess.co.at/ISSUES), likely scenarios of development, or the set of proposed corrective measure in quantitative formats that lend themselves to at least non-parametric statistical analysis. The second important phase after the problem definition stage and the generation of alternatives is the definition of preference structures that lead to the ranking and ultimately selection of alternatives. In any multi-criteria analysis, this final step is not a scientific, but a political exercise of negotiation and trade-off of necessarily subjective values and believes, which makes the direct involvement of end users mandatory to reach meaningful and practical results. This in turn requires an interface that facilitates the direct involvement of stakeholders and thus the integration of the DSS in the decision making process, both in terms of the concepts and the terminology used. User participation is desirable if not essential at several stages of the entire process:

1. identification of the main problems and issues, the problem statement 2. design of the basic simulation model structure and components, representing the

problem situation, ensuring that the baseline scenario as adequate and representative, including all the elements the stakeholders deem important;

3. definition of the set of criteria that will be used to describe and evaluate the performance of the system and the constraints on feasible alternatives; this also includes definition of any penalty functions used;

4. definition of the set of measure and instruments including applicable water technologies but also policies, strategies, and regulatory mechanisms that are believed to affect control parameters; together they define the dimensionality and extent of the search space;

5. selection and setting of the constraints that the define the region of feasible alternatives in the model behaviour space; depending on the mechanism used, this my be an iterative process of tightening and relaxing constraints;

6. definition of a preference structure for the selection of an efficient solution out of a set of feasible, and non-dominated (Pareto optimal) alternatives; the proposed method is a reference point approach.

For all these steps it is essential to find a representation that is intuitively understandable by a non-technical audience. Largely graphical formats, and use of a consistent domain specific rather than technical language are important considerations.

13


Water Resources System Simulation The optimization is based on a web-based water resources management information and simulation system for river basin analysis and management, compliant with the EU Water Framework Directive 2000/60/EC (Fedra, 2004a,b; 2002; Fedra and Jamieson 1996; Jamison & Fedra 1996); . This simulation system is the basis for the optimization and decision support approach developed in OPTIMA. This DSS approach (Fedra, 2000) is implementing concepts of satisficing (Byron, 1998) with a modified reference point paradigm for multi-criteria, multi-objective problems (Wierzbicki, 1998, Simon, 1957, 1969) using an inverse problem solving approach in a simulation based Monte Carlo framework (Fedra, 1983). The basic approach of OPTIMA is one of participatory decision making (Fedra, 2004a) within this multi-criteria framework. The basic DSS approach is based on a satisficing paradigm, chosen and further developed specifically for the participatory approach (See also D03.2, Basic Optimization Model). The basic model paradigm is dynamic mass budget simulation as a straight forward implementation of the conservation laws of classical physics, even though the system representation for the optimization can be understood as a simplified Hamiltonian set of state space “fiber bundles”, and in it possible probabilistic extension as a Riemannian manifold representation of the state space. As part of the mass budget approach, the model calculates a cumulative mass budget error over all nodes and time steps – the mass budget error results from the simplified river channel routing, and the mixed use of difference and differential equation and their numerical integration, respectively. The river basin is represented by a topological network of nodes representing sources of water, storage elements, and demand/consumption, connected by reaches that convey the water between these elements. The model operates on a daily time step, its primary state variables are the content and flow of water between the different elements. Performance of the system is expressed in terms of criteria and indicators (any one of which can be used in the optimization) such as the

1. Overall water budget, balancing all inputs, losses, uses, outflows including export and inter-basin transfer, and change in storage including reservoirs and the groundwater system; additional information relates to the storage/extraction relationships and thus the sustainability of the overall system in a long-term perspective;

2. Technical efficiency of the system: this describes the ratio of useful demand satisfied to the losses through evaporation from reservoirs and seepage losses through the various conveyance systems;

3. Supply demand ratio, globally, for any and all individual demand nodes, or any functional/sectoral grouping; this also includes any environmental water demands, low flow constraints, wetland nourishing, etc.

4. Reliability of supply, measured at any or all demand nodes and control nodes that compare the water available with user defined needs/expectation as constraints;

5. Development potential, which relates the unallocated water summed over all demand points to the total input: this, in principle, defines the amount of water that is available for further exploitation;

14


6. Costs and benefits, derived from useful demand satisfied and the added value derived, versus the costs of shortfalls (again at any or all nodes), as well as the costs of supplying the water in terms of investment (annualized) and operating and maintenance costs of structures and institutions;

7. Groundwater sustainability, which describes the ratio of content to the net withdrawal (balance of recharge, summed over natural and artificial) and evaporative and deep percolation losses, measured in years of reserves at current exploitation levels.

Additional evaluation criteria and cost/benefit input is derived from the parallel water quality simulations. These describe primarily BOD and DO, but also any arbitrary conservative of first-order decaying pollutant that may limit the use of the water. The model system used is very flexible in terms of representing various issues perceived as important by the stakeholders: within the mass budget paradigm, any level of detail in representing individual components of the river basin can be modeled easily in a modular and hierarchical, nested manner. This can be achieved by defining networks with any level of resolution and number of nodes or elements, to make sure all components deemed important by the stakeholders are directly and individually represented; the figures below provides examples that demonstrate a range of complexity and resolutions to illustrate that point.

Basic economics To maintain consistency, and provide a complete set of criteria also at the level of the baseline scenario, we use a number of simple node specific economic variables that together define a basic cost benefit ratio for a scenario. The parameters include:

1. Cost of supply or treatment. All start nodes, storage nodes, recharge nodes, treatment nodes, and diversion nodes can have an associated cost factor, expressed as a linear function of low. This includes annualized investment costs for a simplified accounting. The node class specific cost functions can be adapted from a simple linear function of flow to any more complex construct required. Examples would be to make the cost of pumping groundwater also a function of groundwater level/head, or the cost of treatment a function of recipient water quality. In general the simple linear representation can be exchanged for a piecewise linear representation to approximate non-linear characteristics such as economies of scale.

2. Benefits: all demand nodes can have a monetary factor associated that estimates the benefits from demand met as a linear function of flow again. The same possible node type specific extension and piecewise linear approximation are possible.

3. Penalties: Any violation (demand not met at demand or control nodes, water quality standards not met at control nodes) can be associated with a linear or piecewise linear penalty function.

Together these simple economic accounting factors provide a basic benefit/cost ration:

BCR = Benefits / (Costs + Penalties)

It is important to not that this is not meant as a serious cost/benefit analysis: the role of this factor is restricted to the comparison of basins and scenarios rather than any detailed econometric analysis!

15


Hyperspace and Gestalt Consider the behavior of a river basin or water resources system as a trajectory of daily states in hyperspace, the dimensions being the topology of the system (its connected nodes and reaches) and the multiple parameters related to quantity and quality of water. From this trajectory of a state space fiber bundle, we can derive for any and all arbitrary intervals and groups of dimensions secondary metrics by arbitrary transfer functions, including differentiation and integration. We can now represent the systems behavior over some interval in time, say one year, as a point in this newly created hyperspace, where the choice of transfer functions and intervals lend themselves to the extraction of Gestalt. Gestalt theory has mainly been applied to psychology and cognition, music and the arts, but also has a considerable following in applied mathematics. Ackoff described the synergism of systems as: "the systems approach to problems, which focuses on systems taken as a whole, not on their parts separately. Such an approach is concerned with total-system performance even when a change in only one or a few of its parts is contemplated, because there are some properties of systems that can only be treated adequately from a holistic point of view." (Wertheimer, 1924, p7). We can, for ease of computation, represent any singular scenario as a point in hyperspace, described by a vector of "descriptors"; as an example, such as descriptor can be the amount of water available at a given point in time and space, or an aggregate property such as the total inflow to a reservoir, a derived property such as a sectoral supply/demand ratio, or any more complex construct such as the average reliability of domestic water supply during the summer months. (see, for example: http://iit.ches.ua.edu/systems/gestalt.html).

Using Gestalt for Optimization To illustrate the basic approach, we use a simple example but from the set of models used ion the WaterWare system, namely the rainfall-runoff model that is used to generate flow or runoff from ungaged upstream basins. The model provides a spatially (horizontally) lumped but vertically distributed, dynamic (daily) representation of the water budget of a catchment. In the example, the optimization is used for calibration, i.e., identifying a parameter (control) vector that maximizes an objective function expressed in terms of the model behavior. While classical calibration/optimization in this case would consist of minimizing the (sum of squared) deviation of observed and calculated flow values, we propose an alternative approach that fully recognizes the uncertainty of the observations. The calibration/optimization procedure involves the following steps, which are fully analogous to the more complex case of the water resources system optimization:

1. We defined a set of constraints (following the satisficing approach) on allowable system behavior; this is done by defining "an obstacle course in hyperspace" for the model output. As described in the introduction, this consists of a set of direct and derived observation that aim at capturing the gestalt of the system behavior rather then

16


individual observation, which we know to be associated with an error (both white noise and bias) of unknown magnitude.

The constraints are defined as inequalities over a specified period, from a singly day to the entire time horizon of the simulation period, one year. With the inequalities we can capture elements of the system behavior we consider more invariant than any individual observation. For example, we can state (for a specific river basin in any one of the case studies):

the average runoff during summer (May to September) is less than 0.2 * the average runoff during the winter period of November to February.

SIGMA(i from 151 to 273) Fi /(153) < 0.2 * SIGMA(i from 304 to 365+59) Fi /(120)

where F is the observed flow, and i represents Julian day.

This pattern is assumed to be more reliably observed irrespective of a given year being particularly wet or dry, and constitutes an element of Gestalt. We now compile a set of such statements.

2. We define a repertoire of allowable control parameters, expressed as ranges (or

distributions, PDFs) around a starting value for a set of parameters to calibrate.

3. We sample the parameter space defined by these ranges randomly

4. We run the model with the parameter (or control ort decision vector) and obtain a system response or behavior, which be compared against the set of constraints defined in (1). The result of this classification is either "SUCCESS" (all constraints met) or "FAILURE" (one or more constraints violated).

5. If a reasonably large number of sample result in no "SUCCESS", we relax the constraints

until a "SUCCESS" can be found; 6. If we get "SUCCESS" after even a small number of trials, we tighten the constraints or

increase the set, following the basic strategy of the satisficing methodology. The main advantage of the method is its conceptual simplicity, and parsimony of assumptions., which makes it easy to communicate and understand as it represents basic everyday heuristics. Beside, every element or dimension can be expressed in natural units and concepts the stakeholder can directly understand, in fact contribute directly as requirements of system performance, expressed as constraints. The very complex (and in a mathematical sense, badly structure, non-differentiable) system becomes very simple, expressed fully in the users language and concepts, and a simple to observe and thus directly empirically accessible model: it does, or does not, the "right thing" (meets all constraints), and it keeps trying (exploiting all possibilities) until it does.

17


Specifying the constraints To specify the constraints in a convenient format, a dedicated editor is used. Constraints are expressed as inequalities for a CONCEPT derived from a variable (usually, flow of water) over a period of time. The generic format of a constraint specification in terms of seven variables is thus:

From START time – to END time – for variable name - SCOPE : constrain CONCEPT- between LOWER BOUND – and UPPER BOUND

where SCOPE can be • general (all basin), • sectoral (agriculture, industry, tourism, domestic) , • node/location specific.

and CONCEPT can be one of

• MINimum, • MAXimum, • AVERAGE (arithmetic), • SUM, • VALUES (range).

In the definition of an optimization scenario, the set of constraints (as well as the set of measures or instruments and thus parameter changes) are implemented as sets of OBJECTS. The images show an example of the constraint editor implementation within the RRMCAL system for the Automatic calibration of the rainfall-runoff model that uses the same methodology and concepts as the basic optimization model. Every OBJECT is characterized by a set of META DATA the describe:

• Its name (a short descriptive name of the constraint) • A short, free text description that explains the

rational, e.g., the policy that is expected to affect the parameter used to represent the range of possible changes;

• The user who defined the constraint; • Creation and last modification date of the constraint.

The attributes of a constraint are defined above.

18


Heuristics and genetic programming The simple, blind Monte Carlo method is appealing through its conceptual simplicity. It is also attractive due to the possibility to directly use a complex, non-linear, distributed and dynamic system model, that is clearly non-differentiable and thus does not lend itself to classical optimization methods. The dimensionality and combinatorial extensions of the control space also precludes any method based on direct or implicit enumeration. So for higher dimensionality it becomes increasingly computationally inefficient. The inverse method (Fedra, 1983) has no guarantee of convergence or global optimality, so the only "reassurance" other than the satisfaction of all constraints is in large numbers. The efficiency of the search process can be increased considerably by a range of heuristic strategies and some elements of genetic or evolutionary programming.

Application to the Water Resources System Optimization The basic methodology described above for a simple case of about a dozen parameters and a trajectory in a model behavior space of less than 100 dimensions can be extended in a straight forward manner to the extremely high dimensionality of the distributed water resources model with several hundred control parameters, and several hundred output dimensions. Here, however, the need for increased efficiency through more efficient search strategies should be immediately obvious.

19


The search space The search space is defined by the individual values or value range of parameters that can be modified to represent possible interventions, actions, policies, strategies, or control measures, the instruments available to the decision making process to affect the behavior of the system. In the case of OPTIMA and water resources systems at the river basin scale, illustrative examples of such decision variables or possible measures are:

• building a new reservoir (the engineers preferred solution); • raising the price of water to affect demand (the economists preferred solution); • subsidizing public infrastructure development projects (the politicians preferred solution); • increasing the irrigated area (the farmers preferred solution).

Slightly more structured, the dimensions of the decision space can be organized and expressed in numerous different ways, representing different approach of individual or collective decision making. Control measure can be:

• global, i.e., one decision is supposed to affect all elements in the system equally; an example could be an educational program that is expected to reduce water demand across all sector by some percentage.

• sectoral: the same example would apply to a campaign directed to farmers specifically • individual/local: here the measure applies to a specific location, represented by on node

in the system such as a particular reservoir, irrigation district, industry, or city. Consequently, the control specifications that define the search space are hierarchically grouped, with any lower level inheriting the specification from the parent level, but optionally overloading them with individual specifications. The hierarchical control logic is structured as follows:

GLOBAL PARAMETERS: parameter MIN START MAX parameter ALT1 parameter ALT2 parameter ALT3

SECTORAL PARAMETERS:

sector parameter MIN START MAX sector parameter ALT1 sector parameter ALT2 sector parameter ALT3

LOCAL PARAMETERS:

node parameter MIN START MAX node parameter ALT1 node parameter ALT2 node parameter ALT3 node technology1 MIN MAX node technology2 MIN MAX

20


Water Technologies At the local level, we introduce an additional concept: water technology. To compile and describe water technologies, as web-based data base with interactive interface for import and editing has been develop. The data base currently holds more than 30 technology profiles for the optimization. Water Technologies are convenient containers (or object classes) to specify alternative parameters (technical in combination with economic ones) in a natural grouping: rather than specifying a decrease in the loss rate and an associated investment and operating cost, we specify an applicable technology that itself contains this information, together with a scaling factor that describes the degree, level, or extent of the technology. For example, for the technology "canal lining" which will reduce seepage losses for water conveyance to ademand node, we can specify a range (e.g., 0.1 - 0.5) that would indicate that we can consider applying this technology to any fraction of the canal length associated with the node from a minimum of 10 to a maximum of 50%.

Since the technology specification include a length dependent cost function, the scaling of the economic data is thus performed automatically. Water technologies can be generic, i.e., they apply to all OPTIMA case studies, or they can be made locations specific for an individual case study only.

21


Search strategies The search problem, formally, is simple: Given a search space S and its feasible subpart F ⊆ S, we want to find x ∈ F such that eval( x ) ≤ eval (y) for all y ∈ F. Search strategy depends on the size and structure of the search space. Depending on the dimensionality and combinatorial size of the search space, different search strategies have to be used, (Michalewicz and Fogel, 1998; Conley, 1980). Possible strategies, that in most cases can be combined, include

1. Exhaustive search. While this is guaranteed to lead to an optimal solution, problem dimensionality usually rules out the approach: at about 10 parameters per node and maybe 50 nodes, dimensionality is in the order of 1050 and thus beyond computability. The number of options (the size of the search space) if we only assume, on average, 10 discrete possible values for each parameter reaches 500! (which even as a number let alone complex model simulation runs is beyond any calculator .

2. Gradient search methods: such as hill climbing and stochastic hill climbing, simulated annealing, etc.

3. Heuristic methods. Due to the size and complexity of the water resources system optimization problem, we use a combination of methods around an central Monte Carlo based evolutionary approach, combined with stochastic hill climbing, a greedy algorithm, and taboo search strategies for the neighborhood of solutions. One of the major advantages of the methodology used is its conceptual simplicity, which lends itself to direct and easy computational implementation on distributed hardware (cluster of servers) which provides an excellent scalability for the problem. However, even under the best circumstances, we must expend many hours of CPU time for any one of the computational experiments. Since these still need careful tuning and an adaptive strategy, this leads to a partially supervised and thus overall time consuming process.

Constructing gradients One of the basic requirements of several of the search algorithms used are continuous gradients. While the overall system is non differentiable, we still can expect local gradients. Even in a constraint based system, we can construct gradients by:

1. counting the number of constraint violations of a given run 2. measuring the (cumulative) distance of the system response vector from any or all

constraints. Both methods will result in local gradients that can be exploited, for example, for local stochastic hill climbing in combination with the basic evolutionary and heuristic strategies.

22


Evolutionary Algorithms The basic, innovative if not revolutionary, idea behind evolutionary algorithms and genetic programming is to replace search strategies based on finding and improving a single solution with an approach based on sets of populations. Mimicking evolutionary processes of selection, the approach identifies populations of solutions that are “good enough” rather than one arguable best, and develop or breed them as a set. This concepts fits perfectly with the ideas of satisficing, multi-attribute theory, and a participatory approach where multiple and conflicting objectives, plural rationalities, hidden agenda and many more complication of the naïve rationalist view of optimization lurk (Fedra, 2000). Within OPTIMA, evolutionary algorithms are therefore a perfectly appropriate strategy for the first optimization phase preceding the discrete multi-criteria selection phase that aims at conflict resolution. A basic generic evolutionary algorithm resembles any other iterative search strategy, using the pseudo-code notation from Michalewicz and Fogel (1998):

procedure evolutionary algorithm begin

t 0 initialize P(t) evaluate P(t) while (not termination-condition) do begin

t t+1 select P(t) from P(t-1) alter P(t)

end end

The evolutionary algorithm maintain a population of individual P(t) = {x1,t ….. xn,t} for the iteration t. Each individual x represents a possible solution to the problem (expressed in our set of constraints) and corresponds to an input or control vector for the water resources model. The evaluation step measured “fitness” in terms of the objective functions for the individual case studies: these include some mixture of

• meet as many constraints as possible which is the primary objective in the satisficing approach; please note that all objectives of the form minimize … and maximize … can always be translated into constraints;

• minimize the shortfall at each demand/control node • minimize the penalties • minimize costs • maximize benefits

The alter strategy will modify any or all members of the population. Here elements of genetic programming can be used where new input and control vectors (individuals) are generated either by individual change of one or more properties (in the language of biology, spontaneous

23


mutations) or by combining components of the input vector from several (two or more) individuals into a new one (akin to sexual reproduction and recombination of genetic material). The main problem here is that other than in biological systems, the input vector (the genome) of the model lacks any epigenetic structure, so that the advantage of the recombination over the spontaneous mutation is not at all obvious. A more appropriate strategy is based on heuristics that can either be representing expert opinion (including the stakeholders suggestions on efficient strategies) or can be automatically be learned from the system simulations themselves (in fact, if we would use these heuristics together with principal component analysis, we could evolve and epigenetic structure for the model, which, however, would at the same time lose its mechanistic appeal and explanatory structure. An example of a heuristic strategy would be as follows:

1. For every demand or control node where we define a constraint in terms of a minimal supply/demand ratio or a maximum allowable number of shortfalls (days where the demand target is not met) we can define a set of upstream consumers.

2. For each upstream consumer we can determine its technical or water efficiency and rank all upstream (competing) consumers accordingly;

3. We can then distribute any reductions in water allocation (as but one of the numerous mechanisms or instruments) based on:

a. Any unallocated amounts at any one upstream demand point; b. The efficiency (measure by any one of the applicable criteria) of the demand

node. This will results in a simple strategy that is superior to any random search. There are obviously numerous combinations and variations of the approach. Its main advantage is that the search strategy can be expressed in simple non-technical terms so that stakeholders or their proxies can, in principle, see their suggestions directly reflected which makes the results more acceptable as they involve ownership and responsibility rather than external expert suggestions.

24


References and selected Bibliography Bell, D.E., Keeney, R.L., Raiffa, H. (1977) Conflicting Objectives in Decisions. 442 pp., John Wiley, Chichester. Byron, M. (1998) Satisficing and Optimality. Ethics 109, 67-93. Conley, W. (1980) Computer Optimization Techniques. 266 pp., Petrocelli, New York. Cougar, J.D. and Knapp, R. (1974) Systems Analysis Techniques . John Wiley & Sons. 1974. ISBN 0-471-17735-0 Fedra, K. (2004a) Coastal Zone Resource Management: Tools for a Participatory Planning and Decision Making Process. In: Green, D.R. et al. [eds.]: Delivering Sustainable Coasts: Connecting Science and Policy. Proceedings of Littoral 2004, September 2004, Aberdeen, Scotland, UK., Volume 1, 281-286 pp. Fedra, K. (2004b) Water Resources Management in the Coastal Zone: Issues of Sustainability. In: Harmancioglu, N.B., Fisitikoglu, O., Dlkilic, Y, and Gul, A. [eds.]: Water Resources Management: Risks and Challenges for the 21st Century. Proceedings of the EWRA Symposium, September 2-4, 2004, Izmir, Turkey, Volume I, 23-38 pp. Fedra, K. (2002) GIS and simulation models for Water resources Management: A case study of the Kelantan River, Malaysia. 39-43 pp., GIS Development, 6/8. Fedra, K. (2001) Environmental RTD projects and applications over the Internet: Case studies from Thailand, China, Malaysia and Japan. Paper presented at the Asia eco-Best Roundtable in Lisbon, March 2001. Fedra, K. (2000) Environmental Decision Support Systems: A conceptual framework and application examples. Thése prèsentèe á la Facultè des sciences, de l'Universitè de Genéve pour obtenir le grade de Docteur és sciences, mention interdisciplinaire. 368 pp., Imprimerie de l'Universitè de Genéve, 2000. Fedra, K. (1983) A Monte Carlo Approach to Estimation and Prediction. In: M.B. Beck and G. van Straten [eds.]: Uncertainty and Forecasting of Water Quality, Springer-Verlag, Berlin. pp.259-291. Fedra, K., and Jamieson, D.G. (1996) An object-oriented approach to model integration: a river basin information system example. In: Kovar, K. and Nachtnebel, H.P. [eds.]: IAHS Publ. no 235, pp. 669-676. Gillet, B.E. (1976) Inreoduction to Operations Research. A computer-oriented algorithmic apporach. 617 pp, MacGraw-Hill, New York. Harris, J.W. and Stocker, H. (1998) Handbook of Mathematical and Computational Science. 1028 pp., Spinger, Heidelberg. Jamieson, D.G. and Fedra, K. (1996) The WaterWare decision-support system for river basin planning: I. Conceptual Design. Journal of Hydrology, 177/3-4, pp. 163-175. Loucks, D.P., Stedinger, J.R. and Haith, D.A. (1981) Water resources Systems Planning and Analysis. 559 pp, Prentice Hall, Englewood Cliffs, NJ. Michalewicz, Z. and Fogel, D.B. (1998) How to Solve It: Modern Heuristics. 467 pp, Springer, Berlin. Moore, P., & Fitz, C. (1993). Gestalt Theory and Instructional Design. J. Technical Writing and Communication, 23 (2), 137-157, 1993. Scheepers, L.F. (2000) Optimising the holistic mass balance performance In: ORSSA, "Fostering the use of Operations Research in Development" The Operations Research Society of South Africa. Simon, H.A. (1969) The Science of the Artificial. MIT Press, Cambridge, MA. Simon, H.A. (1957) Models of Man. Social and Rational. Wiley, NY.

25


Wertheimer, W. (1924) Über Gestalttheorie [an address before the Kant Society, Berlin, '7th December, 1924], Erlangen, 1925. In the translation by Willis D. Ellis published in his "Source Book of Gestalt Psychology," New York: Harcourt, Brace and Co, 1938. Wierzbicki, A. (1998) Reference Point Methods in Vector Optimization and Decision Support. IR-98-017, 43 pp ; International Institute for Applied Systems Analysis, Laxenburg, Austria.

26

d03.2 implementation report: basic optimization model

Documents