rights / license: research collection in copyright - non ......giorgos, kostas, nikos, yannis for...

Research Collection

Doctoral Thesis

Constrained optimal control for complex systemsAnalysis and applications

Author(s): Margellos, Kostas

Publication Date: 2012

Permanent Link: https://doi.org/10.3929/ethz-a-007597774

Rights / License: In Copyright - Non-Commercial Use Permitted

This page was generated automatically upon download from the ETH Zurich Research Collection. For moreinformation please consult the Terms of use.

ETH Library

https://doi.org/10.3929/ethz-a-007597774

http://rightsstatements.org/page/InC-NC/1.0/

https://www.research-collection.ethz.ch

https://www.research-collection.ethz.ch/terms-of-use

Doctoral Thesis ETH Zürich No. 20758

Constrained Optimal Controlfor Complex SystemsAnalysis and Applications

A dissertation submitted to the

ETH Zürich

for the degree of

Doctor of Sciences

presented by

Kostas Margellos

Dipl. El.-Eng., University of Patras, Greece

born on 05.12.1984 in Athens

citizen of Greece

accepted on the recommendation of

Prof. Dr. John Lygeros, examiner

Prof. Dr. Peter Caines, co-examiner

Prof. Dr. Maria Prandini, co-examiner

2012

To my wonderful parents,Vasilis and Ioanna,

. . . and my beloved grandmother,Georgia

To my sweet Maria

Abstract

Many applications in control engineering, involve solving a constrained optimal controlproblem. Especially for the case of complex systems, the presence of constraints gives riseboth to theoretical and computational challenges. Developing a general theoretical frame-work is crucial to capture all features and interdependencies among the components ofthe underlying system, but numerical computations for such detailed models might not betractable. Therefore, a trade-off between the generality of the developed theory and thescale of applications that can be addressed numerically needs to be reached. This disser-tation deals with three of these problems, all of which resulting in a constrained optimalcontrol formulation.

The first problem concentrates on the development of an optimal control framework todeal with state constrained reachability problems for continuous and hybrid systems. Themain objective is to design suitable controllers to steer or keep the state of the system ina “safe” part of the state space. The synthesis of such safe controllers relies on the abilityto solve target problems for the case where state constraints are also present, giving rise toreach-avoid calculations. We first focus on continuous time systems and show how suchreach-avoid problems can be related to the viscosity solutions of certain optimal controlproblems. We then extend our framework to reachability/viability problems for hybrid sys-tems and provide a complete characterization based entirely on optimal control, logic the-orems and the definition of executions of hybrid automata. Hybrid automata with nonlin-ear continuous dynamics and bounded competing inputs can be captured by the proposedframework. The theoretical results are illustrated on a 4D trajectory management problemin air traffic control and various benchmark examples.

The second problem involves applying model predictive control for nonlinear, but feedbacklinearizable systems, with input constraints. The challenge is that after feedback lineariza-tion, even though the systems dynamics are rendered linear, the initial input constraintsare mapped to a set of state dependent, and in general nonlinear and possibly non con-

i

vex bounds. To circumvent this difficulty, an iterative scheme is proposed, where a set ofinitially arbitrarily chosen input constraints is continuously refined by solving a sequenceof quadratic or linear problems. Despite the fact that there are still no rigorous guaranteesregarding the convergence properties of the developed algorithm and the optimality of theresulting solution, we evaluate the effectiveness of the scheme by means of several single-input single-output and multi-input multi-output examples. Systems with continuous ordiscrete time, input affine dynamics, that can be at least partially feedback linearized, canbe solved using the proposed methodology.

The last problem investigated in this dissertation deals with chance constrained optimiza-tion. A new method which lies between robust (or worst-case) optimization and scenario-based methods is proposed. It does not require prior knowledge of the underlying proba-bility distribution as in standard robust optimization methods, nor is it based entirely onrandomization as in scenario approaches. It instead involves solving a robust optimizationproblem with bounded uncertainty, where the uncertainty bounds are randomized and arecomputed using the scenario approach. The proposed approach is applied to the prob-lem of reserve scheduling for power networks with renewable generation, whereas genericrobust optimization problems with discrete time, linear dynamics can be casted in the de-veloped framework.

ii

Zusammenfassung

Viele Anwendungen der Regelungstechnik umfassen die Lösung von beschränkten rege-lungstechnischen Problemen. Speziell im Fall von komplexen Systemen ist die Anwesen-heit von Nebenbedingungen mit theoretischen und rechnerischen Herausforderungen ver-knüpft. Die Entwicklung eines allgemeinen theoretischen Rahmens ist essentiell um al-le Eigenschaften und Abhängigkeiten des zu Grunde liegenden Systems zu erfassen, aberdie numerische Lösbarkeit von solchen Modellen ist nicht immer gewährleistet. Aus die-sem Grund muss eine Abwägung zwischen der allgemeinen Anwendbarkeit des theoreti-schen Hintergrundes und der numerischen Realisierbarkeit der modellierten Applikationgefunden werden. Diese Dissertation behandelt drei Problembereiche, wobei jeder in einbeschränktes Regleroptimierungsproblem darstellt.

Der erste Problembereich bezieht sich auf die Entwicklung eines optimalen regelungstech-nischen Rahmens um zustandsbeschränkte Erreichbarkeitsprobleme für kontinuierlicheund hybride Systeme zu behandeln. Die Hauptaufgabe hierbei ist die Entwicklung vonReglern welche die Systemzustände in einem “sicheren” Bereich im Zustandsraum hal-ten. Die Synthese von sicheren Reglern liegt in der Möglichkeit Zielprobleme zu lösen,wobei Zustandsbeschränkungen vorhanden sind. Dies gibt Grundlage für Erreichbarkeit-Vermeidungsprobleme. Der Fokus dieser Arbeit liegt zuerst bei zeitkontinuierlichen Proble-men und es wird gezeigt wie Erreichbarkeits-Vermeidungsprobleme zu Viskositätslösungenvon gewissen regelungstechnischen Problemstellungen in Relation gesetzt werden können.Anschliessend werden die Betrachtungen erweitert für hybride Systeme und eine komplet-te Charakterisierung basierend auf optimaler Regelung, Logik und der Definition von hy-briden Automaten wird gegeben. Hybride Automaten mit nichtlinearen Dynamiken undbeschränkten konkurrierenden Eingängen können mit diesem Ansatz beschrieben werden.Theoretische Ergebnisse werden anhand eines 4D Trajektorie Management Problems in derLuftfahrtkontrolle und anhand von verschiedenen Musterbeispielen gezeigt.

Der zweite Problembereich beinhaltet die Anwendung von prädiktiver Regelung für nicht-

iii

lineare Systeme mit beschränkten Inputgrössen, wobei die Rückkopplung linearisierbar ist.Die Schwierigkeit hierbei ist, dass nach der Linearisierung der Rückkopplung und obwohldie Systemdynamiken linearisiert wurden, die ursprünglichen Eingangsgrössenbeschrän-kungen abgebildet werden in eine Menge mit zustandsabhängigen, generell nichtlinearenund möglichweise nicht konvexen Grenzen. Um diese Schwierigkeit zu umgehen wird einiteratives Verfahren vorgeschlagen, wobei eine Menge von ursprünglich frei gewählten Ein-gangsgrössenbeschränkungen kontinuierlich verfeinert wird durch die Lösung einer Reihevon quadratischen und linearen Programmen. Trotz der Tatsache, dass es bezüglich desKonvergenzverhaltens des entwickelten Verfahrens und der Optimalität der Lösung nochkeinen grundlegenden Beweis gibt, wird die Effektivität des Verfahrens über mehrere ein-fache Eingangs-Ausgangs- und Mehrgrössenprobleme evaluiert. Systeme mit kontinuier-lichem oder diskretem Zeitverhalten und eingangsaffinen Dynamiken, welche zumindestteilweise linearisiert werden können, sind durch die vorgeschlagene Methodik lösbar.

Der letzte Problembereich dieser Dissertation umfasst die wahrscheinlichkeitsbeschränkteOptimierung. Eine neue Methodik, welche zwischen der robusten Optimierung und denszenario basierten Methoden liegt wird vorgeschlagen. Diese beansprucht weder Infor-mation über die zugrunde liegende Wahrscheinlichkeitsverteilung wie in anderen robus-ten Optimierungen, noch wird eine Lösung gänzlich über Randomisierung durch Szena-rios erreicht. Stattdessen ist eine robuste Optimierung mit begrenzter Unsicherheit invol-viert, wobei die Unsicherheitsgrenzen über Randomisierung und Szenariogenerierung er-mittelt werden. Die Methode wird angewendet auf das Problem der Reservedeterminierungin elektrischen Energiesystemen mit fluktuierender Einspeisung, wobei robuste Optimie-rungsprobleme mit zeitdiskreten, linearen Dynamiken in den entwickelten Rahmen umge-wandelt werden können.

iv

Acknowledgements

Many people have accompanied me in the unique experience that this Ph.D was for me.Without their support, help and encouragement the completion of this dissertation wouldnot have been possible.

My deepest gratitude goes to my advisor, Prof. John Lygeros, for his close supervision andhis endless support throughout my Ph.D years. I would like to thank him for giving methe opportunity to get exposed to many different aspects of control engineering and pro-viding me with a huge amount of research opportunities. I consider myself blessed havingworked with a professor who not only shaped my research interests, but also supported allmy steps providing me with a feeling of security. What I appreciate the most, and this prob-ably extends to other aspects of my life as well, is that from his enthusiasm and sharp wayof thinking I learned to love research and get motivated and respect any part of it, from theprocedural to the excited ones.

I am especially grateful to Prof. Peter Caines and Prof. Maria Prandini, not only for serv-ing in my Ph.D exam committee, but most importantly for their detailed comments andsuggestions regarding potential improvements and their very positive attitude against mywork. Special thanks go to Prof. Prandini for our recent collaboration and for giving afriendly flavour to my Ph.D exam.

I would like to thank Prof. Göran Andersson, with whom and his group I had the chanceto collaborate extensively, for providing a different perspective on how parts of this workcould be of interest to the power system community.

There have been many people that I have worked with and learned from them all theseyears. I am grateful to the three amazing post-docs (some of them are professors now), Dr.Debasish Chatterjee, Dr. Peter Hokayem and Dr. Federico Ramponi for generously provid-ing their advice every time I needed some help. Special thanks go to Peter for introducingme to the scenario approach and for assisting me on a preliminary formulation of the workreported in Chapter 7. My thanks go to Dr. Paul Goulart for the nice collaboration we had

v

during the last months of my studies on problems related to Chapter 6. I would also liketo thank Sean Summers, Peyman Mohajerin and Nikos Kariotoglou for discussions relatedto Chapter 2, Dr. Georgios Chaloulos, Dr. Ioannis Lymperopoulos, Alexandra Lotz as wellas all the members of the CATS project consortium for their input regarding the applica-tions of the methods of this thesis on air traffic control, and Andreas Milias for his adviceon various mathematical issues. Moreover, I would like to thank Tobias Haring for trans-lating the thesis abstract in German, and all the bachelor and master students that workedwith me on different projects. My sincere thanks go to all current and former members ofthe Automatic Control Laboratory, professors, post-docs, students and administrative staff,for creating a warm and productive research environment. The multi-disciplinary nature ofthe research carried out in this group together with the “academic” coffee breaks has beena valuable exposure to many interesting control problems.

At a personal level, I am thankful to the many friends that have made my life happy and theSwiss experience memorable. Starting from the Greeks at the lab, many thanks to Andreas,Giorgos, Kostas, Nikos, Yannis for their help and the great moments in Paris and elsewhere.Andreas has been a great officemate and influence, and the one to share my everyday wor-ries with. Vaggeli, Spyro, Sofia, Valia, Evdokia, Marianna (your presence in the lab madeour office alive), Niko (it was always refreshing when you came for lunch with us), Eralia,Yiourka and many others, it has been always great going out with you. Thanks also to Gi-annis, Andreas and Thany, my life long friends from Greece, whose friendship and supporthas been making my holiday breaks relaxing.

Almost at the end of this part, I want to express my gratitude and love to the better half ofme, Maria. She has been much more than a nice colleague and friend, encouraging all mydecisions and ensuring a stable and caring environment in my personal life.

Finally, I want to thank my parents Ioanna and Vasilis for standing by me, for their uncon-ditional love, and for supporting me by all means at every occasion. What I owe them cannot be expressed; this is just an opportunity to admit how important it is having them inmy life. Thanks go also to our sweet Ada and Angeliki, for making our life in Greece happier.My deepest love and gratitude goes to my grandmother Georgia. Her caring attitude andintense presence give rise to feelings that I can not put in words. I just feel blessed that Ihave someone like her in my life.

vi

Contents

Abstract i

Zusammenfassung iii

Acknowledgements v

Contents vii

1 Introduction 11.1 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Problems to be addressed and contributions . . . . . . . . . . . . . . . . . . . 4

1.2.1 Problem 1: Reachability with state constraints . . . . . . . . . . . . . 41.2.2 Problem 2: MPC for feedback linearizable systems with input con-

straints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2.3 Problem 3: Chance constrained optimization . . . . . . . . . . . . . . 5

1.3 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 An optimal control formulation for reach-avoid differential games 92.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2 Differential games and Reach-Avoid problems . . . . . . . . . . . . . . . . . . 11

2.2.1 Differential game problem formulation . . . . . . . . . . . . . . . . . . 112.2.2 Reach-Avoid at the terminal time . . . . . . . . . . . . . . . . . . . . . 122.2.3 Reach-Avoid at any time . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.3 Characterization of the value function . . . . . . . . . . . . . . . . . . . . . . . 142.3.1 Basic properties of V . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.3.2 Variational equation for V . . . . . . . . . . . . . . . . . . . . . . . . . . 152.3.3 Variational equation for V . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.4 Case study: Underwater vehicle motion in the presence of obstacles . . . . . 20

vii

2.4.1 Mathematical modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.4.2 Reach-Avoid formulation . . . . . . . . . . . . . . . . . . . . . . . . . . 212.4.3 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.5 Summary and open problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3 Viable set computation for hybrid systems 273.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2 Viability specifications of hybrid game automata . . . . . . . . . . . . . . . . . 28

3.2.1 Hybrid dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283.2.2 Gaming formulation and input strategies . . . . . . . . . . . . . . . . . 303.2.3 Problem statement and definition of operators . . . . . . . . . . . . . 31

3.3 Hybrid discriminating kernel characterization . . . . . . . . . . . . . . . . . . 333.3.1 Finite time continuous evolution - Finite number of discrete transitions 333.3.2 Finite time continuous evolution - Infinite number of discrete transi-

tions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373.3.3 Infinite time continuous evolution - Infinite number of discrete tran-

sitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413.4 Case studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.4.1 Numerical example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 433.4.2 Voltage stability of a single machine-load system . . . . . . . . . . . . 45

3.4.2.1 System description and mathematical modeling . . . . . . 453.4.2.2 Viability problem and simulation results . . . . . . . . . . . 47


4 Reachability based 4D trajectory management in air traffic control 494.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.2 Mathematical modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

4.2.1 Simulation environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 504.2.2 From way points to TWs . . . . . . . . . . . . . . . . . . . . . . . . . . . 524.2.3 TW tracking controller . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.3 Determining the limits of maneuverability using reachability . . . . . . . . . 534.3.1 Reach-Avoid problem characterization - Extension to time depen-

dent state constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534.3.2 Model abstraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 544.3.3 Reach-avoid tubes for TW tracking and conflict avoidance . . . . . . 58

4.4 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.4.1 Simulation set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604.4.2 Impact of the Time of Arrival controller on the TW size . . . . . . . . . 624.4.3 Reachability calculations . . . . . . . . . . . . . . . . . . . . . . . . . . 654.4.4 Computational issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70


5 MPC for feedback linearizable systems with input constraints 71

viii

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 715.2 Feedback linearization - MPC . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

5.2.1 SISO Feedback linearization . . . . . . . . . . . . . . . . . . . . . . . . 725.2.2 MIMO Feedback linearization . . . . . . . . . . . . . . . . . . . . . . . 735.2.3 Model Predictive Control . . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.3 Proposed iterative scheme . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755.3.1 Problem set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 755.3.2 Iterative algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

5.4 Case studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795.4.1 DC motor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795.4.2 Two-Area Power System . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

5.4.2.1 Physical description and mathematical modeling . . . . . . 815.4.2.2 Nonlinear observer . . . . . . . . . . . . . . . . . . . . . . . . 835.4.2.3 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . 845.4.2.4 Zero dynamics stability analysis . . . . . . . . . . . . . . . . 86

5.4.3 Flight control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 885.5 Summary and open problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

6 Scenario based chance constrained optimization 936.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 936.2 Problem description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 956.3 Method 1: Unstructured Constraints . . . . . . . . . . . . . . . . . . . . . . . . 96

6.3.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 976.3.2 Tractability of the proposed method . . . . . . . . . . . . . . . . . . . . 99

6.4 Method 2: Structured Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.4.1 Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1006.4.2 Tractability of the proposed method . . . . . . . . . . . . . . . . . . . . 103

6.5 Discussion and numerical results . . . . . . . . . . . . . . . . . . . . . . . . . . 1036.5.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1036.5.2 Numerical examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

6.5.2.1 Example 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1056.5.2.2 Example 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108


7 Reserve scheduling for power systems with wind power generation 1117.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1117.2 Problem set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

7.2.1 Definitions and preliminaries . . . . . . . . . . . . . . . . . . . . . . . . 1127.2.2 Reserve representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

7.3 Problem formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1147.4 Simulation study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

7.4.1 Wind power model and Simulation set-up . . . . . . . . . . . . . . . . 1177.4.2 Simulation results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

ix


8 Concluding remarks 1238.1 Reachability with state constraints . . . . . . . . . . . . . . . . . . . . . . . . . 1238.2 MPC for feedback linearizable systems with input constraints . . . . . . . . . 1248.3 Chance constrained optimization . . . . . . . . . . . . . . . . . . . . . . . . . 125

Appendix 127A.1 Additional proofs of Chapters 2 and 3 . . . . . . . . . . . . . . . . . . . . . . . 127

A.1.1 Proof of Proposition 2.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . 127A.1.2 Proof of Lemma 2.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129A.1.3 Proof of Lemma 2.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131A.1.4 Proof of Theorem 3.1. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

A.2 Reach-Avoid Algorithm of Chapter 4 for Conflict Resolution . . . . . . . . . . 135A.3 Additional proofs of Chapter 6 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

A.3.1 Proof of Proposition 6.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . 136

Bibliography 139

Curriculum Vitae 153

x

CHAPTER

1Introduction

1.1 Outline

A common practice in the control design process is to consider a mathematical descrip-tion/abstraction of a given system, which gives rise to a model that will be employed forour control purposes. To achieve the desired control performance it is crucial for the devel-oped model to capture the basic features of the underlying process, while satisfying certainphysical limitations. The latter is of major importance for the control synthesis task, andgives rise to constraints in a control problem formulation. Based on their nature, one candistinguish between state constraints, or in other words constraints that the trajectories ofthe system should not violate, and input constraints that capture the case of limited controlauthority or sensing. If uncertainty is also present we may have constraints in the probabil-ity with which a certain event (state or input constraints) is satisfied, giving rise to chanceconstrained problems. In the context of optimization, we may also want to separate con-straints according to their convexity properties.

In an optimal control framework, once a model is decided (linear vs. nonlinear, discretevs. continuous time , hybrid, etc.), and a list of control objectives is specified, one or moreof these constraints might be present and difficulties of different nature may arise. Espe-cially large scale systems that engineers are usually inclined to, like air traffic control, powersystems, biochemical networks, ground transportation systems, etc., exhibit a quite com-plex structure. To capture then accurately all physical interactions and interconnectionsamong their components, a general theoretical framework is required. As a step towardthis direction, we resort to models based on uncertain hybrid automata, subject to possiblynon-convex constraints. Research over the last decades has provided formal mathematicaltools for the analysis of such systems, enabling us either to verify certain specifications ofthe system under consideration, or to synthesize controllers that achieve the desired con-trol objectives. Despite its general nature, incorporating generic input and state constraints

1

Introduction

can be a challenging task, and requires additional attention. Moreover, even though a veryrich class of systems can be addressed in this theoretical framework, numerical computa-tions are still restricted to systems of lower dimension. Therefore, to ensure tractability forlarge scale applications, appropriate abstractions of the system dynamics need to be per-formed. Model abstraction can be then done either on a systematic way, or on an intuitivebasis. The former consists a separate research area, and is based on maintaining certainproperties of the initial system, whereas the latter requires physical knowledge regardingtime scale separation, coupling among the states, etc..

An alternative way to achieve a tractable problem, is to modify the model of the underlyingprocess, by imposing additional assumptions on the system dynamics. Consider for exam-ple the case of linearizing a system around an operating point, modeling only its steadystate behavior, or applying some convex relaxation. If the resulting problem can be castedinto a convex optimization framework, or it can be formulated in a model predictive controlcontext, then powerful tools are available to tackle problems of high dimension, sometimeseven with low computational cost. Nevertheless, the class of systems that can be addressedin this framework is much more restricted, since a specific structure is now required. More-over, treating uncertainty, especially in a stochastic set-up, requires resorting to robust orchance constrained optimization, giving rise to problems of different nature.

Based on the discussion so far, it should be apparent that there is a trade-off between thegenerality of the developed theory and the scale of applications (in terms of numerical com-putation) that can be addressed in the resulting framework. In any case different theoreticalrestrictions are imposed, and hence different issues may arise. This dissertation will dealwith three of these issues, all of which resulting in a constrained optimal control problem.Moreover, for every case different type of constraints are prominent, in the sense that addi-tional care is required. Each problem is formulated on an individual theoretical basis, andpermits us to address numerically applications of different scale. The main objective andthe basic features of each problem can be then summarized as follows.

Problem 1: An optimal control framework to incorporate state constraints in reachabilitybased verification and controller synthesis problems, is developed. Hybrid automata withnonlinear continuous dynamics and competing inputs (bounded control and disturbanceinputs are considered) can be captured by the proposed framework.

Problem 2: A novel iterative algorithm, which combines standard nonlinear control tech-niques based on feedback linearization, and model predictive control for systems with in-put constraints, is proposed. Systems with continuous or discrete time, input affine dy-namics, that can be at least partially feedback linearized, can be solved using the proposedscheme.

Problem 3: A new method for solving chance constrained optimization problems whichlies between robust (or worst-case) optimization and scenario-based methods is proposed.It does not require prior knowledge of the underlying probability distribution as in stan-

2

Outline

scale of application

ge

ne

ralit

yo

fth

eo

retica

lfr

am

ew

ork

Chapter 2

Problem 1

Chapter 3

Chapter 4

Chapter 5

Problem 2

Chapter 6

Problem 3

Chapter 7

Figure 1.1: Qualitative classification of the dissertation contents - Generality of theoreticalframework vs. Scale of applications in terms of numerical computation.

Dynamics Constraints UncertaintyContinuous time Discrete time Hybrid Input State Worst-case Stochastic

Linear Nonlinear Linear Nonlinear

Problem 1 X X X X X X X XProblem 2 X X X X XProblem 3 X X X X X

Table 1.1: Class of systems that can be casted in the framework of each problem.

dard robust optimization methods, nor is it based entirely on randomization as in the sce-nario approach. It instead involves solving a robust optimization problem with boundeduncertainty, where the uncertainty bounds are randomized and are computed using thescenario approach. The proposed approach is applied to the problem of reserve schedul-ing for power networks with renewable generation, whereas generic robust optimizationproblems with discrete time, linear dynamics can be casted in the developed framework.

Fig. 1.1 classifies qualitatively the contents of this dissertation, in terms of how general thetheoretical formulation of each problem is, and what is the scale of applications that canbe tackled numerically using the corresponding framework. The arrows denote the inter-dependencies between the chapters. Moreover, Table 1.1 highlights the class of systems, interms of dynamics, constraints and uncertainty structure, that can be casted in the frame-work of each problem. In the next section, more details regarding the key issues of eachproblem and the contributions of the dissertation are provided.

3

Introduction

1.2 Problems to be addressed and contributions

1.2.1 Problem 1: Reachability with state constraints

Chapters 2,3 and 4, concentrate on the development of an optimal control framework todeal with state constrained reachability problems for continuous and hybrid systems. Themain aim is to design suitable controllers to steer or keep the state of the system in a “safe”part of the state space. The synthesis of such safe controllers relies on the ability to solvetarget problems for the case where state constraints are also present, giving rise to reach-avoid calculations.

In this direction, Chapter 2, focuses on continuous time systems, and proposes a new op-timal control formulation, in a sense dual to capture basin type of problems in nonsmoothanalysis, that allows for state constraints, and hence enables one to compute reach-avoidsets for a general class of systems with nonlinear dynamics and competing inputs. We firstrestrict our attention to a specific reach-avoid scenario, where the objective of the controlinput is to lead the state trajectories toward the target at the end of the time horizon, with-out violating the state constraints, and for any adversarial action. We then generalize ourapproach to the case where the controller aims to steer the system toward the target notnecessarily at the terminal, but at some time within the specified time horizon. The contri-bution of this chapter is that it provides a proof that the corresponding reach-avoid sets aredetermined by the level sets of nonsmooth value functions, which in turn are the unique,continuous viscosity solutions to appropriate variational equations. Another advantage ofthe proposed approach is that it has very good properties in terms of its numerical solu-tion, since the value function and the Hamiltonian of the system are continuous, enablingthe use of existing tools for solving the problem numerically.

Chapter 3 extends the reachability problem to hybrid systems, and to the best of our knowl-edge provides the first complete characterization relying entirely on optimal control andthe definition of executions of hybrid automata. Reachability for hybrid systems involvesthe alternating application of one continuous and two discrete operators. The former isa reach-avoid computation, whose properties are studied in Chapter 2, whereas the latterrequire the inversion of the reset maps which encode the discrete dynamics of the system.Three different cases are considered, based on whether the horizon of the continuous cal-culation and the number of discrete transitions is finite or infinite. For the case of infinitediscrete transitions, we prove the convergence of the iterative algorithm, following a logicbased approach. This involves the application of a constructive version of Tarski’s fixedpoint theorem, to determine the maximal fixed point of a monotone operator on a com-plete lattice of closed sets.

Chapter 4 demonstrates the application of the theoretical results of Chapters 2 and 3 on a4D trajectory management problem in air traffic control. By appropriately abstracting theaircraft dynamics we show how timing requirements, inherent in 4D trajectory manage-ment, can be formulated in the reachability framework. We also demonstrate how reach-

4

Problems to be addressed and contributions

ability tools can be used to perform conflict resolution while respecting the 4D trajectoryconstraints whenever possible.

1.2.2 Problem 2: MPC for feedback linearizable systems with inputconstraints

This problem is addressed in Chapter 5, which proposes a novel methodology for applyingModel Predictive Control (MPC) for nonlinear, but feedback linearizable systems, with in-put constraints. Due to its ability of handling constraints, MPC provides an efficient wayof solving optimal control problems online. Most real systems though, are governed bynonlinear differential equations, and usually linear constraints. The latter emanates eitherdirectly from the physical laws that govern the behavior of the underlying process, or due toan approximation procedure introduced to ensure tractability of the resulting control de-sign. Applying MPC in this case would in general lead to a difficult optimization problem,with no guarantees that the global optimum will be found. In such systems, and if the non-linear dynamics exhibit a specific structure (input affine systems), feedback linearizationtechniques could be used. That way, by an appropriate nonlinear mapping and feedback,the initial system can be transformed to a linear one, and then MPC could be employedfor the new system. The challenge in this case is that after feedback linearization, the ini-tial input constraints are mapped to a set of state dependent, and in general nonlinear andpossibly non convex bounds.

To circumvent this difficulty, an iterative scheme is proposed, where a set of initially arbi-trarily chosen input constraints is continuously refined by solving a sequence of quadraticor linear problems. If convergence is achieved, the first part of the resulting input sequenceis applied to the system, the horizon is rolled, and the procedure is repeated. The advan-tage of this approach is that non convex problems are replaced by a sequence of quadraticprograms that is easy to solve, thus offering a promising alternative to more direct schemes.The drawback is that it is not clear that the global optimum will be reached. Furthermore,there is no guarantee regarding the number of iterations needed per time-step until con-vergence is achieved. We evaluate the effectiveness of the proposed algorithm, and studyits properties through various examples for Single-Input Single-Output (SISO) and Multi-Input Multi-Output (MIMO) systems.

1.2.3 Problem 3: Chance constrained optimization

Chapter 6 investigates the use of a constraint-sampling technique, the so called scenarioapproach, for chance constrained optimization problems. Under a convexity assumption(with respect to the decision variables), the scenario approach replaces the chance con-straint with a finite number of hard constraints, while offering certain probabilistic guar-antees for the feasibility of the resulting randomized solution. We propose a new method

5

Introduction

for solving chance constrained optimization problems which lies between robust (or worst-case) optimization and the scenario approach. Our method does not require prior knowl-edge of the underlying probability distribution as in standard robust optimization methods,nor is it based entirely on randomization as in the scenario approach. It instead involvessolving a robust optimization problem with bounded uncertainty, where the uncertaintybounds are randomized and are computed using the scenario approach.

We propose two alternatives for dealing with chance constrained optimization problems.The first one is general and applies to optimization problems with very few underlying as-sumptions on the objective functions or constraints. The second applies to problems whoseconstraint functions are structured such that they can be decomposed into a product oftwo functions; one that depends exclusively on the uncertainty, and one that depends ex-clusively on the decision variables. We show that for a suitable choice of the number ofscenarios, both our approach and the scenario approach lead to equivalent probabilisticguarantees. The number of scenarios that need to be generated in our case, however, doesnot depend on the number of decision variables as in the standard scenario approach, buton the dimension of the uncertainty vector or the number of constraints depending onwhich of our two methods is used. The proposed solution methods are compared with thescenario approach by means of numerical examples.

In Chapter 7 we apply our approach to the problem of reserve scheduling for power net-works with renewable generation. It is shown how this problem can be formulated as achance constrained optimization program, thus alleviating the conservatism or intractabil-ity of a robust solution and the sensitivity of a deterministic variant of the problem. Toidentify the minimum cost reserves needed to satisfy a given demand level while avoidingundesirable load shedding and wind generation spillage with a certain probability, we rep-resent the reserves as a linear function of the difference between the current wind powerand its forecasted value. To evaluate the robustness of our approach we compare it with abenchmark reserve scheduling method by means of Monte Carlo simulations.

1.3 Publications

The work presented in this dissertation relies mainly on the following publications.Chapter 2 is based on

1. “Hamilton-Jacobi Formulation for Reach-Avoid Differential Games”, K.Margellos andJ.Lygeros, IEEE Transactions on Automatic Control, vol. 56, no. 8, pp. 1849-1861,2011. ([115])

2. “A Viability Approach for the Stabilization of an Underactuated Underwater Vehicle inthe Presence of Current Disturbances”, D.Panagou, K.Margellos, S.Summers, J.Lygerosand K.Kyriakopoulos, Proceedings of IEEE Control and Decision Conference, Shang-hai, China, pp. 3045-3050, 2009. ([142])

6

Publications

Chapter 3 is based on

1. “Revisiting the Viability Algorithm for Hybrid Systems Using Optimal Control”,K.Margellos and J.Lygeros, Proceedings of IFAC Conference on Analysis and Designof Hybrid Systems, 2012. ([117])

2. “Viable Set Computation for Hybrid Systems”, K.Margellos and J.Lygeros, submitted toNonlinear Analysis: Hybrid Systems, 2012. ([116])


1. “Toward 4D Trajectory Management in Air Traffic Control: A Study Based on MonteCarlo and Reachability Analysis”, K.Margellos and J.Lygeros, to appear in IEEE Trans-actions on Control Systems Technology, 2012. ([118])

2. “Air Traffic Management with Target Windows: An approach using Reachability”,K.Margellos and J.Lygeros, Proceedings of IEEE Control and Decision Conference,Shanghai, China, pp. 3045-3050, 2009. ([112])

3. “Hamilton-Jacobi Formulation for Reach-Avoid Problems with an Application to AirTraffic Management”, K.Margellos and J.Lygeros, Proceedings of American ControlConference, Baltimore, MD, USA, pp. 3045-3050, 2010. ([114])


1. “A Simulation Based MPC Technique for Feedback Linearizable Systems with InputConstraints”, K.Margellos and J.Lygeros, Proceedings of IEEE Control and DecisionConference, Atlanta, Georgia, USA, pp. 7539-7544, 2010. ([113])


1. “On the road between robust optimization and the scenario approach for chance con-strained optimization problems”, K.Margellos, P.Goulart and J.Lygeros, submitted toIEEE Transactions on Automatic Control, 2012. ([110])


1. “A Probabilistic Framework for Reserve Scheduling and N-1 Security Assessment of Sys-tems with High Wind Power Penetration”, M.Vrakopoulou, K.Margellos, J.Lygeros andG.Andersson, submitted to IEEE Transactions on Power Systems, 2012. ([173])

2. “A Robust Reserve Scheduling Technique for Power Systems with High Wind Penetra-tion”, K.Margellos, T.Haring, P.Hokayem, M.Schubiger, J.Lygeros and G.Andersson,Proceedings of Probabilistic Methods Applied to Power Systems Conference, Istan-bul, Turkey, 2012. ([111])

7

Introduction

3. “Probabilistic Guarantees for the N-1 Security of Systems with Wind Power Generation”,M.Vrakopoulou, K.Margellos, J.Lygeros and G.Andersson, Proceedings of Probabilis-tic Methods Applied to Power Systems Conference, Istanbul, Turkey, 2012. ([175])

4. “A Probabilistic Framework for Security Constrained Reserve Scheduling of Networkswith Wind Power Generation”, M.Vrakopoulou, K.Margellos, J.Lygeros and G.An-dersson, Proceedings of IEEE International Conference & Exhibition (ENERGYCON),Florence, Italy, pp. 452-457, 2012. ([174])

5. “Stochastic unit commitment and reserve scheduling: A tractable formulation withprobabilistic certificates”, K.Margellos, V.Rostampour, M.Vrakopoulou, M.Prandini,G.Andersson and J.Lygeros, submitted to European Control Conference (ECC), Zürich,Switzerland, 2012. ([119])

Other publications

1. “Cyber Attack in a Two-Area Power System: Impact Identification using Reachability”,P.Mohajerin, M.Vrakopoulou, K.Margellos, J.Lygeros and G.Andersson, Proceedingsof American Control Conference, Baltimore, MD, USA, pp. 962-967, 2010. ([127])

2. “A robust policy for Automatic Generation Control cyber attack in two area power net-work”, P.Mohajerin, M.Vrakopoulou, K.Margellos, J.Lygeros and G.Andersson, Pro-ceedings of IEEE Control and Decision Conference, Atlanta, Georgia, USA, pp. 5973-5978, 2010. ([126])

3. “Nonlinear control of wind turbines: An approach based on switched linear systemsand feedback linearization”, R.Burkart, K.Margellos and J.Lygeros, Proceedings of IEEEControl and Decision Conference, Orlando, Florida, USA, pp. 5485-5490, 2011. ([40])

4. “Optimal Wind Turbine Placement via Randomized Optimization Techniques”,J.Tzanos, K.Margellos and J.Lygeros, Proceedings of Power Systems Computation Con-ference (PSCC), Stockholm, Sweden, pp. 1-8, 2011. ([166])

8

CHAPTER

2An optimal control formulation forreach-avoid differential games

2.1 Introduction

Reachability for continuous and hybrid systems has been an important topic of researchin the dynamics and control literature. Numerous problems regarding safety of air trafficmanagement systems [164], [165], [112], flight control [163], [101], [19], [83] ground trans-portation systems [94], [99], etc. have been formulated in the framework of reachabilitytheory. In most of these applications the main aim was to design suitable controllers tosteer or keep the state of the system in a “safe” part of the state space. The synthesis of suchsafe controllers for hybrid systems relies on the ability to solve target problems for the casewhere state constraints are also present.

One direct way to tackle such capture basin type of problems is by means of viability the-ory and nonsmooth analysis [14], [51], [53]. Following the same approach, the authors of[13], [70] formulated viability, invariance and pursuit-evasion gaming problems for hybridsystems and used nonsmooth analysis tools to characterize their solutions. Computationaltools to support this approach have been developed by [52].

An alternative, indirect way of characterizing such problems is through the level sets of thevalue function of an appropriate optimal control problem. By using dynamic program-ming, reachability/invariant/viability problems without state constraints can be related tothe solution of a first order partial differential equation in the standard Hamilton-Jacobiform [101], [164]. Numerical algorithms based on level set methods have been developedby [138], [152], have been coded in efficient computational tools by [122], [124] and can bedirectly applied to reachability computations.

In the case where state constraints are also present, this target hitting problem is the solu-tion to a reach-avoid problem in the sense of [101]. Specifically, the authors of [164], [162]developed a reach-avoid computation, whose value function was characterized as a solu-tion to a pair of coupled partial differential equations. In [124], [125], [135] the authors

9

An optimal control formulation for reach-avoid differential games

proposed another characterization, which involved only one Hamilton-Jacobi type partialdifferential equation together with an inequality constraint. These methods however, arehampered both from a theoretical and a numerical point of view by the fact that are limitedto differentiable value functions with continuous Hamiltonians [162]. Therefore, [98], [123],considered the notion of viscosity solutions, but were restricted to the characterization ofthe “reach” part of the operator.

In [88], [86] a scheme based on ellipsoidal techniques to compute reachable sets for controlsystems with linear dynamics and constraints on the state was proposed. In [87], this ap-proach was extended to a list of target problems with state constraints. The calculation of asolution to the equations proposed in [88]-[87] is in general not easy apart from the case oflinear systems, where duality techniques of convex analysis can be used.

In this chapter we propose a new framework of characterizing reach-avoid sets of nonlin-ear control systems as the solution to an optimal control problem [115]. We consider thecase where we have competing inputs and hence adopt the gaming formulation proposedin [66]. Since the work described here was first published related work has been reportedin [33], [34]. We start with a reach-avoid problem where the objective of the control in-put is to lead the state trajectories of the system towards the target at the end of the timehorizon, without violating the state constraints and for any adversarial decision. We thenextend our approach to the case where the controller aims to steer the system towards thetarget not necessarily at the terminal, but at some time within the specified time horizon.In both cases, we provide a clear characterization of two nonlinear reach-avoid problems,and a proof that the corresponding reach-avoid sets are determined by the level sets of non-smooth value functions similar to [87], which in turn are the unique continuous viscositysolutions to variational equations of a form similar to [67], [16]. In addition to theoreticalsupport for the use of computational tools, the numerical advantage of our characterizationis that the properties of the value function and the Hamiltonian (both of them are contin-uous) enable the use of existing tools based on Level Set Methods [125], or other tools forsolving variational equations [67], to compute the solution of the problem numerically. Ourformulation also provides theoretically solid foundations the hybrid systems algorithm of[164], [162], as an alternative to the viability-based algorithm of [70] (see Chapter 3).

To illustrate the theoretical results of this chapter, we consider the motion of an underac-tuated underwater vehicle in the presence of a disturbance current, whose mathematicalmodeling was studied in detail in [142], [141]. Our objective in this case is to determine theset of initial states from which, for any disturbance, the underwater vehicle can hit a targetset, while avoiding fixed obstacles.

The remainder of the chapter is organized as follows. In Section 2.2 we formulate the tworeach-avoid problems for continuous systems with competing inputs and state constraints.Section 2.3 provides the characterization of the value functions of these problems as theviscosity solution to two variational equations. In Section 2.4 we present an application ofthis approach to the navigation of an underactuated underwater vehicle in the presence ofobstacles. Finally, in Section 2.5 we provide a summary and directions for future work.

10

Differential games and Reach-Avoid problems

2.2 Differential games and Reach-Avoid problems

2.2.1 Differential game problem formulation

Consider the continuous time control system x = f (x,u,d), and an arbitrary time horizonT > 0, with x ∈ Rn , u ∈ U ⊆ Rm , d ∈ D ⊆ Rp , and f (·, ·, ·) : Rn ×U ×D → Rn . Let U[t ,t ′],D[t ,t ′] denote the set of Lebesgue measurable functions from the interval [t , t ′] to U, andD respectively. Consider also two bounded, Lipschitz continuous functions l (·) : Rn → R,h(·) : Rn →R to be used to encode the “reach” and “avoid” set respectively.

Assumption 2.1. The sets U ⊆ Rm and D ⊆ Rp are compact. The functions f (x,u,d), l (x)and h(x) are bounded, Lipschitz continuous in x, and continuous in u and v. Moreover, forall x ∈ Rn ,

⋃u∈U f (x,u,d) and

⋃d∈D f (x,u,d) are convex and compact for all d ∈ D and all

u ∈U , respectively.

Under Assumption 2.1 the system admits a unique absolutely continuous solution x(·) :[t ,T ] → Rn for all t ∈ [0,T ], u(·) ∈ U[t ,T ], d(·) ∈ D[t ,T ] and for each initial state x(t ) = x. Forτ ∈ [t ,T ] this solution will be denoted by

φ(τ, t , x,u(·),d(·)) = x(τ). (2.1)

The last part of Assumption 2.1 is only used in the proofs of Proposition 2.1 and 2.2 to guar-antee the existence of an optimal control input. Following Corollary 1.4 (Chapter IV, pp.368) of [15] and using the notion of relaxed controls, it can be shown that the imposed as-sumption is sufficient to guarantee existence of the optimal control input. For a gamingset-up this is also shown in [53]. In case of a single input though this assumption maybe removed, and compactness of the input set suffices to show that the reach-avoid setsof Propositions 2.1 and 2.2 are closed. Let C f > 0 be a bound such that for all x, x ∈ Rn ,u(·) ∈U[t ,T ] and d(·) ∈D[t ,T ], and for all u ∈U and d ∈ D ,

| f (x,u,d)|6C f and | f (x,u,d)− f (x,u,d)|6C f |x − x|.

Let also Cl > 0 and Ch > 0 be such that

|l (x)|6Cl and |l (x)− l (x)|6Cl |x − x|,|h(x)|6Ch and |h(x)−h(x)|6Ch |x − x|.

In a game setting it is essential to define the information patterns that the two players use.Following [167], [66] we restrict the first player to play non-anticipative strategies. A non-anticipative strategy is a function α : D[0,T ] → U[0,T ] such that for all s ∈ [t ,T ] and for alld , d ∈ D, if d(τ) = d(τ) for almost every τ ∈ [t , s], then α[d ](τ) = α[d ](τ) for almost everyτ ∈ [t , s]. We then use A[t ,T ] to denote the class of non-anticipative strategies.

Consider the sets R, A related to the level sets of l (·) : Rn →R and h(·) : Rn →R respectively.For technical purposes assume that R is closed whereas A is open. We show below that

11


this choice ensures that the corresponding reach-avoid set is closed, a crucial fact for thearguments of Chapter 3. Then R and A could be characterized as

R = {x ∈Rn | l (x)6 0},

A = {x ∈Rn | h(x) > 0}.

For the rest of the chapter we will assume that R and A are disjoint sets; this correspondsalso to the only practically relevant cases.

2.2.2 Reach-Avoid at the terminal time

Let R ⊆ Rn represent a set that we would like to reach while avoiding a set A ⊆ Rn . Onewould like to characterize the set R A(t ,R, A) of states x(t ) at some time t ∈ [0,T ] for whichthere exists a choice for the control inputs such that for any disturbance the system trajec-tories reach the set R exactly at the terminal time T without passing through the set A overthe time horizon [t ,T ]. To answer this question one needs to determine whether there ex-ists a choice of α ∈ A[t ,T ] such that for all d(·) ∈ D[t ,T ], the trajectory x(·) satisfies x(T ) ∈ Rand x(τ) ∈ Ac for all τ ∈ [t ,T ]. The set of initial conditions that have this property is then

R A(t ,R, A) = {x ∈Rn | ∃α(·) ∈A[t ,T ], ∀d(·) ∈D[t ,T ], (φ(T , t , x,α(·),d(·)) ∈ R)

∧ (∀τ ∈ [t ,T ],φ(τ, t , x,α(·),d(·)) ∉ A)}. (2.2)

Now introduce the value function V : Rn × [0,T ] →R

V (x, t ) = infα(·)∈A[t ,T ]

supd(·)∈D[t ,T ]

max{l (φ(T , t , x,α(·),d(·))), maxτ∈[t ,T ]

h(φ(τ, t , x,α(·),d(·)))}. (2.3)

V can be thought of as the value function of a differential game, where u is trying to mini-mize, whereas d is trying to maximize the maximum between the value attained by l at theend T of the time horizon and the maximum value attained by h along the state trajectoryover the horizon [t ,T ]. Based on [98], [66] and [67], we will show that the value functiondefined by (2.3) is the unique viscosity solution of the following variational equation.

max{h(x)−V (x, t ),∂V

∂t(x, t )+ sup

d∈Dinfu∈U

∂V

∂x(x, t ) f (x,u,d)} = 0, (2.4)

with terminal condition V (x,T ) = max{l (x),h(x)}. It is then easy to link the set R A(t ,R, A)of (2.2) to the level set of the value function V (x, t ) defined in (2.3). In fact R A(t ,R, A) isrelated to the zero sub-level set of V (x, t ), but for simplicity we will refer to it as level setthroughout the chapter.

Proposition 2.1. R A(t ,R, A) = {x ∈Rn | V (x, t )6 0}.

12

Differential games and Reach-Avoid problems

Proof. We first show that R A(t ,R, A) ⊆ {x ∈ Rn | V (x, t ) 6 0} holds. Consider a point x ∈R A(t ,R, A), and for the sake of contradiction assume that V (x, t ) > 0. The latter implies that∃ε> 0 such that infα(·)∈A[t ,T ] supd(·)∈D[t ,T ]

max{l (φ(T , t , x,α(·),d(·))),maxτ∈[t ,T ] h(φ(τ, t , x,α(·),d(·)))} > 2ε > 0. Equivalently, there exists ε > 0, such that for all α(·) ∈ A[t ,T ], there existsd(·) ∈ D[t ,T ], so that max{l (φ(T , t , x,α(·),d(·))),maxτ∈[t ,T ] h(φ(τ, t , x,α(·),d(·)))} > ε > 0. Thelast statement is equivalent to there exists ε > 0, such that for all α(·) ∈ A[t ,T ], there ex-ists d(·) ∈ D[t ,T ], such that l (φ(T , t , x,α(·),d(·))) > ε > 0 or there exists τ ∈ [t ,T ] such thath(φ(τ, t , x,α(·),d(·))) > ε > 0. Or in other words, there exists ε > 0, such that for all α(·) ∈A[t ,T ], there exists d(·) ∈D[t ,T ], so thatφ(T , t , x,α(·),d(·)) ∉ R or there exists τ ∈ [t ,T ]φ(τ, t , x,α(·),d(·)) ∈ A. The last statement is equivalent to x ∉ R A(t ,R, A), which is a contradiction.

We now show that {x ∈ Rn | V (x, t ) 6 0} ⊆ R A(t ,R, A). Consider (x, t ) such that V (x, t ) 6 0,and for the sake of contradiction assume that x ∉ R A(t ,R, A). This implies that for all α(·) ∈A[t ,T ], there exists d(·) ∈ D[t ,T ], such that φ(T , t , x,α(·),d(·)) ∉ R or there exists τ∗ ∈ [t ,T ]such thatφ(τ∗, t , x,α(·),d(·)) ∈ A. Then, for allα(·) ∈A[t ,T ], there exists d(·) ∈D[t ,T ] and δ> 0such that l (φ(T , t , x,α(·),d(·))) > δ> 0, or there exists τ∗ ∈ [t ,T ] such that h(φ(τ∗, t , x,α(·),d(·))) > δ> 0. But under the convexity part of Assumption 2.1, V (x, t )6 0 implies that thereexists a strategy α(·) ∈A[t ,T ] such that supd(·)∈D[t ,T ]

max{l (φ(T , t , x,α(·),d(·))),maxτ∈[t ,T ] h(φ(τ, t , x,α(·),d(·)))} 6 0. Hence, there exists a strategy α(·) ∈ A[t ,T ] such that for all d(·) ∈D[t ,T ], l (φ(T , t , x,α(·),d(·))) 6 0 and also for all τ ∈ [t ,T ], h(φ(τ, t , x,α(·),d(·))) 6 0. The lastargument implies that l (φ(T , x, t ,α(·),d(·)))6 0, and for all τ ∈ [t ,T ], and so also for τ= τ∗,h(φ(τ∗, x, t ,α(·),d(·)))6 0, and establishes a contradiction.

2.2.3 Reach-Avoid at any time

Consider now the related problem of characterizing the set R A(t ,R, A) of states x(t ) at sometime t ∈ [0,T ] from which trajectories can start, and for any disturbance input reach the setR not necessarily at the terminal, but at some time within the time horizon [t ,T ], and with-out passing through the set A prior to hitting R. In other words, we would like to determinethe set

R A(t ,R, A) = {x ∈Rn | ∃α(·) ∈A[t ,T ], ∀d(·) ∈D[t ,T ],∃τ1 ∈ [t ,T ], (φ(τ1, t , x,α(·),d(·)) ∈ R)

∧ (∀τ2 ∈ [t ,τ1], φ(τ2, t , x,α(·),d(·)) ∉ A)}. (2.5)

Based on [123], define the augmented input as u = (u, u) ∈ U× [0,1] and consider the dy-namics

f (x, u,d) = u f (x,u,d). (2.6)

In Assumption 2.1, f (x,u,d) is assumed to be continuous in u and d , and Lipschitz con-tinuous in x. Hence, since u is the augmented input, u is not binary but takes values in[0,1], and f (x, u,d) is affine in u, if f (x,u,d) satisfies Assumption 1 so will f (x, u,d). Letφ(τ, x, t , u(·),d(·)) denote the solution of the augmented system, and define U , U and A

13


similarly to the previous case. Following [123] for every u ∈ U[t ,T ] the pseudo-time variableσ : [t ,T ] → [t ,T ] is given by

σ(τ) = t +∫ τ

tu(s)d s. (2.7)

Consider σ∗, as it was defined in [123], such that σ(σ∗(τ)) = τ. In Lemma 6 of [123], σ∗ wasproven to be the limit of a convergent sequence of functions, its existence was verified, andit was shown that

φ(σ(τ), x, t ,u(σ∗(·)),d(σ∗(·))) = φ(τ, x, t , u(·),d(·)), (2.8)

for any τ ∈ [t ,T ]. Based on the analysis of [123], equation (2.8) implies that the trajectory φof the augmented system visits only the subset of the states visited by the trajectoryφ of theoriginal system in the time interval [t ,σ(τ)].

Define now the value function

V (x, t ) = infα(·)∈A[t ,T ]

supd(·)∈D[t ,T ]

max{l (φ(T , t , x, α[d ](·),d(·))), maxτ∈[t ,T ]

h(φ(τ, t , x, α[d ](·),d(·)))}.

One can then show that V is related to the set R A.

Proposition 2.2. For τ ∈ [0,T ], R A(τ,R, A) = {x ∈Rn | V (x,τ)6 0}.

The proof of this proposition is given in Appendix A.1.

2.3 Characterization of the value function

2.3.1 Basic properties of V

We first establish the consequences of the principle of optimality for V .

Lemma 2.1. For all (x, t ) ∈Rn × [0,T ] and all k ∈ [0,T − t ]:

V (x, t ) = infα(·)∈A[t ,t+k]

supd(·)∈D[t ,t+k]

max{

maxτ∈[t ,t+k]

h(φ(τ, t ,x,α(·),d(·))),

V (φ(t +k, t , x,α(·),d(·)), t +k)}. (2.9)

Moreover, for all (x, t ) ∈Rn × [0,T ], V (x, t )> h(x) .

The proof for the second part is straightforward and follows from the definition of V . Theproof for the first part is given in Appendix A.1.

Next, we show that V is a bounded, Lipschitz continuous function.

Lemma 2.2. There exists a constant C > 0 such that for all (x, t ), (x, t ) ∈Rn × [0,T ]:

|V (x, t )|6C and |V (x, t )−V (x, t )|6C (|x − x|+ |t − t |).

The proof of this Lemma is given in Appendix A.1.

14

Characterization of the value function

2.3.2 Variational equation for V

We now introduce the Hamiltonian H : Rn ×Rn →R, defined by

H(p, x) = supd∈D

infu∈U

pT f (x,u,d).

Lemma 2.3. There exists a constant C > 0 such that for all p, q ∈Rn , and all x, y ∈Rn :

|H(p, x)−H(q , x)| <C |p −q | and

|H(p, x)−H(p, y)| <C |p||x − y |.

The proof of this fact is straightforward (see [98], or [97] Lemma 2). We are now in a positionto state and prove the following theorem, which is the main result of this section.

Theorem 2.1. The function V is the unique viscosity solution over (x, t ) ∈ Rn × [0,T ] ofthe variational equation

max{h(x)−V (x, t ),∂V

∂t(x, t )+ sup

d∈Dinfu∈U

∂V

∂x(x, t ) f (x,u,d)} = 0,

with terminal condition V (x,T ) = max{l (x),h(x)}.

Proof. Uniqueness follows from Lemma 2.3 and Proposition 1 of [67]. Note also that V (x,T ) =max{l (x),h(x)} by definition of the value function. Therefore it suffices to show that

1. For all (x0, t0) ∈ Rn × (0,T ) and for all smooth W : Rn × (0,T ) → R, if V −W attains alocal maximum at (x0, t0), then

max{h(x0)−V (x0, t0),∂W

∂t(x0, t0)+ sup

d∈Dinfu∈U

∂W

∂x(x0, t0) f (x0,u,d)}> 0.

In this case V is a viscosity subsolution.

2. For all (x0, t0) ∈ Rn × (0,T ) and for all smooth W : Rn × (0,T ) → R, if V −W attains alocal minimum at (x0, t0), then

max{h(x0)−V (x0, t0),∂W

∂t(x0, t0)+ sup

d∈Dinfu∈U

∂W

∂x(x0, t0) f (x0,u,d)}6 0.

In this case V is a viscosity supersolution.

The case t = 0 is automatically captured by [65] (p.546).Part 1. Consider an arbitrary (x0, t0) ∈ Rn × (0,T ) and a smooth W : Rn × (0,T ) → R suchthat V −W has a local maximum at (x0, t0). Then, there exists δ1 > 0 such that for all (x, t ) ∈Rn × (0,T ) with |x −x0|2 + (t − t0)2 < δ1

(V −W )(x0, t0)> (V −W )(x, t ).

15


We would like to show that

max{h(x0)−V (x0, t0),∂W

∂t(x0, t0)+ sup

d∈Dinfu∈U

∂W

∂x(x0, t0) f (x0,u,d)}> 0.

Since by Lemma 1 h(x)−V (x, t )6 0, either h(x0) =V (x0, t0) or, h(x0)−V (x0, t0) < 0. For theformer the claim holds, whereas for the latter it suffices to show that there exists d ∈ D suchthat for all u ∈ U

∂W

∂t(x0, t0)+ ∂W

∂x(x0, t0) f (x0,u,d)> 0.

For the sake of contradiction assume that for all d ∈ D there exists u ∈ U such that for someθ > 0

∂W

∂t(x0, t0)+ ∂W

∂x(x0, t0) f (x0,u,d) <−2θ < 0.

Since W is smooth and f is continuous, then based on [66] we have that

∂W

∂t(x0, t0)+ ∂W

∂x(x0, t0) f (x0,u,ζ) <−3θ

2< 0,

for all ζ ∈ B(d ,r )∩D and some r > 0, where B(d ,r ) denotes a ball centered at d with radiusr . Because D is compact there exist finitely many distinct points d1, ...,dn ∈ D, u1, ...,un ∈ U,and r1, ...,rn > 0 such that D ⊂⋃n

i=1 B(di ,ri ) and for ζ ∈ B(di ,ri )

∂W

∂t(x0, t0)+ ∂W

∂x(x0, t0) f (x0,ui ,ζ) <−3θ

2< 0.

Define g : D → U by setting for k = 1, ...,n, g (d) = uk if d ∈ B(uk ,rk )\⋃k−1

i=1 B(ui ,ri ). Then

∂W

∂t(x0, t0)+ ∂W

∂x(x0, t0) f (x0, g (d),d) <−3θ

2< 0.

Since W is smooth and f is continuous, there exists δ2 ∈ (0,δ1) such that for all (x, t ) ∈Rn × (0,T ) with |x −x0|2 + (t − t0)2 < δ2

∂W

∂t(x, t )+ ∂W

∂x(x, t ) f (x, g (d),d) <−θ < 0.

Finally, define α : D[t0,T ] → U[t0,T ] by α[d ](τ) = g (d(τ)) for all τ ∈ [t0,T ]. It is easy to seethat α is now non-anticipative and hence α(·) ∈A[t0,T ]. So for all d(·) ∈ D[t0,T ] and all (x, t ) ∈Rn × (0,T ) such that |x −x0|2 + (t − t0)2 < δ2,

∂W

∂t(x, t )+ ∂W

∂x(x, t ) f (x,α[d ](·),d(·)) <−θ < 0.

By continuity, there exists δ3 > 0 such that |φ(t , t0, x0,α[d ](·),d(·))− x0|2 + (t − t0)2 < δ2 forall t ∈ [t0, t0 +δ3]. Therefore, for all d(·) ∈ D[t0,T ]

V (φ(t , t0, x0,α(·),d(·)), t )−V (x0, t0)

6W (φ(t , t0, x0,α(·),d(·)), t )−W (x0, t0)

=∫ t

t0

(∂W

∂s(φ(s, t0, x0,α(·),d(·)), s)

+ ∂W

∂x(φ(s, t0, x0,α(·),d(·)), s) f (φ(s, t0, x0,α(·),d(·)),α(·),d(·))

)d s

<−θ(t − t0).

16


Let τ0 ∈ [t0, t0 +δ3] be such that

h(φ(τ0, t0, x0,α(·),d(·))) = maxτ∈[t0,t0+δ3]

h(φ(τ, t0, x0,α(·),d(·))).

Case 1.1: If τ0 ∈ (t0, t0 +δ3], then for t = τ0 we have

V (φ(τ0, t0, x0,α(·),d(·)),τ0)−V (x0, t0) <−θ(τ0 − t0) < 0. (2.10)

Then by the dynamic programming argument of Lemma 2.1 we have:

V (x0, t0)6 supd(·)∈D[t0,t0+δ3]

max{

maxτ∈[t0,t0+δ3]

h(φ(τ, t0, x0,α(·),d(·))),V (φ(τ0, t0, x0,α(·),d(·)),τ0)}.

We can choose d(·) ∈D[t0,t0+δ3] such that

V (x0, t0)6max{

maxτ∈[t0,t0+δ3]

h(φ(τ, t0, x0,α(·),d(·))),V (φ(τ0, t0, x0,α(·),d(·)),τ0)}+ε,

and set ε< θ2 (τ0−t0). Since by Lemma 2.1 h(x)−V (x, t )6 0 for all (x, t ) ∈Rn×(0,T ), we have

that maxτ∈[t0,t0+δ3] h(φ(τ, t0, x0,α(·), d(·))) = h(φ(τ0, t0, x0,α(·), d(·)))6V (φ(τ0, t0, x0,α(·), d(·)),τ0).Hence,

V (x0, t0)6V (φ(τ0, t0, x0,α(·), d(·)),τ0)+ θ

2(τ0 − t0).

Since (2.10) holds for all d(·) ∈ D[t0,T ], it will also hold for d(·), and hence the last argumentestablishes a contradiction.

Case 1.2: If τ0 = t0 then for t = t0 +δ3 we have that for all d(·) ∈ D[t0,T ]

V (φ(t0 +δ3, t0, x0,α(·),d(·)), t0 +δ3)−V (x0, t0) <−θδ3 < 0.

Since by Lemma 2.1

V (x0, t0)6 supd(·)∈D[t0,t0+δ3]

max{

maxτ∈[t0,t0+δ3]

h(φ(τ, t0, x0,α(·),d(·))),

V (φ(t0 +δ3, t0, x0,α(·),d(·)), t0 +δ3)},

then if

V (x0, t0)6 supd(·)∈D[t0,t0+δ3]

V (φ(t0 +δ3, t0, x0,α(·),d(·)), t0 +δ3),

we can choose d(·) ∈D[t0,t0+δ3] such that

V (x0, t0)6V (φ(t0 +δ3, t0, x0,α(·), d(·)), t0 +δ3)+ θδ3

2,

which establishes a contradiction.If we now have

V (x0, t0)6 supd(·)∈D[t0,t0+δ3]

maxτ∈[t0,t0+δ3]

h(φ(τ, t0, x0,α(·),d(·))),

17


then we can choose d(·) ∈D[t0,t0+δ3] such that

V (x0, t0)6 maxτ∈[t0,t0+δ3]

h(φ(τ, t0, x0,α(·), d(·)))+ε,

or equivalently V (x0, t0) 6 h(x0) + ε, since τ0 = t0. Based on our initial hypothesis thath(x0) < V (x0, t0), there exists a δ > 0 such that h(x0)−V (x0, t0) < −2δ. If we take ε < δ weestablish a contradiction.

Part 2. Consider an arbitrary (x0, t0) ∈ Rn × (0,T ) and a smooth W : Rn × (0,T ) → R suchthat V −W has a local minimum at (x0, t0). Then, there exists δ1 > 0 such that for all(x, t ) ∈Rn × (0,T ) with |x −x0|2 + (t − t0)2 < δ1

(V −W )(x0, t0)6 (V −W )(x, t ).

We would like to show that

max{h(x0)−V (x0, t0),∂W

∂t(x0, t0)+ sup

d∈Dinfu∈U

∂W

∂x(x0, t0) f (x0,u,d)}6 0.

Since V (x, t )> h(x) it suffices to show that ∂W∂t (x0, t0)+supd∈D infu∈U

∂W∂x (x0, t0) f (x0,u,d)6

0. This implies that for all d ∈ D there exists a u ∈ U such that

∂W

∂t(x0, t0)+ ∂W

∂x(x0, t0) f (x0,u,d)6 0.

For the sake of contradiction assume that there exists d ∈ D such that for all u ∈ U thereexists θ > 0 such that

∂W

∂t(x0, t0)+ ∂W

∂x(x0, t0) f (x0,u, d) > 2θ > 0.

Since W is smooth, there exists δ2 ∈ (0,δ1) such that for all (x, t ) ∈Rn × (0,T ) with |x −x0|2+(t − t0)2 < δ2

∂W

∂t(x, t )+ ∂W

∂x(x, t ) f (x,u, d) > θ > 0.

Hence, following [66], for d(·) ≡ d and any α(·) ∈A[t0,T ]

∂W

∂t(x, t )+ ∂W

∂x(x, t ) f (x,α(·),d(·)) > θ > 0.

By continuity, there exists δ3 > 0 such that |φ(t , t0, x0,α(·),d(·))− x0|2 + (t − t0)2 < δ2 for allt ∈ [t0, t0 +δ3]. Therefore, for all α(·) ∈A[t0,T ]

V (φ(t0 +δ3, t0, x0,α(·),d(·)), t0 +δ3)−V (x0, t0)

>W (φ(t0 +δ3, t0, x0,α(·),d(·)), t0 +δ3)−W (x0, t0)

=∫ t0+δ3

t0

(∂W

∂t(φ(t , t0, x0,α(·),d(·)), t )

+ ∂W

∂x(φ(t , t0, x0,α(·),d(·)), t ) f (φ(t , t0, x0,α(·),d(·)),α(·),d(·))

)d t

> θδ3.

18


But by the dynamic programming argument of Lemma 2.1 we can choose a α(·) ∈ A[t0,T ]

such that

V (x0, t0)> supd(·)∈D[t0,t0+δ3]

max{

maxτ∈[t0,t0+δ3]

h(φ(τ, t0, x0, α(·),d(·))),

V (φ(t0 +δ3, t0, x0, α(·),d(·)), t0 +δ3)}− δ3θ

2

>max{

maxτ∈[t0,t0+δ3]

h(φ(τ, t0, x0, α(·),d(·))),V (φ(t0 +δ3, t0, x0, α(·),d(·)), t0 +δ3)}− δ3θ

2

>V (φ(t0 +δ3, t0, x0, α(·),d(·)), t0 +δ3)− δ3θ

2.

The last statement establishes a contradiction, and completes the proof.

2.3.3 Variational equation for V

Consider the value function V defined in the previous section. The following theorem pro-poses that V is the unique viscosity solution of another variational equation.

Theorem 2.2. V : Rn × [0,T ] → R is the unique viscosity solution of the variational equa-tion

max{

h(x)− V (x, t ),∂V

∂t(x, t )+min{0,sup

d∈Dinfu∈U

∂V

∂x(x, t ) f (x,u,d)}

}= 0, (2.11)

with terminal condition V (x,T ) = max{l (x),h(x)}.

Proof. By Theorem 2.1, V (x, t ) is the unique viscosity solution of (2.4), subject to V (x,T ) =max{l (x),h(x)}. If we let H(p, x) = supd∈D infu∈U pT f (x,u,d) then, following the proof ofTheorem 2 of [123], we have that

H(p, x) = supd∈D

infu∈U

pT f (x, u,d)

= supd∈D

infu∈U

infu∈U

pT (u f (x,u,d))

= infu∈U

u supd∈D

infu∈U

pT f (x,u,d)

= minu∈U

u supd∈D

infu∈U

pT f (x,u,d)

= min{0, H(p, x)}.

Consequently, the two variational equations (2.4) and (2.11) are equivalent, and so V (x, t )is the viscosity solution of (2.11).

19


Since the solution to (2.11) is unique [67], one could easily show that

V (x, t ) = infα(·)∈A[t ,T ]

supd(·)∈D[t ,T ]

minτ1∈[t ,T ]

max{l (φ(τ1, t , x,α(·),d(·))),

maxτ2∈[t ,τ1]

h(φ(τ2, t , x,α(·),d(·)))}. (2.12)

Remark 2.1. Note that the second term in (2.11) is a modified version of the Hamilton-Jacobi equation, which was proposed in [98], [123], and does not allow states included inthe reachable set once, to leave the set later on. The first term is the same in both (2.4),(2.11), was proposed in [115], and prevents the value function from entering the “avoid”region characterized by h(·). Both modifications encode different kind of obstacles for thevalue function, and hence fall in the framework of [16].

2.4 Case study: Underwater vehicle motion in the presenceof obstacles

To illustrate the theoretical formulation of Section 2.2, we consider the motion of an under-actuated underwater vehicle in the presence of a disturbance current. Based on the mod-eling approach of [142], we focus on the problem of steering the vehicle towards a specifiedtarget set, while avoiding fixed obstacles in the navigation space.

2.4.1 Mathematical modeling

Following the detailed derivation of [142], [141], we consider the motion of a three-degreeof freedom underwater vehicle with two back thrusters but no side thruster. We start withthe kinematics of the underwater vehicle, given by

x1

x2

x3

=

m cos(ψ)m sin(ψ)

0

+

cos(x3)sin(x3)

0

u1 +

001

u2 +

−sin(x3)cos(x3)

0

d1.

The state variables in x = [x1 x2 x3]T represent the cartesian coordinates and the orientationof the vehicle, whereas u1, d1 are the components of the linear velocity vector, and u2 is theangular velocity. Variable m denotes the amplitude and ψ the direction of the disturbancecurrent. We consider, as in [142], that u = [u1 u2]T is the control input, which consists of thevelocities along the two degrees of freedom, and d1 to act as a bounded disturbance, sinceit is the velocity along the unactuated degree of freedom.

20

Case study: Underwater vehicle motion in the presence of obstacles

2.4.2 Reach-Avoid formulation

Our objective in this problem is to identify the set of initial states for which there exists acontrol input u, such that for any disturbance d, the vehicle can reach a target R within somespecified time interval, while avoiding some fixed obstacles denoted by A. This is a reach-avoid at any time problem, and based on the analysis of Section 2.3, the value function V ,that characterizes the desired set, is the viscosity solution of (2.11).

The target set R is characterized by constraints of the form xR1,min 6 x1 6 xR

1,max and xR2,min 6

x2 6 xR2,max. Similarly, A represents the obstacles in the motion space and could be ex-

pressed as A = ∪Ni=1 Ai , where Ai = {x ∈ R3 | x Ai

1,min 6 x1 6 x Ai1,max and x Ai

2,min 6 x2 6 x Ai2,max},

and N denotes the number of obstacles. To encode these constraints in the reach-avoid set-ting of Section 2.2, we define functions l (·) : R3 → R and h(·) : R3 → R, such that l (·) charac-terizes the set R and h(x) = max{h A1 (x), . . . ,h AN (x)}, where h Ai (x) determines the obstaclei . A natural choice is to choose l (·) to be the signed distance to the set Rc . Then

l (x) ={

d(x,R) if x ∈ Rc

−d(x,Rc ) if x ∈ R,

where d(x,R) = infx∈R |x − x| stands for the usual distance to the set R. Similarly, h Ai (·)is defined to be the signed distance to the set Ai respectively. The functions l (·) and h(·)will then be Lipschitz by construction; to keep them bounded, we saturated them at theLipschitz constants Cl and Ch respectively. For the numerical implementation this is notan issue since the computations are performed over compact sets.

The Hamiltonian of the system is H(p, x) = min{0, H(p, x)}, where H(p, x) (defined in Sec-tion 2.3.2) is given by

H(p, x) = supd∈D

infu∈U

((p1 cos(x3)+p2 sin(x3))u1 +p3u2+

(−p1 sin(x3)+p2 cos(x3))d1 +mp1 cos(ψ)+mp2 sin(ψ)).

The input values that optimize H(p, x) can be then easily computed as

u∗1 =

{u1,min if p1 cos(x3)+p2 sin(x3)> 0u1,max if p1 cos(x3)+p2 sin(x3) < 0

,

u∗2 =

{u2,min if p3 > 0u2,max if p3 < 0

,

d∗1 =

{d1,max if −p1 sin(x3)+p2 cos(x3)> 0d1,min if −p1 sin(x3)+p2 cos(x3) < 0

,

where pi = (∂V /∂xi ) for i = 1,2,3 (see [142] for a detailed derivation). Although these in-puts depend in general on the state of the system (through the costate vector p), are notnecessarily feedback, but non-anticipative strategies.

21


(a) (b)

(c) (d)

Figure 2.1: Contour plots of R A(t,R,A) for (a) x3 = (π/2)r ad , (b) x3 = (π/4)r ad , (c) x3 =0r ad . (d) 3D representation of of R A(t,R,A).

By inspection of the Hamiltonian of the system, both H(p, x) and H(p, x) are monotonewith respect to the value function V (x, t ) and V (x, t ), respectively. This fact, together withLemma 2.3, allows us to use level set methods to compute the solution of the problem nu-merically, while having guarantees that the numerical solution tends to the viscosity so-lution as the grid becomes finer. To enforce the constraints represented by h(·) numer-ically, a procedure called “masking” is used in the level set methods, to ensure that thevalue function will not enter in the obstacle region A. Alternatively, numerical tools of[67] for solving variational equations, could be used. In both methods, as also stated in[124], at each timestep t and for all grid points x the value function is computed as V (x, t ) =max(h(x),V (x, t )), where V (x, t ) is the numerical solution of the partial differential equa-tion, which appears as the second term in (2.11). Similar procedure is followed for V (x, t ),where the second term of (2.4) is solved instead.

22

Case study: Underwater vehicle motion in the presence of obstacles

(a) (b)

Figure 2.2: (a) Worst case analysis with ψ as additional disturbance input. (b) Worst caseanalysis with ψ as additional disturbance input.

2.4.3 Simulation results

For the numerical computation we used four fixed obstacles and considered m = 0.2m/s,ψ = 0◦ (aligned with the x1 axis) to be the current disturbance amplitude and orientationrespectively. The orientation of the underwater vehicle can vary in the interval [−π,π].For the numerical simulations, u1,max = 1.6m/s,u1,min = −1.6m/s,u2,max = 0.8rad/s,u2,min =−0.8rad/s,d1,max = 0.8m/s, and d1,min =−0.8m/s were chosen from [142] to be the extrema ofthe control and disturbance inputs. Contour slices of the resulting set R A(t ,R, A), for t = 3sand different values of x3, are shown in Figures 2.1a-2.1c. The reachable sets include allstates inside the area determined by the solid lines, and as expected do not include pointsinside the fixed obstacles denoted by rectangles. The filled square represents the targetset that the vehicle aims to reach, whereas the dashed square indicates the boundary ofthe motion space. For comparison purposes, the dashed lines depict the reachable sets att = 2s.

So far, the disturbance current was assumed to have constant magnitude and direction. Ina worst case setting, the angle ψ of the current can be considered as an additional distur-bance input d2 = ψ, which is also trying to maximize the Hamiltonian of the system. Themaximum value of H is attained for d∗

2 = atan2(p2, p1). The numerically computed reach-able set for this case is depicted in Figure 2.2a. It implies that only the points that belong tothis 3D set can reach the target for any disturbance direction and for any value of d1. Thetransparent cube indicates the boundary of the motion space.

For a more more realistic implementation of the worst case scenario, the state space couldbe augmented with ψ = d2 ∈ [−5◦/s 5◦/s], and so the derivative of the current’s angle, in-stead of the actual angle, could be treated as an additional disturbance input. Figure 2.2bdepicts a 3D projection of the 4D reachable set for t = 3s. This projection represents a

23


3D case 3D case 4D caseψ= 0 d2 =ψ d2 = ψ

Time 305.12s 338.45s 970.15sMemory 216MB 296MB 702MB

Grid 81×81×81 81×81×81 31×31×31×81

Table 2.1: Numerical statistics of the reachability computations.

union over the reachable sets that correspond to each disturbance angle, and as expectedit is a superset of the one of Figure 2.2a (conservative case), since the disturbance does notchange direction instantaneously any more, and subset of that of Figure 2.1d, where thedisturbance was assumed to have constant direction. The main purpose of this examplewas to illustrate the proposed formulation numerically, and from an application point ofview further investigation is required.

All simulations for this study were performed on an Intel(R) Core(TM)2 Duo 2.66GHz pro-cessor running Windows 7, and using the Level Set Method Toolbox [120] (version 1.1) onMATLAB 7.10. Since the the Level Set Method Toolbox is based on gridding the state space,the memory and computational cost grow exponentially with the dimension of the system,and hence the algorithm suffers from the “curse of dimensionality” [124]. On the otherhand, assuming that an accurate enough grid is used, tight approximations of the (in gen-eral irregular and nonconvex) reachable sets can be achieved. The details for the numericalimplementation of the specific example are summarized on Table 2.1. As expected, the firsttwo cases, that were performed on the same grid, required similar time and memory usage,whereas the 4D implementation led to a significant increase both in memory and compu-tational time despite using a sparser grid.

2.5 Summary and open problems

In this chapter, a new framework of controlling nonlinear systems with state constraintsand competing inputs was presented, and a proof that the value function of the resultingreach-avoid problem is the viscosity solution to a variational equation was provided. Theformulation was based on reachability and game theory, and has the advantage of main-taining the continuity in the value function and the Hamiltonian of the system. As a con-sequence, it has very good numerical properties and provides theoretical support for theuse of standard numerical tools. To illustrate the main features of the proposed approach,an example, involving the target hitting problem of an underactuated underwater vehiclein the presence of obstacles, was provided. The developed reach-avoid operator providesan alternative formulation for the viability-based approaches, and could be extended andincorporated in existing algorithms, for verification of hybrid systems (see Chapter 3).

24

Summary and open problems

Although the proposed framework is quite general, the obstacle function was consideredto be static in the sense that no time or control dependency was taken into account. Theformer could be captured by the current formulation in the way presented in Chapter 4,whereas the latter is more demanding to characterize, since one needs to resort to discon-tinuous viscosity solutions [15]. Another issue would be to provide a systematic method-ology in order to construct numerically the theoretically optimal control policy. One wayto achieve this, is to “save” the computed control input at every grid point, the first timethat this point enters the reach-avoid set, and use nearest neighbor or linear interpolationtechniques when applying the control sequence online. Reliable construction of a controlpolicy remains challenging however, since it requires the computation of derivatives of thevalue function, which is a process very sensitive to numerical errors.

25

CHAPTER

3Viable set computation for hybridsystems

3.1 Introduction

The problem of synthesizing controllers for hybrid systems has attracted considerable at-tention both from the automatic control and the computer science community [157], [154],[153], [32], [92], [106]. In this direction, [109], [74], [161] considered reachability and vi-ability type of problems for a class of hybrid automata whose continuous dynamics aregoverned by time invariant vector fields.

In [101], [164] the authors characterize the maximal control invariant set (viability kernelin the sense of [14]) for a more general class of hybrid systems, with nonlinear dynamicsand competing inputs [18]. The proposed procedure was based on the alternating applica-tion of one continuous and two discrete operators. The former involves what was referredin [101] as reach-avoid computation, whereas the latter requires the inversion of the resetmaps which encode the discrete behavior of the system. There are however certain limi-tations in the iterative procedure of [101]. The first is that there is no guarantee that theprocess reaches a fixed point, hence the algorithm might not converge to the desired viabil-ity kernel. Moreover, the continuous part of the algorithm involves a reach-avoid compu-tation, which was limited to differentiable value functions with continuous Hamiltonians[101]. The authors of [98], [123], extended this to the notion of viscosity solutions, but wererestricted to the characterization of the “reach” part of the operator. To overcome these lim-itations and achieve a complete characterization of the algorithm, [70], [13], considered thesame problem from a viability theory perspective. Moreover, they introduced a theoreti-cally sound notion of hybrid strategies in a gaming context, and using nonsmooth analysistools they proved convergence of the iterative scheme.

In this chapter we provide an alternative characterization [117], in a sense dual to that of[70], which offers a complete framework for addressing viability problems for hybrid sys-tems in an optimal control context, and supports theoretically the use of existing numerical

27

Viable set computation for hybrid systems

tools [125] for hybrid reachability calculations. The proposed formulation is entirely basedon optimal control and the properties of the hybrid system executions. We first restrict ourattention to the case where only a finite number of transitions is allowed, and a finite timecontinuous reach-avoid computation is performed. A similar problem was addressed in[62]. For the finite time reach-avoid characterization we adopt the formulation of Chapter2 which relates the problem of reaching a target, while avoiding an unsafe set of states, tothe viscosity solution of a quasi-variational inequality of the form of [67], [86]. Both thevalue function and the Hamiltonian of the optimal control problem are continuous, andhence the reach-avoid computation does not suffer from the drawbacks of [101], [164]. Wethen consider the problem where the continuous calculation is still of finite time, but aninfinite number of transitions is allowed. To show convergence of the algorithm we treatthe problem as the one of determining the maximal fixed point of a monotone operator ona complete lattice of sets [159], [60], and use a constructive version of Tarski’s theorem [57]to characterize this fixed point. This approach, which is fundamentally different from theone adopted in [70], is related to the approach proposed by [130] for a more restricted classof hybrid dynamics. The last and more general case, deals with the problem where we mayhave both infinite time continuous evolution and infinite number of discrete transitions.In this case, we use the infinite time counterpart of the reach-avoid operator of [115], andfollow a procedure similar with [67]. The case of infinite continuous evolution and finitenumber of discrete transitions follows then directly from the results of Sections 3.3.1 and3.3.3.

We demonstrate some features of the proposed algorithms by means of a numerical exam-ple and an application to the problem of voltage stability for a single machine-load systemin case of a line fault [158], [58]. The objective is to determine the set of initial conditions,from which the system trajectories can start, and despite a line fault, the voltage will remainwithin the safety margins both during the transient phase and after the reclosure of the line.

The chapter is organized as follows. Section 3.2 states the main assumptions, describes thehybrid dynamics, and poses the viability problem. Section 3.3 deals with the characteriza-tion of the continuous part of the proposed reachability algorithm, and the convergence ofthe iterative process for three different cases. Section 3.4 shows a numerical example andthe application of the viability algorithm to a power system case study. Finally, Section 3.5summarizes our results and provides a list of open problems.

3.2 Viability specifications of hybrid game automata

3.2.1 Hybrid dynamics

We consider dynamical systems, whose state vector comprises both a discrete componentq , and a continuous component x. The trajectories of the state vector are governed by con-trol and disturbance inputs. Adopting the notation of [70], let v and u denote the control,

28

Viability specifications of hybrid game automata

and δ, d the disturbance inputs (discrete and continuous respectively). Note that, when-ever possible, for the continuous dynamics we adopted the notation of Chapter 2. The sys-tem can be then described by a hybrid automaton H , which is the collection of the followingelements.

Definition 3.1. A hybrid automaton H is the collection of

• discrete state variables q ∈Q and continuous state variables x ∈ X ,

• control inputs v ∈V and u ∈U ,

• disturbance inputs δ ∈∆ and d ∈ D,

• vector field f (·, ·, ·, ·) : Q ×X ×U ×D → X ,

• domain set, Dom(·) : Q → 2X ,

• edges, E ⊆Q ×Q,

• guard condition G(·) : E → 2X ,

• reset function r (·, ·, ·, ·) : E ×X ×V ×∆→ X .

The necessary properties needed to ensure that the system is well posed will be listed inAssumption 3.1. Before introducing the properties of the executions accepted by the hybridautomaton H , we provide the definition of a hybrid time set [100].

Definition 3.2. A hybrid time set τ= {Ii }Ni=0 is a finite or infinite sequence of intervals of the

real line, such that for all i < N , Ii = [τi ,τ′i ], if N <∞, IN = [τN ,τ′N ) (possibly with τ′N =∞),or IN = [τN ,τ′N ], and for all i , τi 6 τ′i = τi+1.

We are now in a position to define the class of executions accepted by the automaton H .

Definition 3.3. Consider the sequence of functions q = {qi (·)}Ni=0, x = {xi (·)}N

i=0, v = {vi (·)}Ni=0,

u = {ui (·)}Ni=0,δ = {δi (·)}N

i=0,d = {di (·)}Ni=0, where qi (·) : Ii → Q, xi (·) : Ii → X , vi (·) : Ii →

V ,ui (·) : Ii →U ,δi (·) : Ii →∆,di (·) : Ii → D, respectively. An execution accepted by the hybridautomaton H with initial condition (q0(τ0), x0(τ0)), is a hybrid trajectory (τ, q , x, v ,u,δ,d)over its state and input variables that satisfies the following conditions:

• Discrete evolution: For all i < N ,

1. (qi (τ′i ), qi+1(τi+1)) ∈ E.

2. xi (τ′i ) ∈G(qi (τ′i ), qi+1(τi+1)).

3. xi+1(τi+1) = r (qi (τ′i ), qi+1(τi+1), xi (τ′i ), vi+1(τi+1), δi+1(τi+1)).

• Continuous evolution: For all i with τi < τ′i29


1. ui (·) ∈U and di (·) ∈D are Lebesgue measurable functions on Ii .

2. qi (t ) = qi (τi ), υi (t ) = vi (τi ) and δi (t ) = δi (τi ) for all t ∈ Ii .

3. xi (·) : Ii → X is the solution of the differential equation

xi (t ) = f (qi (t ), xi (t ),ui (t ),di (t )),

over the interval Ii with initial condition xi (τi ).

4. xi (t ) ∈ Dom(qi (t )) for all t ∈ [τi ,τ′i ).

The executions accepted by H may be finite if τ is a finite sequence and its last interval isclosed, finite-open if τ is a finite sequence and its last interval is open, infinite if τ is aninfinite sequence or

∑∞i=1(τ′i −τi ) =∞, and Zeno if it is infinite but

∑∞i=1(τ′i −τi ) <∞. The

convergence of the viability algorithm, and the validity of the results presented in the nextsection, rely on a series of assumptions on the hybrid automaton H and its executions.

Assumption 3.1. 1. The continuous state space is X = Rn . The set Q is finite. The sets U ,V , D and ∆ are compact subsets of Euclidean spaces.

2. For all q ∈ Q the function f (q , x,u,d) is globally Lipschitz continuous in x, contin-uous in u and d and bounded. For all (q , x) ∈ Q × X , the sets

⋃u∈U f (q , x,u,d) and⋃

d∈D f (q , x,u,d) are convex and compact for all d ∈ D and all u ∈U , respectively.

3. For all q ∈Q, the set Dom(q) is open and Dom(q)∪⋃q ′∈Q G(q , q ′) = X .

4. For all q , q ′ ∈Q with (q , q ′) ∈ E, the function r (q , q ′, x, v ,δ) is continuous in x, v and δ.

5. For all q , q ′ ∈Q with (q , q ′) ∈ E, the set G(q , q ′) is open, possibly the empty set.

The first two parts of Assumption 3.1 ensure that the game between the control and distur-bance inputs is well-posed (the convexity part is mainly used in Section 3.3.3), whereas part3 guarantees that H is non-blocking [100]. Statements 4,5 ensure continuity of the discreteoperators of Section 2.3. Note that Zeno executions are not excluded from this formulation.However, in certain cases, the Zeno behavior can be eliminated by performing dynamicregularization (spatial or temporal) of the hybrid automaton [80]. Given this framework,and under Assumption 3.1, it was shown in [70] that for every admissible initial conditionand input trajectories there exists an infinite execution (possibly Zeno) for H .

3.2.2 Gaming formulation and input strategies

In purely continuous differential games [66], [167], it is standard to consider the notion ofnonanticipative strategies. A function α(·, ·, ·) : D ×Q × X → U is called non-anticipative(with respect to the first variable) if for all (q , x) ∈ Q × X , d(·),d ′(·) ∈ D and T > 0, if d(t ) =d ′(t ) for almost every t ∈ [0,T ] then α(d , q , x)(t ) = α(d ′, q , x)(t ) for almost every t ∈ [0,T ].Note that this definition is identical to the one of Section 2.2.1, with the difference that the

30

Viability specifications of hybrid game automata

nonanticipative strategy in this case depends also on (q , x), which play the role of initialcondition.

Let then A denote the class of nonanticipative strategies. Following [70], we can now definea hybrid strategy (α,γ) for the control inputs (u, v) as a pair whose first element is a nonan-ticipative strategy α(·, ·, ·) : D ×Q × X → U for the continuous input, and γ(·, ·) : Q × X → Vis a feedback strategy for the discrete one (a feedback strategy is sufficient for the discreteinputs). To ensure that the game is well-posed we do not allow transitions between discretemodes to be forced by the inputs. Moreover, the new state after the transition is not deter-mined by the control inputs. This restriction serves as a sufficient condition to ensure thatthe players are not second guessing one another in the discrete game. We could relax thisassumption if the sets of states where the control and the disturbance inputs can force tran-sitions are disjoint. However, autonomous transitions of the non-deterministic automatonare captured by the proposed framework.

3.2.3 Problem statement and definition of operators

The main objective is to characterize the viability kernel of a given closed set F ⊆Q×X . Thisis often referred as the maximal control invariant set [164] or hybrid discriminating kernel[70]. Formally this set is defined as follows (see Definitions 8 and 9 of [70] for details).

Definition 3.4. Let N and T , both possible infinite, denote the number of allowed discretetransitions and the total time of continuous evolution respectively. The hybrid discriminat-ing kernel Viab(N ,T )

F (finite or infinite time) of a closed set F ⊆ Q × X is the set of (q , x) ∈ F ,for which there exists a hybrid strategy (α,γ) for the control inputs, such that for all n 6 Nand any disturbance d(·) ∈ D, {δi }n−1

i=0 , all executions of the hybrid automaton starting from(q0(τ0), x0(τ0)) = (q , x) ∈ F with

∑n−1i=0 τ

′i −τi 6 T are such that (qi (t ), xi (t )) ∈ F for all Ii ∈ τ

and all t ∈ Ii with i < n, and (qn(τn), xn(τn)) ∈ F .

For technical purposes, in Definition 3.4 we consider only executions that terminate witha transition followed by an interval of zero continuous evolution. More general class ofexecutions can be captured as well. For the viability algorithm, which will be defined inthe next section, the computation of two discrete and one continuous operator is required.Following [70], for an arbitrary set of states K , we define

Pre∃(K ) = {(q , x) ∈ K | [x 6∈ Dom(q)]∧ [∃v ∈V , ∀δ ∈∆,

∀q ′ ∈Q, (q , q ′) ∈ E , x ∈G(q , q ′) ⇒ (q ′,r (q , q ′, x, v ,δ)) ∈ K ]}, (3.1)

Pre∀(K ) = K c ∪ {(q , x) ∈ K | ∀v ∈V , ∃δ ∈∆,∃q ′ ∈Q,

[(q , q ′) ∈ E , x ∈G(q , q ′)]∧ [(q ′,r (q , q ′, x, v ,δ)) 6∈ K ]}. (3.2)

The set Pre∃(K ) contains all states in K for which continuous evolution is not possible (x ∉Dom(q)), and there exists a choice for the discrete control input v ∈ V such that for alldisturbance inputs δ ∈ ∆, the state remains in K after one transition. On the other hand,

31


Pre∀(K ), contains all states that are either outside K , plus all states for which for all discretecontrol inputs v ∈ V there exists at least one choice for the disturbance δ ∈∆ such that thestate of the system leaves K after a transition.

For all t ∈ [0,T ] (if T = ∞ we use [0,T )) we define the continuous operator Reach(t , ·, ·) :2Q×X ×2Q×X → 2Q×X , such that the set Reach(t ,R, A) includes all states (q , x) ∈Q×X so thatif the state starts at (q , x) at time t there exist a nonanticipative strategyα for the control in-puts, such that for any disturbance d , the system trajectories either reach R before passingthrough A, or remain outside A over the time interval [0,T ]. In other words, Reach(t ,R, A)contains all states that are viable under continuous evolution, plus those that can reach Rprior to hitting A. The latter was referred to as reach-avoid computation in [164]. Noticethat the discrete state q remains constant along this computation, since only continuousdynamics are involved. Hence, for each q ∈ Q we can define Rq = {x ∈ X | (q , x) ∈ R} andAq = {x ∈ X | (q , x) ∈ A}. Then, as in [70], treating one discrete state at a time,

Reach(t ,R, A) = ⋃q∈Q

{q}×Reachq (t ,Rq , Aq ), (3.3)

where

Reachq (t ,Rq , Aq )

= {x ∈ X | ∃α(·, q , x) ∈A , ∀d(·) ∈D, [∃t1 ∈ [t ,T ], φ(t1, t , q , x,α(·, q , x),d(·)) ∈ Rq

∧∀t2 ∈ [t , t1], φ(t2, t , q , x,α(·, q , x),d(·)) 6∈ Aq ]},

∪ {x ∈ X | ∃α(·, q , x) ∈A , ∀d(·) ∈D, [∀t3 ∈ [t ,T ], φ(t3, t , q , x,α(·, q , x),d(·)) 6∈ Aq ]}. (3.4)

We can then compute Reachq (t ,Rq , Aq ) separately for each mode q ∈Q, and take their dis-joint union (3.3) to construct Reach(t ,R, A). Note that variable φ(·, t , q , x,α(·, q , x),d(·)) de-notes the trajectory of the system and is the solution of the continuous vector field at thediscrete mode q , which remains constant along this computation. Reach(t , ·, ·) has the fol-lowing basic properties, which are shown in [70] and are repeated here, since they will beused in the sequel.

Proposition 3.1. For all t ∈ [0,T ],

1. If R is closed and A is open, then Rq is closed, Aq is open and Reachq (t ,Rq , Aq ) is closedfor all q ∈Q.

2. For all K ⊆Q ×X , Pre∃(K ) ⊆ Reach(t ,Pre∃(K ),Pre∀(K )).

3. For all K ⊆Q ×X , Reach(t ,Pre∃(K ),Pre∀(K )) ⊆ K .

4. Reach(t ,Pre∃(·),Pre∀(·)) is a monotone operator, i.e. for all K1 ⊆ K2 ⊆Q ×X ,Reach(t ,Pre∃(K1),Pre∀(K1)) ⊆ Reach(t ,Pre∃(K2),Pre∀(K2)).

The first part of Proposition 3.1 can be easily shown using induction and relies on the factthat under Assumption 3.1 (part 1) Q is finite, and that Reachq (t ,Rq , Aq ) is closed if Rq isclosed and Aq is open. The latter is shown in Proposition 3.3 of Section 3.3.1. A directconsequence of Proposition 3.1 is that Reach(t ,R, A) is closed as well.

32

Hybrid discriminating kernel characterization

Algorithm 3.1 Finite time viability algorithm1 Initialization:. W0 = F × [0,T ], i = 0.2 repeat

Wi+1 = Reach(0,Pre∃(Wi ),Pre∀(Wi )),i = i +1.

3 until Wi =Wi−1 or i = N .4 Viab(N ,T )

F×[0,T ] =Wi .

5 Viab(N ,T )F = {(q , x) ∈ F |(q , xz) ∈ Viab(N ,T )

F×[0,T ] and z = 0}.

3.3 Hybrid discriminating kernel characterization

3.3.1 Finite time continuous evolution - Finite number of discretetransitions

Consider first the case where both T and N are finite. To keep track of the elapsed timewe augment the hybrid automaton by introducing an additional continuous state and setxz = [

x z] ∈ X×R. For all q ∈Q, u ∈U , d ∈ D , the vector field of the augmented dynamics is

given by f z(q , xz ,u,d) = [f (q , x,u,d) 1

] ∈ X ×R, and for all q , q ′ ∈Q with (q , q ′) ∈ E , v ∈V ,δ ∈∆ the reset map is given by r z(q , q ′, x, v ,δ) = [

r (q , q ′, xz , v ,δ) z] ∈ X ×R. In words, the

new state component z ∈ R counts forward in time and remains unaffected after discretetransitions.

The objective is to compute the finite time hybrid discriminating kernel (see Definition3.4 with T , N <∞) of a desired set F × [0,T ] ⊆ Q × X ×R. Given the discrete and continu-ous operators of Section 3.2.3, Algorithm 3.1 [70] summarizes the steps needed to computeViab(N ,T )

F×[0,T ]. Note that, unlike [70], the algorithm will terminate after at most N discretetransitions, even if a fixed point is not reached (i.e. Wi 6=Wi−1).

Proposition 3.2. Denote by Wk with k 6 N the set returned by Algorithm 3.1. The finitetime hybrid discriminating kernel of F ∈ Q × X is given by Viab(N ,T )

F = {(q , x) ∈ F |(q , xz) ∈Wk and z = 0}.

Proof. It suffices to show that Viab(k,T )W0

= Wk for all k 6 N . If k = N we directly have that

Viab(N ,T )W0

= WN , whereas if k < N notice that Viab(N ,T )W0

= WN = Wk . The last equality holdssince if the algorithm terminates at some k < N , then Wi =Wk for all i > k.Part 1: We first show that Viab(k,T )

W0⊆ Wk . Since Viab(k,T )

W0⊆ W0 and Wk ⊆ W0, it suffices to

show that W0 \ Wk ⊆ W0 \ Viab(k,T )W0

. Take (q , xz) ∈ W0 \ Wk . Fix any hybrid strategy (α,γ).Similarly to [70] we show that we can find an admissible execution (in the sense of Def-inition 3.4) starting at (q0(τ0), xz

0 (τ0)) = (q , xz) leaving W0 in finite time and after a finite

number of discrete transitions, thus proving that (q , xz) 6∈ Viab(k,T )W0

. Since (q , xz) 6∈ Wk ,

there exists i < k such that (q , xz) 6∈ Wi = Reach(0,Pre∃(Wi−1),Pre∀(Wi−1)). By Proposi-tion 3.1 (part 2) we have that (q , xz) 6∈ Pre∃(Wi−1). Therefore, either xz ∈ Dom(q), or there

33


exists δ and q ′ such that xz ∈ G(q , q ′) and (q ′,r z(q , q ′, xz ,γ(q , xz), δ)) 6∈ Wi−1. In the lat-ter case, set τ′0 = 0, q1(τ1) = q ′, xz

1 (τ1) = r z(q , q ′, xz ,γ(q , xz), δ)) and notice that τ1 = 0and (q1(τ1), xz

1 (τ1)) 6∈ Wi−1. If now xz ∈ Dom(q), and since under Assumption 3.1 (part 3)Dom(q) is open, there exists d(·) such that the solution φ(·, q , xz ,α,d) reaches Pre∀(Wi−1)without first reaching Pre∃(Wi−1). Since the admissible executions are limited to at most Ttime of continuous evolution, it suffices to consider the case where there exists t1 ∈ [0,T ]such that xz(t1) ∈ Pre∀(Wi−1) and for all t2 ∈ [0, t1), xz(t2) ∈ Dom(q) \ Pre∃(Wi−1). Let xz

0 (t ) =xz(t ) for all t ∈ [0, t1]. By the definition of Pre∀, either (q0(t1), xz

0 (t1)) 6∈ Wi−1 or there exist δand q ′ such that xz

0 (t1) ∈ G(q , q ′) and (q ′,r z(q , q ′, xz0 (t1),γ(q , xz

0 (t1)), δ)) 6∈ Wi−1. In the lat-ter case, set τ′0 = t1, q1(τ1) = q ′ and xz

1 (τ1) = r z(q , q ′, xz0 (t1),γ(q , xz

0 (t1)), δ) and notice thatτ1 = t1 6 T <∞ and (q1(τ1), xz

1 (τ1)) 6∈Wi−1.

Overall, starting from (q , xz) ∈W0 \Wi we constructed an admissible run that leaves Wi−1 inless than T time of continuous evolution and after at most one discrete transition. Iteratingi times we can construct a run that leaves W0 in less than T time of continuous evolutionand after at most i 6 k discrete transitions. For every iteration the above arguments remainthe same with the modification stated below. Assume that at iteration j < k the system is at(q j (τ j ), xz

j (τ j )) = (q , xz) with z j (τ j ) = τ j > 0. Everything remains the same apart from thecase where xz ∈ Dom(q). For an execution to be admissible we need to consider only thecase where there exists t1 ∈ [0,T −τ j ] so that the solution φ(·, q , xz ,α,d) reaches Pre∀(Wi−1)without first reaching Pre∃(Wi−1). Letting now xz

j (τ j + t ) = xz(t ) for all t ∈ [0, t1], we canshow as before that either (q j (τ j + t1), xz

j (τ j + t1)) 6∈Wi−1 or (q j+1(τ j+1), xzj+1(τ j+1)) 6∈Wi−1),

for τ j+1 = τ j + t1. Hence for any hybrid strategy we have found discrete and continuousdisturbance inputs such that the associated run starting from (q , xz), leaves W0 via an ad-missible execution, which in turn implies that (q , xz) 6∈ Viab(k,T )

W0.

Part 2: We will show that W1 ⊆ Viab(1,T )W0

. We prove that if (q , xz) ∈W1 then (q , xz) ∈ Viab(1,T )W0

.

Since W1 = Reach(0,Pre∃(W0),Pre∀(W0)), for any (q0(τ0), xz0 (τ0)) = (q , xz) ∈ W1 we can dis-

tinguish two cases.Case 1: For any (q , xz) ∈ W1 there exists an nonanticipative strategy α for the continu-ous controls such that for any continuous disturbance d(·) ∈ D, (q , xz(t )) 6∈ Pre∀(W0) forall t ∈ [0,T ] (note that we are interested to executions with τ′0 − τ0 6 T ). Choose thenan arbitrary t ∈ [0,T ]. Therefore, there exist α such that for any d , (q , xz(t )) ∈ W0 andthere also exists v ∈ V such that for all δ ∈ ∆ and q ′ ∈ Q with (q , q ′) ∈ E , xz(t ) ∈ G(q , q ′)and (q ′,r z(q , q ′, xz(t ), v ,δ)) ∈ W0. Choose q ′ ∈ Q, set τ′0 = t , xz

0 (t ) = xz(t ) for all t ∈ [0, t ],q1(τ1) = q ′, γ(q ′, xz(t )) = v for all t ∈ [0, t ], xz

1 (τ1) = r z(q , q ′, xz(t ),γ(q ′, xz(t )),δ), and no-tice that τ1 = t and (q1(τ1), xz

1 (τ1)) ∈ W0. Since t ∈ [0,T ] was arbitrary, we have shown thatthere exists a hybrid strategy (α,γ), such that for any disturbance d and δ, all executionswith τ′0 − τ0 6 T starting from (q0(τ0), xz

0 (τ0)) ∈ W1 ⊆ W0 are such that (q0(t ), xz0 (t )) ∈ W0

for all t ∈ [τ0,τ′0] and (q1(τ1), xz1 (τ1)) ∈W0. By definition 3.4, the last statement implies that

(q , xz) ∈ Viab(1,T )W0

.Case 2: For any (q , xz) ∈ W1 there exists an nonanticipative strategy α for the continuouscontrols such that for any continuous disturbance d(·) ∈D, there exists t1 ∈ [0,T ] such that

34


(q , xz(t1)) ∈ Pre∃(W0) and (q , xz(t2)) 6∈ Pre∀(W0) for all t2 ∈ [0, t1]. As in the previous case,we restricted t1 ∈ [0,T ] since all admissible executions are such that τ′0 −τ0 6 T . Considerfirst any execution with τ′0 ∈ [0, t1). Since (q , xz(t )) 6∈ Pre∀(W0) for all t ∈ [0,τ′0], following thesame arguments as in Case 1 we can show that the system executions stay in W0 via con-tinuous evolution and one discrete transition, i.e. (q0(t ), xz

0 (t )) ∈ W0 for all t ∈ [τ0,τ′0] and(q1(τ1), xz

1 (τ1)) ∈ W0. If τ′0 = t1, a transition is forced to occur since (q , xz(τ′0)) ∈ Pre∃(W0).This implies that there exist a v ∈ V such that for any δ ∈ ∆ and q ′ ∈ Q with (q , q ′) ∈ E ,xz(τ′0) ∈ G(q , q ′) and (q ′,r z(q , q ′, xz(τ′0), v ,δ)) ∈ W0. Choose q ′ ∈ Q, set q1(τ1) = q ′, xz

0 (t ) =xz(t ) for all t ∈ [0, t1], γ(q ′, xz(τ′0)) = v , xz

1 (τ1) = r z(q , q ′, xz(τ′0),γ(q ′, xz(τ′0)),δ), and noticethat τ1 = t1 and (q1(τ1), xz

1 (τ1)) ∈W0. The last statement implies that (q , xz) ∈ Viab(1,T )W0

.

Part 3: We now show that Wk ⊆ Viab(k,T )W0

. To achieve this we will use induction. Fork = 1 the claim follows from Part 2. Assume that the statement holds for some j < k, i.e.W j ⊆ Viab( j ,T )

W0. We should show that W j+1 ⊆ Viab( j+1,T )

W0. By the last part of Proposition 3.1,

Reach(0,Pre∃(W j ),Pre∀(W j )) ⊆ Reach(0,Pre∃(Viab( j ,T )W0

),Pre∀(Viab( j ,T )W0

)).

Since W j+1 = Reach(0,Pre∃(W j ),Pre∀(W j )), it suffices to show that

Reach(0,Pre∃(Viab( j ,T )W0


)) ⊆ Viab( j+1,T )W0

.

Following the same arguments as in Part 2 with Viab( j ,T )W0

in place of W0, we can show thatthere exist continuous controls such that for any disturbance input and any admissible ex-ecution starting from (q0(τ0), xz

0 (τ0)) ∈ Reach(0,Pre∃(Viab( j ,T )W0


)) ⊆ Viab( j ,T )W0

,

(q0(t ), xz0 (t )) ∈ Viab( j ,T )

W0for all t ∈ [τ0,τ′0] and (q1(τ1), xz

1 (τ1)) ∈ Viab( j ,T )W0

. Since (q1(τ1), xz1 (τ1))

∈ Viab( j ,T )W0

, following Definition 3.4 viability should be ensured for all executions with n 6 jand

∑ni=1τ

′i −τi 6 T (the first interval of those executions was assumed to be [τ1,τ′1]). To

achieve this, and since W0 = F × [0,T ], for all such executions the last component of thecontinuous state should not exceed T , i.e. zn(τn) 6 T for all n 6 j . But zn(τn) = τn =∑n

i=0τ′i − τi . Therefore, all admissible executions that lead to (q1(τ1), xz

1 (τ1)) ∈ Viab( j ,T )W0

should be restricted to one discrete transition and one interval [τ0,τ′0] of continuous evolu-tion such that

∑ni=0τ

′i −τi 6 T for all n 6 j .

Overall, starting from (q0(τ0), xz0 (τ0)) ∈ Viab( j ,T )

W0⊆ W0 there exists a continuous control in-

put such that for any disturbance, all executions with n 6 j +1 (one transition is needed toreach Viab( j ,T )

W0) and

∑ni=0τ

′i −τi 6 T are such that (qi (t ), xi (t )) ∈W0 for all Ii ∈ τ and all t ∈ Ii

with i < n, and (qn(τn), xn(τn)) ∈ W0. By Definition 3.4 this implies that (q0(τ0), xz0 (τ0)) ∈

Viab( j+1,T )W0

and concludes the induction proof.

Part 4: We will now show that Viab(N ,T )F = {(q , x) ∈ F |(q , xz) ∈ Wk and z = 0}. Parts 1 and

3 leads to Wk = Viab(k,T )W0

= Viab(N ,T )W0

, where W0 = F × [0,T ]. Therefore, the set W tk = {(q , x) ∈

F |(q , xz) ∈ Wk and z = t } for t ∈ [0,T ], denotes the states that remain in F following anyexecution restricted to T − t time of continuous evolution. Notice that Viab(N ,T )

F contains

35


all states that remain in F for any execution restricted to T time of continuous evolution.Therefore Viab(N ,T )

F =⋃Tt=0 W t

k =W 0k .

We will now characterize Reachq (t ,Rq , Aq ) (for the augmented system) which based on (3.3)determines Reach(t ,R, A) for all t ∈ [0,T ]. The dependency of the nonanticipative strategyon the initial state of the system has been neglected for simplicity. Notice that (3.4) can bewritten as Reachq (t ,Rq , Aq ) = R A(t ,Rq , Aq )∪N (t , Aq ). R A(t ,Rq , Aq ) is a finite time reach-avoid computation (as this was introduced in Chapter 2.2.3), and includes all states fromwhich trajectories can start, and reach Rq at some time within [t ,T ], and without passingthrough Aq until reaching Rq . On the other hand, N (t , Aq ) is a finite time viability calcu-lation in the sense of [98], and contains all states that do not leave Ac

q during the interval[t ,T ]. As in Chapter 2, and under the assumption that Rq is closed and Aq is open, considertwo Lipschitz continuous functions l (·), h(·), whose level sets characterize the sets Rq andAq respectively. Therefore,

Rq = {xz ∈ X ×R | l (xz)6 0},

Aq = {xz ∈ X ×R | h(xz) > 0}.

The reach-avoid set R A(t ,Rq , Aq ) was related in Chapter 2 to the sublevel set of V , whichis in turn the unique continuous viscosity solution over (xz , t ) ∈ X ×R× [0,T ] of (2.12). Itsuffices then to characterize the set N (t , Aq ). Following [98], the set N (t , Aq ) = {xz ∈ X ×R | V (xz , t ) 6 0} is the outcome of a viability type calculation (with Ac

q as the terminal set),where

V (xz , t ) = infα(·)∈A

supd(·)∈D

maxt3∈[t ,T ]

h(φ(t3, t , xz ,α(·),d(·))). (3.5)

It can be then shown that the function V is the unique continuous viscosity solution over(xz , t ) ∈ X ×R× [0,T ] of

∂V

∂t(xz , t )+max{0,sup

d∈Dinfu∈U

∂V

∂xz(xz , t ) f (xz ,u,d)} = 0, (3.6)

with V (xz ,T ) = h(xz). Note that the freezing term in this partial differential equation pre-vents a state that left the safe set once to reenter it later on.

Proposition 3.3. Reachq (t ,Rq , Aq ) = {xz ∈ X ×R | V (xz , t )6 0}∪ {xz ∈ X ×R | V (xz , t )6 0}.

Proof. Following Proposition 2.2 of Chapter 2, R A(t ,Rq , Aq ) = {xz ∈ X ×R | V (xz , t ) 6 0}and N (t , Aq ) = {xz ∈ X ×R | V (xz , t ) 6 0}. The statement follows then from the fact thatReachq (t ,Rq , Aq ) = R A(t ,Rq , Aq )∪N (t , Aq ).

One can now apply Algorithm 3.1 and compute a finite time approximation of Viab(N ,T )F×[0,T ],

where in contrast to [164], the value functions V and V are both continuous, allowing theuse of existing numerical tools.

36


3.3.2 Finite time continuous evolution - Infinite number of discretetransitions

The continuous evolution is once again of finite time, and hence the characterization ofthe Reach operator remains unchanged. The viability algorithm is then the same with theone of Section 3.1 without the test i = N test when exiting the repeat-until loop1. Since thealgorithm will terminate only if Wi = Wi−1, we need to show that the algorithm converges,and also that this fixed point is the viability kernel of F × [0,T ]. In [70] a proof based onnonsmooth analysis tools was provided; here, however, we follow a logic based approach.

Consider the complete lattice of closed subsets of W0 = F × [0,T ]. By Proposition 3.1, Reach(t ,Pre∃(·),Pre∀(·)) is a monotone operator on this lattice, and hence by Knaster-Tarski theo-rem [159], [60], it admits a maximal fixed point. Unfortunately, Tarski’s fixed point theoremthat could be used to characterize this fixed point is not constructive, unless the monotoneoperator is continuous. To overcome this difficulty, we adopt the constructive approachof [57], which relates the minimal and maximal fixed points of a monotone operator on acomplete lattice to the limits of stationary transfinite iteration sequences. Following thisapproach no continuity assumption for the Reach operator is needed, but the limit of theresulting transfinite iteration might not be

⋂∞i=0 Wi since sets corresponding to higher limit

ordinals may be required. Therefore, we relate the hybrid discriminating kernel to the fixedpoint of the set-valued operator Reach, but do not establish convergence of a countableintersection of sets to the operator’s fixed point, since in theory higher limit ordinals maybe needed. However, the argument provides a correctness proof for cases where conver-gence is reached after a finite or countable number of iterations (the only practically rele-vant cases) without additional assumptions on the system dynamics. In certain cases how-ever, where the objective is the continuation of the hybrid system executions through theZeno time [177], the introduction of transfinite induction and higher limit ordinals is justi-fied.

We first recall the definition of a complete lattice [60], and provide the definition of an upperand lower iteration sequence and their limits [57]. Consider an arbitrary set K . An order(or ordering relation) on K is a binary relation 6 such that for all x, y , z ∈ K , x 6 x, x 6 yand y 6 x imply x = y , x 6 y and y 6 z imply x 6 z; a set K with an ordering relation iscalled an ordered set. Note that the standard inclusion ⊆ is an order for sets. An orderedset has a top element > if there exists > ∈ K such that x 6 > for all x ∈ K . The bottomelement ⊥ is defined in a similar way. For an arbitrary subset S of an ordered set K , if{x ∈ K |s 6 x for all s ∈ S} has a least element, then this element is called the supremum of S,denoted by ∨S. The infimum, ∧S of S is defined analogously.

Definition 3.5. A non-empty ordered set L is called a complete lattice if for all S ⊆ L, ∨S and∧S exist. Denote then the complete lattice as L(6,>,⊥,∨,∧).

1Note that Zeno executions belong to this category.

37


Let now λ denote the smallest ordinal such that {i : i ∈ λ} has cardinality strictly greaterthat the one of L. This assumption enables us to consider monotone sequences of setson a complete lattice, whose number of elements is higher than the cardinality of lattice.Therefore, as it will be shown in Lemma 3.2 such a sequence is confined to be stationarywithout imposing any continuity assumption.

Definition 3.6. The λ-termed lower (dually for the upper) iteration sequence for a monotoneoperator P (·) : L → L starting with a set W0 is the sequence ⟨Wi , i ∈λ⟩ of elements of L, definedby the transfinite recursion

• Wi = P (Wi−1)6Wi−1 for every successor ordinal i ∈λ,

• Wi =⋂j<i W j for every limit ordinal i ∈λ.

Definition 3.7. A sequence ⟨Wi , i ∈ λ⟩ is stationary if and only if there exists k ∈ λ such thatfor all j ∈ λ with j > k, W j = Wk . Wk is then the limit of the sequence. Denote by liml

P (W0)(limu

P (W0)) the limit of a lower (upper) stationary sequence of a monotone operator P, start-ing with W0.

Adopting the notation of [60], consider the sets of pre- and post-fixed points of P 2.

Pre(P ) = {x ∈ X | P (x)6 x}, (3.7)

Post(P ) = {x ∈ X | x 6 P (x)}. (3.8)

Following Tarski’s theorem [60], the maximal and minimal fixed points of P are denotedby gfp = ∨Post(P ) and lfp = ∧Pre(P ) respectively. To relate gfp(P ) (lfp(P )) to the limits of alower (upper) iteration sequence, the following Lemma is needed (dual to Lemma 3.1 andTheorem 3.2 (part 1) of [57]).

Lemma 3.1. Let ⟨Wi , i ∈λ⟩ be aλ-termed lower iteration sequence for the monotone operatorP (·) : L → L, on the complete lattice L, starting with W0 ∈ L. Then, ifω ∈λ is the smallest limitordinal,

1. For all x ∈ L with W0 > x and x ∈ Post(P ), we have that Wi > x for all i ∈λ.

2. For all i ∈ λ let a 6 i and b < ω, such that i = aω+b. Then, for all a′ > a and for alla′ω6 k 6 a′ω+b, Wi >Wk .

Proof. Part 1: Let x ∈ L such that W0 > x and x ∈ Post(P ), i.e. P (x) > x. Assume that for allj ∈ λ with j < i , we have W j > x. If i is a successor ordinal, then we have Wi−1 > x. Then,since P is a monotone operator and by Definition 3.6, Wi = P (Wi−1) > P (x) > x. If i is alimit ordinal, then by the induction hypothesis

⋂j<i W j > x, and hence by Definition 3.6,

Wi > x. Therefore, by transfinite induction, for all i ∈λ, Wi > x.

2Note that in some references (e.g. [57]), Pre(P ) is defined by (3.8) and Post(P ) by (3.7) instead.

38


Reach(t, P re∃(·), P re∀(·)) is a monotone operator on thislattice, and hence by Knaster-Tarski theorem Tarski[1955], Davey and Priestley [2002], it processes a maximalfixed point. The drawback is that Tarski’s fixed pointtheorem is not constructive, unless a continuity argumentfor the monotone operator is made. To overcome thisdifficulty, we adopt the constructive approach of Cousotand Cousot [1979], which relates the minimal and maximalfixed points of a monotone operator on a complete latticeto the limits of stationary transfinite iteration sequences.The advantage of this approach is that no continuity as-sumption for the Reach operator is needed (which wouldrequire rather complicated mathematical arguments), andalso one does not need to resort to nonsmooth analysis(known for its technical difficulties) to achieve a charac-terization similar to Gao et al. [2007]. We first recall thedefinition of a complete lattice, and provide the definitionof an upper and lower iteration sequence and their limitsCousot and Cousot [1979] (Definitions 2.1 & 2.2).

Definition 5. A non-empty ordered set L is called a com-plete lattice if for all S ⊆ L, ∨S and ∧S exist. Denote thenthe complete lattice as L(≤,�,⊥,∨,∧).

Note that “≤” denotes the ordering relation (standardinclusion “⊆” for sets), “�”, “⊥” denote the top andbottom element of the lattice, and “∨”, “∧” its suppremumand infimum respectively (see Davey and Priestley [2002]).Let then λ denote the smallest ordinal such that {i : i ∈ λ}has cardinality greater that the one of L.

Definition 6. The λ-termed upper (lower) iteration se-quence for a monotone operator P (·) : L → L startingwith a set W0 is the sequence 〈Wi, i ∈ λ〉 of elements of L,defined by the transfinite recursion

• Wi = P (Wi−1) ≥ Wi−1 for every successor ordinal i ∈ λ.(upper iteration)

• Wi = P (Wi−1) ≤ Wi−1 for every successor ordinal i ∈ λ.(lower iteration)

• Wi =⋃

j<iWj for every limit ordinal i ∈ λ (upper iteration).

• Wi =⋂

j<iWj for every limit ordinal i ∈ λ (lower iteration).

Definition 7. A sequence 〈Wi, i ∈ λ〉 is stationary if andonly if there exists k ∈ λ such that for all j ∈ λ with j ≥ k,Wj = Wk. Wk is then the limit of the sequence. Denote

by limuP (W0) (liml

P (W0)) the limit of an upper (lower)stationary sequence of a monotone operator P , startingwith W0.

Adopting the notation of Davey and Priestley [2002],consider the sets of pre- and post-fixed points of P 3 .

Pre(P ) = {x ∈ X | P (x) ≤ x}, (10)

Post(P ) = {x ∈ X | x ≤ P (x)}. (11)

Following Tarski’s theorem Davey and Priestley [2002],the maximal and minimal fixed points of P are denotedby gfp = ∨Post(P ) and lfp = ∧Pre(P ) respectively.To relate then gfp(P ) (lfp(P )) to the limits of a lower(upper) iteration sequence, the following Lemma is needed(the proof is dual to Lemma 3.1 and Theorem 3.2 (part 1)of Cousot and Cousot [1979], and is based on transfiniteinduction).

3 Note that in some references (as in Cousot and Cousot [1979]),Pre(P ) is defined by (11) and Post(P ) by (10) instead.

�

gfp(P )

lfp(P )

⊥

Pre(P )

x = P (x)

Post(P )

•W0

.

.

.〈Wi, i ∈ λ〉

Complete lattice L

Fig. 1. Pictorial representation of the set of pre-fixed pointsPre(P ) and post-fixed points Post(P ) of a monotoneoperator P on a complete lattice L. Starting froman initial set W0 ∈ Pre(P ), a stationary, loweriteration sequence is constructed, which converges tothe maximal fixed point gfp(P ) of P .

Lemma 5. Let 〈Wi, i ∈ λ〉 be a λ-termed lower iterationsequence for the monotone operator P (·) : L → L, on thecomplete lattice L, starting with W0 ∈ L. Then, if ω ∈ λis the smallest limit ordinal,

(1) For all x ∈ L with W0 ≥ x and x ∈ Post(P ), we havethat Wi ≥ x for all i ∈ λ.

(2) For all i ∈ λ let a ≤ i and b < ω, such that i = aω+b.Then, for all a′ > a and for all a′ω ≤ k ≤ a′ω + b,Wi ≥ Wk.

The first part of Lemma 5 shows that for x = W0 ∈Post(P ), the lower iteration sequence 〈Wi, i ∈ λ〉 startingfrom W0, can only reach Post(P ) at some k ∈ λ withWk = P (Wk) = Wk+1 (i.e. stationarity). Otherwise, ifW0 ∈ Post(P ), the chain is already stationary. The secondpart shows that for a lower iteration sequence 〈Wi, i ∈ λ〉,〈Wiω , i ∈ λ〉 is also a decreasing chain.

We are now in a position to show that if we start a loweriteration sequence from an initial set W0 ∈ Pre(P ), then astationary decreasing chain is constructed, and its limit isthe greatest fixed point (and also post fixed point) of P lessthat or equal to W0 Cousot and Cousot [1979] (Theorem3.2). A pictorial representation of this procedure is givenin Figure 1.

Lemma 6. A λ-termed lower iteration sequence 〈Wi, i ∈ λ〉for the monotone operator P (·) : L → L on the completelattice L starting with W0 ∈ Pre(P ), is a stationary

decreasing chain, and its limit limlP (W0) is the greatest

fixed point of P , less than or equal to W0 (i.e. gfp(P ) =

limlP (W0)).

Proof. SinceW0 ∈ Pre(P ), and P is a monotone operatoron L, 〈Wi, i ∈ λ〉 is a decreasing chain. For the sake ofcontradiction, assume that this chain is strictly decreasing.By the definition of λ, this would imply that Card(〈Wi, i ∈λ〉) = Card({i ∈ λ}) ≥ Card(L) (Card(A) denotes thecardinality of a A). On the other hand, we have that forall i ∈ λ, Wi ∈ L, hence Card(〈Wi, i ∈ λ〉) ≤ Card(L).The last argument establishes a contradiction, and showsthat 〈Wi, i ∈ λ〉 is a stationary decreasing chain. This

Figure 3.1: Pictorial representation of the set of pre-fixed points Pre(P ) and post-fixedpoints Post(P ) of a monotone operator P on a complete lattice L. Starting from an initialset W0 ∈ Pre(P ), a stationary, lower iteration sequence is constructed, which converges tothe maximal fixed point gfp(P ) of P .

Part 2: For all i ∈λ there exist unique a,b such that i = aω+b, with a > i and b <ω [57]. If iis a limit ordinal, then b = 0 and for all a′ > a, i = aω< a′ω. But a′ω is a limit ordinal, henceby Definition 3.6, Wi = ⋂

j<i W j >⋂

j<a′ωW j = Wa′ω. If b 6= 0, then i is a successor ordinal,and i−1 = aω+b−1. Assume now that for all a′ > a and for all a′ω6 k 6 a′ω+b−1, we haveWi−1 > Wk . Then, by monotonicity of P , we have Wi = P (Wi−1) > P (Wk ) = Wk+1. Then,since Wi > Wa′ω, with k ′ = k +1 we have that for all a′ > a and for all a′ω6 k ′ 6 a′ω+b,Wi >Wk ′ . Therefore, by transfinite induction, the last statement concludes the proof.

The first part of Lemma 3.1 shows that for x = W0 6∈ Post(P ), the lower iteration sequence⟨Wi , i ∈ λ⟩ starting from W0, can only reach Post(P ) at some k ∈ λ with Wk = P (Wk ) =Wk+1

(i.e. stationarity). The second part shows that for a lower iteration sequence ⟨Wi , i ∈ λ⟩,⟨Wiω, i ∈ λ⟩ is also a decreasing chain. We are now in a position to show that if we start alower iteration sequence from an initial set W0 ∈ Pre(P ), then a stationary decreasing chainis constructed, and its limit is the greatest fixed point of P less that or equal to W0 [57](Theorem 3.2). A pictorial representation is given in Fig. 3.1.

Lemma 3.2. A λ-termed lower iteration sequence ⟨Wi , i ∈λ⟩ for the monotone operator P (·) :L → L on the complete lattice L starting with W0 ∈ Pre(P ), is a stationary decreasing chain,and its limit liml

P (W0) is the greatest fixed point of P, less than or equal to W0 (i.e. gfp(P ) =liml

P (W0)).

Proof. Since W0 ∈ Pre(P ), and P is a monotone operator on L, ⟨Wi , i ∈ λ⟩ is a decreasingchain. For the sake of contradiction, assume that this chain is strictly decreasing. By thedefinition of λ, this would imply that Card(⟨Wi , i ∈ λ⟩) = Card({i ∈ λ}) > Card(L) (Card(A)

39


denotes the cardinality of a A). On the other hand, we have that for all i ∈ λ, Wi ∈ L, henceCard(⟨Wi , i ∈ λ⟩) 6 Card(L). The last argument establishes a contradiction, and shows that⟨Wi , i ∈ λ⟩ is a stationary decreasing chain. This implies that there exists k ∈ λ, such thatWk = Wk+1. Since W0 > Wk = Wk+1 = P (Wk ), Wk is a fixed point (and also a post-fixedpoint) of P less than or equal W0. If k is a limit ordinal, denote k = aω. Then, for all a′ > awe have a′ω > aω. By Definition 3.6, Wa′ω = ⋂

j<a′ωW j = Waω∩⋂aω< j<a′ωW j . But Waω =

Wk ∈ Post(P ), so by Lemma 3.1.2, for all j ∈λ (and also for all j > aω), W j >Waω. Therefore,Wa′ω = Waω for all a′ > a. Hence, the sequence ⟨W jω, j ∈ λ⟩ is stationary decreasing (it isdecreasing due to Lemma 3.1.2) with limit Waω =Wk .

Let now x ∈ L be such that W0 > x and x ∈ Post(P ). Then, by Lemma 3.1.1, Wk > x, whichimplies that Wk is the greatest fixed point of P less than or equal W0. Hence, gfp(P ) =liml

P (W0) =∨Post(P ).

Let now L to be the complete lattice of closed subsets of W0 = F × [0,T ], and set

P = Reach(t ,Pre∃(·),Pre∀(·)).

The previous lemma leads then to the main result of this section.

Lemma 3.3. There exists a limit ordinal k ∈λ such that for all j > k,

Reach(Pre∃(W j ),Pre∀(W j )) =Wk .

Moreover, Wk is the largest set such that this holds.

Proof. Reach(t ,Pre∃(·),Pre∀(·)) is a monotone operator on the complete lattice L. Therefore,by Knaster-Tarski theorem [60], it processes a maximal fixed point. Since Wi+1 ⊆ Wi , W0 ∈Pre(Reach(t ,Pre∃(·),Pre∀(·))). By Lemma 3.2, it follows that gfp(Reach(Pre∃(·),Pre∀(·))) =liml

W0(Reach(t ,Pre∃(·),Pre∀(·))). Therefore, by Definition 3.7 (and since for all j ∈λ we have

that W j+1 = Reach(t ,Pre∃(W j ),Pre∀(W j ))), there exists a limit ordinal k ∈ λ such that for allj > k, we have Wk = Reach(t ,Pre∃(W j ),Pre∀(W j )). In Lemma 3.2 it was also shown that Wk

is the greatest fixed point of Reach(t ,Pre∃(·),Pre∀(·)). Hence, Wk is the largest set such thatthis holds.

Note that Lemma 3.3 guarantees the existence of a limit ordinal such that the viability algo-rithm converges, but not necessarily the least one. It remains to show that the fixed pointWk of the algorithm is the hybrid discriminating kernel of W0. This can be done as in The-orem 2 of [70].

Theorem 3.1. Wk is the hybrid discriminating kernel of W0 = F × [0,T ] (i.e Wk = Viab(N ,T )W0

with N possibly infinite), with k ∈λ such that Wk = Reach(t ,Pre∃(Wk ),Pre∀(Wk )).

40


The proof follows from Theorem 2 of [70] and is based on the properties of the Reach opera-tor, the executions of the hybrid automaton, and the definition of the hybrid discriminatingdomain3. For the sake of completeness though, we provide the proof in Appendix A.1.

3.3.3 Infinite time continuous evolution - Infinite number of discretetransitions

This is the most general case, since both the number of discrete transitions and the horizonof the continuous evolution may be infinite. For the discrete part of the hybrid algorithmthe same procedure as in the previous case can be followed. It remains then to characterizethe infinite time version of the continuous operator. To achieve this, define as in [123] theaugmented input u = (u, u) ∈ U × [0,1], and consider the dynamics f (x, u,d) = u f (x,u,d)(see also Section 2.2.3 of Chapter 2). Note that since the horizon of the continuous evolu-tion is allowed to be infinite, there is no need to augment the state space with an additional“timer” as in the previous subsections. Denote by V (x) and V (x) the infinite horizon valuefunctions, which correspond to (2.12) and (3.5) respectively. We will only provide the char-acterization of V (x); the same for V (x). The infinite horizon value function V (x) is givenby

V (x) = infα(·)∈A

supd(·)∈D

max{l (φ(t , x,α(·),d(·))), maxτ∈[0,t ]

h(φ(τ, x,α(·),d(·)))},

for all t > 0. The map φ(·, x,α(·),d(·)) : [0,∞) → X denotes the trajectory of the augmentedsystem, starting from the initial condition x with inputs α(·), d(·). Unlike the finite timecase of Section 3.3.1, there is no explicit dependency of φ on the initial time. Moreover, V (·)is not necessarily continuous.

Lemma 3.4. The function V (x) is upper semicontinuous.

Proof. Consider an arbitrary x0 ∈ X . It suffices to show that for all ε > 0 there exists δ > 0such that

V (x)− V (x0) < ε for all |x −x0| < δ.

By the definition of V (x), for all ε > 0 there exists T ∈ [0,∞) and α(·) ∈ A such that for alld(·) ∈D,

V (x0) > max{l (φ(T , x0, α(·),d(·))), maxτ∈[0,T ]

h(φ(τ, x0, α(·),d(·)))}− ε

4. (3.9)

Due to the continuous dependence of finite time trajectories on initial conditions (recallthat both l (·), h(·) are Lipschitz continuous), there exists δ> 0 with |x−x0| < δ, such that for

3A closed set K ⊆ Q × X is called hybrid discriminating domain if there exists a hybrid strategy (α,γ) forthe control inputs, such that for all n 6 N and any disturbance d(·) ∈ D, {δi }n−1

i=0 , all executions of the hybrid

automaton starting from (q0(τ0), x0(τ0)) ∈ K with∑n−1

i=0 τ′i −τi 6 T are such that (qi (t ), xi (t )) ∈ K for all Ii ∈ τ

and all t ∈ Ii with i < n, and (qn(τn), xn(τn)) ∈ K .

41


all α(·) ∈A and d(·) ∈D,∣∣∣ maxτ∈[0,T ]

h(φ(τ, x,α(·),d(·)))− maxτ∈[0,T ]

h(φ(τ, x0,α(·),d(·)))∣∣∣< ε

4, (3.10)∣∣∣l (φ(τ, x,α(·),d(·)))− l (φ(τ, x0,α(·),d(·)))

∣∣∣< ε

4. (3.11)

But, by the definition of V (x), we have that

V (x) < supd(·)∈D

max{l (φ(T , x, α(·),d(·))), maxτ∈[0,T ]

h(φ(τ, x, α(·),d(·)))}.

Hence, there exists d(·) ∈D such that

V (x) < max{l (φ(T , x, α(·), d(·))), maxτ∈[0,T ]

h(φ(τ, x, α(·), d(·)))}+ ε

2.

Since (3.9), (3.10), (3.11) hold for any d(·) ∈ D, they would also hold for d(·). We can thendistinguish two cases. If V (x) < l (φ(T , x0, α(·), d(·))), then statements (3.9), (3.11) lead toV (x)− V (x0) < ε. Else, if V (x) < maxτ∈[0,T ] h(φ(τ, x0, α(·), d(·))), statements (3.9), (3.10) leadto V (x)− V (x0) < ε, and conclude the proof.

Proposition 3.4. The function V (x) is a viscosity solution of

max{h(x)− V (x),min{0,supd∈D

infu∈U

∂V

∂x(x) f (x,u,d)}} = 0. (3.12)

Note that (3.12) is the stationary version of (2.11) for the initial (not the augmented) system(see [115] for details). Following a proof similar to the finite horizon case (see Theorem 2.1of Chapter 2) it is straightforward to show that V (x) is a viscosity subsolution of (3.12). Un-der the convexity part of Assumption 3.1 we can also show that the lower semicontinuousenvelope of V (x) (and hence also V (x)) is a viscosity supersolution of (3.12); this can bedone then as in the finite horizon case. Note that instead of imposing the second part ofAssumption 3.1, one could enlarge the set of admissible control functions from the classof measurable functions to that of relaxed controls (see Chapter 3 of [15]). Unfortunately,the comparison principle does not hold in this case and hence the viscosity solution is notunique [67]. However, V (x) is the maximal upper semicontinuous viscosity subsolution.

Proposition 3.5. For all upper semicontinuous viscosity subsolutions W (x) of (3.12) suchthat W (x)6 l (x) for all x ∈ X , V (x)>W (x) for all x ∈ X .

Proof. The proof is analogous to that of Proposition 5 of [67], and is based on the com-parison principle for discontinuous viscosity solutions [15]. For any T ∈ [0,∞), let w(x, t ) =W (x) for all t ∈ [0,T ] and x ∈ X . We have w(x,T )6 l (x), and hence w(x,T )6max{l (x),h(x)}= V (x,T ). Theorem 3.4 of [17] leads then to V (x, t ) > w(x, t ) = W (x) on X × [0,T ]. There-fore, T →∞ leads to V (x) = limT→∞ V (x,T )>W (x), and concludes the proof.

42

Case studies5

existing set-up

Δf,ΔPtie−linen generators participating

in AGC

d1

d2

dn

outage

Pw

Reformulation 1 or 2

distribution vectord = [d1, . . . , dn]T

AGC

Fig. 1. Schematic diagram of the security constrained reserve schedulingalgorithm.

(P1) (P′1)

(P2) (P3) (P4)scenario approach

scenario approach

robust problemδ ∈ Δ ∩ B∗

Proposed methodology

Fig. 2. Schematic diagram highlighting the connections between problems(P1) − (P3).

A. Proposed reformulation 1

Assume that in the case where i ∈ IG we can distinguishbetween the mismatch that corresponds to wind deviation andthe one which occurs due to a generator outage. For i ∈ IG,we would thus have

Rit = d1,iup,t max

+(P f

w,t − Pw,t)

− d1,idown,t max+(Pw,t − P f

w,t) + d2,iup,tPiG,t, (16)

where no d2,idown,t vector is introduced, since it was assumedthat the network is not congested, i.e constraints similar to(9), (10) hold. In the opposite case, Ri

t could be definedas in (5), with different distribution vectors for wind devi-ation and generator outages. By considering the optimiza-tion problem that corresponds to (6)-(13) if the additionaldistribution vectors are introduced, d2,iup,tP

iG,t becomes the

only bilinear term, which appears both in the constraintsand the objective function. Setting z i

t = d2,iup,tPiG,t ∈ RNG

as a new decision variable, and defining the new decisionvector xt = [PG,t, dup,t, ddown,t, [d

1,iup,t]i∈IG , [d

1,idown,t]i∈IG ,

[zit]i∈IG , Rup,t, Rdown,t]T ∈ R3N2

G+5NG , the resulting prob-lem is linear in zit, and hence convex (with a chance con-straint). The new optimization problem is of the same structurewith (6)-(13), with the difference that instead of (10), (12), we

q1 q2

x = 0 x = u + ( 12 − x)d

x ∈ R x ∈ (−∞, 0)

x ∈ R

x ∈ (−1,∞)

x := 2xδ

x := x

Fig. 3. Two-state hybrid automaton.

now have

d1,iup,t, d1,idown,t ≥ 0, for all i ∈ IG, (17)

1T d1,iup,t = 1,1T d1,idown,t = 1, for all i ∈ IG, (18)

1T zit = P iG,t, for all i ∈ IG. (19)

Moreover, the left-hand side of the last two constraints in(13) is replaced by the corresponding terms of (16). Oncethe solution to this problem is computed, d2,iup,t is calculatedas d2,iup,t = zit/P

iG,t if P i

G,t is not equal to zero, and is set tozero otherwise. Note that the sum of the elements of d2,iup,t isconfined to be one, since z i

t, i ∈ IG satisfies (19).For real time operation, the look-up table interpretation

(discussed in Section III.C) may be adopted. Given then amismatch P i

m,t = (Pw,t − P fw,t) − P i

G,t, the participation ofeach unit in compensating P i

m,t can be determined a posterioriby Ri

t/1T Ri

t. The latter requires knowledge of the mismatchterms.

Using this reformulation, a convex problem is achieved atthe expense of a more conservative reserve schedule. This isdue to the fact that Pw,t − P f

w,t, PiG,t are treated separately,

leading to reserves of higher cost. To see this, consider thecase where Pw,t−P f

w,t ≥ 0. The proposed formulation wouldlead to |Pw,t − P f

w,t|+ |P iG,t| MW of reserves, whereas only

|Pw,t − P fw,t − P i

G,t| MW are needed.

B. Proposed reformulation 2

In this subsection we overcome the bilinearity problem byusing an iterative algorithm (see Algorithm 1). We first attemptto identify a feasible solution of the problem, starting from anarbitrarily chosen power schedule P 0

G,t. At iteration k of thealgorithm, we fix P k,i

G,t only in (5) to the value obtained in theprevious iteration. Solving then (6)-(13) a new solution x k

t iscomputed, and P k

G,t is updated accordingly. If the algorithmconverges, its fixed point xk∗

t will be a feasible solution of theinitial problem.

At a second step, we use an alternating iterativescheme to refine the resulting feasible solution in termsof cost. At iteration k we first fix dk,iup,t, d

k,idown,t to the

values obtained at the previous step of the algorithm,and obtain

[P kG,t, d

kup,t, d

kdown,t, R

kup,t, Rk

down,t

]Tby

solving (6)-(13). We then fix P kG,t to the computed

value in all equations it appears, and solve for[dkup,t, d

kdown,t, [d

k,iup,t]i∈IG , [d

k,idown,t]i∈IG , R

kup,t, Rk

down,t

]T.

The entire process is then repeated until convergence. For abetter understanding, Fig. ?? shows how the power dispatch

Figure 3.2: Two-state hybrid automaton.

3.4 Case studies

3.4.1 Numerical example

Consider the hybrid automaton of Fig. 3.2 with Q = {q1, q2}, X =R, U = D =V = [−1,1], ∆=[0,1], f (q1, x,u,d) = 0 and f (q2, x,u,d) = u+( 1

2 −x)d . Notice also that E = {(q1, q2), (q2, q1)},Dom(q1) = R, Dom(q2) = (−∞,0), G(q1, q2) = R, G(q2, q1) = (−1,∞), r (q1, q2, x, v ,δ) = 2xδand r (q2, q1, x, v ,δ) = x. Clearly, the hybrid automaton satisfies Assumption 3.1.

We will now apply the infinite time counterpart of Algorithm 3.1 (i.e. T =∞ and terminationoccurs only if Wi =Wi−1) with W0 = {q1}×[−1,1]∪{q2}×[−1,1]. Since we do not require thetime of continuous evolution to be finite, the additional state that was appended to thecontinuous state vector to track time is no longer needed. For all i = 0,1, . . ., let Wi ,q1 = {x ∈R|(q1, x) ∈Wi }, Wi ,q2 = {x ∈R|(q2, x) ∈Wi }, and define

Si (q1) = {x ∈R|∃v ∈V ,∀δ ∈∆, x ∈G(q1, q2) ⇒ r (q1, q2, x, v ,δ) ∈Wi ,q2 },

Ti (q1) = {x ∈R|∀v ∈V ,∃δ ∈∆, x ∈G(q1, q2)∧ r (q1, q2, x, v ,δ) 6∈Wi ,q2 }.

Si (q1) contains the states x ∈ R for which there exists a choice for the discrete control in-put v ∈ V such that for all disturbance inputs δ ∈ ∆, the continuous state remains in Wi ,q1

after a transition. On the other hand, Ti (q1) contains all states x ∈ R for which for all dis-crete control inputs v ∈ V there exists at least one choice for the disturbance δ ∈ ∆ suchthat the continuous state leaves Wi ,q1 after a transition. The sets Si (q2),Ti (q2) are definedanalogously. Treating each mode q ∈Q separately, notice from (3.1), (3.2) that

Pre∃(W0,q1 ) = Domc (q1)∩S0(q1) =;∩ [−1

2,

1

2] = [−1

2,

1

2],

Pre∀(W0,q1 ) =W c0,q1

∪T0(q1) = ((−∞,−1)∪ (1,∞)

)∪ ((−∞,−1

2)∪ (

1

2,∞)

)= (−∞,−1

2)∪ (

1

2,∞),

Pre∃(W0,q2 ) = Domc (q2)∩S0(q2) = [0,∞)∩ (−1,1] = [0,1],

Pre∀(W0,q2 ) =W c0,q2

∪T0(q2) = ((−∞,−1)∪ (1,∞)

)∪ (1,∞)

= (−∞,−1)∪ (1,∞).

43


The fact that S0(q1) = [−12 , 1

2 ] follows from the definition of S0, where we seek to determinethe set of x ∈ R such that r (q1, q2, x, v ,δ) ∈ Wi ,q2 for all δ ∈ ∆. The latter implies that for allδ ∈ [0,1], 2xδ ∈ [−1,1]. We can now compute W1 = {q1}×W1,q1 ∪ {q2}×W1,q2 , where

W1,q1 = Reachq1 (Pre∃(W0,q1 ),Pre∀(W0,q1 )) = [−1

2,

1

2],


2,1].

For the computation of W1,q2 it suffices to notice that for all x > 1/2 there exists u such thatfor all d , f (q2, x,u,d)> 0.

Proceeding on a similar way, the second iteration of the algorithm results in

Pre∃(W1,q1 ) = Domc (q1)∩S1(q1) =;∩ [−1

4,

1

2] = [−1

4,

1

2],

Pre∀(W1,q1 ) =W c1,q1

∪T1(q1) = ((−∞,−1

2)∪ (

1

2,∞)

)∪ ((−∞,−1

4)∪ (

1

2,∞)

)= (−∞,−1

4)∪ (

1

2,∞),

Pre∃(W1,q2 ) = Domc (q2)∩S1(q2) = [0,∞)∩ [−1

2,

1

2] = [0,

1

2],

Pre∀(W1,q2 ) =W c1,q2

∪T1(q2) = ((−∞,−1

2)∪ (1,∞)

)∪ ((−1,−1

2)∪ (

1

2,∞)

)= (−∞,−1

2)∪ (

1

2,∞).

Therefore, W2 = {q1}×W2,q1 ∪ {q2}×W2,q2 , where


4,

1

2],


2,

1

2].

In general, it is easy to show that for i = 1,2, . . . , W2i−1 = {q1}×W2i−1,q1 ∪ {q2}×W2i−1,q2 andW2i = {q1}×W2i ,q1 ∪ {q2}×W2i ,q2 , where

W2i−1,q1 = [− 1

2i,

1

2i], W2i ,q1 = [− 1

2i+1,

1

2i],

W2i−1,q2 = [− 1

2i,

1

2i−1], W2i ,q2 = [− 1

2i,

1

2i].

Hence, the sequence {Wi }i converges asymptotically (i.e. N =∞) to Viab(N ,T )W0

= {q1}× {0}∪{q2}× {0}, which is the hybrid discriminating kernel of W0. Notice that for all executionsstarting at (q ,0) with q ∈Q the continuous state remains at zero either via continuous evo-lution or after a discrete transition. In particular, there exists the choice of a Zeno execution,taking at τ′0 = τ0 = 0 an infinite number of transitions between the discrete states q1 and q2,without time progressing further.

44

Case studies

0 1 3 5 7 9 11 13 15 17 190

0.5

1

1.5

2

# iterations

set v

olum

e

Pre∃q

1

Reachq

1

Pre∃q

2

Reachq

2

Figure 3.3: Volume of Pre∃(Wi ,q ) and Reachq (Pre∃(Wi ,q ),Pre∀(Wi ,q )) for every itera-tion i = 0,2, . . . and each mode q ∈ Q. Note that for all i = 0,2, . . ., Pre∃(Wi ,q1 ) =Reachq1 (Pre∃(Wi ,q1 ),Pre∀(Wi ,q1 )).

It should be also noted that, as stated in Proposition 2 and 4 of [70] and Proposition 3.1,the sets Reach(Pre∃(Wi ),Pre∀(Wi )), Pre∃(Wi ) are closed and their volume decreases with thenumber of iterations (see Fig. 3.3), whereas Pre∀(Wi ) is open and its volume increases withthe number of iterations. The latter is not included in Fig. 3.3 since it extends to infinity.

3.4.2 Voltage stability of a single machine-load system

3.4.2.1 System description and mathematical modeling

We consider a standard single machine-load system, as shown in Fig. 3.4, including the dy-namics of the Automatic Voltage Regulator (AVR). E , E ′ and E f d denote the voltage at theload bus, the voltage behind the generator’s transient reactance, and the field excitation re-spectively. The voltage dynamic behavior for this network was studied in detail in [168], andit was assumed to be isolated from the frequency dynamics. The objective of the AVR con-trol loop is to regulate E f d at a specified reference value Er , using as feedback the measuredvalue of Eg . Following [168], the system is represented by a set of differential equations thatgovern the response of E ′, E f d and an algebraic equation that couples E ′ with E . The dif-ferential equation that describes the evolution of E ′ corresponds to a one axis generatormodel, whereas the equation for E f d is due to the first degree model that was used for thecontrol dynamics. The algebraic equation that couples E ′ with E emanates from the powerflow balance equations at every bus of the network. By solving the algebraic equation with

45


in Thm 2 of Gao et al. [2007]. The fact that Wk is thelargest viability kernel, follows from Lemma 5. �

3.3 Infinite time continuous evolution - Infinite number ofdiscrete transitions

This is the most general case, since both the number ofdiscrete transitions and the horizon of the continuous evo-lution may be infinite. For the discrete part of the hybridalgorithm the same procedure as in the previous case couldbe followed. It remains then to characterize the infinitetime version of the continuous operator. To achieve this,define as in Mitchell et al. [2005] the augmented input u =

(u, u) ∈ U × [0, 1], and consider the dynamics f(x, u, d) =uf(x, u, d). Note that since the horizon of the continuousevolution is allowed to be infinite, there is no need to aug-ment the state space with an additional “timer” as in the

previous subsections. Denote by V1(x) and V2(x) the infi-nite horizon value functions, which correspond to (5) and(7) respectively. To the interest of space, we will only pro-

vide the characterization of V1(x); the same for V2(x). The

infinite horizon value function V1(x) is given by V1(x) =

infα(·)∈A supd(·)∈D max{l(φ(t, x, α(·), d(·))),maxτ∈[0,t] h(

φ(τ, x, α(·), d(·)))}, for all t > 0. The map φ(·, x, α(·), d(·)) :[0,∞) → X denotes the trajectory of the augmented sys-tem, starting from the initial condition x with inputs α(·),d(·). Unlike the finite horizon case, V1(·) is not necessarilycontinuous.

Lemma 7. The function V1(x) is upper semicontinuous.

Proof. Consider an arbitrary x0 ∈ X . It suffices toshow that for all ε > 0 there exists δ > 0 such thatV1(x) − V1(x0) < ε for all |x − x0| < δ. By the defi-

nition of V1(x), for all ε > 0 there exists T ∈ [0,∞)

and α(·) ∈ A such that for all d(·) ∈ D, V1(x0) >

max{l(φ(T, x0, α(·), d(·))),maxτ∈[0,T ] h(φ(τ, x0, α(·), d(·))}−ε/4. Due to continuous dependence on initial conditions,there exists δ > 0 with |x − x0| < δ, such that for all

α(·) ∈ A and d(·) ∈ D, |maxτ∈[0,T ] h(φ(τ, x, α(·), d(·))) −maxτ∈[0,T ] h(φ(τ, x0, α(·), d(·)))| < ε/4 and |l(φ(τ, x, α(·),d(·)))− l(φ(τ, x0, α(·), d(·)))| < ε/4. But, by the definition

of V1(x) we have that V1(x) < supd(·)∈D max{l(φ(T, x, α(·),d(·))),maxτ∈[0,T ] h(φ(τ, x, α(·), d(·)))}. Hence, there existsd(·) ∈ D such that V1(x) < max{l(φ(T, x, α(·), d(·))),maxτ∈[0,T ] h(φ(τ, x, α(·), d(·)))} + ε/2. For d(·) = d(·)we can then distinguish two cases; namely V1(x) <

l(φ(T, x, α(·), d(·))), and V1(x) < maxτ∈[0,T ] h(φ(τ, x, α(·),d(·))). In both cases we end up with V1(x) − V1(x0) < ε,and conclude the proof. �Proposition 8. V1(x) is a viscosity solution of max{h(x)−V1(x),min{0, supd∈D infu∈U

∂V1

∂x (x)f(x, u, d)}} = 0.

Note that this is the stationary version of (6) for theinitial (not the augmented) system (see Margellos andLygeros [2011] for details). Under the convexity part of

Assumption 1, the proof that V1(x) is a viscosity solutionis similar to the finite horizon case Margellos and Lygeros[2011]. Unfortunately, it is easy to see that the viscosity

AVR

P + jQ

E

XE′

Xd, X′d

Er

Eg

Eg

Efd

Fig. 1. Single machine-load power system with AVR.

solution is not unique in this case. However, V1(x) isthe maximal upper semicontinuous viscosity solution. Theproof is analogous to that of Proposition 5 of Fialho andGeorgiou [1999], and is based on the comparison principlefor discontinuous viscosity solutions Bardi and Capuzzo-Dolcetta [2008].

Proposition 9. If there exists an upper semicontinuousviscosity subsolutionW (x) of the stationary equation such

that W (x) ≤ l(x) for all x ∈ X , then V1(x) ≥ W (x) for allx ∈ X .

4. CASE STUDY: VOLTAGE STABILITY OF ASINGLE MACHINE-LOAD SYSTEM

4.1 System description and mathematical modeling

We consider a standard single machine-load system, asshown in Fig. 2, including the dynamics of the AutomaticVoltage Regulator (AVR). The voltage dynamic behaviorfor this network was studied in detail in Venkatasubra-manian et al. [1992], and it was assumed to be isolatedfrom the frequency dynamics. That way, the system isrepresented by a set of differential equations that governthe response of E′, Efd and an algebraic equation thatcouples E′ with E. E, E′ and Efd denote the voltage atthe load bus, the voltage behind the generator’s transientreactance, and the field excitation respectively. By solvingthe algebraic equation with respect to E′, we get thefollowing system (see Venkatasubramanian et al. [1992]).

E =(− x− xd

Tdx′ g1(E) +xd − x′

d

Tdx′E2

g1(E)

+xd − x′

d

Td

Q(E)

g1(E)+

Efd

Td

)∂g1∂E

−1

, (9)

Efd = (−Efd + E0fd)u1 + (−g2(E) + Er)u2, (10)

where

g1(E) = E′ =1

E

√x′2(P 2 +Q(E)2) + 2x′Q(E)E2 + E4,

g2(E) = Eg =1

E

√x2(P 2 +Q(E)2) + 2xQ(E)E2 + E4,

Q(E) = Q0 +HE +BE2.

P , Q are the active and reactive parts of the load, x′ = x+x′d, whereas all undefined parameters are constants and

their values were retrieved from Venkatasubramanian et al.[1992]. A voltage dependent load was considered, usingH = B = 0.1. Variables u1 ∈ [0.1, 1], u2 ∈ [−10, 10]are gains treated as control inputs so as to regulate Efd

to its reference value Er. Note that (9) becomes singularalong ∂g1/∂E = 0, but this occurs for an unacceptablylow voltage value, outside the region of interest.

We consider the case that due to a fault, one of thetwo lines that connect the generator with the load is

Figure 3.4: Single machine-load power system with AVR.

q1 q2

x = f1(q1, x, u) x = f2(q2, x, u)

x3 < tf ∨x3 > tc − ε

x3 < tc + ε

tf − ε < x3 < tc

x3 > tcx := x

x := x

Fig. 2. Two-state hybrid automaton.

tripped at tf = 4s. The line closes then automaticallyafter the fault is cleared at tc = 5s. The overall systemcan be described by the two-state automaton of Fig. 3(no disturbance inputs in this case, although one couldconsider load, parameter uncertainty, etc.). Define x =

[x1 x2 x3]T

= [E Efd z]T ∈ R3, where z is a “timer”,

and is appended to the system dynamics to capture thetimed transitions between the two discrete modes. Thevector fields f1, f2 ∈ R3 represent (9),(10), augmentedwith x3 = 1, with the difference that the value of thereactance x is doubled once the line is tripped in mode q2.The hysteresis ε > 0 is added to the guard and domainconditions of the automaton, to avoid Zeno phenomenaand ensure that Assumptions 1.3, 1.5 are satisfied.

4.2 Viability problem and simulation results

Considering the hybrid automaton of Fig. 3, the mainobjective is to determine the set of initial operating con-ditions, from which the system trajectories can start, anddespite the line failure, there exists a control action suchthat the voltage remains within its safety limits both dur-ing the transient phase and after the reclosure of the line.A similar problem, but from a reachability perspective,was investigated in Suzuki and Hikihara [2007], Cross andMitchell [2008]. In these references the authors attemptedto identify the time that the voltage exceeds the safetymargins (i.e. voltage instability) after a fault. They repre-sented the system by an acyclic graph (no line reclosurewas considered), and hence a sequence of continuous cal-culations was applied instead.

The set W0 = {q1} × {x ∈ R3|0.9 ≤ x1 ≤ 1.1} ∪ {q2} ×{x ∈ R3|0.8 ≤ x1 ≤ 1.2} encodes the safety limits ofE. Using the Level Set Method Toolbox Mitchell [2002],we apply the viability algorithm, which reaches a fixedpoint after two iterations, since at most two transitionsmay occur. The continuous calculation at each mode wascarried out until the viability sets had saturated. V iabW0

is then illustrated in Fig. 4a. For any starting point in thelow “green” region of q1 there exists a control sequence,such that the corresponding trajectory remains safe untiltransiting to the safe part of q2 (“red”) at x3 = tf , andthen return to the upper “green” region of q1 at x3 = tc,remaining in W0 for ever while evolving continuously.

Fig. 4b shows how the volume of Pre∃(Wi), Pre∀(Wi), and

Reach(Pre∃(Wi), P re∀(Wi)), computed as the numberof grid points inside each set (normalized by the initialvolume with a 41×41×41 grid), changes at every iterationi = 1, . . . , 3 for each mode. As expected (see Proposition

2 & 4 of Gao et al. [2007]), Reach(Pre∃(Wi), P re∀(Wi)),

Pre∃(Wi) shrink, whereas the size of Pre∀(Wi) increaseswith the number of iterations.

5. CONCLUDING REMARKS

In this paper, the problem of computing viability sets forhybrid systems with competing inputs was revisited. Tothe best of our knowledge, this is the first complete charac-terization of the problem, based entirely on optimal controland the definition of executions of hybrid automata. Threedifferent cases, based on whether the horizon of the con-tinuous calculation and the number of discrete transitionswas finite or infinite, were considered, and the algorithmwas employed to comment on the voltage stability for asingle machine-load system in case of a line fault.

In future work, we plan to extend this approach to dealwith the probabilistic version of such problems. Moreover,the class of hybrid systems considered in this work, doesnot allow control inputs to force discrete transitions.Current work concentrates toward this direction.

REFERENCES

J. P. Aubin, J. Lygeros, M. Quincampoix, S. Sastry, andN. Seube. Impulse differential inclusions: A viabilityapproach to hybrid systems. IEEE Transactions onAutomatic Control, 47(1):2–20, 2002.

J.P. Aubin. Viability Theory. Boston:Birkhauser, 1991.M. Bardi and I. Capuzzo-Dolcetta. Optimal control andviscosity solutions of Hamilton-Jacobi-Bellman equa-tions. Boston Birkhauser, 2008.

E. Barron and H. Ishii. The Bellman equation for mini-mizing the maximum cost. Nonlinear Analysis: Theory,Methods and Applications, 13(9):1067–1090, 1989.

T. Basar and G. J. Olsder. Dynamic Noncooperative GameTheory. Academic Press, New York, 1982.

A. M. Bayen, I. M. Mitchell, M. Oishi, and C. Tomlin.Aircraft Autolander Safety Analysis Through OptimalControl-Based Reach Set Computation. Journal ofGuidance, Control, and Dynamics, 30(1):68–77, 2007.

P. Cousot and R. Cousot. Constructive versions of Tarski’sfixed point theorems. Pacific Journal of Mathematics,82(1):43–57, 1979.

E. Cross and I. Mitchell. Level set methods for com-puting reachable sets of systems with differential alge-braic equation dynamics. American Control Conference,pages 2260–2265, 2008.

B. Davey and H. Priestley. Introduction to Lattices andOrder. 2nd edition, Cambridge University Press, 2002.

L. Evans and P. Souganidis. Differential games and rep-resentation formulas for solutions of Hamilton-Jacobi-Isaacs equations. Indiana University of MathematicsJournal, 33(5):773–797, 1984.

I. J. Fialho and T. Georgiou. Worst Case Analysis ofNonlinear Systems. IEEE Transactions on AutomaticControl, 44(6):1180–1196, 1999.

Y. G. Gao, J. Lygeros, and M. Quincampoix. On theReachability Problem for Uncertain Hybrid Systems.IEEE Transactions on Automatic Control, 52(9):1572–1586, 2007.

M. Heymann, F. Lin, and G. Meyer. Synthesis andviability of minimally interventive legal controllers forhybrid systems. Discrete Event Dyn.Syst.: Theory andApplicat., 8(2):105–135, 1998.

K. Johansson, M. Egerstedt, J. Lygeros, and S. Sastry.Regularization of Zeno hybrid automata. Systems andControl Letters, pages 141–150, 1999.

Figure 3.5: Two-state hybrid automaton for the voltage control problem.

respect to E ′, we get the following system (see [168]).

E =(− X −Xd

Td X ′ g1(E)+ Xd −X ′d

Td X ′E 2

g1(E)+ Xd −X ′

d

Td

Q(E)

g1(E)+ E f d

Td

)∂g1

∂E

−1

,

E f d = (−E f d +E 0f d )u1 + (−g2(E)+Er )u2, (3.13)

where

g1(E) = E ′ = 1

E

√X ′2(P 2 +Q(E)2)+2X ′Q(E)E 2 +E 4,

g2(E) = Eg = 1

E

√X 2(P 2 +Q(E)2)+2XQ(E)E 2 +E 4,

Q(E) =Q0 +HE +BE 2.

P , Q are the active and reactive parts of the load, where the latter was assumed to be volt-age dependent. It consists of a constant power source Q0, a current source HE and animpedance load BE 2, where H ,B ∈ R are constant coefficients. Variables X , Xd , X ′

d denotethe transmission reactance, the generator’s d-axis reactance and transient reactance re-spectively, and X ′ = X + X ′

d . Td is the open-circuit transient time constant. For the voltagedependent load we considered H = B = 0.1, whereas variables u1 ∈ [0.1,1], u2 ∈ [−10,10]

are gains treated as control inputs (i.e. u = [u1 u2

]T) so as to regulate E f d to its reference

value Er . In this case no disturbance inputs are present, although one could consider load,parameter uncertainty, etc.. Numerical values for the remaining parameters were retrievedfrom [168]. Note that (3.13) becomes singular along ∂g1/∂E = 0, but this occurs for an un-acceptably low voltage value, outside the region of interest.

46

Case studies

We consider a case where due to a fault, one of the two lines that connect the generator withthe load is tripped at t f = 4s. The line closes then automatically after the fault is clearedat tc = 5s. The overall system can be described by the two-state automaton of Fig. 3.5.

Define x = [x1 x2 x3

]T = [E E f d z

]T ∈R3, where z is a “timer”, and is appended to the systemdynamics to capture the timed transitions between the two discrete modes. The vectorfields f1, f2 ∈ R3 represent (3.13), augmented with x3 = 1, with the difference that the valueof the reactance X is doubled once the line is tripped in mode q2. The hysteresis ε > 0 isadded to the guard and domain conditions of the automaton, to avoid Zeno phenomenaand ensure that Assumptions 3.1.3, 3.1.5 are satisfied.

3.4.2.2 Viability problem and simulation results

Considering the hybrid automaton of Fig. 3.5, the main objective is to determine the set ofinitial operating conditions, from which the system trajectories can start, and despite theline failure, there exists a control action such that the voltage remains within its safety lim-its both during the transient phase and after the reclosure of the line. A similar problem,but from a reachability perspective, was investigated in [158], [58]. In these references theauthors attempted to identify the time that the voltage exceeds the safety margins (i.e. volt-age instability) after a fault. They represented the system by an acyclic graph since no linereclosure was considered, and hence a sequence of continuous calculations was appliedinstead. In our case the system is effectively also acyclic due to the timing constraints ofthe fault, and could be represented by a three-mode automaton whose third mode wouldbe a sink state with the same continuous dynamics as the first one. Nevertheless, we keepthe representation of Fig. 3.5 to illustrate some of the properties of the iterative procedureoutlined in Algorithm 3.1.

The set W0 = {q1}× {x ∈R3|0.96 x1 6 1.1}∪ {q2}× {x ∈R3|0.86 x1 6 1.2} encodes the safetylimits of E . Using the Level Set Method Toolbox [125] (version 1.1) on MATLAB 7.10 (at anIntel(R) Core(TM)2 Duo 2.66GHz processor running Windows 7), we applied the viabilityalgorithm, which reached a fixed point after two iterations, since at most two transitionsmay occur (i.e. N = 2). The continuous calculation at each mode was carried out until theviability sets had saturated (i.e. T =∞). V i ab(N ,T )

W0is then illustrated in Fig. 3.6a. For any

starting point in the low “green” region of q1 there exists a control sequence, such that thecorresponding trajectory remains safe until transiting to the safe part of q2 (“red”) at x3 = t f ,and then return to the upper “green” region of q1 at x3 = tc , remaining in W0 for ever whileevolving continuously.

Fig. 3.6b shows how the volume of Pre∃(Wi ), Pre∀(Wi ), and Reach(Pre∃(Wi ),Pre∀(Wi )), com-puted as the number of grid points inside each set (normalized by the initial volume witha 41×41×41 grid), changes at every iteration i = 1, . . . ,3 for each mode. As expected (seeProposition 2 and 4 of [70] and Proposition 3.1), Reach(Pre∃(Wi ),Pre∀(Wi )), Pre∃(Wi ) shrink,whereas the size of Pre∀(Wi ) increases with the number of iterations.

47


(a)

0 1 2 30.3

0.4

0.5

0.6

0.7

0.8

0.9

1

1.1

1.2

1.3

# iterations

set v

olum

e

Pre∀q1

Pre∃q1

Reachq1

Pre∀q2

Pre∃q2

Reachq2

(b)

Figure 3.6: a) Hybrid discriminating kernel for each discrete mode. b) Volume of Pre∃(Wi ),Pre∀(Wi ), and Reach(Pre∃(Wi ),Pre∀(Wi )) for every iteration i = 1, . . . ,3 and each mode.


In this chapter, the problem of computing viability sets for hybrid systems with competinginputs was investigated. To the best of our knowledge, this is the first complete character-ization of the problem, based entirely on optimal control and the definition of executionsof hybrid automata. Three different cases, based on whether the horizon of the continuouscalculation and the number of discrete transitions was finite or infinite, were considered,and algorithm was applied to a numerical example and to the problem of voltage stabilityfor a single machine-load system in case of a line fault.

Although the results reported here, together with those in the literature address a wide va-riety of viability type problems for hybrid systems, there are still a few open issues. Thefirst is that no transitions forced by the inputs are allowed, hence a wide range of prob-lems is excluded. Moreover, it was shown that the viability algorithm terminates at someordinal number (see Section 3.2.2), but not necessarily the least one, which hampers theapplicability of the method. Finally, for the infinite horizon case, the value function is semi-continuous, and hence uniqueness is no longer guaranteed.

48

CHAPTER

4Reachability based 4D trajectorymanagement in air traffic control

4.1 Introduction

Air traffic is expected to increase rapidly over the coming decades. This increase is expectedto lead to further departure delays and en-route congestion [5], [8], which in turn mightcause additional safety problems and lead to an increased number of conflicts comparedto the current situation. A major issue when attempting to deal with this scenario is uncer-tainty about the future evolution of flights. The focus to reduce this uncertainty has shiftedtowards the so called 4D trajectory management, which is widely regarded as the basis forthe next generation in air traffic management systems envisioned by the Single EuropeanSky Air Traffic Management Research (SESAR) [3] and the Next Generation Air Transporta-tion System (NextGen) [8] projects.

The Contract-based Air Transportation System (CATS) research project [7] proposed onepossible implementation of the 4D trajectory management concept, called the Contractof Objectives (CoO) [4]. The CoO provides objectives for each actor involved in a flight(air traffic controllers, airports, airlines, air navigation service providers), which representtheir commitment to deliver a particular aircraft inside specific temporal and spatial con-straints called Target Windows (TWs) [10]. The hope is that the TWs will help to increasepredictability, punctuality and safety during the flight. A detailed methodology for the com-putation of the TWs based on the constraints imposed by the various actors can be foundin [10], and their importance from an airline perspective is investigated in [54].

The presence of TWs could potentially increase predictability, but also imposes additionalconstraints that air traffic controllers should respect when issuing conflict resolution ma-neuvers. TWs therefore represent a trade-off between the predictability of flights (whichtends to make the task of air traffic controllers simpler) and their maneuverability (whichtends to make air traffic control more complex). As part of the validation effort of the CATSproject this trade-off was investigated through Human in the Loop simulations (HIL) [9],

49

Reachability based 4D trajectory management in air traffic control

[6]. Even though HIL simulations provide the most realistic way for validation of the overallconcept, they have several limitations, most notably their inability to quantitatively assessthe probability of rare events such as conflicts.

In this work we conduct a complementary, computational study of this trade-off betweenpredictability and maneuverability achieved through TWs. We first use Monte Carlo simula-tions to assess the probability of flights meeting their TW constraints and the probability ofconflict under TW operation in the realistic scale set-up used in the CATS HIL experiments.Monte Carlo simulations have been used extensively in air traffic control for optimizationpurposes [90], safety verification [31], [30], and to estimate the conflict probability due towind uncertainty [55], [139], [140], [75], [146]. Even though highly sophisticated MonteCarlo methods [31] are not necessary in our case since the events of interest have substan-tial probability, deployment of Monte Carlo methods still requires novel developments suchas a feedback controller to emulate the actions air traffic controllers and pilots might taketo meet TW constraints. Our results indicate that TW tracking is feasible even with the cur-rent fleet of aircraft which are mostly equipped with the so-called 3D flight managementsystems that do not automatically correct for timing deviations.

To assess the maneuvering freedom afforded by TWs, we use the reach-avoid frameworkof Chapters 2 and 3, highlighting the applicability of the theoretical results to a realisticcase study [118]. In this context, following the work of [112], [114], perform a reach-avoidcomputation to construct conflict free tubes. We then manually place an artificial TW insideeach tube to initiate a resolution maneuver for each aircraft. Although not implemented inthis work, an optimization step could be included here to place the additional TWs in anoptimal way and avoid situations where a subsequent conflict might occur. Automationin this process is not the main purpose of this study though, which aims to illustrate howthe maneuvering bounds determined by the reach-avoid tubes could be used as a decisionsupport tool for air traffic controllers. The bounds can also serve as feedback to the TWgeneration process [10], to design less conservative TWs.

The chapter is organized in five sections. Section 4.2 contains details regarding the mod-eling of the aircraft, the TWs and the TW tracking controller. In Section 4.3, our simplifiedassumptions for the hybrid abstraction of the aircraft dynamics are stated, and the 4D tra-jectory management problem is linked to the reachability framework of Chapter 2. Section4.4 summarizes the simulation results obtained from the TW concept evaluation and theconflict reachability calculations. Finally, in Section 4.5 we provide some concluding re-marks and directions for future work.

4.2 Mathematical modeling

4.2.1 Simulation environment

Both the continuous dynamics for the aircraft motion and the discrete events triggered bythe FMS and the flight plan, are described in detail in [1]. Using this as a starting point,

50

Mathematical modeling

[105], [73], [102] develop a stochastic hybrid model for the flight dynamics. Following [1],we consider a six-state, flat earth, trimmed, point mass model for the aircraft dynamics.

xyzVψ

m

=

V cosψcosγ+wx

V sinψcosγ+wy

V sinγ+wz

−CD Sρ(z)2

V 2

m − g sinγ+ Tm

CLSρ(z)2

Vm sinφ

−ηT

. (4.1)

The states are the cartesian coordinates x and y , the altitude z, the true airspeed V (i.e. thespeed of the aircraft relative to the surrounding air), the heading angle ψ, and the mass m.The aerodynamic lift and drag coefficients CD and CL , the surface area of the wings S, thefuel flow coefficient η, and the air density ρ(h) are obtained from the Base of Aircraft Data

(BADA) [1]. Let[x1 x2 x3 x4 x5 x6

]T = [x y z V ψ m

]Tdenote the state variables and notice

that the aircraft engine thrust T , the flight path angle γ, and the bank angleφ, represent thecontrol inputs in the above set of equations, which, due to aerodynamic limitations, haveto satisfy constraints of the form T 6 T 6 T , γ6 γ6 γ, ϕ6ϕ6ϕ. The wind speed vector

is denoted by d = [wx wy wz

]T ∈R3.

To ensure that the aircraft does not stray too far off its reference path in the x − y coordi-nates, the bank angle φ is set based on the heading error and the cross track deviation ofthe aircraft from the reference path. In [103], [104] a nonlinear controller was developed toemulate the lateral corrective actions of the aircraft flight management system. To counter-act for along track errors, we also implement here a Time of Arrival controller, as describedin Section 4.2.3.

The stochastic nature of the model developed in [103], [104], enters through the wind com-ponent d ∈ R3. The wind that an aircraft encounters comprises a deterministic compo-nent, available through meteorological forecasts, and a stochastic wind prediction error.The wind prediction error that an aircraft encounters is correlated to the wind experiencedby all other aircraft at a given time instance, but also at earlier and later times. By takingthis into account, realistic weather scenarios can be generated as described in [103], [104].

The flight management system determines the control input according to different discretemodes [1]. In the context of this paper, the main task of the FMS of each aircraft j = 1, . . . , N ,is to track the flight plan, which can be thought of as a sequence of way points. Way points

are characterized by their three dimensional coordinates O(i , j ) =[

Ox(i , j ) Oy

(i , j ) Oz(i , j )

]T ∈ R3+,where i = 1, ..., M j . They define M j −1 straight line flight segments, and give rise to a dis-crete state i , which stores the segment of the flight that aircraft j is currently in. The de-cision making logic of the FMS also involves additional discrete states whose dynamicsare expressed in terms of finite state machines. One example is the flight level, which ischaracterized by the altitude z, and determines the nominal airspeed of the aircraft Vnom .Additionally, the flight phase (cruise, climb, descent) determines the thrust input T , which

51


Adjacent TW

Superimposed TW

sector I sector II

sector III sector IV

(a) (b)

Figure 4.1: a) Superimposed and Adjacent TWs placed at the boarders between verticallyand horizontally separated control sectors respectively. b) Two flights with their TWs. Thefirst TW of the “green” flight is superimposed, whereas all the others are adjacent.

is set accordingly, to track the nominal airspeed Vnom . Further discussion and a detaileddescription of these and other discrete modes of the FMS is given in [1].

4.2.2 From way points to TWs

TWs can be thought of as space rectangles that each aircraft must hit within a specifiedtime interval. Following the CATS concept of operations [10] we assume that the TWs arelocated on the boundary between air traffic control sectors, airspace regions where a dif-ferent control authority (i.e. air traffic controller) is responsible for the safe and efficientmanagement of flights. If the sectors are superimposed vertically the TWs are called super-imposed, otherwise they are called adjacent (Fig. 4.1a). In Fig. 4.1b two sample flights withtheir TWs, from the CATS HIL experiments used in our validation simulations below, are de-picted. Motivated by the procedures adopted for the CATS HIL simulations, we introducetwo assumptions.

1. The center of every TW is always a waypoint in the flight plan.

2. TWs do not overlap, in space and time.

4.2.3 TW tracking controller

A Time of Arrival controller was designed to regulate along track errors and enable the sim-ulated aircraft to meet the timing constraints imposed by their TWs. Our design is inspiredby [78], [79], and can be found in detail in [96]. It aims to mimic the potential actions that airtraffic controllers or pilots may take to track TWs in the simulation environment of Section

52

Determining the limits of maneuverability using reachability

4.2.1. As such it should not be considered as an attempt to design a full scale 4D FMS. Ourchoices are partly dictated by observations of air traffic controllers and pilots that took partin the CATS HIL experiments [9]. The output of the proposed control scheme is the sum ofthe nominal speed (determined by the flight level and retrieved from [1]) and a correctionterm

Vd (t ) =Vnom +K e, (4.2)

where e denotes the tracking error as a function of the continuous state vector of the sys-tem, as well as the wind information and the flight plan. Variable Vd represents the desiredspeed that the thrust command will try to track. For simplicity, in the computation of thetracking error we assume that the current airspeed and wind speed is maintained constantthroughout the remaining part of the flight path until the following TW. Similar assump-tions hold also for the current wind speed value. The tracking error is then defined as

e = dr − (td − t )(x4 cos(x5 −θ1)cos

(θ2 −γ

)+wp), (4.3)

where dr denotes the distance to the center of the next TW along the flight plan, td is thedesired time of arrival, which in this case was considered to be the middle of the TWs timeinterval, and K is a gain. The term x4 cos(x5 −θ1)cos

(θ2 −γ

)is the projection of the speed

x4 along the flight plan (see Fig. 4.2), and θ1 and θ2 are defined as

θ1 = tan−1

Oy

(i+1, j)−x2

Ox(i+1, j)−x1

,

θ2 = tan−1

Oz(i+1, j)−x3√(

Ox(i+1, j)−x1

)2 +(Oy

(i+1, j)−x2

)2

.

The wind speed vector wp is similarly defined as the projection of the wind speed vectoralong the flight plan.

4.3 Determining the limits of maneuverability usingreachability

4.3.1 Reach-Avoid problem characterization - Extension to timedependent state constraints

The main objective is to determine the set of initial conditions from which aircraft can startand reach their TWs, while avoiding conflict with other aircraft. To achieve this, we showin Section 4.3.3 that it is required to perform the two reach-avoid computations defined inChapter 2. The “avoid” region is the area where an aircraft might be in conflict with other

53


8

x2

x3

x1

x4 cos(x5 − θ1)

x4

x4 cos(x5 − θ1) cos(θ2 − γ)

θ1

θ2

O(i+1,j)

O(i,j)

drTW center

Fig. 2. Projection of the speed x4 along the flight plan segment O(i,j) − O(i+1,j). Variable dr denotes the distance to the

center of the next TW along the flight plan

A. Reach-Avoid problem characterization

We first consider the case of static “avoid” sets, which encode state constraints that do not

evolve with time. Motivated by [17], consider the continuous time system

x = f(x, u, v), (4)

with x ∈ Rn, f(·, ·, ·) : Rn × U × V → Rn, and u ∈ U ⊆ Rm, v ∈ V ⊆ Rp 1. Let U[t,t′], V[t,t′]

denote the set of Lebesgue measurable functions from the interval [t, t′] to U, and V respectively,

and let T ≥ 0 to be an arbitrary time horizon. Consider also the functions l(·) : Rn → R and

h(·) : Rn → R, that will be used to characterize the “reach” and the “avoid” set respectively.

Assumption 1. The sets U ⊆ Rm and V ⊆ Rp are compact. The functions f(x, u, v), l(x) and

h(x) are bounded, Lipschitz continuous in x, and continuous in u and v.

For t ∈ [0, T ], u(·) ∈ U[t,T ] and v(·) ∈ V[t,T ], let φ(·, t, x, u(·), v(·)) : [t, T ] → Rn denote the

solution to (4), which under Assumption 1, is unique for each initial state x. Following [36],

[52], we restrict the first player to play non-anticipative strategies. A non-anticipative strategy is

1Note that x in this case is the state vector and should not be related to the cartesian coordinate defined in the previous

section. Throughout the paper it will always be clear from the context to which x we refer.

July 11, 2012 DRAFT

Figure 4.2: Projection of the speed x4 along the flight plan segment O(i , j )−O(i+1, j ). Variabledr denotes the distance to the center of the next TW along the flight plan

aircraft, and therefore evolves with time. Following [172], the reach-avoid framework ofChapter 2 can be extended to capture cases where the state constraints evolve with time.For t ∈ [0,T ] consider the time dependent “avoid” set At ⊆ Rn , where At characterizes theregion of the state space, through which system trajectories should not pass at time t . Wecan now augment the state equations with a “timer” z, and define the new state vector1

x z = (x, z) ∈Rn+1 with f z = ( f ,1) ∈Rn+1. In the augmented space, the “avoid” region can bedefined as

A = ⋃t∈[0,T ]

At × {t },

allowing us to use the formulation of Chapters 2 and 3, with x z in place of x, and h(·) :Rn+1 → R to characterize A. Note that x z 6∈ A is equivalent to x(t ) 6∈ At for all t ∈ [0,T ]. Forthe rest of the paper we will use h(x, t ) to make this dependency explicit.

4.3.2 Model abstraction

The aircraft and FMS model described so far, is adequate for simulation purposes, but itis computationally expensive to analyze. Most of the reachability numerical methods arebased on gridding the state space, so the memory and time necessary for the computa-tion grow exponentially in the state dimension. Therefore, using a full six-state, point massmodel of the aircraft, like the one described in Section 4.2.1, would not be feasible. To makethe reachability computations tractable, we perform a series of simplifications:

1. We eliminate the speed equation from (4.1), and use V as a control input. Using air-craft parameter values for an airbus A330 cruising at 35000ft (taken from [1]), an air-craft would need ∼ 16 seconds to change from the minimum to the maximum value

1Note that x in this case is the state vector and should not be related to the cartesian coordinate definedin the previous section. Throughout this section it will always be clear from the context to which x we refer.

54


of the considered speed envelope (±10% of the nominal one) under full thrust. Inthis time it would cover a distance of less than 2km which is relatively small com-pared to the time scale (∼ 25 minutes) and distances (∼ 350km) of the reachabilitycalculations. This observation is also supported by a study into the time scale sepa-ration of flight dynamics [83], where speed is used as control variable in reachabilitycalculations involving position variables (albeit for a different class of flight vehicleswith very similar dynamics). In specific cases, however, where the outcome of thereachability calculations needs to be refined, we could apply our methodology with-out performing this simplification.

2. We assume that the aircraft perfectly track the flight plan laterally. This is not unrea-sonable since cross-track errors are in general much smaller than along-track errors[55]; indeed modern aircraft with an advanced flight management system laterallytrack their flight plan to within ±1nmi for 95% of the time [2]2. For the simulationmodel outlined in Section 4.2 this is also justified by inspection of Fig. 4.3, whichshows a segment of 1000 simulated trajectories for one of the flights of Fig. 4.11, cor-responding to different uncertainty realizations, as these will be defined in the nextsection (±1nmi lateral accuracy [2]).

In the notation of [70], the dynamics of aircraft j can be modeled by a hybrid automatonH j = (X j ,Q j , Init j , f j ,Dom j ,G j ,R j ), with

• continuous states x j =[s j z j t

]T ∈R3+ = X j .

• discrete states i ∈ {0, ..., M j −1} =Q j .

• control inputs u j =[b j γ j

]T ∈ [−1,1]× [−γ j ,γ j]=U j .

• disturbance inputs d = [wx wy wz

]T ∈R3 = D.

• vector field f j : Q j ×X j ×U j ×D → X j ,

f j (i , s j , z j , t ,u j ,d) =

s j

z j

t

=

(1+0.1b j )g (z j ,γ j )cosγ j +wx cosΨ(i , j ) +wy sinΨ(i , j )

(1+0.1b j )g (z j ,γ j )sinγ j +wz

1

.

• domain Dom j = {(i , s j , z j , t ) | s j 6 d(i , j )}.

• guards G j (i , i +1) = {(s j , z j , t ) | s j > d(i , j )}.

• reset map R j (i , i +1, s j , z j , t ) = {(0, z j , t )}.

The schematic diagram of Fig. 4.4, illustrates the modes of the hybrid automaton for thesimplified aircraft model.

2Note that 1nmi = 1852m, and 1ft = 0.305m.

55


-7 -6.5 -6 -5.5 -5 -4.5 -4 -3.5 -3

x 105

-8

-7.5

-7

-6.5

-6

-5.5

-5

-4.5

-4x 10

5

x (m)

y(m

)

-4.14 -4.12 -4.1 -4.08 -4.06 -4.04 -4.02

x 105

-6.05

-6

-5.95

-5.9

-5.85

-5.8x 10

5

x (m)

y(m

)

Figure 4.3: Flight track projections on the x − y axis for one of the flights of Fig. 4.11, for1000 different uncertainty realizations (wind, mass and entry time uncertainty, as these aredefined in Section 4.4.2). The “black” line is the flight plan, whereas the “red” dotted linescorrespond to ±1nmi Required Navigation Performance values.

13

⎡⎢⎢⎣

sj

zj

t

⎤⎥⎥⎦=fj(i, xj , uj , d)

⎡⎢⎢⎣

sj

zj

t

⎤⎥⎥⎦=fj(i, xj , uj , d)

⎡⎢⎢⎣

sj

zj

t

⎤⎥⎥⎦=fj(i, xj , uj , d)

sj := 0 sj := 0sj := 0

segment 1 segment 2 segment 3

sj ≥ d(1,j) sj ≥ d(2,j)

. . .

sj ≤ d(1,j) sj ≤ d(2,j) sj ≤ d(3,j)

Fig. 4. Hybrid automaton for the simplified aircraft model. Each discrete mode corresponds to the flight segment that the

aircraft is currently in.

can defineΨ(i,j) = tan−1

(Oy

(i+1,j)−Oy

(i,j)

Ox(i+1,j)

−Ox(i,j)

),

Γ(i,j) = tan−1(

Oz(i+1,j)

−Oz(i,j)

d(i,j)

),

where d(i,j) =√

(Ox(i+1,j) −Ox

(i,j))2 + (Oy

(i+1,j) − Oy(i,j))

2 is the length of the projection of

segment i on the horizontal plane. Since we assume that lateral tracking is perfect, it suffices to

track the distance of each segment covered on the horizontal plane, denoted by sj ∈ R+ in Fig.

5. The cartesian coordinates can be then computed by⎡⎣x(i,j)(sj)y(i,j)(sj)

⎤⎦ =

⎡⎣O

x(i,j)

Oy(i,j)

⎤⎦+

⎡⎣cosΨ(i,j)

sinΨ(i,j)

⎤⎦ sj.

To approximate accurately the physical model, the flight path angle γj is fixed according to

the angle Γ(i,j) that the segment forms with the horizontal plane. If Γ(i,j) = 0 the aircraft will

be cruising (γj = 0) at that segment, whereas if it is positive or negative it will be climbing

(γj ∈ [0, γj ]) or descending (γj ∈ [−γj , 0]) respectively.

As discussed in Section II.A, the nominal airspeed of the aircraft depends on the flight level

and whether the aircraft is climbing, cruising or descending. We approximate this dependence

by a linear interpolation of the speed-altitude values of [43], denoted by the function g(zj, γj).

In general, aircraft fly faster at higher altitudes, hence g(·, γj) is a non-decreasing function. For

our simulations, assume that the actual airspeed (treated as a control input in the simplified

model) is allowed to vary within ±10% of the nominal one; this is reflected by the control input

bj ∈ [−1, 1].

August 28, 2012 DRAFT

Figure 4.4: Hybrid automaton for the simplified aircraft model. Each discrete mode corre-sponds to the flight segment that the aircraft is currently in.

For each aircraft j , Ψ(i , j ) denotes the angle that the segment i forms with the x axis, andΓ(i , j ) the flight path angle that it forms with the horizontal plane (Fig. 4.5). For i = 1, ..., M j −1 we can define

Ψ(i , j ) = tan−1(

Oy(i+1, j )−O

y(i , j )

Ox(i+1, j )−Ox

(i , j )

),

Γ(i , j ) = tan−1(

Oz(i+1, j )−Oz

(i , j )

d(i , j )

),

56


13

way point

Global Coordinate Frame

xj

yj

O(i−1,j)

O(i+1,j)

O(i,j)

Ψ(i,j)

d(i,j)sj (x(i,j)(sj), y(i,j)(sj))

ith segment

(a)

way point

ith segment

O(i−1,j)

O(i+1,j)

O(i,j)

zj

zj

sj

Γ(i,j)

(b)

Fig. 4. a) Flight plan projection on the horizontal plane. The aircraft does not deviate from the nominal flight plan, due to the

simplifying assumption of constant heading angle at each segment. b) Flight plan projection on the z-s plane.

Apart from sj , the other two continuous states are the altitude zj , and the time t. The last

equation was included in order to track the TW temporal constraints. As already stated, the wind

speed v is assumed to act as a bounded disturbance with −v ≤ v ≤ v. For our simulations we

considered v = 12m/s, corresponding to 3σ of the values in [46]. Since the flight path angle γj

does not exceed 5◦, for simplicity we can assume that sin γj ≈ γj and cos γj ≈ 1.

The constraint set for each aircraft j (corresponding to the TW located at O(i,j)) can be then

defined as Kj = (d(i,j), [z(i,j)+z(i,j), z(i,j)+z(i,j)], [tj, tj ]) if the TW is adjacent (z(i,j), z(i,j) denote

the minimum and maximum TW’s height), and Kj = ([d(i,j)+s(i,j), d(i,j)+s(i,j)], z(i,j), [tj , tj]) if

the TW is superimposed (s(i,j), s(i,j) denote the minimum and maximum TW’s horizontal length).

By tj and tj we denote the temporal extremities of the TWs. Note that using this abstracted

version of the model, TWs are represented as line segments.

C. Reach-avoid tubes for TW tracking and conflict avoidance

In air traffic, conflict refers to the loss of minimum separation between two aircraft. Each

aircraft is surrounded by a protected zone, which is generally thought of as a cylinder of radius

5nmi and height 2000ft centered at the aircraft1. If this zone is violated by another aircraft, then

a conflict is said to have occurred.


February 12, 2012 DRAFT

(a)

13

way point

Global Coordinate Frame

xj

yj

O(i−1,j)

O(i+1,j)

O(i,j)

Ψ(i,j)

d(i,j)sj (x(i,j)(sj), y(i,j)(sj))

ith segment

(a)

way point

ith segment

O(i−1,j)

O(i+1,j)

O(i,j)

zj

zj

sj

Γ(i,j)

(b)

Fig. 4. a) Flight plan projection on the horizontal plane. The aircraft does not deviate from the nominal flight plan, due to the

simplifying assumption of constant heading angle at each segment. b) Flight plan projection on the z-s plane.

Apart from sj , the other two continuous states are the altitude zj , and the time t. The last

equation was included in order to track the TW temporal constraints. As already stated, the wind

speed v is assumed to act as a bounded disturbance with −v ≤ v ≤ v. For our simulations we

considered v = 12m/s, corresponding to 3σ of the values in [46]. Since the flight path angle γj

does not exceed 5◦, for simplicity we can assume that sin γj ≈ γj and cos γj ≈ 1.

The constraint set for each aircraft j (corresponding to the TW located at O(i,j)) can be then

defined as Kj = (d(i,j), [z(i,j)+z(i,j), z(i,j)+z(i,j)], [tj, tj ]) if the TW is adjacent (z(i,j), z(i,j) denote

the minimum and maximum TW’s height), and Kj = ([d(i,j)+s(i,j), d(i,j)+s(i,j)], z(i,j), [tj , tj]) if

the TW is superimposed (s(i,j), s(i,j) denote the minimum and maximum TW’s horizontal length).

By tj and tj we denote the temporal extremities of the TWs. Note that using this abstracted

version of the model, TWs are represented as line segments.

C. Reach-avoid tubes for TW tracking and conflict avoidance

In air traffic, conflict refers to the loss of minimum separation between two aircraft. Each

aircraft is surrounded by a protected zone, which is generally thought of as a cylinder of radius

5nmi and height 2000ft centered at the aircraft1. If this zone is violated by another aircraft, then

a conflict is said to have occurred.


February 12, 2012 DRAFT

(b)

Figure 4.5: a) Flight plan projection on the horizontal plane. The aircraft does not deviatefrom the nominal flight plan, due to the simplifying assumption of constant heading angleat each segment. b) Flight plan projection on the z-s plane.

where d(i , j ) =√

(Ox(i+1, j ) −Ox

(i , j ))2 + (Oy

(i+1, j ) −Oy(i , j ))

2 is the length of the projection of seg-

ment i on the horizontal plane. Since we assume that lateral tracking is perfect, it sufficesto track the distance of each segment covered on the horizontal plane, denoted by s j ∈ R+in Fig. 4.5. The cartesian coordinates can be then computed by

[x(i , j )(s j )y(i , j )(s j )

]=

[Ox

(i , j )

Oy(i , j )

]+

[cosΨ(i , j )

sinΨ(i , j )

]s j .

To approximate accurately the physical model, the flight path angle γ j is fixed according tothe angle Γ(i , j ) that the segment forms with the horizontal plane. If Γ(i , j ) = 0 the aircraft willbe cruising (γ j = 0) at that segment, whereas if it is positive or negative it will be climbing(γ j ∈ [0,γ j ]) or descending (γ j ∈ [−γ j ,0]) respectively.

As discussed in Section 4.2.1, the nominal airspeed of the aircraft depends on the flightlevel and whether the aircraft is climbing, cruising or descending. We approximate thisdependence by a linear interpolation of the speed-altitude values of [1], denoted by thefunction g (z j ,γ j ). In general, aircraft fly faster at higher altitudes, hence g (·,γ j ) is a non-decreasing function. For our simulations, assume that the actual airspeed (treated as acontrol input in the simplified model) is allowed to vary within ±10% of the nominal one;this is reflected by the control input b j ∈ [−1,1].

Apart from s j , the other two continuous states are the altitude z j , and the time t . The lastequation was included in order to track the TW temporal constraints. As already stated,the wind speed d is assumed to act as a bounded disturbance with −d 6 d 6 d . For oursimulations we considered d = 12m/s, corresponding to 3σ of the values in [102]. Since theflight path angle γ j does not exceed 5◦, for simplicity we can assume that sinγ j ≈ γ j andcosγ j ≈ 1.

57


The constraint set for each aircraft j (corresponding to the TW located at O(i , j )) can be thendefined as K j = (d(i , j ), [z(i , j ) + z(i , j ), z(i , j ) + z(i , j )], [t j , t j ]) if the TW is adjacent (z(i , j ), z(i , j )

denote the extremities of the TW in the vertical direction), and K j = ([d(i , j ) + s(i , j ),d(i , j ) +s(i , j )], z(i , j ), [t j , t j ]) if the TW is superimposed (s(i , j ), s(i , j ) denote the extremities of the TW

along the s direction). By t j and t j we denote the temporal extremities of the TWs. Notethat using this abstracted version of the model, TWs are represented as line segments. Thisis used only for the reachability calculations of Section 4.4.3, whereas for the Monte Carloanalysis of Section 4.4.2, as well as for the CATS HIL experiments the initial TW representa-tion is employed.

4.3.3 Reach-avoid tubes for TW tracking and conflict avoidance

In air traffic, conflict refers to the loss of minimum separation between two aircraft. Eachaircraft is surrounded by a protected zone, which is generally thought of as a cylinder ofradius 5nmi and height 2000ft centered at the aircraft. If this zone is violated by anotheraircraft then a conflict is said to have occurred.

We now show how the problem of meeting TW constraints while avoiding conflicts canbe formulated as a reach-avoid problem. Note that, even though the simplified model isstrictly speaking hybrid, the discrete modes are visited sequentially. Hence, we can per-form a sequential calculation involving only the continuous dynamics, treating the initialset computed for each flight plan segment as the terminal condition for the previous seg-ment. In the opposite case, the assumptions and algorithm of Chapter 3 should be fol-lowed instead. For the rest of the paper we will refer to the computed sets as reachabletubes3. In the process we show how the state variable introduced to keep track of tem-poral constraints can in fact be analytically eliminated due to the structure of TWs, givingrise to an equivalent but computationally simpler reach-avoid problem. Similar to the def-inition of the set K j , we define for each aircraft j the spatial constraints of a TW centeredat the way point O(i , j ) as R j = (d(i , j ), [z(i , j ) + z(i , j ), z(i , j ) + z(i , j )]) if the TW is adjacent, and

R j = ([d(i , j ) + s(i , j ),d(i , j ) + s(i , j )], z(i , j )) if the TW is superimposed. Let also [t j , t j ] denote the

temporal extent of R j .

Stage 0: For each aircraft j apply the procedure outlined in Stage 1 and 2 without “avoid”regions. We can thus compute the reachable tubes for each aircraft and identify for everytime instance the states that correspond to conflicting situations. Specifically, for t ∈ [t j , t j ],

let A j i ,t denote all states x j for which the aircraft j is in conflict with another aircraft i 6= j attime t , and use h j i (x j , t ) to characterize this set. We can then define the obstacle functionh j (x j , t ) = maxi 6= j h j i (x j , t ) to characterize the “avoid” region A j ,t = ⋃

i 6= j A j i ,t , capturingthe case of multiple conflicts. Similar calculations for t 6 t j give rise to the “avoid” regionA j ,t . Note that by defining any region of conflict as an “avoid” set for all involved aircraft

3Using the terminology of [121] the outcome of the first stage of the proposed methodology should bereferred to as reachable tubes, whereas the outcome of the second stage should be referred to as reachablesets.

58


may be conservative; if an automated conflict resolution procedure is employed the con-servatism of the resulting solution may be reduced.

Stage 1: Compute for each aircraft j the set R j of states x j at time t j (beginning of TW)from which there exist a nonanticipative strategy for the control input that for all wind re-alizations can lead the aircraft inside R j at least once within the time interval [t j , t j ], whileavoiding conflict with other aircraft. This is a reach-avoid at any time calculation. The cor-responding set R A j (t , R j , A j ,t ) is the zero sublevel set of V , which is the solution of

max{h j (x j , t )− V (x j , t ),∂V

∂t(x j , t )+min{0, Hi j (p, x j )}} = 0,

where

Hi j (p, x) = supd∈D

infu j∈U j

(p1(1+0.1b j )g (z j ,γ j )

+p2(1+0.1b j )g (z j ,γ j )γ j +p1 cosΨ(i , j )wx +p1 sinΨ(i , j )wy +p2wz

),

is the Hamiltonian of the system, with u j =[b j γ j

]T and v = [wx wy wz

]T . Since TWs donot overlap, the terminal condition is V (x j , t j ) = l j (x j ). The function l j (·) can be set equalto the signed distance to the set Rc

j , i.e.

l (x j ) ={ infx j∈R j

|x j − x j |, if x j ∈ Rcj

− infx j∈R j|x j − x j |, if x j ∈ R j

.

Similarly, h j (·, t ) is defined to be the signed distance to the set A j ,t that was computed atStage 0. The functions l j (·) and h j (·, t ) are Lipschitz by construction; to ensure that they arebounded and satisfy Assumption 1, we saturate them at their Lipschitz constants by settingthem equal to their Lipschitz constant wherever they are greater than this value and minusthe Lipschitz constant wherever they are less than this value. For the numerical implemen-tation this is not an issue since the computations are performed over compact sets.

Stage 2: Compute for each aircraft j the set of all states at time t 6 t j for which there exists anonanticipative strategy for the control input such that for all wind realizations the systemcan reach the set R j (determined at Stage 1) at time t j , while avoiding conflict with otheraircraft. Based on the analysis of Section 2.2, this is a reach-avoid at the terminal time setR A j (t ,R j , A j ,t ), and can be computed by solving

max{h j (x j , t )−V (x j , t ),∂V

∂t(x j , t )+Hi j (p, x j )} = 0,

with terminal condition V (x j , t j ) = max{V (x j , t j ),h j (x j , t )}. Based on the computation of

Stage 1, the set R j = {x j ∈ Rn | V (x j , t j )) 6 0}, whereas A j ,t depends once again on the ob-

stacle function h j (x j , t ), and is defined similarly to A j ,t .

The optimal control and disturbance inputs needed in the calculation can be analyticallycomputed by inspecting the Hamiltonian of the system for each segment i . The optimal

59


value for wx is therefore given by

w∗x =

{w x if p1 cosΨ(i , j ) > 0,

−w x if p1 cosΨ(i , j ) < 0.

In a similar way we can define w∗y , and w∗

z . Likewise, and since (1+0.1b j )g (z j ,γ j ) > 0, wehave

γ∗j =

γ j if (p2 6 0∧Γ(i , j ) > 0),

0 if (p2 > 0∧Γ(i , j ) > 0)∨ (p2 6 0∧Γ(i , j ) < 0),−γ j if (p2 > 0∧Γ(i , j ) < 0).

b∗j =

{1 if p1 +p2γ

∗j 6 0,

−1 if p1 +p2γ∗j > 0.

For the case where Γ(i , j ) = 0 (cruising), b∗j = 1 if p1 6 0 and b∗

j = −1 if p1 > 0, where p =∂V /∂x j is the costate vector.

The steps of the reach-avoid computation for each aircraft j are summarized in the Algo-rithm 4.1 of Appendix A.2.

4.4 Simulation results

4.4.1 Simulation set-up

We consider 50 flights with 117 TWs whose mean temporal size is ∼ 7 minutes, extractedfrom the HIL experiments conducted within the CATS project [9]. The HIL experimentsconcentrated on the interface between the Geneva and Milan control centers. Each of theflights used comes with a flight plan, coordinates for the TW extremities and the time widthof each TW. For illustrative purposes Fig. 4.6 gives an overview of the simulated flights andthe geographical area of interest.

In [6], no Time of Arrival controller was used, whereas four different sources of uncertaintyare considered and their effect on the TW size and the probability of conflict was analyzedby means of Monte Carlo simulations. Specifically, the sources of uncertainties consideredwere:

1. Wind uncertainty. We used the weather forecast error statistics of [56], as encoded forsimulation in [103], [104].

2. Aircraft mass uncertainty. We assumed a gaussian distribution, with mean equalto the nominal aircraft mass mnom provided by BADA [1] and standard deviationσ= min{mmax −mnom ,mnom −mmin}/3. All samples outside the minimum and max-imum value of the aircraft mass (mmin and mmax respectively, based on BADA) werediscarded.

60

Simulation results

30° E

Figure 4.6: Flight plan overview in the simulated airspace, of the 50 simulated flights ex-tracted from the HIL experiments.

3. Entry time uncertainty. We considered a uniform distribution between ±1 minutefrom the center of the temporal interval of each aircraft’s entry TW. Note that therange of values for the entry time uncertainty is less than the width of many TWs,to take implicitly into account the effect of speed adjustments by the air traffic con-trollers.

4. Nominal speed uncertainty. We assumed a uniform distribution between ±6% of thenominal speed value (not the actual one as in Section 4.3.2), provided by BADA [1].This range of values was dictated by the air traffic controllers that participated in theCATS HIL experiments. The extracted value was kept constant for every minute, andthen a new sample was generated. It provides a naïve way of representing speed ad-justments that air traffic controllers and/or pilots might perform. Subsequently thissource of uncertainty will be replaced by the more sophisticated Time of Arrival con-troller of Section 4.2.3.

The analysis of [6] implies that uncertainty in the wind and at the time an aircraft entersthe controlled sector play a major role both in the TW hitting and the conflict probability.If wind is the only uncertainty source, its effect on the TW hitting probability distribution isstill major, since the outcome of our simulation based study would differ significantly fromthe deterministic case. Nevertheless, our results suggest that if wind is the only uncertaintysource, the TWs used for the CATS HIL experiments [9] are rather conservative, as in manycases their size can be significantly reduced without a negative impact on the probability ofmeeting the TW constraints. This is not the case though when there is uncertainty on thetime an aircraft enters the controlled sector, where more generous TW values are needed.This is reasonable, since an initial time deviation may lead to accumulated time errors, andhence the aircraft might violate subsequent TW constraints. This motivates the use of aTime of Arrival controller bellow, to avoid accumulating temporal deviations when movingfrom one TW to the next.

61


0 20 40 60 80 1000

0.2

0.4

0.6

0.8

1

TWs

prob

abili

ty o

f mis

sing

TW

probability of missing TW in spaceprobability of missing TW in time

Figure 4.7: Probability of missing each TW in space (“red dots”) and time (“green dia-monds”). The results are based on 1000 Monte Carlo simulations, corresponding to differ-ent uncertainty realizations (wind, mass, and entry time uncertainty), and it is always morelikely to violate the temporal instead of the spatial requirements of each TW. The flightsare sorted based on their departure time, and for each flight its TWs are listed accordingto the sequence with which they occur. The “red dots” indicate the probability of violatingthe spatial requirements of the TW constraints, whereas the “green diamonds” indicate theprobability of missing a TW in time.

Motivated by these results we concentrate on the case where both wind, mass and entrytime uncertainty is present, taking into account the two most important sources of uncer-tainty. In Section 4.4.2 we investigate the effect that the use of a Time of Arrival controllermight have on the TW size and the probability of meeting the TW requirements. Finally,for the same scenario and for the flights that were found to be in conflict with high prob-ability, in Section 4.4.3 we apply the proposed reachability based algorithm to assess thelimits of maneuverability that can be exploited to resolve the conflict while preserving theexisting TW requirements. To illustrate how the algorithms developed here could be usedfor conflict resolution we select by hand additional TWs within the maneuverability limitsand demonstrate using Monte-Carlo simulation that they indeed resolve the conflicts.

4.4.2 Impact of the Time of Arrival controller on the TW size

Considering both wind, mass and entry time uncertainty, and in the absence of Time ofarrival control, we attempt first to identify the probability of missing a TW in space andtime, using 1000 Monte Carlo simulations for the 117 TWs of the simulated flights (Fig. 4.7).Fig. 4.7, shows that it is always more likely to violate the temporal, instead of the spatial,constraints of the TWs (some TWs are almost always missed in time). This is mainly due tothe fact that TWs were designed for the HIL experiments and are rather generous in space[10] to leave enough maneuvering freedom to air traffic controllers in case they need toissue resolution maneuvers.

62

Simulation results

100% 90% 80% 70% 60% 50% 40% 30% 20% 10%0

0.2

0.4

0.6

0.8

1

TW time width

prob

abili

ty o

f hitt

ing

TW

(a)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.80

5

10

15

20

25

hitting time spread / TW time width

# T

W

(b)

100% 90% 80% 70% 60% 50% 40% 30% 20% 10%0

0.2

0.4

0.6

0.8

1

TW time width

prob

abili

ty o

f hitt

ing

TW

(c)

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.80

5

10

15

20

25

hitting time spread / TW time width

# T

W

(d)

Figure 4.8: a) TW hitting probability with wind, mass, and entry time uncertainty, with allTWs reduced to a percentage of their original temporal size without Time of Arrival control.b) TW size indicator with wind, mass, and entry time uncertainty, and without Time ofArrival control. c) TW hitting probability with wind, mass, and entry time uncertainty, withall TWs reduced to a percentage of their original temporal size, using Time of Arrival control.d) TW size indicator with wind, mass, and entry time uncertainty, using Time of Arrivalcontrol.

In view of determining the effect of the Time of Arrival controller on TW hitting probabilityand the TW size (mainly the temporal component due to the implications of Fig. 4.7), Fig-ures 4.8a, 4.8b correspond to the uncontrolled case, whereas Fig. 4.8c, 4.8d show the resultswhen the Time of Arrival controller was used. Each bar in Fig. 4.8a, 4.8c corresponds to adifferent TW time width (as a percentage of the nominal one), and its height indicates thecorresponding TW hitting probability. Note that all TWs were reduced by the same fraction,shown on the horizontal axis of Fig. 4.8a, 4.8c. In both cases, the probability of meetingthe TW requirements is decreasing as we reduce the TW width. From a comparison thoughbetween Fig. 4.8a and Fig. 4.8c it is clear that by using a Time of Arrival controller we can

63


0 20 40 60 80 100−8

−6

−4

−2

0

2

4

6

8

TWs

Min

imum

− M

axim

um T

W h

ittin

g tim

e re

lativ

e to

the

TW

cen

ter

TW time widthwithout ToA controllerwith ToA controller

Figure 4.9: Minimum and maximum TW hitting times for each TW, relative to the TW cen-ter. For visualization purposes all TW are shifted so that they are centered to zero. The “red”bars (with “point” extremities) correspond to the case where Time of Arrival control is used,whereas the “blue” bars (with “diamond” extremities) depict the uncontrolled case; the ex-tremities correspond to the minimum and maximum TW hitting times. As a result of theimprovement afforded by the use of Time of Arrival control, the spread of the “red” is lessthan the one of the uncontrolled case. The “black” crosses denote the extremities of theTWs time width, as this was designed for the CATS HIL experiments.

achieve a very high TW hitting probability, even if we reduce the TWs time width to 50%of its initial value. This was not the case though in the uncontrolled scenario (Fig. 4.8a),where a significant reduction in the TW hitting probabilities was obtained. Note that theTWs time width is an indication though of how predictable a flight is; very tight intervalsincrease predictability on the one hand, but lead to smaller reachability envelopes on theother (less maneuverability).

This is also implied by the distributions of Fig. 4.8b and Fig. 4.8d, which show the TW sizeindicator. This is defined as the ratio of the spread of the TW hitting time distributions overthe time width of each TW, and hence gives an indication of what the potential temporalsize of the TW can be. The lower this value, the more the width of the TW can be reduced.In the case where a Time of Arrival controller is used, this ratio shifts to lower values (Fig.4.8d compared to Fig. 4.8b), so the time width of the TWs can be reduced significantly.The spread of the TW hitting time distribution was calculated as the difference betweenthe extreme TW hitting times, whereas to avoid outliers, the distribution could have beentruncated at certain values. Note that a TW size indicator value less than one does notnecessarily imply that the probability of missing this TW is zero. It could be the case that theTW hitting distribution is not “contained” in the TW (hence the TW hitting probability is lessthan one), but its spread is less than the TW’s time width (TW size indicator less than one).In such cases, the corresponding TWs should be shortened and also shifted in time. For

64

Simulation results

0 2 4 6 8 10 12 14 16 180

0.05

0.1

0.15

0.2

0.25

0.3

# conflicts

Con

flict

pro

babi

lity

(a)

0 2 4 6 8 10 12 14 16 180

0.05

0.1

0.15

0.2

0.25

0.3

# conflicts

Con

flict

pro

babi

lity

(b)

Figure 4.10: Conflict probability distribution, after 1000 Monte Carlo simulations, for thecase where both uncertainty in the wind, the mass and the time an aircraft enters the con-trol sector are considered. a) Without Time of Arrival control. b) With Time of Arrival con-trol.

a better understanding, Fig. 4.9 illustrates the minimum and maximum TW hitting times(denoted by the extremities of the “blue” and “red” bars respectively) relative to the TWcenter. Specifically, the average reduction in the TW hitting time spread is ∼ 1.41 minutesand the standard deviation is σ = 1.04 (the minimum and maximum reduction was 0.08and 7.37 minutes respectively). Note that the width of the region where the TWs overlapwith the spread of the “blue” (“red”) bars (as a percentage of the TWs time width), does notnecessarily indicate the probability of hitting the TWs in time, as this is shown in Fig. 4.7.This is due to the fact that the extremities of each bar might correspond to outliers, andthe probability distribution might be skewed differently. The actual TW hitting probabilitydistributions for some of the simulated TW can be found in [6].

4.4.3 Reachability calculations

For the 50 simulated flights we performed conflict detection so as to identify the flightswith high conflict probability. We considered the case where wind, mass, and entry timeuncertainty are present, and carried out 1000 Monte Carlo simulations with and withoutTime of Arrival control. Figure 4.10a depicts the conflict probability distribution for the casewhere no Time of Arrival control is used. We encountered a median of 10 conflicts, whereasif a Time of Arrival controller is employed, the median number of conflicts is reduced to7 (see Fig. 4.10b). In general, using a Time of Arrival controller improves the punctualityof the flights with respect to their schedule and reduces the along track error, but does notnecessarily lead to a lower number of conflicts.

Based on the conflict statistics of Fig. 4.10b (with Time of Arrival control) we selected thepairs of flights that are most likely to be involved in a conflict, and carried out the reachabil-

65


−7

−6

−5

−4

−3

x 105

−8

−7

−6

−5

−4

x 105

5800

6000

6200

6400

6600

6800

7000

7200

y (m)x (m)

z (m

)

(a) (b)

Figure 4.11: a) Flight plans for the two aircraft scenario of Fig. 4.11. The “red” and “blue”rectangles represent the TWs of each flight. b) Flight plan top view for the same two aircraftcase. Note that for the reachability calculations the projection of the TWs on the horizontalplane is line segments, which are aligned with the flight plan.

ity computation for them. Figure 4.11 illustrates the two flights most likely to be in conflictclose to their third way point. The TWs are centered at the last way point of each flight plan.For comparison purposes, Fig. 4.12a, 4.12b show a snapshot of the two-stage backwardreachability computation, i.e. if the avoid region is the empty set. The tubes at this figureinclude all the states that each aircraft could start from and fulfill its TW constraints, ignor-ing for the time being the fact that there may be conflicts along the way. Since aircraft flyfaster at higher flight levels there are more states that can reach each TW at high altitudes.The numerical values next to the way points in Fig. 4.12b indicate the corresponding alti-tude. Note that the x-y projection of the reachable tubes coincides with the projection ofthe flight plans on the horizontal plane (Fig. 4.11b), since in the hybrid model we assumedperfect lateral tracking.

Fig. 4.12c shows the same set, after removing all points where the two aircraft are in conflictat some time instance. For each aircraft j , the “hole” around the intersection point is theunion of all “avoid” regions A j ,t . For a better understanding, Fig. 4.13 shows the reach-avoid tubes on the s − z plane, as well as the time evolution of the conflict zone of eachaircraft for four time instances separated by 3 minutes. These sets can be thought of aslevel sets of the time dependent obstacle function h j (x j , t ) at four different time instances.

By applying now the reach-avoid approach of Section 4.3.3 we can construct the conflictfree tubes of Fig. 4.12d. As expected, the set of states that could reach the target whileavoiding conflict with the other aircraft, excludes the conflict zone of Fig. 4.12c, as well asall other states that would end up in this zone for some wind realizations irrespective of thecontrol inputs of the hybrid automaton.

The reach-avoid tubes provide an indication of the locations where aircraft can be andavoid conflict while meeting their TW constraints. An air traffic controller can then use this

66

Simulation results

−7

−6

−5

−4

−3

x 105

−8

−7

−6

−5

−4

x 105

5000

5500

6000

6500

7000

7500

8000

y (m)x (m)

z (m

)

(a)

−7 −6.5 −6 −5.5 −5 −4.5 −4 −3.5 −3

x 105

−8

−7.5

−7

−6.5

−6

−5.5

−5

−4.5

−4x 10

5

x (m)

y (m

)

6900m

6900m6600m

7000m

6900m

6000m

6500m

(b)

−7

−6

−5

−4

−3

x 105

−8

−7

−6

−5

−4

x 105

5000

5500

6000

6500

7000

7500

8000

y (m)x (m)

z (m

)

(c)

−7

−6

−5

−4

−3

x 105

−8

−7

−6

−5

−4

x 105

5000

5500

6000

6500

7000

7500

8000X: −3.769e+005Y: −5.385e+005Z: 7480

y (m)x (m)

z (m

)

(d)

Figure 4.12: a) Backward reachable tubes, including all states from which each aircraft canreach its TW. b) Projection of the reachable tubes on the horizontal plane. The values nextto the way points indicate the corresponding altitude. c) Conflict zone including all statesthat are in conflict at some time instance, calculated based on the minimum separationstandards. d) Reach-avoid tubes. The “black” lines correspond to sample simulated trajec-tories, after introducing an artificial TW.

information to reroute the flight. To illustrate how this can be done, we place an artificialTW inside the reach-avoid tubes of Fig. 4.12d by hand.

An air traffic controller could do something similar given the information generated by thereachability calculation. They could also issue standard maneuvers, e.g. vectoring or flightlevel changes, leading to changes in the flight plan. In this case the reachability calcula-tion can be repeated for the new flight plan providing feedback to the air traffic controllerwhether the proposed maneuver resolves the conflict and respects the existing TW con-straints. The updated flight plan with the new TW sequence is imported in the detailedsimulation environment of Section 4.2.1 and is tracked by the FMS, which is equipped with

67


Figure 4.13: Reach-avoid tubes and time evolution of the conflict zone for the aircraft ofFig. 4.12 in the s − z plane, for four time instances separated by 3 minutes. The “green”regions denote the conflict zones, whereas the numbering indicates the sets of states foreach aircraft that are in conflict at the same time instance. The dotted rectangles denotethe boundary of the conflict zone for each aircraft of Fig. 4.12c on the s − z plane.

Table 4.1: Conflict resolution results.

Conflict probability without resolution Conflict probability after resolution

Case 1 100% 0%Case 2 100% 0%Case 3 99% 15%

the Time of Arrival controller of Section 4.2.3. We then run Monte Carlo simulations con-sidering again uncertainty in the wind, the mass, and the time an aircraft enters the controlsector, to evaluate the effect this action had on the probability of conflict.

We applied this procedure to the three pairs of flights with the highest conflict probability,but any pair of flights that is in conflict could be considered instead. The results are summa-rized in Table 4.1. Case 1 corresponds to the two aircraft example of Fig. 4.12, where one ofthe aircraft was rerouted by placing a 2 minute TW at the position shown in Fig. 4.12d. The“black” lines in Fig. 4.12d represent 20 sample simulated trajectories (out of 1000 Monte

68

Simulation results

0 5 10 15 20 25 300

200

400

600

time (min)

Ver

tical

sep

arat

ion

(m)

Prior to resolution

0 5 10 15 20 25 300

2

4

6x 10

4

time (min)

Hor

izon

tal s

epar

atio

n (m

)

No conflictConflict1000 ft

No conflictConflict5 nmi

0 5 10 15 20 25 300

200

400

600After resolution

time (min)

Ver

tical

sep

arat

ion

(m)

No conflict1000 ft

0 5 10 15 20 25 300

2

4

6x 10

4

time (min)

Hor

izon

tal s

epar

atio

n (m

)

No conflict5 nmi

Figure 4.14: Time evolution of the horizontal and vertical separation between the two air-craft of Fig. 4.11 for different uncertainty realizations, before and after the proposed reso-lution maneuver (left and right panels respectively). The “red” line segments correspond totime instances where a conflict is encountered, whereas “green” lines represent cases wherea safe separation is achieved. The “blue” dotted lines indicate the minimum distance valuesthat ensure safe separation.

Carlo simulations), which correspond to different uncertainty realizations. For the casewhere no resolution was performed, the simulated tracks follow closely the nominal flightplans, deviating slightly at the “climb” and “descent” phase as expected. In the case wherea maneuver is initiated all simulated trajectories pass from the artificial TW and no conflictoccurs. Fig. 4.14 shows the time evolution of the horizontal and vertical separation be-tween the two aircraft for different uncertainty realizations, before and after the proposedresolution maneuver (left and right panels respectively). After the resolution maneuver ofFig. 4.12d no conflict is encountered.

Despite the fact that the reachability analysis assumed a worst case setting, with the windacting as a bounded disturbance input, the probability of conflict in case 3 of Table 4.1is not reduced to zero. This is justified by the fact that the rerouted track was computedbased on the reach-avoid tubes, which include only bounded wind uncertainty, whereasthe maneuver was executed on the detailed simulator of Section 4.2, and in the presence ofwind, mass, and entry time uncertainty.

69


4.4.4 Computational issues

All simulations were performed on an Intel(R) Core(TM)2 Duo 2.66GHz processor runningWindows 7. The reachability calculations were implemented using the Level Set MethodToolbox [125] (version 1.1) on MATLAB 7.10. Since the terminal sets of the reachability cal-culation (the TWs) are small compared to the simulated space, a 501× 501 grid was usedinitially to avoid degeneracy. When the sets had increased enough, the grid was changed to251×251 to increase the computational efficiency. Specifically, for the two aircraft example(25 minutes of flight) of Fig. 4.12, 8.22 minutes were needed to complete the reachabilitycomputation, whereas the memory usage of MATLAB was 744MB. The Monte Carlo simu-lations of the 50 flights of Fig. 4.6 were carried out in the detailed simulation environmentof Section 4.2.1, which is coded in JAVA, with a MATLAB interface. Every simulation (for allflights) required approximately 9.3 seconds, and 2MB memory.


This chapter performed an analysis for a new paradigm for 4D trajectory management in airtraffic control, based on the notion of TWs. We evaluated the concept by estimating the TWhitting and conflict probability by means of Monte Carlo simulations. An abstraction of theaircraft dynamics was performed, and we showed how to exploit the reachability frameworkof Chapters 2 and 3 to compute the maneuvering bounds of each aircraft in the presenceof TW constraints. A resolution procedure was then carried out, based on placing artificialTWs on the reach-avoid tubes. Indeed, the TW placement was not automated, showinghow an air traffic controller could make use of the information provided by the reachabilitycalculation.

In the current implementation no optimization was employed when placing the artificialTWs in the reach-avoid tubes. Moreover, we treated each conflict pair separately, and thepossibility that the resolution command causes subsequent conflicts was not consideredin the reachability calculations. For a more realistic implementation, vectoring commandsshould also be taken into account. Follow on research would be necessary to address theselimitations.

70

CHAPTER

5MPC for feedback linearizablesystems with input constraints

5.1 Introduction

Model Predictive Control (MPC) provides an efficient way of solving optimal control prob-lems on line [107]. The objective is normally to track a reference signal, by minimizing theerror between the predicted and the actual reference value, subject to input and/or stateconstraints [21]. In the case where both the model and the constraints are linear, and thecost is quadratic, the resulting optimization problem is a quadratic program that can beefficiently solved on-line.

Most real systems are governed by nonlinear differential equations, and usually linear con-straints. Applying MPC in this case would in general lead to a difficult optimization prob-lem, with no guarantees that the global optimum will be found [108], [46]. In such systems,and if the nonlinear dynamics exhibit a specific structure (input affine systems), feedbacklinearization techniques could be used [77], [82], [151]. That way, by an appropriate nonlin-ear mapping and feedback, the initial system can be transformed to a linear one, and thenMPC could be employed for the new system.

Such an approach has been studied in [145], [134], [132], [131], [61]. The main difficulty isthat the original nonlinear system which is subject to linear constraints, is mapped to a lin-ear one but with state dependent and in general nonlinear and/or non convex constraints.To overcome this difficulty, in [136] an alternative method based on feedback linearizationand reachability analysis was proposed. The limitation of this approach is that since it isbased on griding the state space it suffers from the “curse of dimensionality”.

In this chapter we concentrate on feedback linearizable systems subject to input constraintsand propose an iterative scheme [113], where a set of initially arbitrarily chosen input con-straints is continuously refined by solving a series of quadratic problems. If convergence isachieved, the first part of the resulting control sequence is applied to the system, the hori-zon is rolled, and the procedure is repeated. The advantage of this approach is that non

71

MPC for feedback linearizable systems with input constraints

convex problems are replaced by a sequence of quadratic programs that is easy to solve,thus offering a promising alternative to more direct schemes. The drawback is that it is notclear that the global optimum will be reached. Furthermore, there is no guarantee regardingthe number of iterations needed per time-step until convergence is achieved.

The proposed method is applied to several benchmark examples. The first deals with thespeed tracking problem of a dc motor, a topic extensively studied in the nonlinear con-trol literature [82], [151]. The second example deals with the stabilization of the frequencyand power exchange in an interconnected two area power network [85], [11], where thestandard PI based Automatic Generation Control in one area is replaced by the proposedMPC method. Finally, we consider the problem of flight plan tracking, when using a MIMOmodel for an aircraft in level flight.

The remainder of the chapter is organized as follows: Section 5.2 provides some back-ground information regarding feedback linearization and MPC, whereas Section 5.3 intro-duces the proposed algorithm, and discusses its main properties. Section 5.4 demonstratesthe performance of the new method in different case studies, and presents a reachabilityanalysis for the stability of the zero dynamics. Finally, Section 5.5 provides some directionsfor future work and a list of open problems.

5.2 Feedback linearization - MPC

5.2.1 SISO Feedback linearization

Consider a SISO system of the form

x = f (x)+ g (x)u,

y = h(x), (5.1)

where x ∈Rn is the state vector, u ∈R is the control input, y ∈R is the output, f (·), g (·) : Rn →Rn are vector fields, and h(·) : Rn → R is the output function. We assume that all functionsand vector fields are smooth enough for all necessary Lie derivatives and brackets to bedefined.

Let x0 denote an equilibrium point of the autonomous system. Then, following [77], [82],[151], (5.1) is said to have a strict relative degree γ at x0 ∈Rn , if there exists a neighborhoodX of x0 such that

Lg L jf h(x) ≡ 0, ∀x ∈ X , j = 0, ...,γ−2,

Lg Lγ−1f h(x0) 6= 0.

L f h(·) : Rn → R and Lg h(·) : Rn → R denote the Lie derivatives of h with respect to f andg respectively1. It can be shown that if we have a strict relative degree then one can find a

1The Lie derivative L f h(·) of a function h(·) with respect to f (·) is defined as L f h(x) = ∂h(x)∂x f (x).

72

Feedback linearization - MPC

diffeomorphism [151] of the form

Φ : x →

h(x)L f h(x)

...

Lγ−1f h(x)

η

.

We refer to the γ new variables h,L f h, . . . ,Lγ−1f h as z. Then the system in the new coordi-

nates can be expressed as

z1 = z2

... (5.2)

zγ = b(z,η)+a(z,η)u

η= q(z,η)

y = z1

where z = [z1 . . . zγ]T ∈Rγ is the new state vector, η ∈Rn−γ represents the zero dynamics,

b(z,η) = Lγf h(Φ−1(z,η)) and a(z,η) = Lg Lγ−1f h(Φ−1(z,η)). The feedback linearizing control

law is given by

u = 1

Lg Lγ−1f h(x)

(−Lγf h(x)+ v), (5.3)

where v ∈R represents the input in the new coordinates. To comment on the stability of thezero dynamics, it suffices to solve the output-zeroing problem [151] and investigate stabilityof η= q(0,η).

5.2.2 MIMO Feedback linearization

Feedback linearization can be applied to general form MIMO systems. To simplify notationwe consider the following system with two inputs u1,u2 ∈R and outputs y1, y2 ∈R.

x = f (x)+ g1(x)u1 + g2(x)u2,

y1 = h1(x),

y2 = h2(x), (5.4)

where g1(·), g2(·) : Rn → Rn and h1(·),h2(·) : Rn → R. Let u = [u1 u2]T denote the controlinput vector.

Similar to the SISO case, the system (5.4) is said to have a vector relative degree [γ1,γ2] atthe equilibrium point x0 if for i = 1,2

Lgi L jf hi (x) ≡ 0, 06 j 6 γi −2,

73


and the matrix

A(x0) =[

Lg1 Lγ1−1f h1(x0) Lg2 Lγ1−1

f h1(x0)

Lg1 Lγ2−1f h2(x0) Lg2 Lγ2−1

f h2(x0)

],

If A(x0) is nonsingular the vector relative degree of (5.4) is well defined and γ1 +γ2 < n, wecan define the following change of coordinates

z11 = h1, z2

1 = L f h1, . . . , zγ11 = Lγ1−1

f h1,

z12 = h2, z2

2 = L f h2, . . . , zγ22 = Lγ2−1

f h2.

Applying this coordinate transformation, and the nonlinear feedback

[u1

u2

]= A−1(x)

(−b(x)+

[v1

v2

]), (5.5)

with b =[

Lγ1

f h1(x)

Lγ2

f h2(x)

], results in the following system.

z11 = z2

1 ,

z21 = z3

1 ,

...

zγ11 = v1,

z12 = z2

2 ,

z22 = z3

2 ,

...

zγ22 = v2,

η= q(z,η)+P (z,η)

[u1

u2

], (5.6)

where q ∈ Rn−γ1−γ2 , P ∈ R(n−γ1−γ2)×2, and v1, v2 ∈ R are the components of the new inputvector. That way, Φ : x → (z,η) can be defined to be a diffeomorphic mapping from the ini-tial to the new coordinate frame. In analogy to the SISO case, solving the output-zeroingproblem could help proving stability of the zero dynamics. If A(x) is nonsingular, this re-sults in showing that

η= q(0,η)−P (0,η)A−1(Φ−1((0,η)))b(Φ−1((0,η))),

is a stable system.

74

Proposed iterative scheme

5.2.3 Model Predictive Control

Conventionally MPC is formulated in discrete time [21]. Assume that the plant to be con-trolled is described by the linear discrete time difference equations

z(k +1) = Ad z(k)+Bd v(k),

y(k) =Cd z(k), (5.7)

where z(k) ∈ Rn , v(k) ∈ R, y(k) ∈ R denote the state, control input, and output respectively.A receding horizon implementation is typically based on the solution of an open loop opti-mization problem [21], [107],

minVk

J (k) = minVk={v(k+i |k)}

Np−1

i=0

(zT (Np )Q0z(Np )+

Np−1∑i=0

zT (k + i |k)Qz(k + i |k)+ vT (k + i |k)Rv(k + i |k)), (5.8)

subject to

z(k + i +1|k) = Ad z(k + i |k)+Bd v(k + i |k), for all i = 0, . . . , Np −1, (5.9)

z(k|k) = z0(k), (5.10)

v(k + i |k) ∈ Vk+i |k , for all i = 0, . . . , Np −1. (5.11)

Variable Np denotes the prediction horizon, Q0,Q,R are weighting matrices of appropriatedimension, and z0(k) is the initial state of the system. Moreover, in view of the algorithmthat will be presented in the next section, only input constraints are considered (5.11), withVk+i |k denoting the input constraint set at time-step k+i . Although the optimization proce-

dure at time k results in an optimal sequence Vk = {v(k+i |k)}Np−1i=0 of Np present and future

control inputs, only the first component v(k|k) is applied over the time interval [k, k +1](assuming zero-order hold (ZOH)). At the next time-step the horizon is rolled by one stepand the same procedure is repeated.

5.3 Proposed iterative scheme

5.3.1 Problem set-up

The proposed scheme will be described for the case where the initial system is governedby continuous time MIMO dynamics of the form (5.4). Applying then the algorithm forSISO systems would be a straightforward extension. Assume that feedback linearization isapplied to (5.4). By ignoring the zero dynamics and performing time discretization (ZOH)with Ts being the temporal step, we derive (5.7).

75


MPCeq. (5.3)

oreq. (5.5)

Nonlinearsystem

Feedback linearized system

MPCparameters

ZOH

sampler

eq. (5.12)

Figure 5.1: MPC-Feedback linearization block diagram, adopted from [134].

Assume that the input u of the initial system is subject to polyhedral constraints of the formu ∈ U . For simplicity, we consider time-invariant input constraints, though the proposedapproach could be easily extended to time-varying constraints. Integration of the controlinput obtained from feedback linearization in the MPC framework poses a new challenge inthis case [134], [132], [61]; after feedback linearization the optimization problem involves alinear system (5.8)-(5.11) with state dependent and in general nonlinear input constraints.To make this state dependency explicit, notice that by (5.5) and for all i = 0, . . . , Np −1,

Vk+i |k = {v(k + i |k) = A(x(k + i |k))u(k + i |k)+b(x(k + i |k)) | u(k + i |k) ∈U }. (5.12)

Note the dependency of Vk+i |k on the state x(k + i |k). For illustrative purposes, Fig. 5.1(modified version of [134]), depicts the control structure obtained from a combination ofMPC and feedback linearization.

As stated in [134], the resulting optimization problem will be of the same complexity asthe initial one, unless the nonlinear constraints happen to be convex. Since this is not thecase in general, there is no guarantee that the global optimum will be found. The MPC-Feedback linearization approach, that has been studied in the literature, is limited by twobasic factors.

• Consider first the case that the system is not fully feedback linearizable. Due to thepresence of the zero dynamics, the map from the x to the z coordinates is not analyt-ically known, and hence the state of the initial system x(k+i |k) at time k+i with i > 0can not be computed. As a consequence the input constraint set Vk+i |k , i > 0 can notbe determined.

• In the case where the system is fully feedback linearizable, for all i > 0 the constraintset Vk+i |k of the input v could be a nonlinear and/or non convex function of the newstate z (after composition with Φ−1), and hence the optimization is challenging. Inthat case the problem is reduced to that of a linear system with nonlinear constraints.

The main contribution of the algorithm that will be presented in the next section, is thatthis generally non-convex problem is substituted by an iterative procedure, where at each

76

Proposed iterative scheme

eq. (5.3)or

eq. (5.5)

Nonlinearsystem

Feedback linearized system

ZOH

sampler

eq. (5.12)

Algorithm 5.1

MPCFeedbacklinearizedsystemMPC

parameters

sampler

...repeat

initial

trajectory

(t=0)

ZOH

Figure 5.2: Block diagram representation of the proposed algorithm.

time-step a sequence of convex problems is solved. Note that a similar algorithm was inde-pendently developed by [133].

5.3.2 Iterative algorithm

Since the proposed algorithm is based on an iterative procedure, we introduce the super-script j in the variables defined in the previous section to indicate the number of iteration,whereas T denotes the optimization horizon.

At the initialization step of the algorithm we start with an arbitrarily chosen, but feasible,input i.e. u(t ) ∈U for t ∈ [0,T ]. Applying this input to (5.4), we get a trajectory of the system,

which we sample to generate {x0(k + i |k)}Np−1i=0 . These samples are then inserted in (5.12) to

construct the initial constraint sets {V 0k+i |k }

Np−1i=0 . At iteration j of the algorithm, we com-

pute the optimal input sequence {v j (k + i |k)}Np−1i=0 , by solving (5.8)-(5.11) with {V j

k+i |k }Np−1i=0 .

By (5.5), the input sequence {u j (k+ i |k)}Np−1i=0 is computed, and it is then applied in a piece-

wise constant fashion in (5.4). Since time-invariant input constraints U are considered,the piecewise constant input trajectory is guaranteed to be feasible. Sampling the resulting

trajectory yields {x j (k + i |k)}Np−1i=0 , which is in turn used to compute the new input bounds

{V j+1k+i |k }

Np−1i=0 .

The process is repeated until convergence is achieved, i.e.∣∣∣∣{v j (k + i |k)}

Np−1i=0 − {v j−1(k +

i |k)}Np−1i=0

∣∣∣∣∞6 ε for a given tolerance ε> 0. We then apply the first input component for thetime period Ts , roll the horizon, and repeat the entire process for the next time-step. To fa-cilitate convergence, the first Np −1 elements of the input sequence u j used to initialize theprocess at the next time-step, are the Np −1 last elements of the sequence generated at theprevious step, whereas the last element is considered to be the same with the last elementof the curtailed sequence (feasibility is guaranteed since U is time-invariant). Algorithm5.1 summarizes the steps of the proposed methodology (see also Fig. 5.2).

The proposed algorithm has been applied to several examples, some of them reported inSection 5.4. Although in all these case studies convergence is achieved, there are no theo-

77


Algorithm 5.1 Iterative MPC using feedback linearization1 Initialization.2 Set k = 0.3 Consider an arbitrary input u(t ) ∈U for t ∈ [0,T ], and simulate (5.4).

4 Generate {x0(k + i |k)}Np−1i=0 by sampling the simulated trajectory.

5 Construct {V 0k+i |k }

Np−1i=0 using (5.12).

6 Define convergence tolerance 06 ε¿ 17 For t = 0 until T8 Set j = 0, k = t/Ts .

9 While∣∣∣∣{v j (k + i |k)}

Np−1i=0 − {v j−1(k + i |k)}

Np−1i=0

∣∣∣∣∞ > ε or j = 010 Set j = j +1.

11 Construct {V jk+i |k }

Np−1i=0 using (5.12).

12 Compute the optimal sequence {v j (k + i |k)}Np−1i=0 solving (5.8)-(5.11).

13 Determine {u j (k + i |k)}Np−1i=0 from (5.5), and simulate (5.4) using ZOH.

14 Generate {x j (k + i |k)}Np−1i=0 by sampling the resulting trajectory.

15 end while16 Apply the first input component u j (k|k) and simulate (5.4) until t +Ts .17 t = t +Ts .18 end for

retical guarantees that this is always the case. One approach toward this direction wouldbe to concentrate on fully feedback linearizable systems, with Φ known analytically, andexploit the idea of multi-parametric programming. At every iteration j of the algorithm,

the solution depends on the constraint sets {V jk+i |k }

Np−1i=0 . By an appropriate parametriza-

tion of each set Vj

k+i |k , every MPC problem of Algorithm 5.1 could be casted as a parametricquadratic program. Following then [35], [20], its solution can be written in explicit form,and turns out to be piecewise affine in the parameter space. With a slight abuse of no-tation, the last statement implies that for each region r = 1, . . . , Nr over which the controlinput is constant, and for all i = 0, . . . , Np −1,

v j (k + i |k) = F ri pi +Gr

i , (5.13)

where pi denotes the parameter vector, and F ri ,Gr

i are matrices of appropriate dimension.Moreover, pi = fp (v j−1(k + i |k)), where fp is a nonlinear map, emanating from (5.4), (5.5)and the analytic expression of the diffeomorphism Φ. Hence, for all i = 0, . . . , Np − 1, andr = 1, . . . , Nr

v j (k + i |k) = F ri fp (v j−1(k + i |k))+Gr

i . (5.14)

Under this setting, convergence of the iterative algorithm is equivalent to proving stabilityof (5.14). The latter is not necessarily easier though, since it requires showing stability for anonlinear, switched system.

78

Case studies

Remark 5.1. In the numerical examples, convergence is always achieved, apart from a sin-gle instance, where the algorithm oscillates between two different input sequences. To al-leviate this, we introduced a penalty term in the objective function of each MPC problem,

which is proportional to∣∣∣∣{v j (k + i |k)}

Np−1i=0 − {v j−1(k + i |k)}

Np−1i=0

∣∣∣∣22.

Remark 5.2. Even if the algorithm converges, there are no guarantees that the global opti-mum of the initial problem will be found. This will be the case though, if the system is giveninitially in discrete time (no discretization mismatch), it is fully feedback linearizable, andthe input constraints in the new coordinates are convex.

5.4 Case studies

5.4.1 DC motor

Modeling and analysis of a DC motor is a topic that has been studied extensively in nonlin-ear control literature [77], [82], [151], [63], [68]. A simplified model for the dynamics of themotor is described by the following SISO system.

x1 =−θ1x1 −θ2x2u +θ3,

x2 =−θ4x2 +θ5x1u,

y = x2, (5.15)

where x1 represents the armature current, x2 is the rotational speed of the motor, u is thefield current and θi , i = 1, . . . ,5 are constants depending on the resistance, inductance androtor inertia. For a detailed derivation the reader is referred to [82].

The system is bilinear but not fully feedback linearizable. The objective is to stabilize x2 (therotor speed) to a reference value z∗

1 . It is easy to check that the system has relative degreeγ= 1 in a region D0 = {x ∈ R3 | x1 6= 0}, and in the new coordinate system, after a nonlineartransformation, can be represented by

z1(t ) = v(t ),

y(t ) = z1(t ),

and the feedback linearizing control law (5.3) is given by u = θ4x2+vθ5x1

. As stated also in [77],the zero dynamics of this system, given by

η=−θ1η−θ2θ4(z∗

1 )2

θ5η+θ3,

and with the physical restriction θ23θ

25 > 4θ1θ2θ4θ5(z∗

1 )2, have two equilibria 0 < ηa < ηb .Then η is negative for 0 < η< ηa , positive for ηa < η< ηb , and negative for η> ηb . Hence, ηa

is an unstable and ηb is a stable equilibrium of the zero dynamics. So, for initial conditionsη0 > ηa , η will converge asymptotically to ηb .

79


0 5 10 150.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

time

x1

(a) x1 time response.

0 5 10 1540

60

80

100

120

140

160

time

x2

(b) x2 time response.

0 5 10 150.015

0.02

0.025

0.03

0.035

0.04

time

u

(c) input response.

Figure 5.3: System response starting from [0.5 50]T (“blue”) and [0.5 130]T (“red”).

Since MPC is implemented in discrete time, the above system is discretized to the followingdifference equation z1(k+1) = z1(k)+v(k). In view of (5.8), the prediction horizon was cho-

sen to be Np = 15, and the cost J (k) was defined by J (k) = ∑Np−1i=0 (z1(k + i |k)− z∗

1 )T (z1(k +i |k)− z∗

1 ). Following [82], the parameters were chosen to be θ1 = 60,θ2 = 0.5,θ3 = 40,θ4 =6,θ5 = 40000. The input was saturated between 0 and 40mA, and z∗

1 was selected to be100rpm. Fig. 5.3 shows the obtained results for a randomly chosen initial input trajec-tory (same in all cases) and different initial conditions [0.5 50]T (“blue”), [0.5 130]T (“red”).The convergence tolerance was set to ε = 0.005. For a given initial condition, we repeatedthe algorithm for 1000 different initial input trajectories. Remarkably, the same optimalcontrol sequence was obtained independently of the arbitrarily chosen initial trajectory. Itshould be also noted, that for this example convergence was achieved after 3 iterations pertime-step, irrespectively of the initial condition. This number varies though with the inputbounds; if these bounds get narrower more iterations per step are needed until convergenceis achieved.

80

Case studies

Figure 5.4: Two-Area Power System with AGC.

5.4.2 Two-Area Power System

5.4.2.1 Physical description and mathematical modeling

Power networks are normally divided into independently controlled areas. In every area,in stable steady-state conditions, the total generated active power is the same as the con-sumed one. Any disturbance of this balance results in a deviation of the system frequencyfrom its set-point value. The behavior of the frequency in area i with respect to the powerbalance can be approximated by∆ fi = f0

2Hi SBi(∆Pmi −∆Pei ), where fi is the frequency of the

i area (Hz), Hi denotes its inertia (sec MW/MVA), Pmi is the generated power (MW), Pei isthe consumed power (MW), and SBi is the power base (MVA). The 0 index stands for thenominal value of each variable and the ∆ operator denotes the deviation from its nominalvalue.

After an increase in the power demand Pei , the rotating parts of the generators will startlosing their kinetic energy until the consumed and the produced power are equal and anew equilibrium is reached. This stabilizing effect is normally too small to be able to keepthe frequency within reasonable bounds. Therefore, in order to keep the frequency devia-tion at an acceptable level, generators are equipped with a regulating unit (governor) thatperforms automatic primary frequency control, using proportional feedback. This primaryfrequency control law is of the form ∆Ppi = − 1

Si∆ fi , where the proportional gain Si is re-

ferred to as speed droop or speed regulator.

Since the governor only involves proportional control, the frequency will in general not re-turn to its nominal value after a deviation. Also, in an interconnected system with two ormore independently controlled areas, the scheduled power exchange between these areaswill be in general violated, even after the activation of the primary control. Hence, sup-plementary control action is needed; this is provided by the Automatic Generation Control(AGC). Consider the system of Fig. 5.4, which consists of two areas connected with a tie line,each one equipped with its own AGC. Following [11], [85] each area is approximated by an

81


�

�

ACEiΔP AGCi ΔP m,AGCi

p i

Σgain

PI

Figure 5.5: PI controller with anti-wind up.

equivalent generating unit equipped with an equivalent primary frequency control. TheAGC of area i is typically a proportional-integral (PI) controller. To avoid wind up in case ofsaturation, an anti-wind up circuit (Fig. 5.5) is also used [68]. All undefined parameters areconstant gains.

The Area Control Error signal (AC Ei ), is defined as AC Ei = ∆Pi j + 1Si∆ fi , where Pi j is the

power transmitted from area i to area j , and P0i j denotes its scheduled value. Assume thatthe AGC of the second area is replaced by a control input u ∈ [∆P min

AGCi∆P max

AGCi] generated by

the Algorithm 5.1, which is applied to stabilize the frequency and power exchange errors tozero. Based on the detailed derivation of [114], the model of the two-area power system forthis case could be described by the following SISO system,

∆ f1 = A1(∆Pm,p1 +∆Pm,AGC1 − c∆ f1 −d sin∆φ),

∆ f2 = A2(∆Pm,p2 +u − c∆ f2 +d sin∆φ),

∆φ= 2π(∆ f1 −∆ f2),

∆P AGC1 =−k1∆ f1 −k2∆Pm,p1 −k2∆Pm,AGC1

+k3 sin∆φ+k4(∆ f1 −∆ f2)cos∆φ−k5p1, (5.16)

where ∆ f1,∆ f2 denote the frequency deviation of each area, ∆φ is the angle deviation, and∆Pm,AGC1 is the deviation of the AGC signal of area 1 from its scheduled value. The lumpedparameters A1, A2,c,d ,ki , i = 1, . . . ,5 are kept constant throughout the simulations. Due tothe saturations, the system exhibits a hybrid behavior. In particular,

∆Pm,pi =

∆P min

piif ∆P pi 6∆P min

pi

∆Ppi if ∆P minpi

<∆Ppi <∆P maxpi

∆P maxpi

if ∆P pi >∆P maxpi

In a similar way, ∆Pm,AGCi = ∆P AGCi if ∆P AGCi ∈ [∆P minAGCi

∆P maxAGCi

], and is restricted to its

saturation limits ∆P minAGCi

, ∆P maxAGCi

in an opposite case. By obtaining the block diagram

of Fig. 5.5, p1 = 0 if ∆P AGCi ∈ [∆P minAGCi

∆P maxAGCi

]. The latter implies normal operation,whereas p1 = ∆P AGCi −∆P m,AGCi in any other case. For the rest of this section considerx = [x1 x2 x3 x4]T = [∆ f1 ∆ f2 ∆φ ∆P AGC1 ]T .

82

Case studies

5.4.2.2 Nonlinear observer

For a more realistic implementation, we consider the case where full knowledge of the stateis not available. To construct an estimate for the unmeasured states, a standard nonlinearobserver is designed, based on the closed-loop extended Luenberger observer proposedby [28], [29], which is a relaxation of the normal form observer developed by [84]. Theformulation considered in this section is in general valid only for autonomous systems.Nevertheless, in the approach of [28], [29], input-driven systems have also been taken intoaccount, and the nonlinear observer formula can be reduced to the one adopted in thiswork, since in the examined example the function g is independent of the state x.

Following [64], consider a multi-output system with y = [h1(x) h2(x) . . . hp (x)]T . Then the

observability map is given by q(x) = [h1(x) . . . Lp1−1f h1(x) . . . hp (x) . . . L

pp−1f hp (x)]T , where

p1+. . .+pp = n. That way the system is decomposed into p decoupled subsystems, each onewith dimension pi . The system is said to be locally observable if the observability matrix Q,given by the following equation, has full rank.

Q(x) = ∂q(x)

∂x= [dh1(x) . . . dLp1−1

f h1(x) . . . dhp (x) . . . dLpp−1f hp (x)]T , (5.17)

where dLif denotes the differential of the Lie derivative. The dynamics of the observer are

then described by

˙x = f (x)+ g (x)u +L(x)(h(x)−h(x)). (5.18)

As also stated in [150],

B = ∂h(x)

∂x[ad p1−1

f ◦ s1 . . . adpp−1f ◦ sp ],

L(x) = [a1(ad f )◦ s1 . . . ap (ad f )◦ sp ]B−1.

In the above equations ad if s := [ f , ad i−1

f s], for i > 0, is the ith Lie Bracket, with ad 0f s := s

and [ f , s] = ∂s∂x f (x)− ∂ f

∂x s(x). The vector si is the ki vector of Q(x)−1, where ki = ∑ij=1 p j .

ai (λ) = ci 0+ci 1λ+ . . .+ci pi−1λpi−1+λpi is the characteristic polynomial of the i subsystem,

and the coefficients ci j are design parameters that are selected so as to place the eigenval-ues at the desired position.

For the system of Section 5.4.2.1, using y = [h1(x) h2(x)]T = [x1 x3]T , leads to a state es-timate x, which can be in turn used in Algorithm 5.1 in place of x. Note that this outputdoes not necessarily need to be the same with the one used for the feedback linearizationpurposes, since the latter is defined so as to derive an appropriate change of coordinates.

83


0 5 10 15−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

time

estim

ated

x1,

x2

x1 x2

(a) x1, x2 time response.

0 5 10 15−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

0.4

0.5

0.6

time

estim

ated

x3

(b) x3 time response.

0 5 10 15

−300

−200

−100

0

100

200

300

time

estim

ated

x4

(c) x4 time response.

0 5 10 15−400

−300

−200

−100

0

100

200

300

time

u

(d) control input.

Figure 5.6: State and input response for the case where x0 = [0.5 0.5 0.6 70]T .

5.4.2.3 Simulation results

Feedback linearization of (5.16), with y = x3 leads to the following two-state linear system(relative degree γ= 2),

z(t ) =[

0 10 0

]z(t )+

[01

]v(t ),

y(t ) = [1 0

]z(t ), (5.19)

where the feedback linearizing control law (5.3) is given by v = b(x)+a(x)u, where

a(x) =−2πA1,

b(x) = 2πA1(∆Pm,AGC1 +∆Pm,p1 − cx1 −d sin x3)

−2πA2(∆Pm,p2 − cx2 +d sin x3),

and ∆Pm,p1 , ∆Pm,p2 , and ∆Pm,AGC1 are calculated as described in the previous subsectionwith x (estimated via the observer of Section 5.4.2.2) in place of x.

84

Case studies

0 5 10 15−0.05

0

0.05

0.1

0.15

0.2

time

e1

(a) x1 − x1 time response.

0 5 10 15−0.02

0

0.02

0.04

0.06

0.08

0.1

time

e2

(b) x2 − x2 time response.

0 5 10 15−0.1

−0.08

−0.06

−0.04

−0.02

0

0.02

0.04

time

e3

(c) x3 − x3 time response.

0 5 10 15−150

−100

−50

0

50

100

150

200

250

300

time

e4

(d) x4 − x4 time response.

Figure 5.7: Response of the observer error e = x − x.

Since MPC is implemented in discrete time, the above system is discretized using ZOH tothe following difference equations

z(k +1) =[

1 Ts

0 1

]z(k)+

[T 2

s /2Ts

]v(k).

For the simulation set-up we chose Ts = 1, Np = 15, and J (k) =∑Np−1i=0 z1(k+i |k)T z1(k+i |k).

Fig. 5.6 shows the response of the system starting from x0 = [0.6Hz 0.6Hz 0.5rad 60MW]T

with x0 = [0.5Hz 0.5Hz 0.6rad 70MW]T , and A1 = A2 = 5 · 10−4, c = 200, d = 1000, k1 =116.67,k2 = 0.25,k3 = 216.67,k4 = 628.32,k5 = 3.34, and ε = 0.05. The saturation boundswere considered to be ∆P min

pi=−75MW,∆P max

pi= 75MW and ∆P min

AGCi=−350MW,∆P max

AGCi=

350MW.

The observer errors have a transient response with some initial peaks, which then convergeto zero. The results, obtained with observer gains c10 = 10, c20 = 300, c21 = 200, c22 = 15as defined in Section 5.4.2.2, are shown in Fig. 5.7.

85


20 25 30 35 40 45 50 55 600

50

100

150

200

250

300

# iterations

# si

mul

atio

ns

Figure 5.8: Number of iterations for the first time-step after 1000 simulations.

Step#2 Step#3 Step#4 Step#5 Step#6 Step#7 Step#8Iterations 4 3 7 5 5 10 6

Step#9 Step#10 Step#11 Step#12 Step#13 Step#14 Step#15Iterations 7 4 4 3 3 3 3

Table 5.1: Iterations per time step.

In order to evaluate the performance and robustness of the proposed scheme, we per-formed Monte Carlo simulations for 1000 different initial input trajectories and the sameinitial condition. Based on the simulation results, all scenarios converged to the same con-trol sequence after the first time-step. Hence, the number of iterations needed for the re-maining steps was the same for all simulations, and is reported in Table 5.1. For the firststep we obtained the distribution of Fig. 5.8, which shows that an average of 27 iterationswas needed so as to achieve convergence.

5.4.2.4 Zero dynamics stability analysis

To ensure that the closed loop system is stable, the behavior of the zero dynamics shouldbe also investigated. In this case the zero dynamics are represented by a switched linearsystem, due to the saturations imposed by the primary and secondary control actions. Ingeneral the system is of the form η= Asη+Bs with varying As ,Bs . For the case where bothsaturations are inactive we get the following linear system.

η=[−A1(1/S1 + c) A1

−k1 +k2/S1 −k2

]η= Aη. (5.20)

For S1 = 2 10−4, which is a typical value for the speed droop, the eigenvalues of the systemare both negative, and hence (5.20) is exponentially stable. In order to analyze the stabilityof the zero dynamics, for the switched linear system, a two stage reachability computation

86

Case studies

(a) Level set of invariance computation.

t = 15

n1

n2

−2 −1 0 1 2

−1000

−500

0

500

1000

(b) Level set of reachability computation.

Figure 5.9: Zero dynamics stability analysis following the two-stage approach.

is performed.Stage 1. An invariant calculation is carried out to determine the set of states that the systemcould start from and remain in the region where no saturation is active. To carry out thisinvariance computation, the framework of Chapter 2 may be adopted. Invariance is relatedto reachability in a complementary way [98]. Therefore, in the absence of state constraints,invariance for the autonomous system (5.20) appears as a special case of (2.11). The desiredset is then characterized by the zero super-level set of V (η, t ) = mint∈[0,T ] l (η(t )), which isproven to be the unique viscosity solution of

∂V (η, t )

∂t+min{0,

∂V (η, t )

∂ηAη} = 0, (5.21)

with terminal condition V (η,T ) = l (η,T ) (T is the end of the time horizon). The functionl (·) can be chosen to be a bounded, Lipschitz continuous function, that characterizes theset R = {η ∈ R2 | |η1|6 0.015 and |η2|6 350} i.e. no saturation is active; here we chose thesaturated signed distance to R. The result of this computation is depicted in Fig. 5.9a.Under equation (5.20), all states in this set converge to zero exponentially.

Stage 2. Given the set of Fig. 5.9a, a backward reachability calculation was carried out,so as to determine the states that at some time can reach this set. The new set is in turncharacterized by the sub-level of V , which is the solution of (5.21) with Asη+Bs in placeof Aη, and with different terminal condition, since l (·) determines now the complement ofthe invariant set of Fig. 5.9a. Therefore, all points that can reach this set, can then convergeto zero, and hence the zero dynamics are stable. The simulation results indicate that after50sec, all possible initial conditions enter this backward reachable set. A snapshot of thisset at t=15sec is given in Fig. 5.9b.

Via the two-stage computation described above, it can be shown numerically that from anyinitial condition the system trajectories end up in the invariant set of Fig. 5.9a, and then

87


Figure 5.10: Numerically constructed Lyapunov function for the zero dynamics at t = 15sec.

converge to zero exponentially. An alternative way to verify exponentially stability, is in-spired from [14], and is based on augmenting the state vector with V =−α1V , together withthe constraint V >α2‖η‖2 for all t ∈ [0,T ]. The constants α1,α2 are positive design param-eters. We seek then to find the set of states that could reach a ball centered at the origin,while satisfying the constraints of V . This is a reachability problem with state constraints,and based on the formulation of Chapter 2 (see equation (2.11)), the desired set is the zerosub-level set of the solution of

max{

h(η)− V (η, t ),∂V (η, t )

∂t+min{0,

∂V (η, t )

∂η(Asη+Bs)}

}= 0, (5.22)

with terminal condition V (η, t ) = max{l (η),h(η)}. The function l (·) is the signed distanceto the ball around the origin, whereas h(·) characterizes the state constraints2. A snapshotof this numerically constructed Lyapunov function is shown in Fig. 5.10 for t = 15sec, andimplies that the zero dynamics of the system are exponentially stable. We chose α1 = α2 =0.1, whereas the reader is referred to [151] for details on how these constants are relatedto the exponential decay rate (in general they are required to be small). All reachabilitycalculations were performed by the Level Set Method Toolbox [125].

5.4.3 Flight control

In this section we apply Algorithm 5.1 to an air traffic control problem. Specifically, weconsider the problem of navigating an aircraft so as to track its flight plan when cruising.Here we focus only on level flights, therefore, the aircraft dynamics described by (4.1) aremodified as follows.

2Note that h(·) characterizes here the state constraints, and should not be related to 5.1.

88

Case studies

We first neglect the vertical motion, set the flight path angle to zero (i.e. γ = 0) in the re-maining equations, and eliminate the last equation of (4.1), assuming that the aircraft massdoes not change significantly for the time scale of our control scheme. Subsequently, weassume that the engine thrust is T = T0 +u1, where T0 represents the nominal thrust value,and u1 ∈ [−∆T , ∆T ] denotes the deviation from T0. Using a small angle approximation forthe bank angle φ (i.e. sinφ ' φ), we define u2 = φ ∈ [−∆φ, ∆φ] to be an additional controlinput. Under these assumptions, denote by x = [x1 x2 x3 x4]T the cartesian coordinates ofthe aircraft, the true airspeed and the heading angle respectively. The input vector is de-noted by u = [u1 u2]T . The resulting aircraft model, in the absence of wind uncertainty, isdescribed by

x1 = x3 cos x4,

x2 = x3 sin x4,

x3 =−(C1 +C2C 2

L(x3)) ρS

2mx2

3 +T0

m+ 1

mu1,

x4 = g

x3u2, (5.23)

with output

y1 = x1,

y2 = x2. (5.24)

The lift coefficient is defined by CL(x3) = 2mgρSx2

3, whereas CD =C1+C2C 2

L(x3) denotes the drag

coefficient of (4.1). For a detailed description and physical interpretation of the undefinedvariables and parameters in (5.23), the reader is referred to Section 4.2.1. Since the resultingmodel is a MIMO system, following the feedback linearization procedure of Section 5.2.2,results in

z(t ) =

0 1 0 00 0 0 00 0 0 10 0 0 0

z(t )+

0 01 00 00 1

v(t ),

y(t ) = [1 0 1 0

]z(t ), (5.25)

where the vector relative degree of (5.23) is [2, 2], and the feedback linearizing control lawis given by (5.5), with

A(x) =[ 1

m cos x4 −g sin x41m sin x4 g cos x4

],

b(x) =[

cos x4(T0

m − g ( C1CL(x3) +C2CL(x3))

)sin x4

(T0m − g ( C1

CL(x3) +C2CL(x3)))] .

89


0 1 2

x 104

0

1

2

x 104

x1(m)

x 2(m)

aircraft trajectoryflight plan segment

(a)

0 1 2 3 4

x 104

−1

0

1

2

3

4

x 104

x1(m)

x 2(m)

(b)

Figure 5.11: a) Flight plan (“black”) and the resulting system trajectory (“blue”). b) Systemtrajectories for each iteration and every time-step (“green”). The trajectories generated atthe first iteration of each time-step are indicated with “red”, whereas the dashed “red” linecorresponds to the input sequence that was arbitrarily selected at step 3 of Algorithm 5.1.

0 1 2 3 4 5 6 7 8 9 10−1

−0.6

−0.2

0.2

0.6

1x 10

5

time−step

u1

(a)

0 1 2 3 4 5 6 7 8 9 10−0.4

−0.2

0

0.2

0.4

time−step

u2

(b)

Figure 5.12: a) Converged input sequence u1. The “red” dotted lines denote the bounds∆T .b) Converged input sequence u2. The “red” dotted lines denote the bounds ∆φ.

Discretizing (5.25) using ZOH leads to

z(k +1) =

1 Ts 0 00 1 0 00 0 1 Ts

0 0 0 1

z(k)+

T 2

s /2 0Ts 00 T 2

s /20 Ts

v(k),

where the elements z1(k), z3(k) correspond to the cartesian coordinates of the aircraft. Inthis study, we focus on the problem of tracking a single segment of the aircraft’s flight plan;

90


to track the entire flight plan, the proposed scheme can be applied in a sequential fash-ion. Therefore, in the objective function of the MPC formulation, we penalize any deviationfrom the line that contains this segment. To achieve this, we define

J (k) =Np−1∑

i=0

∣∣∣∣z3(k + i |k)−αs z1(k + i |k)∣∣∣∣2

2,

where αs denotes the slope of the corresponding flight plan segment. For the simulations,we used Ts = 15, Np = 10, αs = tan(π/4), ∆T = 94.7kN, ∆φ = π/9rad, g = 9.81m/s2, m =3 ·105kg, ρ = 1.225kg /m2, S = 512m2, T0 = 474kN, C1 = 0.022, and C2 = 0.045.

Applying Algorithm 5.1 with x0 = [1000 − 1500 218 −π/18]T , yields the tracking perfor-mance shown in Fig. 5.11a. The flight plan segment is denoted with “black”, whereas thetrajectory produced by the rolling horizon implementation of our algorithm is shown with“blue”. For illustrative purposes, Fig. 5.11b depicts, for every time-step, the system trajecto-ries generated by applying the input sequence obtained at each iteration of our algorithm(“green”). The trajectories corresponding to the first iteration of each time-step are indi-cated with “red”, whereas the dashed “red” line corresponds to the input sequence that wasarbitrarily selected at step 3 of Algorithm 5.1. Therefore, and since the initial orientation ofthe aircraft is −π/18rad, the dashed trajectory is far off the desired performance. Fig. 5.12depicts the final sequences for the inputs u1 and u2. It should be also noted that for a giveninitial condition, the algorithm converged to the same solution, irrespective of the initialinput sequence.


In this chapter, a novel method was provided in order to apply standard MPC techniques forfeedback linearizable systems with input constraints. To achieve this, an iterative processwas followed, and at every step the resulting optimization problem involved only linear dy-namics and constraints. The efficiency of the proposed scheme was verified via simulationson various examples. Moreover, for the two area power system case study a reachabilitybased approach was followed to show stability of the zero dynamics.

Even though the proposed algorithm has been successfully applied to more examples (see[176]), there is no guarantee that convergence is achieved. Moreover, in case of conver-gence, the resulting solution may not coincide with the global optimum of the initial prob-lem. As a preliminary step toward this direction, it was shown in [176] that the error be-tween two successive iterates remains bounded. Another problem is that the method relieson the stability of the zero dynamics, which might not be always easy to show. Consider-ing also the fact that its dimension tends to increase for systems of higher dimension, theapplicability of the proposed technique is limited. In addition to these issues, the sensitiv-ity of the algorithm with respect to model mismatch and parameter uncertainty should beinvestigated, while for the case of bounded uncertainty, disturbance rejection techniquesmay be used [151].

91


From an application point of view, in the example of Section 5.4.2 an alternative way of sec-ondary frequency control (AGC) was provided. This method is valid in the two-area case,but in a multi-area setting it will not necessarily perform as standard AGC control tech-niques. The reason is that in the MPC objective function only the angle difference, andhence the power exchange, is minimized. In practice AGC should also stabilize the devi-ation of the area frequency to zero. In the examined scenario this is implicitly achieved,since one of the areas is equipped with conventional AGC. In a multi-area scenario this isnot sufficient, hence MIMO feedback linearization should be used, so as both frequencyand angle deviation appear as states of the linear system in the new coordinate frame.

92

CHAPTER

6Scenario based chance constrainedoptimization

6.1 Introduction

In recent years robust optimization has attracted increasing attention due to its ability tooffer performance guarantees for control synthesis problems in the presence of uncer-tainty. Restricting our attention to optimization over continuous spaces, robust controldesign requires constructing a control input such that the constraints in an optimizationproblem are satisfied for all admissible values of some uncertain parameter. The appli-cability of such approaches is hampered by the fact that they may lead to a conservativesolution or to an intractable problem. For such problems, [22, 72, 27, 26] provide condi-tions on the structure of the uncertainty set (polyhedral, ellipsoidal, etc.) under which therobust variants of standard programming problems (linear programming, quadratic con-strained quadratic programming, second order cone programming, semidefinite program-ming, etc.) are tractable.

An alternative approach is to interpret robustness in a probabilistic sense, allowing for con-straint violation with a low probability. Although this potentially reduces the conservatismof a worst-case design, it results in chance-constrained optimization problems [147]. Asidefrom a few special cases [39], such problems are computationally intractable in general,since they require the computation of multi-dimensional probability integrals. To over-come this difficulty, [27, 25] follow a different approach. Instead of solving a chance con-strained program, they formulate a robust problem with bounded uncertainty, where theuncertainty bounds are chosen to provide probabilistic guarantees on the feasibility of theresulting robust solution, under certain assumptions on the underlying probability distri-bution. This is conservative in general, since it requires imposing assumptions on the mo-ments of the probability distribution of the uncertainty and it may involve arbitrary trun-cation.

Randomization of uncertainty offers an alternative way to provide probabilistic performance

93

Scenario based chance constrained optimization

guarantees, without prior knowledge of the probability distribution [45, 43, 81, 160, 169,170, 171, 91, 155, 71]; see also [59, 44] for extensive surveys. Randomization typically in-volves sampling the uncertain parameter and substituting the chance constraint with a fi-nite number of hard constraints, corresponding to the different uncertainty realizations. Toprovide probabilistic guarantees based on a finite number of samples, [42, 48, 41, 49] con-centrate on problems that are convex with respect to the decision variables, and introducethe so called scenario approach. For a given probability level ε and confidence β, the so-lution of the corresponding scenario program, with confidence at least 1−β, violates theconstraints of the system with probability at most ε. To achieve such probabilistic guaran-tees the number of uncertainty realizations N that should be generated is given by [48], [50]as,

N > 2

ε

(n + ln

1

β

), (6.1)

where n is the number of decision variables. By characterizing the probability of violationonly for the optimal solution of the resulting problem, this bound does not require anyprior knowledge of the underlying probability distribution of the uncertainty, and does notdepend on the dimension of the uncertainty vector or the number of constraints. Moreover,it scales linearly in 1/ε and n, and logarithmically in β. While very high confidence levelsmay be achieved with a modest increase of the computational effort, problems with manydecision variables may become computationally expensive, since the number of scenariosthat need to be generated to achieve the desired probabilistic performance grows linearlywith the number of decision variables. Moreover, the number of decision variables remainsthe same, whereas the number of constraints scales linearly with the number of scenarios.

The required number of scenarios specified by (6.1) is tight, in the sense that there are prob-lems where it holds with equality [47]. This is not always the case though, hence by exploit-ing the structure of the problem tighter bounds may be achieved [129]. Here we proposea mixed procedure, which does not rely entirely on randomization of the constraints as inthe case of the standard scenario approach, does not require knowledge about the uncer-tainty probability distribution and does not resort to ad-hoc truncation as in the case ofstandard robust methods. Specifically, as in [27], a robust problem with bounded uncer-tainty is solved, but the uncertainty bounds are computed at an intermediate step usingthe scenario approach.

Following this methodology, we propose two alternatives to the scenario approach. Thefirst one is general and applies to optimization problems with very few underlying assump-tions on the objective functions or constraints. The second applies to problems whose con-straint functions are structured such that they can be decomposed into a product of twofunctions; one that depends exclusively on the uncertainty, and one that depends exclu-sively on the decision variables.

We show that for a suitable choice of the number of scenarios, both our approach and thescenario approach lead to equivalent probabilistic guarantees. The number of scenariosthat need to be generated in our case, however, does not depend on the number of deci-sion variables, but on the dimension of the uncertainty vector or the number of constraints

94

Problem description

depending on which of our two methods is used. This fact leads to guidelines under whicheach of the methods, when applicable, is preferable in terms of providing less conservativeguarantees or reducing the computational cost. Moreover, though related to [27], the pro-posed approach provides probabilistic guarantees without relying on the probability distri-bution of the uncertainty, since at the intermediate step of our methodology the scenarioapproach is employed. Therefore, the distribution-free feature of [42] is maintained.

In Section 6.2, we state the main problem, and recall the standard scenario approach. InSections 6.3 and 6.4 we introduce our two alternative solution methods, prove their equiv-alence with the scenario approach in terms of their probabilistic guarantees, and discussthe conditions under which each method is tractable. Section 6.5 compares the proposedapproaches with the scenario approach, and demonstrates some of their basic features bymeans of numerical examples. Section 6.6 provides some concluding remarks and direc-tions for future work.

6.2 Problem description

Consider the chance constrained optimization problem

minx∈Rnx

J (x)

subject to:P(δ ∈∆ | max

j=1,...,nm

g j (x,δ)6 0)> 1−ε

(P1)

where δ ∈ ∆ ⊆ Rnδ , J : Rnx → R, and g j : Rnx ×∆→ R for all j = 1, . . . ,nm . The variable x isreferred to as the decision or design variable, while δ is referred to as the uncertain parame-ter. Moreover, any solution that satisfies the chance constraint of P1 is referred to as ε-levelfeasible. It is assumed that ∆ is endowed with a σ-algebra D, and that P is a probabilitymeasure defined over D. For all x ∈ Rnx and all j = 1, . . . ,nm , the functions g j (x, ·) : ∆→ R

are assumed to be measurable with respect to D and the Borel σ-algebra over R.

The standard scenario approach [42] substitutes the chance constraint in P1 with a finitenumber of hard constraints each corresponding to a different realization δ(k), k = 1, . . . , Nof the uncertain parameter δ, extracted according to P. This leads to the following scenarioprogram.

minx∈Rnx

J (x)

subject to:max

j=1,...,nm

g j (x,δ(k))6 0, for k = 1, . . . , N ,(P′

1)

where N is selected according to (6.1) with n = nx . We denote an optimal solution to P′1

as x∗, and note that x∗ is itself a random variable since the constraints in P′1 are defined in

terms of a collection of samples δ(k).

95


We then have the following theorem due to [42]:

Theorem 6.1. ([42, Thm. 1]) If J (·) is convex and g j (·,δ), j = 1, . . . ,nm , is convex for everyδ ∈ ∆, and N is selected according to (6.1) with n = nx , then the optimal solution x∗ of P′

1 isan ε-level feasible solution for P1 with probability at least 1−β.

If we denote the probability of constraint violation by

V (x) =P(δ ∈∆ | max

j=1,...,nm

g j (x,δ) > 0), (6.2)

the above statement is equivalent to

PN((δ(1), . . . ,δ(N )) ∈∆N | V (x∗)6 ε

)> 1−β, (6.3)

where PN is the product probability measure.

The main idea of our work is to eschew the direct application of Theorem 6.1 to problemP1, and instead construct and solve a robust version of P1 in the form:

minx∈Rnx

J (x)

subject to:max

j=1,...,nm

maxδ∈∆∩B∗ g j (x,δ)6 0,

(Pr ob)

where the set B∗ ⊂ Rnδ is to be determined. Under certain assumptions on the the prob-ability distribution of the uncertainty, [27] followed a similar approach, and solved P1 byclipping the probability distribution at some level set. Here we follow a different method,which is independent of the underlying probability distribution.

In the remainder of the paper we describe a method for constructing B∗ such that any so-lution x∗ of Pr ob will also be an ε-level feasible solution for P1 with probability at least 1−β.Our method for constructing the set B∗ is itself based on Theorem 6.1, but with the criti-cal feature that the required number of samples is based either on the dimension nδ of theuncertainty set or the number nm of constraints, rather than the number nx of the decisionvariables. We suggest two variations on this approach in the following sections, dependingon the structure of the constraint functions g j .

6.3 Method 1: Unstructured Constraints

We consider first the general case of problems where the constraint functions g j do notpossess any particular convexity or separability properties.

96

Method 1: Unstructured Constraints

6.3.1 Formulation

For simplicity of exposition, we will construct the set B∗ for Pr ob as a hyper-rectangle withoutward normals aligned with the canonical basis vectors in Rnδ . To this end, define con-stants εi ∈ (0,1) for i = 1, . . . ,nδ, such that

∑i εi = ε. We seek element-wise bounds τi :=

(τi , τi ) ∈ R2 such that δi ∈ [τi , τi ] with probability at least 1− εi , where δi ∈ R denotes thei th element of the uncertainty vector δ. For each i , we therefore consider the uncertainoptimization problem

minτi∈R2

(τi −τi

)subject to:

P(δ ∈∆ | δi ∈ [τi , τi ]

)> 1−εi .

(P2)

Since the objective function in P2 is convex and the probabilistic constraints are linear, wecan construct a solution using a scenario approach based on the results of Theorem 6.1.Define additional constants β ∈ (0,1) for i = 1, . . . ,nδ, such that

∑i βi = β, and choose Ni

from (6.1) with n = 2 (since P2 has only two decision variables). Consider now the followingscenario program corresponding to P2:

minτi∈R2

(τi −τi

)subject to:

δ(k)i ∈ [τi , τi ], for k = 1, . . . , Ni

(P3)

where δ(k)i denotes the element i of the sample k. In order to solve this collection of prob-

lems for i = 1, . . . ,nδ, a total of N = maxi Ni samples must be extracted, and for each suchproblem we choose arbitrarily a subset of these samples with cardinality Ni .

For each i = 1, . . . ,nδ, the optimal solution τ∗i := (τ∗i , τ∗i ) of P3 is εi -level feasible (in thesense of [42]) for P2 with probability at least 1−βi . This implies that

PNi((δ(1), . . . ,δ(Ni )) ∈∆Ni | V (τ∗i )6 εi

)> 1−βi , (6.4)

where V (·) is the probability of constraint violation, which in this case is given by

V (τi ) =P(δ ∈∆ | δi ∉ [τi , τi ]

). (6.5)

Finally, we construct the hyper-rectangle B∗ :=×nδi=1[τ∗i , τ∗i ], and pose the following robust

version of P1:

minx∈Rnx

J (x)

subject to:max

j=1,...,nm

maxδ∈∆∩B∗ g j (x,δ)6 0.

(P4)

Note that P4 is not a random program, but rather a robust optimization problem with theuncertainty taking values in the intersection of∆with the set B∗ that was constructed froma randomized solution of P2. The connection between P1-P4 is illustrated in Fig. 6.1.

97


5

existing set-up

Δf,ΔPtie−linen generators participating

in AGC

d1

d2

dn

outage

Pw

Reformulation 1 or 2

distribution vectord = [d1, . . . , dn]T

AGC

Fig. 1. Schematic diagram of the security constrained reserve schedulingalgorithm.

(P1) (P′1)

(P2) (P3) (P4)scenario approach

scenario approach

robust problemδ ∈ Δ ∩ B∗

Proposed methodology

Fig. 2. Schematic diagram highlighting the connections between problems(P1) − (P3).

A. Proposed reformulation 1

Assume that in the case where i ∈ IG we can distinguishbetween the mismatch that corresponds to wind deviation andthe one which occurs due to a generator outage. For i ∈ IG,we would thus have

Rit = d1,iup,t max

+(P f

w,t − Pw,t)

− d1,idown,t max+(Pw,t − P f

w,t) + d2,iup,tPiG,t, (16)

where no d2,idown,t vector is introduced, since it was assumedthat the network is not congested, i.e constraints similar to(9), (10) hold. In the opposite case, Ri

t could be definedas in (5), with different distribution vectors for wind devi-ation and generator outages. By considering the optimiza-tion problem that corresponds to (6)-(13) if the additionaldistribution vectors are introduced, d2,iup,tP

iG,t becomes the

only bilinear term, which appears both in the constraintsand the objective function. Setting z i

t = d2,iup,tPiG,t ∈ RNG

as a new decision variable, and defining the new decisionvector xt = [PG,t, dup,t, ddown,t, [d

1,iup,t]i∈IG , [d

1,idown,t]i∈IG ,

[zit]i∈IG , Rup,t, Rdown,t]T ∈ R3N2

G+5NG , the resulting prob-lem is linear in zit, and hence convex (with a chance con-straint). The new optimization problem is of the same structurewith (6)-(13), with the difference that instead of (10), (12), we

now have

d1,iup,t, d1,idown,t ≥ 0, for all i ∈ IG, (17)

1T d1,iup,t = 1,1T d1,idown,t = 1, for all i ∈ IG, (18)

1T zit = P iG,t, for all i ∈ IG. (19)

Moreover, the left-hand side of the last two constraints in(13) is replaced by the corresponding terms of (16). Oncethe solution to this problem is computed, d2,iup,t is calculatedas d2,iup,t = zit/P

iG,t if P i

G,t is not equal to zero, and is set tozero otherwise. Note that the sum of the elements of d2,iup,t isconfined to be one, since z i

t, i ∈ IG satisfies (19).For real time operation, the look-up table interpretation

(discussed in Section III.C) may be adopted. Given then amismatch P i

m,t = (Pw,t − P fw,t) − P i

G,t, the participation ofeach unit in compensating P i

m,t can be determined a posterioriby Ri

t/1T Ri

t. The latter requires knowledge of the mismatchterms.

Using this reformulation, a convex problem is achieved atthe expense of a more conservative reserve schedule. This isdue to the fact that Pw,t − P f

w,t, PiG,t are treated separately,

leading to reserves of higher cost. To see this, consider thecase where Pw,t−P f

w,t ≥ 0. The proposed formulation wouldlead to |Pw,t − P f

w,t|+ |P iG,t| MW of reserves, whereas only

|Pw,t − P fw,t − P i

G,t| MW are needed.

B. Proposed reformulation 2

In this subsection we overcome the bilinearity problem byusing an iterative algorithm (see Algorithm 1). We first attemptto identify a feasible solution of the problem, starting from anarbitrarily chosen power schedule P 0

G,t. At iteration k of thealgorithm, we fix P k,i

G,t only in (5) to the value obtained in theprevious iteration. Solving then (6)-(13) a new solution x k

t iscomputed, and P k

G,t is updated accordingly. If the algorithmconverges, its fixed point xk∗

t will be a feasible solution of theinitial problem.

At a second step, we use an alternating iterativescheme to refine the resulting feasible solution in termsof cost. At iteration k we first fix dk,iup,t, d

k,idown,t to the

values obtained at the previous step of the algorithm,and obtain

[P kG,t, d

kup,t, d

kdown,t, R

kup,t, Rk

down,t

]Tby

solving (6)-(13). We then fix P kG,t to the computed

value in all equations it appears, and solve for[dkup,t, d

kdown,t, [d

k,iup,t]i∈IG , [d

k,idown,t]i∈IG , R

kup,t, Rk

down,t

]T.

The entire process is then repeated until convergence. For abetter understanding, Fig. ?? shows how the power dispatchof each unit, and the obtained objective value change periteration, for the benchmark problem introduced in the nextsection. After 3 iterations the first part converges, whereasfor the second one only one iteration is needed. As expected,the cost is decreasing monotonically in the second part.

Note that the first part of Algorithm 1 is a heuristic schemeapplied to identify a feasible solution, and no convergenceguarantees can be provided. The second part of the algorithmconverges monotonically, since it is a bilinear descent iteration;the limit point however is not guaranteed to be the global

Figure 6.1: Schematic diagram illustrating the connections between problems P1-P4.

Our choice of a hyper-rectangular representation for B∗ is motivated by a desire to mini-mize the number of variables in P2, and hence the number of extracted scenarios requiredfor P3. One could alternatively construct a set B∗ using some other simple representation,e.g. by computing a spherical or ellipsoidal cover for the extracted scenarios in P3. Thenumber of variables required to parameterize any particular geometric representation forB∗ will then dictate both the number of scenarios required and the conservativeness of theresulting robust problem P4. Our approach and the corresponding proofs can be extendedto such cases.

We next provide conditions under which one can provide probabilistic guarantees aboutsolutions to the robust problem P4 akin to those provided by standard scenario approaches.Specifically, we provide conditions under which an optimal solution of P4 is feasible for thestochastic problem P1:

Proposition 6.1. Suppose that ε, β ∈ (0,1) and εi , βi ∈ (0,1) ,i = 1, . . . ,nδ, are chosen suchthat

ε=nδ∑

i=1εi , β=

nδ∑i=1

βi , (6.6)

and Ni is chosen according to (6.1) with n = 2. If x∗ is the optimal solution of P4, then x∗ isalso an ε-level feasible solution of P1, with probability at least 1−β.

Proof. It suffices to show that for N = maxi=1,...,nδ Ni ,

PN((δ(1), . . . ,δ(N )) ∈∆N | V (x∗)6 ε

)> 1−β, (6.7)

where V (x∗) is the probability of constraint violation for P1 (evaluated at x∗) and is definedby (6.2). If x∗ is the optimal solution of P4 then it will satisfy its constraints, so

maxj=1,...,nm

maxδ∈∆∩B∗ g j (x∗,δ)6 0, (6.8)

where B∗ is constructed from the values τ∗i , which are εi -level feasible solutions of P2 withconfidenceβi . Since the two max operators can be interchanged, the last statement implies

98

Method 1: Unstructured Constraints

that if δ ∈∆∩B∗ then max j=1,...,nm g j (x∗,δ)6 0. Hence,

1− V (x∗) =P(δ ∈∆ | max

j=1,...,nm

g j (x∗,δ)6 0),

>P(δ ∈∆ | δ ∈ B∗

),

= 1−P( nδ⋃

i=1

(δ ∈∆ | δi ∉ [τ∗i , τ∗i ]

)),

> 1−nδ∑

i=1P(δ ∈∆ | δi ∉ [τ∗i , τ∗i ]

). (6.9)

But by (6.5) we have that V (τ∗i ) = P(δ ∈∆ | δi ∉ [τ∗i , τ∗i ]

). The last statement, together with

(6.9), implies that

V (x∗)6nδ∑

i=1V (τ∗i ). (6.10)

Hence, for a multi-sample (δ(1), . . . ,δ(N )) ∈∆N we have

PN((δ(1), . . . ,δ(N )) ∈∆N | V (x∗)6 ε

),

>PN((δ(1), . . . ,δ(N )) ∈∆N |

nδ∑i=1

V (τ∗i )6 ε),

>PN((δ(1), . . . ,δ(N )) ∈∆N | V (τ∗i )6 εi , for all i = 1, . . . ,nδ

),

= 1−PN( nδ⋃

i=1

((δ(1), . . . ,δ(N )) ∈∆N | V (τ∗i ) > εi

)),

> 1−nδ∑

i=1PN

((δ(1), . . . ,δ(N )) ∈∆N | V (τ∗i ) > εi

),

= 1−nδ∑

i=1PNi

((δ(1), . . . ,δ(Ni )) ∈∆Ni | V (τ∗i ) > εi

),

> 1−β, (6.11)

where the first inequality is valid due to (6.10), and the last three follow from the subaddi-tivity of P, and (6.4), (6.6) respectively. The selection of the first Ni samples in the aboveprocedure was arbitrary, and any subset of δ(1), . . . ,δ(N ) with cardinality Ni could have beenchosen instead. The interpretation of this derivation is that the probability of all viola-tion probabilities V (τ∗i ) being simultaneously bounded by the corresponding εi is at least1−β.

6.3.2 Tractability of the proposed method

Proposition 6.1 suggests that for the class of systems which exhibit the structure of P1, onecan solve the problem P4 instead of using a scenario approach directly while achieving the

99


same probabilistic guarantees. Problem P4 is a robust optimization program with intervalbounded uncertainty, where the interval bounds are computed at an intermediate random-ized step solving P3.

In contrast to the scenario approach, we do not require convexity of P1 with respect to thedecision variables. The reason is that the scenario approach is only adopted to solve P2,which is trivially convex. Even though strictly speaking convexity is not an issue, ensuringP4 is tractable often requires all functions to be convex, since most of the known tractablerobust optimization problems are convex [22, 27, 24, 23]. Moreover, P4 requires δ to lieinside the intersection of∆with the bounds computed by P3. Therefore, unlike the scenarioapproach which makes no assumptions on ∆, our approach requires ∆ to be “nice” enoughso that the resulting robust problem is tractable. Still, our approach is applicable to classesof non-convex problems which, though of exponential complexity in general, can often besolved effectively by existing numerical tools, for example mixed integer linear problems[93]. Note that the standard scenario approach is inapplicable in this case.

For certain problem classes (linear programs, quadratic constrained quadratic programs,second order cone programs, semidefinite programs), it is shown in [27] that their robustcounterparts are tractable, and in the same program class as the original problem, i.e. ro-bust linear programs remain linear programs, etc.. This is achieved under the assumptionthat the constraint functions are concave and homogeneous with respect to the uncertaintyvector, and at the expense of introducing additional variables and constraints. Hence, if P4

satisfies these assumptions, and can be cast in one of the cases described in [27], tractabil-ity of our method is ensured. Moreover, since the uncertainty elements in P4 are subjectedto infinity norm bounds, solving P4 would require nm(nδ+1) additional decision variables,and nm(2nδ+1) linear constraints [27]. Therefore, following the proposed approach for thisclass of systems results in a robust optimization problem with the same complexity as [27],but with natural probabilistic guarantees without assumptions on the probability distribu-tion and arbitrary truncation.

6.4 Method 2: Structured Constraints

We next consider the particular case of problems where the constraint functions g j possessa separable structure in (x,δ).

6.4.1 Formulation

We make the following assumption:

Assumption 6.1. For all j = 1, . . . ,nm , g j (x,δ) := p j (x)q j (δ), where p j : Rnx → R and q j :∆→R.

100

Method 2: Structured Constraints

The results of this section are easily generalized to the case where p j : Rnx → R` and q j :∆→R`, and g j (x,δ) := ⟨p j (x)q j (δ)⟩, thus allowing for systems which are affine with respectto the uncertainty functions q j` (note that some of the q j`(·) could be made trivial). Thenumber of scenarios that is required to be extracted in this case would depend on the totalnumber of uncertainty functions, i.e. nm`. To simplify notation, we restrict our attention tothe simpler constraints of Assumption 6.1.

In a manner similar to that in Section 6.3 and using similar notation1, we will construct ahyper-rectangle B∗

q such that it encloses the image of a collection of samples δ(k) under thefunction q(δ) = (

q1(δ), . . . , qnm (δ)). B∗ will then be the pre-image of B∗

q under q . To thisend, we consider the uncertain optimization problems

minτ j∈R2

(τ j −τ j

)subject to:

P(δ ∈∆ | q j (δ j ) ∈ [τ j , τ j ]

)> 1−ε j ,

(P′2)

and the corresponding scenario problems

minτ j∈R2

(τ j −τ j

)subject to:

q j (δ(k)j ) ∈ [τ j , τ j ], for k = 1, . . . , N j ,

(P′3)

where once again we choose N j from (6.1) with n = 2, since P′2 has only two decision vari-

ables. A total of N = max j N j samples must be extracted, and for each such problem wechoose arbitrarily a subset of these samples with cardinality N j .

For each j = 1, . . . ,nm , the optimal solution τ∗j := (τ∗j , τ∗j ) of P′3 is ε j -level feasible (in the

sense of [42]) for P′2 with probability at least 1−β j . This is equivalent to

PN j((δ(1), . . . ,δ(N j )) ∈∆N j | V (τ∗j )6 ε j

)> 1−β j , (6.12)

where V (·) is the probability of constraint violation, which in this case is given by

V (τ j ) =P(δ ∈∆ | q j (δ) ∉ [τ∗j , τ∗j ]

). (6.13)

Finally, we construct a hyper-rectangle B∗q := ×nm

j=1[τ∗j , τ∗j ] from the solutions to P′3, and

define its pre-image under q as B∗ :={δ ∈Rnδ

∣∣∣ q(δ) ∈ B∗q

}. We can then pose the following

robust version of P1:

minx∈Rnx

J (x)

subject to:max

j=1,...,nm

maxδ∈∆∩B∗ p j (x)q j (δ)6 0.

(P′4)

1Note, that in problems P2-P3, we construct a hyper-rectangle B∗ ⊂ Rnδ by solving a collection of prob-lems indexed by i = 1, . . . ,nδ. In the analogous problems P′

2-P′3 of the present section, we instead construct B∗

by solving a collection of problems indexed by j = 1, . . . ,nm .

101


The connection between P′2-P′

4 is similar to that of P2-P4, as shown in Fig. 6.1.

We next provide conditions under which one can provide probabilistic guarantees aboutsolutions to the robust problem P′

4 akin to those provided based on standard scenario ap-proaches. Specifically, we provide conditions under which an optimal solution of P′

4 is fea-sible for the stochastic problem P1. In addition, we determine the probability with whichthe optimal solution of P1 is feasible for P′

4.

Proposition 6.2. Suppose throughout that Assumption 6.1 holds. Then:

1) Assume that ε, β ∈ (0,1) and ε j , β j ∈ (0,1) , j = 1, . . . ,nm , are chosen such that

ε=nm∑j=1

ε j , β=nm∑j=1

β j , (6.14)

and N j is chosen according to (6.1) with n = 2. If x∗ is an optimal solution of P′4, then x∗ is

also an ε-level feasible solution of P1, with probability at least 1−β.

2) Assume that x∗ is an ε-level optimal solution of P1, and select any β ∈ (0,1) and an integerN such that ε = 1− (1−β)1/N . Select any (ε j ,β j ), j = 1, . . . ,nm , such that N j 6 N and 6.14holds, and use these values to construct the set B∗ from P′

3. Then x∗ is a feasible solution ofP′

4 with probability at least 1−β.

Proof. 1) The proof of this part is similar to that of Proposition 6.1, and is included in Ap-pendix A.3 for the sake of completeness.

2) If x∗ is an ε-level optimal solution of P1, then

P(δ ∈∆ | max

j=1,...,nm

p j (x∗)q j (δ)6 0)> 1−ε. (6.15)

Select any β ∈ (0,1), and an integer N such that ε6 1− (1−β)1/N . Then, for N independentuncertainty extractions δ(k), with k = 1, . . . , N ,

P(δ(k) ∈∆ | max

j=1,...,nm

p j (x∗)q j (δ(k))6 0)> (1−β)1/N . (6.16)

Due to independence, for the joint event we have that

PN((δ(1), . . . ,δ(N )) ∈∆N | max

j=1,...,nm

p j (x∗)q j (δ(k))6 0, for all k = 1, . . . , N)> 1−β. (6.17)

Select now ε j , β j , j = 1, . . . ,nm , such that N j 6 N , and solve P′3. Let τ∗j ,τ∗i denote the solu-

tion of P′3, use it to construct q−1[B∗], and formulate P′

4. The argument inside the probabil-ity of (6.17) implies that for all j = 1, . . . ,nm and k = 1, . . . , N , p j (x∗)q j (δ(k)) 6 0. Therefore,it also holds that for all j = 1, . . . ,nm and k = 1, . . . , N j , p j (x∗)q j (δ(k)) 6 0; the choice of thefirst N j samples is arbitrary, and any subset of δ(1), . . . ,δ(N ) with cardinality N j could havebeen selected instead. The latter implies that

PN((δ(1), . . . ,δ(N )) ∈∆N | max

j=1,...,nm

maxδ∈∆∩B∗ p j (x∗)q j (δ)6 0

)> 1−β, (6.18)

102

Discussion and numerical results

where the last step holds since, under Assumption 6.1, the constraint functions are linearwith respect to q j (·). The last statement implies that with probability at least 1−β,

maxj=1,...,nm

maxδ∈∆∩B∗ p j (x∗)q j (δ)6 0, (6.19)

or in other words, with probability at least 1−β, x∗ is a feasible solution for P′4.

Note that an argument similar to the second part of Proposition 6.2 does not in generalhold for the more general problem described in Section 6.3. The reason is that the stepfrom (6.17) to (6.18) would no longer be valid, since the fact that for all j = 1, . . . ,nm andk = 1, . . . , N j , g j (x∗,δ(k))6 0, does not necessarily imply that maxδ∈∆∩B∗ g j (x∗,δ)6 0 for allj = 1, . . . ,nm . An analogous statement can still be made if δ is a scalar uncertainty, and forall j = 1, . . . ,nm , g j (·, ·) is monotone with respect to δ.

6.4.2 Tractability of the proposed method

The first part of Proposition 6.2 has an interpretation similar to that of Proposition 6.1, andimplies that under Assumption 6.1, solving P′

4 instead of P1 provides the same probabilisticguarantees with the standard scenario approach P′

1. As in the unstructured case of Section6.3, if P′

4 can be cast in the class of problems described in [27] then its robust counterpartis tractable, and its size is the same with the one reported in Section 2.3. An additionalfeature of the structured case addressed in this section is that in P′

4 we treat each functionq j (δ), j = 1, . . . ,nm as an uncertainty input. It is then often the case that the assumption of[27] regarding concavity and homogeneity of the constraints with respect to the uncertaintyvector is trivially satisfied.

6.5 Discussion and numerical results

6.5.1 Discussion

We first compare the scenario approach and the proposed methods of Sections 6.3-6.4 interms of the number of samples that they require to provide certain probabilistic guaran-tees. In the case addressed in Section 6.3 we used the scenario approach to solve P2 sepa-rately for each i = 1, . . . ,nδ. This requires generating in total maxi=1,...,nδ Ni samples, usingNi of them for each individual problem. The selection of these Ni scenarios could be arbi-trary, and could be carried out even in a greedy way, as in [49].

Under these conditions, the optimal solutions τ∗i of each problem would violate the corre-sponding constraint at most by εi . This provides additional design freedom, and allows usto introduce different levels of violation for each uncertainty element. Alternatively, for agiven ε and βwe could compute simultaneously bounds for all elements of the uncertaintyvector by solving P2 with n = 2nδ. The term ε would bound in this case the probability that

103


at least one of the constraints is violated, but we would have no guarantees regarding theviolation probability of each individual constraint. Following the same reasoning for thecase addressed in Section 6.4, the number of scenarios that need to be generated in thiscase is given by

N u > 2

ε

(2nδ+ ln

1

β

), (6.20)

N s > 2

ε

(2nm + ln

1

β

), (6.21)

for the formulations of Sections 6.3 and 6.4, respectively. It is easy to show that N u is alwaysless than or equal to the number of scenarios that should be extracted if for all i = 1, . . . ,nδ,the same violation level εi and confidenceβi was selected. The latter is in turn always lowerthan the case where an uneven distribution of εi and βi is used. A similar statement holdsfor N s .

Figure 6.2a shows how N , N u (respectively N s) depend on nx and nδ (nm). The “blue”surface shows that N increases linearly with nx , whereas it remains constant with respectto nδ (nm). Similarly, the “red” surface, which is of higher “slope”, highlights the fact thatN u (N s) increases linearly with nδ (nm), without depending on nx . Figure 6.2b shows howN u and N s depend on nδ and nm respectively, for a given number nx of decision variables.In the region of Figure 6.2a where the “blue” surface is above the “red” one, choosing one ofour two proposed methods instead of the scenario approach may be of advantage, since itleads to N u < N (N s < N ). In particular, if nδ < nm , the unstructured formulation should beemployed with the number of scenarios selected based on (6.20), whereas in the oppositecase, and if Assumption 6.1 is satisfied, the structured formulation should be employedwith the number of scenarios selected according to (6.21).

Although choosing the approach that results in generating fewest scenarios prevents usfrom over-sampling, it does not necessarily lead to a computationally simpler problem.This depends on the structure (i.e. the number and type of decision variables and con-straints) of the resulting robust problem, and how run-time depends on the number ofdecision variables and constraints. For the standard scenario approach, the number of de-cision variables remains equal to nx , whereas the number of constraints is nm N . On theother hand, both of our approaches result in a robust program with interval bounds on theuncertainty. If the problem can be classified in the framework of [27], the number of deci-sion variables will in general be higher than the scenario approach, whereas the number ofconstraints may also be higher, even if N u and N s are smaller than N (cf. the discussion inSection 6.3.2). Therefore, selecting the most computationally effective approach dependson the number of scenarios and the size of the resulting problem.

The proposed approach exhibits some common features with [129], where the feasibilityproblem for systems that are affine with respect to the decision variables and the uncer-tainty is investigated. Under certain assumptions on the underlying probability distribu-tion, the authors of [129] show that the number of scenarios that should be generated to

104


(a)

020

4060

80100

020

4060

80100

0

1000

2000

3000

4000

5000

nδn

m

# sc

enar

ios

Nu

Ns

(b)

Figure 6.2: a) Dependency of the number of scenarios N (“blue”), N u(N s) (“red”) on nx andnδ(nm) respectively, for ε = 10−2 and β = 10−5. b) Dependency of the number of scenariosN u (“red”), N s (“green”) on nδ and nm respectively, for ε= 10−2, β= 10−5, and nx = 1.

achieve the desired probabilistic guarantees, depends logarithmically on both ε and β, re-sulting in tighter bounds compared to the our approach, independent of nx , nδ or nm . Apartfrom requiring certain structure for the probability distribution of the uncertainty, extend-ing the approach of [129] to the case where an objective criterion needs to be simultane-ously optimized is not straightforward.

6.5.2 Numerical examples

We investigate the performance of our approach by means of two numerical examples; amore realistic application can be found in [174]. All simulations were carried out on an IntelCore 2 Duo 2.66-GHz processor running Windows 7, using the solver CPLEX [76] under theMATLAB interface YALMIP [95].

6.5.2.1 Example 1

To compare the approach of Section 6.3 with the standard scenario approach, consider theproblem

minx∈Rnx

‖x‖1


j=1,...,nm

(aT

j x +bTj δ

)6 0

)> 1−ε

(6.22)

where δ ∈ Rnδ is normally distributed with zero mean and identity covariance matrix. Weconsider problem instances with nx = nm = 1, . . . ,20, nδ = 1, . . . ,5, and solve for each one the

105


(a) (b)

Figure 6.3: Example 1: a) Expected (over 100 multi-samples), empirical probability of con-straint violation, computed for 10,000 uncertainty realizations, using both our approachand the standard scenario approach with ε= 10−2 and β= 10−5. b) Average (over 100 multi-samples) computational time, obtained using our approach and the scenario approach.

0 5 10 15 20

05

1015

20

−8

−6

−4

−2

0

2

4

x 10−3

nm

nx

Diff

eren

ce in

E(p

roba

bilit

y of

con

stra

int v

iola

tion)

met

hod

2 −

sce

nario

app

roac

h

(a) (b)

Figure 6.4: Example 2: a) Difference between the expected (over 100 multi-samples), em-pirical probability of constraint violation obtained when using our approach and the sce-nario approach, computed for 10,000 uncertainty realizations (only cases with nm 6 nx

were considered). b) Average (over 100 multi-samples) computational time, obtained usingour approach and the scenario approach (only cases with nm 6 nx were considered).

corresponding scenario program with ε= 10−2 and β= 10−5, where for all j = 1, . . . ,nm , thevectors a j ∈Rnx and b j ∈Rnδ have all of their elements uniformly distributed in [−1, 1]. Foreach case, we use the standard scenario approach with N given by (6.1) with n = nx , andour approach with N u chosen according to (6.20). Although Assumption 6.1 is satisfied, wedo not apply the approach of Section 6.4, since N s > N for every nm = nx .

106


We computed for each case the empirical probability of constraint violation, using 10000uncertainty realizations, not including the ones used for the optimization procedure. More-over, the entire process was repeated for 100 different multi-sample extractions. For eachnm , coefficients a j ,b j , j = 1, . . . ,nm were generated randomly and were kept constant for alluncertainty realizations and multi-sample extractions. Fig. 6.3a shows the expected value(over the 100 multi-samples) of the empirical probability of constraint violation. Note thatin all cases the empirical probability of constraint violation is much smaller than the theo-retical guarantees.

Consider first the case of scalar uncertainty, i.e. nδ = 1. For the scenario approach, theprobability of constraint violation is related to the robustness of the solution of the cor-responding scenario program, which is computed based on N scenarios. Increasing thenumber of scenarios results in a more robust solution, which in turn leads to a lower vio-lation probability. Hence, when using the scenario approach the probability of constraintviolation decreases with respect to nx , since the number of scenarios increases with the di-mension of the decision vector. For the case where our approach is employed, the interpre-tation of the result is similar. Since N u is independent of nx , the probability of constraintviolation does not change significantly with the number of decision variables. Moreover,for the case where N u < N , our approach results in a higher violation probability comparedto the scenario approach, hence it leads to a less conservative solution (see Figure 6.3a fornδ = 1).

As nδ increases, our approach leads to a much lower probability of constraint violation rel-ative to the scenario approach. This is due to the fact that the solution of the scenario ap-proach is guaranteed to be robust only with respect to N uncertainty realizations, indepen-dent of nδ. Therefore, for uncertainties of high dimension, the extracted scenarios sparselycover the uncertainty space, thus increasing the probability of constraint violation. In con-trast, using our approach a robust problem with bounded uncertainty is solved. These un-certainty bounds are computed based on N u scenarios, but the solution of P4 will also berobust with respect to many more uncertainties beyond the N u samples. This leads to lowprobabilities of constraint violation even in the case where N u < N . Hence, for nδ > 1, ourapproach results in a more conservative performance, although it requires fewer samplesfrom the probability distribution of the uncertainty. Using an alternative parametrizationfor the uncertainty set (e.g. ellipsoidal) would not necessarily lead to a less conservativeperformance since the number of scenarios that should be extracted (and enclosed by thisset) may be higher compared to the case where a hyper-rectangle is selected. In particular,the number of scenarios depends on the number of parametrization variables.

To compare our approach and the scenario approach in terms of cost, we compute for everyproblem instance the minimum (over the 100 multi-samples) value of the objective func-tion (6.22) for the case where our method was employed, and the maximum value when thescenario approach was used. The resulting cost surfaces follow a pattern similar to Figure6.3a, since the cost is related to the level of conservatism of each approach, i.e. the moreconservative the solution the higher the cost incurred. The average cost difference between

107


our approach and the scenario approach is independent of the number of decision vari-ables, and increases with the dimension of the uncertainty vector (for nδ = 5 the averagecost increases by 100%).

Figure 6.3b shows the average (based on the 100 multi-samples) total computational time(i.e. sample extraction time plus time to solve the resulting program) for the scenario ap-proach and our method. In all cases and for all multi-sample extractions, our approachrequired considerably less computation time.

6.5.2.2 Example 2

To compare the approach of Section 6.4 with the standard scenario approach consider theproblem

minx∈Rnx

‖x‖1


j=1,...,nm

(aT

j x +b jδsin(kiπδ))6 0

)> 1−ε,

(6.23)

where δ ∈ R is normally distributed with zero mean and unit covariance. For ε = 10−2 andβ = 10−5, we consider problem instances with nx = 1, . . . ,20, nm = 1, . . . ,nx , where for allj = 1, . . . ,nm , a j ∈Rnx , b j ∈R and ki ∈Rhave their elements uniformly distributed in [−1, 1];if nm > nx feasibility problems may occur due to the random entries of the constraint matri-ces. We then apply the scenario approach and our robust approach based on the structuredmethod of Section 6.4 (cf. the discussion following Assumption 6.1 regarding the applicabil-ity of this method) with N s chosen based on (6.21). Using instead the approach of Section6.3 would require fewer scenarios since δ is a scalar, but results in an intractable robustproblem due to the nonlinear uncertainty function.

We again compute for every problem instance the empirical probability of constraint vio-lation using 10000 uncertainty realizations. Repeating the process for 100 different multi-sample extractions, the expected value of the empirical probability of constraint violationwas determined. For each nm , coefficients a j ,b j , j = 1, . . . ,nm were generated randomlyand were kept constant for all uncertainty realizations and multi-sample extractions. Bothapproaches result in very low violation probabilities (a maximum value of 0.019 was en-countered), while exhibiting a similar pattern. Figure 6.4a shows the difference in empiri-cal constraint violation probability between our approach and the scenario approach. Byinspection of Figure 6.4a, the difference is positive if nm 6 nx/2. The latter coincides withintuition, since in this case our approach results in the generation of fewer scenarios com-pared to the scenario approach (N s < N ), which in turn leads to a higher probability ofconstraint violation with our approach.

Both our approach and the scenario approach result in similar objective values (maximumdifference of 10%) for all problem instances and multi-sample extractions. This is becausethe uncertainty functions are not coupled through the constraints, hence the conservatism

108


inherent in the robust program P′4 is not pronounced. Figure 6.4b shows the average com-

putational times of our approach and the scenario approach. In all cases and for all multi-samples, our approach resulted in lower computation time.


An alternative approach for solving chance constrained optimization problems was pre-sented. The proposed method bridges robust optimization and the scenario approach, andinvolves solving a robust problem with interval bounded uncertainty, where the uncertaintybounds are themselves computed probabilistically using the scenario approach. Two newmethods for computing solutions to the original problem were proposed, and it was shownthat for a suitable choice of the number of scenarios they exhibit equivalent probabilis-tic performance to the standard scenario approach. Moreover, although both methods re-sulted in a robust problem, we obtain probabilistic guarantees without any assumptions onthe underlying probability distribution.

Current research concentrates on investigating alternative ways of parameterizing the un-certainty set at the first step of the proposed methodology. Specifically, we focus on charac-terizing the number of scenarios that should be generated so that their convex hull containsa certain percentage of the probability mass. Our aim is a reduction in the conservatism ofthe solution of the resulting robust problem, exhibited in Figures 6.3 and 6.4. Moreover, inview of tightening the bound offered by the scenario approach for a general class of systems,the potential of extending the work of [129] will be investigated.

109

CHAPTER

7Reserve scheduling for power systemswith wind power generation

7.1 Introduction

The expected increase in the installed wind power capacity highlights the necessity of revis-iting certain operational concepts, like reserve scheduling, to take into account the unpre-dictable nature of wind. In a liberalized power market, the problem of reserve procurementis a task of the Transmission System Operator (TSO). In the event of system imbalances theTSO has to counter those by reserves procured in an auction ahead of the physical deliveryof energy. Practice shows various design options of this process [148], [149]. However, therising share of wind power and other unpredictable power sources, highlights the necessityof improved methods for reserve power determination.

The optimization of reserve power always includes the trade-off between the costs of hav-ing energy imbalances, leading to either load shedding or energy spillage, and the cost ofreserves. To tackle this trade-off, in most cases a stochastic optimization framework is used.The underlying optimization may consist of a security-constrained market-clearing proce-dure incorporating the unit commitment and reserve power determination, with an objec-tive function which seeks to maximize an expected social welfare [69], [37], [38], [36], [137].In this context, [69], [128] used a multi-stage stochastic program to obtain a solution for thecombined problem. In [144], a similar framework is considered, without taking the networkconstraints into account.

Following [174], [173], in this work we adopt the probabilistic framework of Chapter 6 tosolve the problem of designing a day-ahead dispatch for the generating units, while deter-mining the minimum cost reserves. In [111], an alternative algorithm was developed, wherethe so called unit-commitment problem was separated from the reserve scheduling, and itwas solved in a deterministic set-up. The main drawback in that formulation was that theunit-commitment problem gives rise to a mixed integer quadratic program, since the on-off status of each generating unit needs to be determined. Using the appropriate alternative

111

Reserve scheduling for power systems with wind power generation

of the scenario approach, as this was introduced in the previous subsection, both the unit-commitment and the reserve scheduling problem can be tackled simultaneously. This isnot possible with the standard scenario approach since it requires convexity of the under-lying problem with respect to the decision variables. Moreover, to achieve a more realisticrepresentation of the reserves compared to [111], we model the steady state behavior of thesecondary frequency controller. This leads to representing the reserves as a linear functionof the total generation-load mismatch, which may occur due to the difference between theactual wind from its forecasted value [174], [175]. A Markov chain based model is employedto generate the wind power realizations [111], [143] that are needed for the scenario-baseddesign, and the resulting problem can be solved easily by existing numerical tools [76],

Section 7.2 introduces the problem and provides some basic definitions, whereas Section7.3 provides a detailed description of the proposed probabilistic formulation. In Section7.4 we provide information regarding the simulation set-up, and present the simulationresults of a Monte Carlo study. Finally, Section 7.5 provides some concluding remarks anddirections for future work.

7.2 Problem set-up

7.2.1 Definitions and preliminaries

In this section we formulate the problem of reserve scheduling for networks with windpower generation as a chance constrained optimization program, and apply the approachof Chapter 6 to solve it numerically. We consider a power network comprising NG gener-ating units, NL loads, Nl lines, and Nb buses, and base our work on the following assump-tions.

1. A standard DC power flow approach [12] is adopted.

2. Wind generation is located at a single bus of the network.

3. No load uncertainty is considered.

4. No component outages (line, load, or generator) are considered.

The first assumption is rather standard for this type of problems, whereas the second andthird one are included to simplify the presentation of our results and be can be easily re-laxed. The last assumption excludes the case of a component outage, which were investi-gated in a similar way in [174], [173].

Under these assumptions, the power flow equations and the power injection vector can bewritten as

P f = B f θ, (7.1)

P = BBU Sθ, (7.2)

112

Problem set-up

where P f ∈ RNl contains the power flows of each line, and P ∈ RNb and θ ∈ RNb denotethe active power injections and the voltage angles at every bus of the network respectively.Moreover, B f = Y Mc ∈RNl×Nb and BBU S = M T

c Y Mc ∈RNb×Nb , where Y ∈RNl×Nl denotes theadmittance matrix, and Mc ∈ RNl×Nb is the adjacency matrix of the power network. Giventhat the network is represented by a directed graph, the entries of Mc are 1, −1 or 0, and thesum of the elements of each row is zero.

We first eliminate θ from (7.1), (7.2) to represent the power flows P f as a function of thepower injections P . Since BBU S is singular with rank Nb −1, we choose one angle as a ref-erence and set it to zero. Without loss of generality let θNb = 0 and BBU S ∈ R(Nb−1)×(Nb−1),θ ∈ RNb−1, P ∈ RNb−1 denote the remaining parts of BBU S , θ, and P respectively. We then

have θ = (BBU S)−1P , and using θ = [θ 0

]T, we get

P f = B f

[(BBU S)−1P

0

]. (7.3)

The power injection vector P can be written in a generic form as

P =[CG (PG +R)+Cw Pw −CLPL

]Nb−1

, (7.4)

where [·]Nb−1 denotes the first Nb − 1 rows of the quantity inside the brackets. PG ∈ RNG ,Pw ∈R, and PL ∈RNL denote the generation dispatch, the wind power in-feed and the load,respectively. R ∈ RNG is a power correction term, which is related to the reserves of eachgenerator and will be defined in the next subsection. Matrices CG ,Cw ,CL are of appropriatedimension, and their element (i , j ) is “1” if generator j (respectively wind power/load) isconnected to the bus i , and zero otherwise.

7.2.2 Reserve representation

Reserves are needed to balance generation-load mismatches, which may occur due to adifference between the actual wind power and its forecast value, or as an effect of a genera-tor/load loss (not considered in this work). Such imbalances between load and generationinduce frequency deviations and activate the primary frequency controller. Secondary fre-quency control (or Automatic Generation Control (AGC)) is then activated and adjusts theproduction of the generators to compensate for the remaining frequency error (for detailssee [11], and the example in Section 5.4.2). The AGC output is distributed in a weighted wayto the participating generators. Hence, in the new steady state, the power setpoint of thesegenerators is changed by a certain percentage of the active power imbalance. In the currentenergy market, this percentage is the result of contracting agreements between producersand the TSO concerning the secondary frequency control reserves. The product of theseweights with the worst case imbalance results in the amount of reserves that each generat-ing unit may be called upon to provide. In the sequel we will refer to the vector that includesthese weights as the distribution vector.

113


To encode the change of the generating output, we define the power correction term R tobe a linear function of the total generation-load mismatch, which is the difference of thewind power from its forecast value. Therefore,

R = dup max(− (Pw −P f

w ),0)−ddown max

(Pw −P f

w ,0), (7.5)

where Pw −P fw ∈ R denotes the deviation of the wind power Pw from the forecast P f

w . Thisterm is directly related to the reserves since for every mismatch, it shows the amount ofpower with which each generator should adjust its production. Vectors dup ∈RNG , (ddown ∈RNG ) represent the distribution vectors. The sum of their elements is one, and if a generatoris not contributing to the AGC, the corresponding element in the vector is zero. The indicesup and down are used to distinguish between the up and down spinning reserves. If thedeviation Pw −P f

w is negative, up-spinning reserves are provided, and the production ofthe generators is increased accordingly, whereas in the opposite case the second term of(7.5) is active, providing down-spinning reserves. Introducing different distribution vectorsfor up-spinning and down-spinning reserves provides us with more degrees of freedom,thus avoiding feasibility issues.

Notice that dup , ddown are allowed to have negative elements as well; the interpretation of

some of the elements of dup being negative for Pw < P fw is that the corresponding gener-

ators should provide down-spinning reserves so that congestion is relieved, while the restof the generators would provide up-spinning. Still, more up-spinning reserves will be pro-vided, i.e. 1T dup max

(− (Pw −P fw ),0

)> 0. If no congestion occurs, the elements of the

distribution vectors will always be positive, since this would lead to a solution with lowercost.

7.3 Problem formulation

The main objective is to design a minimum cost day-ahead dispatch and reserve schedule.To achieve this, we also need to solve a unit-commitment problem, which involves a binarydecision about the “on-off” status of each generating unit. We consider an optimizationhorizon Nt = 24 with hourly steps, and introduce the subscript t in our notation to charac-terize the value of the quantities defined in the previous section for a given time instancet = 1, . . . , Nt . Moreover, let C1,C2,Cup ,Cdown ∈ RNG be generation and reserve cost vectors,and [C2] denote a diagonal matrix with vector C2 on the diagonal. As also defined in [156],a quadratic form for the production cost is considered, whereas motivated by [128] the re-serve cost was considered to be linear.

For each step t of the optimization problem, define the vector of decision variables to be

xt =[PG ,t ,dup,t ,ddown,t ,C SU

t ,ut ,Rup,t , Rdown,t]T ∈ R7NG , where C SU

t ∈ RNG is the start-upcost vector, ut ∈ {0,1}NG denotes the status of each generator, whereas Rup,t ,Rdown,t ∈ RNG

denote the probabilistically worst case up-down spinning reserves that the system operator

114

Problem formulation

needs to purchase. The resulting optimization problem is given by

min{xt }

Ntt=1

Nt∑t=1

(C T

1 PG ,t +P TG ,t [C2]PG ,t +1T C SU

t +C Tup Rup,t +C T

downRdown,t

), (7.6)

subject to

1. Power balance constraints: For all t = 1, . . . , Nt

1T (CG PG ,t +Cw P fw ,t −CLPL,t ) = 0. (7.7)

Following the discussion of the previous subsection, this constraint encodes the factthat the power balance in the network should be always satisfied when Pw ,t = P f

w ,t . Inother words, the sum of all generation dispatches of the conventional units and theforecast wind power production, should match the total load of the system. If loaduncertainty is also taken into account, the equality constraint should be satisfied forthe forecast load value.

2. Start-up cost constraints: For t = 1, . . . , Nt ,

C SUt >λSU

t (ut −ut−1), (7.8)

C SUt > 0, (7.9)

where λSUt ∈ RNG×NG is a diagonal matrix including the start-up costs. Note that (7.8)

and (7.9) impose a lower bound on C SUt . It is then (7.6) which implies that C SU

t willalways be zero unless the corresponding generator changes status from “off” to “on”within two consecutive periods [128].

3. Generation and transmission capacity constraints: For all t = 1, . . . , Nt

ut Pmin 6 PG ,t 6 ut Pmax,

−P l i ne 6B f

[(BBU S)−1Pt

0

]6 P l i ne , (7.10)

where Pmin,Pmax ∈ RNG denote the minimum and maximum generating capacity ofeach unit, P l i ne denotes the line limits, and Pt is given by (7.4) with Pw ,t = P f

w ,t . Thesetwo constraints denote the generation and transmission capacity constraints for thedeterministic case where the wind power is equal to its forecast value.

4. Distribution vector constraints: For all t = 1, . . . , Nt

1T dup,t = 1,

1T ddown,t = 1, (7.11)

This constraint encodes the fact that the elements of the distribution vectors shouldsum up to one.

115


5. Probabilistic constraints: For all t = 1, . . . , Nt

P(Pw ,t ∈R | −P l i ne 6B f

[(BBU S)−1Pt

0

]6 P l i ne ,

ut Pmin 6 PG ,t +Rt 6 ut Pmax,

−Rdown,t 6Rt 6Rup,t

)> 1−ε, (7.12)

where Pt is defined in (7.4) and Rt is given by (7.5). Probability in (7.12) is meantwith respect to the probability distribution of the wind power production Pw ,t ∈ R.The first constraint inside the probability encodes the standard transmission capac-ity constraints, whereas the second one provides guarantees that the scheduled gen-eration dispatch plus the reserve contribution Rt will not result in a new operatingpoint outside the generation capacity limits. The last constraint of (7.12) is includedto determine the reserves Rup,t ,Rdown,t as the worst case, in a probabilistic sense,value of the power correction term Rt . For the sake of simplicity, we considered thesame probability level ε for each time-step t = 1, . . . , Nt , while the proposed frame-work could also capture the case of different probability levels per stage.

The resulting problem is a multi-stage, chance constrained mixed-integer quadratic pro-gram, whose stages are only coupled by the binary variables, which encode the “on-off”status of each generating unit. Here, at the expense of a suboptimal solution, we treat themin a sequential manner, resulting in problems of smaller size. Therefore, each individualproblem is of the form of (P1) with δ = Pw ,t ∈ R, and ut−1 is substituted by the optimal so-lution of the previous time-step. Due to the presence of the binary variables, the resultingproblem is not convex with respect to the decision vector xt , hence the standard scenarioapproach of Section 6.2 can not be applied. By contrast, the method proposed in Chapter6 does not require convexity of the underlying problem, as long as the robust counterpartof the original problem is tractable. For our case this is a robust mixed-integer quadraticprogram with scalar uncertainty, and can be easily solved using existing numerical tools[76]. Due to the structure of the problem, we employ Method 1 of Section 6.3, resulting inN u = ⌈2

ε

(2+ ln 1

β

)⌉scenarios (see (6.20) with nδ = 1), where d·e denotes the smaller integer

greater than or equal of its argument. Following this sequential approach, with probabilityat least 1−β, the constraints of every hour will be satisfied with probability at least 1− ε.Therefore, with probability at least 1− Ntβ, the resulting solution will satisfy the chanceconstraint (7.12) of the day-ahead problem.

Note that if we did not treat each stage separately, we should generate scenarios based on(6.20) with nδ = 24, resulting in a more conservative solution. To solve then the resultingrobust problem, the tractable reformulation of [93] may be adopted, leading to the formu-lation of [119]. Therefore, the choice of solving the multi-stage program directly or in asequential manner, is based on a trade-off between the optimality of the resulting solutionand the conservatism of the probabilistic guarantees we can offer.

116

Simulation study

7.4 Simulation study

7.4.1 Wind power model and Simulation set-up

To apply our scenario based method, we need a model to generate scenarios of the uncer-tain parameter i.e. the wind power. To achieve this, we assume that the wind power is thesum of a deterministic component, which is the available forecast, and a stochastic one,which models the error between the forecast and the actual wind power. To construct arealistic wind power error generator, we employed the method introduced in [143], whichproposes a Markov chain-based model to generate wind power time series that take intoaccount the temporal correlation of the wind power error (i.e. the difference between theforecast and the actual wind power).

We used five-year, hourly measured, normalized wind power data (both actual and fore-casted values), for Germany over the period 2006-2011. Discretizing the wind power error,the transition probability matrix Ptr for the wind power error was constructed. Fig. 7.1adepicts the transition probability matrix using a Markov chain with 41 states for the windpower error. The pronounced block-triangular structure reveals the strong auto-correlationof the wind power error. For illustrative purposes, Fig. 7.1b shows the forecast and the ac-tual wind power for a single day, as well as 5,000 wind power realizations generated by theMarkov chain-based wind power generator.

To evaluate the effectiveness and the robustness of the proposed methodology, we compareit against a benchmark approach for reserve scheduling, which is based on considering thestationary distribution of the wind power error (i.e. forecast-actual wind power). The sta-tionary distribution π of the wind power error is computed as a vector whose entries areall non-negative, sum up to one, and satisfy π= P T

trπ. Fig. 7.1c shows the stationary distri-bution π of the wind power error, where the “shaded” regions denote the quantiles whichcontain an ε fraction of the probability mass. Two extreme scenarios are then considered;a low one corresponding to the forecast plus the ε

2 % percentile of the error distribution,and a high one corresponding to the forecast plus the (1− ε

2 )% percentile of the error dis-tribution (to provide the same ε-guarantees with the method of Section 6.3). Treating nowthe wind power as a bounded uncertainty, with the bounds corresponding to these two ex-treme cases, we compute the generation dispatch and the reserves by solving the robustcounterpart of (7.6)-(7.12).

To collect statistical results regarding the performance of our robust reserve schedulingmethodology (RRS) compared with the benchmark approach (BRS), we perform MonteCarlo simulations repeating the scheduling procedure for different forecast and actual windpower data. We the determined the amount of deployed reserves, load shedding and windgeneration spillage based on the actual wind power values. Specifically, for different val-ues of ε, we considered forecast and actual wind power data for 90 days of the period2007−2008; this dataset was not including when training the Markov chain based model. All

117


(a)

2 4 6 8 10 12 14 16 18 20 22 2420

30

40

50

60

70

80

90

time (h)

Win

d P

ower

(M

W)

actualforecast

(b)

5 10 15 20 25 30 35 400

0.05

0.1

0.15

0.2

0.25

Markov chain states

prop

abili

ty

ε/2 1 − ε/2

(c)

Figure 7.1: a) Transition probability matrix for the wind power error, using a Markov chainwith 41 states. b) Forecast (“blue”), actual (“red”) wind power, and 5,000 wind power sce-narios based on different error realizations, initialized with the actual wind power. c) Sta-tionary distribution of the wind power error. The “shaded” regions denote the quantiles,which contain an ε fraction of the probability mass, whereas the arrows indicate the ε/2and 1−ε/2 percentiles.

optimization problems were solved using the solver CPLEX [76] via the MATLAB interfaceYALMIP [95].

7.4.2 Simulation results

In this section we evaluate the performance of our approach by applying it to the IEEE 30-bus network [178]. The benchmark includes Nb = 30 buses, NG = 6 generators, Nl = 41lines, and is modified to include a wind power generator connected to bus 22. All numer-ical values for the network data and the production cost vectors are retrieved from [178],whereas the reserve costs Cup ,Cdown were assumed to be equal, and were defined by scal-ing down C1 by a factor of ten [128]. As in most of the current market structures, we did not

118

Simulation study

1% 5% 10% 15% 20%3000

4000

5000

6000

7000

8000

9000

10000

probability level ε

Tot

al c

ost w

ith R

RS

($)

(a)

1% 5% 10% 15% 20%3000

4000

5000

6000

7000

8000

9000

10000


Tot

al c

ost w

ith B

RS

($)

(b)

Figure 7.2: Total cost for 90 different days and ε= 1%,5%,10%,15%,20%: a) Calculated withRRS. b) Calculated with BRS. The “red” line corresponds to the median value, the edges ofthe box correspond to the 25th and 75th percentiles, whereas the whiskers extend to a 99%coverage. The “red” marks denote the data outliers.

1% 5% 10% 15% 20%0

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2


Pro

babi

lity

of lo

ad s

hedd

ing

or w

ind

gene

ratio

n sp

illag

e

BRSRRStheoretical guarantees

Figure 7.3: Empirical probability of load shedding or wind generations spillage for RRS(“blue”) and BRS (“red”). The dotted “green” line corresponds to the theoretical probabilis-tic guarantees.

consider any cost for wind generation spillage. For the scenario generation we consideredβ= 10−4.

For 90 different days, and for probability levels ε = 1%,5%,10%,15%,20%, Fig. 7.2a showsthe total cost when our proposed approach (RRS) is employed, calculated as the sum of theproduction and reserves cost, the cost of the reserves that were actually deployed once theactual wind was realized, and the cost due to load shedding actions. As expected, for highervalues of ε, it is more likely to resort to load shedding to meet the supply-demand balance,thus leading to a higher total cost. Similarly, Fig. 7.2b shows the same results when the

119


2 4 6 8 10 12 14 16 18 20 22 240

2

4

6Lo

ad s

hedd

ing

(MW

)

2 4 6 8 10 12 14 16 18 20 22 240

5000

10000

15000

Tot

al c

ost (

$)

2 4 6 8 10 12 14 16 18 20 22 240

10

20

30

time (h)

Win

d po

wer

(M

W)

BRSRRS

BRSRRSno load shedding

forecastactualquantilesscenarios

Figure 7.4: Load shedding, total cost, and wind power (forecast, actual, quantiles and gen-erated scenarios) for one of the simulated days, and for both RRS and BRS.

benchmark approach (BRS) is used. The total cost is in general higher compared to the RRSapproach, since load shedding is more frequent in this case, even though both methodsprovide the same ε-type guarantees. The better performance of RRS, highlights the impor-tance of an accurate wind power modeling taking temporal correlation into account.

Fig. 7.3 depicts the empirical probability of load shedding or wind generation spillage, cal-culated as the number of hours that one of these actions had to be executed over the totalnumber of simulated hours (90×24). For both methods, this value is lower compared to thetheoretical guarantees of our method (“green”), and is in general increasing with respect toε. Nevertheless, RRS (“blue”) leads systematically to a more robust performance comparedto BRS (“red”). For illustrative purposes, Fig. 7.4 shows the cost, the amount of load shed-ding and the wind power for one of the simulated days. The last subplot depicts the fore-cast (“blue”), the actual wind power (“red”), the generated wind power scenarios (“green”)for ε = 20%, and the corresponding quantiles for the BRS approach (“black”). It should beremarked that load shedding occurs with BRS at the time instances where the actual windpower is lower than the lowest quantile. Since RRS provides the same guarantees but withsampling, it spans a wider range of values (not restricted inside the quantile region), andhence it can capture such cases avoiding shedding load. The total cost follows the load pat-tern of each approach, where the dotted “green” line in the second subplot corresponds to

120


the cost that would occur if no load was shed.

Even though RRS seems more conservative in some cases, its scenario based structure pro-vides a more general framework to handle uncertainty, since it takes into account the tem-poral correlation of the wind power error. The latter is of major importance especially if theoptimization stages are coupled, e.g. ramping constraints for the generators are taken intoaccount.


In this chapter a novel algorithm for scheduling reserves for power systems with wind powergeneration was developed. We formulated the problem as a chance constrained mixed-integer quadratic program, and used the methods developed in Chapter 6 to solve it. Todemonstrate the robustness of the obtained solution, a Monte Carlo study was carried out,and the proposed methodology was compared with a benchmark reserve scheduling ap-proach.

Current work concentrates on integrating the proposed reserve scheduling formulation ina security constrained optimization framework as proposed in [175], [174], [173], and onapplying the approach of [93] for solving the robust counterpart of the resulting multi-stagestochastic unit commitment problem. Moreover, instead of using a linearized model for thenetwork, a convex relaxation for AC power flow type of problems may be adopted [89].

121

CHAPTER

8Concluding remarks

Many applications in control engineering, involve solving a constrained optimal controlproblem. Especially for the case of complex systems, the presence of constraints gives riseboth to theoretical and computational challenges. On the one hand, developing a generaltheoretical framework is necessary to capture all features and interdependencies amongthe components of the underlying system. On the other hand, numerical computations forsuch detailed models might not be tractable. Therefore, a trade-off between the generalityof the developed theory and the scale of applications that can be addressed numericallyneeds to be reached. This dissertation concentrated on three of these problems, all of whichresulting in a constrained optimal control formulation.

8.1 Reachability with state constraints

The problem of computing reachable sets for continuous and hybrid systems with stateconstraints was investigated in Chapters 2,3 and 4. In Chapter 2, an optimal control frame-work was developed, enabling us to solve target hitting problems with state constraints andcompeting inputs, which in turn give rise to reach-avoid calculations. The main contribu-tion of this work is that the corresponding reach-avoid sets are related to the level sets of twovalue functions, that are the unique continuous viscosity solutions of two quasi-variationalinequalities. Moreover, continuity of the value function and the Hamiltonian of the system,enables the use of existing tools to solve the problem numerically.

Chapter 3 employs this formulation, and deals with the problem of computing viability setsfor hybrid systems. We provided a complete characterization of the problem based entirelyon optimal control, and consider three cases, according to whether the horizon of the con-tinuous calculation and the number of discrete transitions is finite or infinite. Typically, toaddress this problem an iterative algorithm is used; to prove its convergence we followed alogic based approach using Tarski’s fixed point theorem on lattices.

123

Concluding remarks

Finally, Chapter 4 demonstrated the applicability of the theoretical results of Chapters 2and 3 on a 4D trajectory management problem in air traffic control. To perform conflictavoidance while enabling the use of reachability based computational tools, an abstractionof the aircraft dynamics was presented.

Although these results, together with those in the literature, address a wide variety of reach-ability problems for hybrid systems, there are still a few open issues. The first is that thecontinuous time reach-avoid computation does not capture the case of input dependentstate constraints. Moreover, no transitions forced by the inputs are allowed, hence a widerange of problems is excluded. Finally, using transfinite induction we show that the itera-tive algorithm terminates at some ordinal number, but not necessarily the least one. Froman application point of view, in the conflict avoidance algorithm of Chapter 4, each conflictpair was treated separately, therefore the possibility that the proposed resolution causessubsequent conflicts is not excluded.

8.2 MPC for feedback linearizable systems with inputconstraints

The main objective of this problem, which was addressed in Chapter 5, was to combine astandard nonlinear control technique, namely feedback linearization, with model predic-tive control (MPC), to regulate feedback linearizable systems with input constraints. Thechallenge in that case lies in the fact that most systems are governed by nonlinear differen-tial equations, and usually linear constraints. Applying directly MPC would in general leadto a difficult optimization problem, with no guarantees that the global optimum will befound. An alternative approach would be to first apply feedback linearization, and then useMPC in the new coordinates. The problem in this case is that after feedback linearization,the initial input constraints are mapped to a set of state dependent, and in general nonlin-ear and possibly non convex bounds. To alleviate this problem, an iterative approach wasproposed, where at every step the resulting optimization problem involved only linear dy-namics and constraints. The efficiency of the proposed scheme was verified via simulationson various SISO and MIMO systems. Specifically, we investigated the problem of stabilizingthe the speed of a DC motor to the desired value, the frequency control problem for a twoarea power network, and the flight plan tracking problem for an aircraft during level flights.For the second example, we demonstrated how stability of the zero dynamics can be provenusing a reachability based approach.

Although the proposed iterative scheme reached a fixed point in all simulated examples,there are no formal guarantees regarding the convergence properties of the algorithm. An-other issue is that the performance of this approach relies on the stability of the zero dy-namics, which might not be always straightforward to show. In addition to this, the dimen-sion of the zero dynamics tends to increase for systems of high dimension, hence limitingthe applicability of this method. Moreover, to gain more insight regarding its robustness,

124

Chance constrained optimization

our algorithm should be tested against model and parameter uncertainty. The work of [176]provides a step toward this direction.

8.3 Chance constrained optimization

In Chapter 6, a novel approach for solving chance constrained optimization problems waspresented. The proposed method bridges robust optimization and the scenario approach,and involves solving a robust problem with interval bounded uncertainty, with boundscomputed probabilistically using the scenario approach. Two new methods for comput-ing solutions to the original problem were proposed, and it was shown that for a suitablechoice of the number of scenarios they exhibit equivalent probabilistic performance to thestandard scenario approach. The number of scenarios that need to be generated in ourcase, however, does not depend on the number of decision variables as in the scenarioapproach, but on the dimension of the uncertainty vector or the number of constraints de-pending on which of our two methods is used. Moreover, although both methods resultedin a robust problem, we obtain probabilistic guarantees without any assumptions on theunderlying probability distribution.

The basic properties of the proposed reformulations were demonstrated by means of nu-merical examples, whereas a more realistic application was provided in Chapter 7, wherethe new approach was applied to the problem of reserve scheduling for power networkswith wind power generation. Specifically, a novel reserve scheduling algorithm was devel-oped, and its efficiency was compared against a benchmark approach in terms of cost andperformance (i.e. load shedding and wind generation spillage) via Monte Carlo simulations.

Current research concentrates on investigating alternative ways of parameterizing the un-certainty set at the first step of the proposed methodology. Specifically, aiming in reducingthe conservatism of the solution of the resulting robust problem, we focus on characteriz-ing the number of scenarios that should be generated so that their convex hull contains acertain percentage of the probability mass. Moreover, in view of tightening the bound of-fered by the scenario approach for a general class of systems, the potential of extending thework of [129] will be investigated. From an application perspective, instead of using a lin-earized model for the network, a convex relaxation for AC power flow type of problems maybe adopted [89]. Moreover, the proposed reserve scheduling formulation can be integratedin a security constrained optimization framework as proposed in [175], [174], [173], and theapproach of [93] for solving the robust counterpart of the resulting multi-stage stochasticunit commitment problem may be adopted [119].

125

Appendix

A.1 Additional proofs of Chapters 2 and 3

A.1.1 Proof of Proposition 2.2.

Proof. Part 1. Following Lemma 8 of [123] we first show that R A(τ,R, A) ⊆ {x ∈Rn | V (x,τ)60}. Consider x ∈ R A(τ,R, A) and for the sake of contradiction assume that V (x, t ) > 0. Thenthere exists ε> 0 such that for all α(·) ∈ A[t ,T ], supd(·)∈D[t ,T ]

max{l (φ(T , t , x, α[d ](·),d(·))),

maxτ∈[t ,T ] h(φ(τ, t , x, α[d ](·),d(·)))} > 2ε > 0. This in turn implies that for all α(·) ∈ A[t ,T ]

there exists d(·) ∈ D[t ,T ] such that either l (φ(T , t , x, α[d ](·), d(·))) > ε > 0 or there exists τ ∈[t ,T ] such that h(φ(τ, t , x, α[d ](·), d(·))) > ε> 0.

Consider now the implications of x ∈ R A(τ,R, A). Equation (2.5) implies that there exists aα(·) ∈ A[t ,T ] such that for all d(·) ∈ D[t ,T ], and so also for d(·), we can define u(·) = α[d ](·).Then, for this u(·) and d(·) there exists τ1 ∈ [t ,T ] such that φ(τ1, x, t ,u(·), d(·)) ∈ R and for allτ2 ∈ [t ,τ1] φ(τ2, t , x,u(·), d(·)) ∉ A. Choose the freezing input signal as

u(τ) ={

1 for τ ∈ [t ,τ1],0 for τ ∈ [τ1,T ].

If we combine u(·) with u(·), we can get the input u(·), which will generate a trajectory

φ(τ, x, t , u(·), d(·)) ={

φ(τ, x, t ,u(·), d(·)) if τ ∈ [t ,τ1]φ(τ1, x, t ,u(·), d(·)) if τ ∈ [τ1,T ]

(A.1.1)

Case 1.1: Consider first the case where for all α(·) ∈ A[t ,T ] l (φ(T , t , x, α[d ](·), d(·))) > ε > 0.For τ= T we have that

φ(T , x, t , u(·), d(·)) =φ(τ1, x, t ,u(·), d(·)).

Appendix

Since x ∈ R A(τ,R, A), we showed before thatφ(τ1, x, t ,u(·), d(·)) ∈ R, i.e. l (φ(τ1, x, t ,u(·), d(·)))6 0. So from (A.1.1) we have that l (φ(T , x, t , u(·), d(·))) 6 0. Since u(·) = α[d ](·) is alreadynon-anticipative, and a non-anticipative strategy for u(·) can be designed, u(·) will also benon-anticipative. Therefore, the previous statement establishes a contradiction.

Case 1.2: Consider now the case where for all α(·) ∈ A[t ,T ] there exists τ ∈ [t ,T ] such thath(φ(τ, t , x, α[d ](·), d(·))) > ε> 0. Since we showed that for all τ ∈ [t ,τ1], φ(τ, t , x,u(·), d(·)) ∉A, we can conclude from (A.1.1) that for all τ ∈ [t ,τ1]

h(φ(τ, x, t , u(·), d(·)))6 0.

If τ ∈ [τ1,T ], we have that

φ(τ, x, t , u(·), d(·)) =φ(τ1, x, t ,u(·), d(·)).

So h(φ(τ, x, t , u(·), d(·))) = h(φ(τ1, x, t ,u(·), d(·))) 6 0. Hence, for all τ ∈ [t ,T ] we have thath(φ(τ, x, t , u(·), d(·)))6 0. Since in Case 1.1, u(·) was shown to be non-anticipative, we havea contradiction.

Part 2. Next, we show that {x ∈ Rn | V (x,τ) 6 0} ⊆ R A(τ,R, A). Consider (x, t ) such thatV (x, t ) 6 0 and assume for the sake of contradiction that x ∉ R A(τ,R, A). Then for all α(·) ∈A[t ,T ] there exists d(·) ∈ D[t ,T ] such that either for all τ1 ∈ [t ,T ], φ(τ1, t , x,α(·),d(·)) ∉ R orthere exists τ2 ∈ [t ,T ] such thatφ(τ2, t , x,α(·),d(·)) ∈ A∧∀τ′2 ∈ [t ,τ2],φ(τ′2, t , x,α(·),d(·)) ∉ R.

Consider the strategy α(·) ∈ A[t ,T ] (note that α(·) ∈ A[t ,T ] follows from the implications ofV (x, t ) 6 0 and will be defined in the sequel), which consists of a strategy α(·) ∈ A[t ,T ], asdefined in Section IIB, and an additional scalar component, that corresponds to u. Fol-lowing Lemma 8 of [123], by eliminating this scalar component, we can extract α(·) ∈A[t ,T ]

from α(·) ∈ A[t ,T ]. By the implications of x ∉ R A(τ,R, A), we can then choose the d(·) ∈D[t ,T ]

that corresponds to thatα(·). In Lemma 4 of [123], it was proven that the set of states visitedby the augmented trajectory is a subset of the states visited by the original one. We thereforehave that for all τ1 ∈ [t ,T ]

φ(τ1, x, t ,α[d ](·), d(·)) ∉ R =⇒ φ(τ1, x, t , α[d ](·), d(·)) ∉ R, (A.1.2)

or there exists τ∗2 ∈ [t ,T ] such that

φ(τ∗2 , x, t ,α[d ](·), d(·)) ∈ A =⇒ φ(τ∗2 , x, t , α[d ](·), d(·)) ∈ A, (A.1.3)

and similarly, ∀τ′2 ∈ [t ,τ∗2 ], φ(τ′2, x, t , α[d ](·), d(·)) ∉ R. By (A.1.2),(A.1.3), and based on thedefinition of R and A, we conclude that there exists a δ> 0 such that either for all τ1 ∈ [t ,T ]

l (φ(τ1, x, t , α[d ](·), d(·))) > δ> 0, (A.1.4)

or for some τ∗2 ∈ [t ,T ]

h(φ(τ∗2 , x, t , α[d ](·), d(·))) > δ> 0 and

∀τ′2 ∈ [t ,τ∗2 ] l (φ(τ′2, x, t , α[d ](·), d(·))) > δ> 0. (A.1.5)

128

Appendix

Under the convexity part of Assumption 2.1 V (x, t ) 6 0 implies that there exists a non-anticipative strategy α(·) ∈ A[t ,T ] such that supd(·)∈D[t ,T ]

, max{l (φ(T , t , x, α[d ](·),d(·))),maxτ∈[t ,T ] h(φ(τ, t , x, α[d ](·),d(·)))}6 0. Hence for all d(·) ∈D[t ,T ], l (φ(T , t , x, α[d ](·),d(·)))60 and for all τ ∈ [t ,T ], h(φ(τ, t , x, α[d ](·),d(·)))6 0. For d(·) = d(·) the last argument impliesthat

l (φ(T , x, t , α[d ](·), d(·)))6 0,

and for τ= τ∗2h(φ(τ∗2 , x, t , α[d ](·), d(·)))6 0.

The last statements contradict (A.1.4),(A.1.5) and complete the proof.

A.1.2 Proof of Lemma 2.1.

Proof. Following Theorem 3.1 of [66] we can define

W (x, t ) = infα(·)∈A[t ,t+k]

supd(·)∈D[t ,t+k]

max{

maxτ∈[t ,t+k]

h(φ(τ, t , x,α(·),d(·))),

V (φ(t +k, t , x,α(·),d(·)), t +k)}.

We will then show that for all ε > 0, V (x, t ) 6 W (x, t )+2ε and V (x, t ) > W (x, t )−3ε. Thensince ε> 0 is arbitrary, V (x, t ) =W (x, t ).

Case 1: V (x, t )6W (x, t )+2ε. Fix ε> 0 and choose α1(·) ∈A[t ,t+k] such that

W (x, t )> supd1(·)∈D[t ,t+k]

max{

maxτ∈[t ,t+k]

h(φ(τ, t , x,α1(·),d1(·))),

V (φ(t +k, t , x,α1(·),d1(·)), t +k)}−ε.

Similarly, choose α2(·) ∈A[t+k,T ] such that

V (φ(t +k, t , x,α1(·),d1(·)), t +k)

> supd2(·)∈D[t+k,T ]

max{l (φ(T , t +k,φ(t +k, t , x,α1(·),d1(·)),α2(·),d2(·))),

maxτ∈[t+k,T ]

h(φ(τ, t +k,φ(t +k, t , x,α1(·),d1(·)),α2(·),d2(·)))}−ε.

For any d(·) ∈ D[t ,T ] we can define d1(·) ∈ D[t ,t+k] and d2(·) ∈ D[t+k,T ] such that d1(τ) = d(τ)for all τ ∈ [t , t +k) and d2(τ) = d(τ) for all τ ∈ [t +k,T ]. Define also α(·) ∈A[t ,T ] by

α[d ](τ) ={α1[d1](τ) if τ ∈ [t , t +k),α2[d2](τ) if τ ∈ [t +k,T ].

It easy to see thatα : D[t ,T ] →U[t ,T ] is non-anticipative. By uniqueness,φ(τ, t , x,α(·),d(·)) =φ(τ, t , x,α1(·),d1(·)) in the case where τ ∈ [t , t + k), and also φ(τ, t , x,α(·),d(·)) = φ(τ, t +k,φ(t +k, t , x,α1(·),d1(·)),α2(·),d2(·)) if τ ∈ [t +k,T ].

129

Appendix

Hence,

W (x, t )> supd1(·)∈D[t ,t+k]

supd2(·)∈D[t+k,T ]

max{

maxτ∈[t ,t+k]

h(φ(τ, t , x,α1(·),d1(·))),

l (φ(T , t +k,φ(t +k, t , x,α1(·),d1(·)),α2(·),d2(·))),

maxτ∈[t+k,T ]

h(φ(τ, t +k,φ(t +k, t , x,α1(·),d1(·)),α2(·),d2(·)))}−2ε

> supd(·)∈D[t ,T ]

max{l (φ(T , t , x,α(·),d(·))), max

τ∈[t ,T ]h(φ(τ, t , x,α(·),d(·)))

}−2ε

>V (x, t )−2ε.

Therefore, V (x, t )6W (x, t )+2ε.

Case 2: V (x, t )>W (x, t )−3ε. Fix ε> 0 and choose now α(·) ∈A[t ,T ] such that

V (x, t )> supd(·)∈D[t ,T ]

max{l (φ(T , t , x,α(·),d(·))), max

τ∈[t ,T ]h(φ(τ, t , x,α(·),d(·)))

}−ε. (A.1.6)

By the definition of W (x, t )

W (x, t )6 supd(·)∈D[t ,t+k]

max{

maxτ∈[t ,t+k]

h(φ(τ, t , x,α(·),d(·))),V (φ(t +k, t , x,α(·),d(·)), t +k)}.

Hence there exists a d1(·) ∈D[t ,t+k] such that

W (x, t )6max{

maxτ∈[t ,t+k]

h(φ(τ, t , x,α(·),d1(·))),V (φ(t +k, t , x,α(·),d1(·)), t +k)}+ε. (A.1.7)

Let d(τ) = d1(τ) for all τ ∈ [t , t +k) and d(τ) = d ′(τ) for all τ ∈ [t +k,T ]. Let also α′ ∈A[t+k,T ]

to be the restriction of the non-anticipative strategy α(·) over [t + k,T ]. Then, for all τ ∈[t +k,T ], we define α′[d ′](τ) =α[d ](τ). Hence

V (φ(t +k, t , x,α(·),d1(·)), t +k)

6 supd ′(·)∈D[t+k,T ]

max{l (φ(T , t +k,φ(t +k, t , x,α(·),d1(·)),α′(·),d ′(·))),

maxτ∈[t+k,T ]

h(φ(τ, t +k,φ(t +k, t , x,α(·),d1(·)),α′(·),d ′(·)))}.

and so there exists a d2(·) ∈D[t+k,T ] such that

V (φ(t +k, t , x,α(·),d1(·)), t +k)

6max{l (φ(T , t +k,φ(t +k, t , x,α(·),d1(·)),α′(·),d2(·))),

maxτ∈[t+k,T ]

h(φ(τ, t +k,φ(t +k, t , x,α(·),d1(·)),α′(·),d2(·)))}+ε. (A.1.8)

We can define

d(τ) ={

d1(τ) if τ ∈ [t , t +k),d2(τ) if τ ∈ [t +k,T ].

Therefore, from (A.1.7) and (A.1.8)

W (x, t )6max{l (φ(T , t , x,α(·),d(·))), max

τ∈[t ,T ]h(φ(τ, t , x,α(·),d(·)))

}+2ε,

which together with (A.1.6) implies V (x, t )>W (x, t )−3ε.

130

Appendix

A.1.3 Proof of Lemma 2.2.

Proof. Since l , and h are bounded, V is also bounded. For the second part fix x, x ∈Rn andt ∈ [0,T ]. Let ε> 0 and choose α(·) ∈A[t ,T ] such that

V (x, t )> supd(·)∈D[t ,T ]

maxτ∈[t ,T ]

max{l (φ(T , t , x, α(·),d(·))),h(φ(τ, t , x, α(·),d(·)))}−ε.

By definition

V (x, t )6 supd(·)∈D[t ,T ]

maxτ∈[t ,T ]

max{l (φ(T , t , x, α(·),d(·))),h(φ(τ, t , x, α(·),d(·)))}.

We can choose d(·) ∈D[t ,T ] such that

V (x, t )6 maxτ∈[t ,T ]

max{l (φ(T , t , x, α(·), d(·))),h(φ(τ, t , x, α(·), d(·)))}+ε,

and hence

V (x, t )−V (x, t )6 maxτ∈[t ,T ]

max{l (φ(T , t , x, α(·), d(·))),h(φ(τ, t , x, α(·), d(·)))}

− maxτ∈[t ,T ]

max{l (φ(T , t , x, α(·), d(·))),h(φ(τ, t , x, α(·), d(·)))}+2ε.

For all τ ∈ [t ,T ]:

|φ(τ, t , x, α(·), d(·))−φ(τ, t , x, α(·), d(·))|

=∣∣∣(x − x)+

∫ T

t[ f (φ(s, t , x, α(·), d(·)))− f (φ(s, t , x, α(·), d(·)))]d s

∣∣∣6 |x − x|+

∫ T

t| f (φ(s, t , x, α(·), d(·)))− f (φ(s, t , x, α(·), d(·)))|d s

6 |x − x|+C f

∫ T

t|φ(s, t , x, α(·), d(·))−φ(s, t , x, α(·), d(·))|d s,

where C f is the Lipschitz constant of f . By the Gronwall-Bellman Lemma [151] (p.86), thereexists a constant Cx > 0 such that for all τ ∈ [t ,T ]

|φ(τ, t , x, α(·), d(·))−φ(τ, t , x, α(·), d(·))|6Cx |x − x|.

Let τ0 ∈ [t ,T ] be such that

h(φ(τ0, t , x, α(·), d(·))) = maxτ∈[t ,T ]

h(φ(τ, t , x, α(·), d(·))).

Then,

V (x, t )−V (x, t )6max{l (φ(T , t , x, α(·), d(·))),h(φ(τ0, t , x, α(·), d(·)))}

−max{l (φ(T , t , x, α(·), d(·))),h(φ(τ0, t , x, α(·), d(·)))}+2ε.

131

Appendix

Case 1. l (φ(T , t , x, α(·), d(·)))> h(φ(τ0, t , x, α(·), d(·)))

V (x, t )−V (x, t )

6 l (φ(T , t , x, α(·), d(·)))−max{l (φ(T , t , x, α(·), d(·))),h(φ(τ0, t , x, α(·), d(·)))}+2ε

6 l (φ(T , t , x, α(·), d(·)))− l (φ(T , t , x, α(·), d(·)))+2ε

6ClCx |x − x|+2ε.

Case 2. l (φ(T , t , x, α(·), d(·))) < h(φ(τ0, t , x, α(·), d(·)))

V (x, t )−V (x, t )

6 h(φ(τ0, t , x, α(·), d(·)))−max{l (φ(T , t , x, α(·), d(·))),h(φ(τ0, t , x, α(·), d(·)))}+2ε

6 h(φ(τ0, t , x, α(·), d(·)))−h(φ(τ0, t , x, α(·), d(·)))+2ε

6ChCx |x − x|+2ε.

So in any case V (x, t )−V (x, t ) 6 max{Cl ,Ch}Cx |x − x| + 2ε. The same argument with theroles of x, x reversed establishes that V (x, t )−V (x, t )6max{Cl ,Ch}Cx |x − x|+2ε. Since ε isarbitrary,

|V (x, t )−V (x, t )|6max{Cl ,Ch}Cx |x − x|.Finally consider x ∈ Rn and t , t ∈ [0,T ]. Without loss of generality assume that t < t . Letε> 0 and choose α(·) ∈A[t ,T ] such that

V (x, t )> supd(·)∈D[t ,T ]

maxτ∈[t ,T ]

max{l (φ(T , t , x,α(·),d(·))),h(φ(τ, t , x,α(·),d(·)))}−ε

> maxτ∈[t ,T ]

max{l (φ(T , t , x,α(·),d(·))),h(φ(τ, t , x,α(·),d(·)))}−ε.

By definition,

V (x, t )6 supd(·)∈D[t ,T ]

maxτ∈[t ,T ]

max{l (φ(T , t , x, α(·),d(·))),h(φ(τ, t , x, α(·),d(·)))}.

So we can choose d(·) ∈D[t ,T ] such that

V (x, t )6 maxτ∈[t ,T ]

max{l (φ(T , t , x, α(·), d(·))),h(φ(τ, t , x, α(·), d(·)))}+ε,

where α ∈ A[t ,T ] is the restriction of α(·) over [t ,T ]. Then, for all τ ∈ [t ,T ], we defineα[d ](τ) = α[d ](τ), and d(τ) = d(τ+ t − t ). By uniqueness, for all τ ∈ [t ,T ] we have thatφ(τ, t , x, α(·), d(·)) =φ(τ+ t − t , t , x,α(·),d(·)).

V (x, t )−V (x, t )> maxτ∈[t ,T ]

max{l (φ(T , t , x,α(·),d(·))),h(φ(τ, t , x,α(·),d(·)))}

− maxτ∈[t ,T ]

max{l (φ(T , t , x, α(·), d(·))),h(φ(τ, t , x, α(·), d(·)))}−2ε.

132

Appendix

Case 1. l (φ(T , t , x, α(·), d(·)))>maxτ∈[t ,T ] h(φ(τ, t , x, α(·), d(·)))

V (x, t )−V (x, t )

> maxτ∈[t ,T ]

max{l (φ(T , t , x,α(·),d(·))),h(φ(τ, t , x,α(·),d(·)))}− l (φ(T , t , x, α(·), d(·)))−2ε

> l (φ(T , t , x,α(·),d(·)))− l (φ(T , t , x, α(·), d(·)))−2ε

= l (φ(T , t , x,α(·),d(·)))− l (φ(T + t − t , t , x,α(·),d(·)))−2ε

>−ClC f |T −T − t + t |−2ε

=−ClC f |t − t |−2ε,

where Cl is the Lipschitz constant of l (·).

Case 2. l (φ(T , t , x, α(·), d(·))) < maxτ∈[t ,T ] h(φ(τ, t , x, α(·), d(·)))

V (x, t )−V (x, t )

> maxτ∈[t ,T ]

max{l (φ(T , t , x,α(·),d(·))),h(φ(τ, t , x,α(·),d(·)))}− maxτ∈[t ,T ]

h(φ(τ, t , x, α(·), d(·)))−2ε

> maxτ∈[t ,T ]

h(φ(τ, t , x,α(·),d(·)))− maxτ∈[t ,T ]

h(φ(τ, t , x, α(·), d(·)))−2ε.

Let τ0 ∈ [t ,T ] be such that

h(φ(τ0, t , x, α(·), d(·))) = maxτ∈[t ,T ]

h(φ(τ, t , x, α(·), d(·))).

Then

V (x, t )−V (x, t )> maxτ∈[t ,T ]

h(φ(τ, t , x,α(·),d(·)))−h(φ(τ0, t , x, α(·), d(·)))−2ε

> h(φ(τ0, t , x,α(·),d(·)))−h(φ(τ0, t , x, α(·), d(·)))−2ε

= h(φ(τ0, t , x,α(·),d(·)))−h(φ(τ0 + t − t , t , x,α(·),d(·)))−2ε

>−ChC f |τ0 −τ0 − t + t |−2ε

=−ChC f |t − t |−2ε,

where Ch is the Lipschitz constant of h(·). In any case we have that

V (x, t )−V (x, t )>−max{Cl ,Ch}C f |t − t |−2ε.

A symmetric argument shows that V (x, t )−V (x, t ) 6 max{Cl ,Ch}C f |t − t |+2ε, and since εis arbitrary this concludes the proof.

A.1.4 Proof of Theorem 3.1.

Proof. By Lemma 3.3, there exists k ∈λ such that for all j > k, Reach(t ,Pr e∃(W j ),Pr e∀(W j ))= Wk . We show first that Viab(N ,T )

W0⊆ Wk ; this part is similar to the first part of Proposition

133

Appendix

3.2. Since Viab(N ,T )W0

⊆ W0 and Wk ⊆ W0, it suffices to show that W0 \ Wk ⊆ W0 \ Viab(N ,T )W0

.Take (q , x) ∈W0 \Wk . Fix any hybrid strategy (α,γ). We show that we can find a closed looprun starting at (q0(τ0), x0(τ0)) = (q , x) leaving W0 in finite time and after a finite number ofdiscrete transitions, thus proving that (q , x) 6∈ Viab(N ,T )

W0.

Since (q , x) 6∈Wk , there exists i < k such that (q , x) 6∈Wi = Reach(t ,Pr e∃(Wi−1),Pr e∀(Wi−1))(Definition 3.6). By Proposition 3.1 (part 2), (q , x) 6∈ Pr e∃(Wi−1). Therefore, either x ∈Dom(q), or there exists δ and q ′ such that x ∈ G(q , q ′) and (q ′,r (q , q ′, x,γ(q , x), δ)) 6∈ Wi−1.In the latter case, set τ′0 = 0, q1(τ1) = q ′, x1(τ1) = r (q , q ′, x,γ(q , x), δ)) and notice that τ1 = 0and (q1(τ1), x1(τ1)) 6∈ Wi−1. If now x ∈ Dom(q), there exists d(·) such that the solutionφ(·, q , x,α,d) reaches Pr e∀(Wi−1) without first reaching Pr e∃(Wi−1). Hence, there existst1 ∈ [0,T ] such that x(t1) ∈ Pr e∀(Wi−1) and for all t2 ∈ [0, t1), x(t2) ∈ Dom(q) \ Pr e∃(Wi−1).Let x0(t ) = x(t ) for all t ∈ [0, t1]. By the definition of Pr e∀, either (q0(t1), x0(t1)) 6∈ Wi−1 orthere exist δ and q ′ such that x0(t1) ∈ G(q , q ′) and (q ′,r (q , q ′, x0(t1),γ(q , x0(t1)), δ)) 6∈ Wi−1.In the latter case, set τ′0 = t1, q1(τ1) = q ′ and x1(τ1) = r (q , q ′, x0(t1),γ(q , x0(t1)), δ) and noticethat τ1 = t1 <∞ and (q1(τ1), x1(τ1)) 6∈Wi−1.

Overall, starting from (q , x) ∈ W0 \ Wi we have constructed a run that leaves Wi−1 after afinite amount of time and at most one discrete transition. Iterating i times we construct arun that leaves W0 in finite time and after at most i discrete transitions. Hence for any αwehave found d and {δi }i such that the associated run starting from (q , x), leaves W0 in finitetime. Hence (q , x) 6∈ Viab(N ,T )

W0.

For the second part of the proof, we need to show that Wk ⊆ Viab(N ,T )W0

. Following [70],any closed hybrid discriminating domain (see footnote at the end of Section 3.3.2) con-tained in W0 = F × [0,T ] is also contained in Viab(N ,T )

W0. By Proposition 3.1, and since Wi+1 =

Reach(t ,Pr e∃(Wi ),Pr e∀(Wi )), Wi is closed for all i ∈λ. By Theorem 1 of [70], to prove thatWk is a discriminating domain, it suffices to show that Wk = Reach(t ,Pr e∃(Wk ),Pr e∀(Wk )).The last argument was proven in Lemma 3.3, and hence shows that Viab(N ,T )

W0= Wk . More-

over, by the second statement of Lemma 3.3 and Theorem 1 of [70], Wk is the largest hybriddiscriminating domain (see Definition 3.5) contained in F × [0,T ]. The last statement con-cludes the proof.

134

Appendix

A.2 Reach-Avoid Algorithm of Chapter 4 for ConflictResolution

In this Appendix, the algorithm that summarizes the steps of the reach-avoid calculation ofChapter 4, Section 4.3.3, is presented. For each aircraft j , this algorithm is solved backwardsin time between two consecutive TWs.

Algorithm 4.1 Reach-Avoid computation for each aircraft j .

1 Let [t j , t j ] denote the TW temporal constraints of aircraft j ,

2 Stage 0. B Solve from t = t j until the previous TW:3 for t ∈ [t j t j ]

4 Solve ∂V∂t (x j , t )+min{0,supd∈D infu j∈U j

∂V∂x j

(x j , t ) f j (x j ,u j ,d)} = 0,

5 with boundary condition V (x j , t j ) = l (x j ), B it was assumed that TW do not overlap.6 for i 6= j7 C j i = {x j | i , j are in conflict at time t},8 Define A j i ,t = {x j | h j i (x j , t ) > 0} ⊇C j i to be the smallest box containing C j i .9 end for

10 if A j ,t =⋃i 6= j A j i ,t 6= ;B captures the case of multiple conflicts.

11 h j (x j , t ) = maxi 6= j h j i (x j , t ), B characterizes A j ,t .12 end if13 end for14 for t 6 t j

15 Solve ∂V∂t (x j , t )+ supd∈D infu j∈U j

∂V∂x j

(x j , t ) f j (x j ,u j ,d) = 0,

16 with boundary condition V (x j , t j ) = V (x j , t ).17 Repeat steps 6−12 and compute A j ,t .18 end for19 Stage 1. B For t ∈ [t j t j ] repeat steps 4−5 and compute V (x j , t ).

20 if A j ,t 6= ;21 V (x j , t ) = max(h j (x j , t ),V (x j , t )). B performs the “masking” operation of Section 2.3.22 end if23 Stage 2. B For t 6 t j repeat steps 15−16 and compute V (x j , t ).

24 Repeat steps 20−22 with V instead of V and avoid region A j ,t .

135

Appendix

A.3 Additional proofs of Chapter 6

A.3.1 Proof of Proposition 6.2.1

Proof. It suffices to show that for N = max j=1,...,nm N j

PN((δ(1), . . . ,δ(N )) ∈∆N | V (x∗)6 ε

)> 1−β, (A.3.1)

where V (·) is given by (6.2). If x∗ is the optimal solution of P′4, it will satisfy its constraints,

so

maxj=1,...,nm

maxδ∈∆∩B∗ p j (x∗)q j (δ)6 0, (A.3.2)

where B∗ is the pre-image of B∗q , which is in turn constructed from the values of τ∗j , which

are ε j -level feasible solution of P′2 with confidence β j . The last statement implies that if

δ ∈∆∩B∗, then

maxj=1,...,nm

p j (x∗)q j (δ)6 0.

Hence,

1− V (x∗) =P(δ ∈∆ | max

j=1,...,nm

p j (x∗)q j (δ)6 0),

>P(δ ∈∆ | δ ∈ B∗

),

=P(δ ∈∆ | q j (δ) ∈ [τ∗j ,τ∗j ]

),

= 1−P( nm⋃

j=1

(δ ∈∆ | q j (δ) ∉ [τ∗j ,τ∗j ]

)),

> 1−nm∑j=1P(δ ∈∆ | q j (δ) ∉ [τ∗j ,τ∗j ]

). (A.3.3)

But by (6.13) we have that V (τ∗j ) =P(δ ∈∆ | q j (δ) ∉ [τ∗j ,τ∗j ]

). Therefore,

V (x∗)6nm∑j=1

V (τ∗j ). (A.3.4)

136

Appendix

Hence, for a multi-sample (δ(1), . . . ,δ(N )) ∈∆N we have

PN((δ(1), . . . ,δ(N )) ∈∆N | V (x∗)6 ε

),

>PN((δ(1), . . . ,δ(N )) ∈∆N |

nm∑j=1

V (τ∗j )6 ε),

>PN((δ(1), . . . ,δ(N )) ∈∆N | V (τ∗j )6 ε j , for all j = 1, . . . ,nm

),

= 1−PN( nm⋃

j=1

((δ(1), . . . ,δ(N )) ∈∆N | V (τ∗j ) > ε j

)),

> 1−nm∑j=1PN

((δ(1), . . . ,δ(N )) ∈∆N | V (τ∗j ) > ε j

),

= 1−nm∑j=1PN j

((δ(1), . . . ,δ(N j )) ∈∆N j | V (τ∗j ) > ε j

),

> 1−β, (A.3.5)

where the first inequality is valid due to (A.3.4), and the last three follow from the subaddi-tivity of P, and (6.12), (6.14) respectively. The selection of the first N j samples in the aboveprocedure was arbitrary, and any subset of δ(1), . . . ,δ(N ) with cardinality N j could have beenchosen instead.

137

Bibliography

[1] “Eurocontrol Experimental Centre. User manual for the base of aircraft data (BADA)revision 3.3,” 2002. [Online]. Available: http://www.eurocontrol.fr/projects/bada/

[2] “Advanced Required Navigation Performance (A-RNP) Real-Time Simulation,” April2010. [Online]. Available: http://www.eurocontrol.int/crds/gallery/content/public/reports/10067-CRDSSIMRTS-FEU_01.01.pdf

[3] “SESAR The ATM Target Concept - Deliverable D3,” December 2006. [Online]. Avail-able: http://www.eurocontrol.int/sesar/public/standard_page/documentation.html

[4] “Validation Strategy and Plan - Deliverable D1.3.1,” December 2008. [Online].Available: http://www.cats-fp6.aero/cats-fp6/public_deliverables.html

[5] “SESAR Definition Phase - Deliverable D1 - Air Transport Framework - The currentsituation,” July 2006. [Online]. Available: http://www.eurocontrol.int/sesar/public/standard_page/documentation.html

[6] “Report on modelling and quantitative risk analysis - Deliverable D2.2.1.2,” July 2010.[Online]. Available: http://www.cats-fp6.aero/cats-fp6/public_deliverables.html

[7] “CATS State of the Art - Deliverable D1.1,” June 2008. [Online]. Available:http://www.cats-fp6.aero/cats-fp6/public_deliverables.html

[8] “Next Generation Air Transportation System (NGATS) Air Traffic Management (ATM)- Airportal Project,” May 2007. [Online]. Available: http://www.aeronautics.nasa.gov/nra_pdf/airspace_project_c1.pdf

[9] “HIL2 primary results Analysis - Deliverable D2.1.2,” May 2010. [Online]. Available:http://www.cats-fp6.aero/cats-fp6/public_deliverables.html

139

http://www.eurocontrol.fr/projects/bada/

http://www.eurocontrol.int/crds/gallery/content/public/reports/10067-CRDSSIMRTS-FEU_01.01.pdf

http://www.eurocontrol.int/crds/gallery/content/public/reports/10067-CRDSSIMRTS-FEU_01.01.pdf

http://www.eurocontrol.int/sesar/public/standard_page/documentation.html


http://www.cats-fp6.aero/cats-fp6/public_deliverables.html





http://www.aeronautics.nasa.gov/nra_pdf/airspace_project_c1.pdf

http://www.aeronautics.nasa.gov/nra_pdf/airspace_project_c1.pdf


[10] “Report on Target Window Modeling - Deliverable D2.2.4.3,” October 2010. [Online].Available: http://www.cats-fp6.aero/cats-fp6/public_deliverables.html

[11] G. Andersson, Dynamics and Control of Electric Power Systems. Lecture Notes, ETHZurich, 2009.

[12] ——, Modeling and Analysis of Power Systems. Lecture Notes, ETH Zürich, 2011.

[13] J. P. Aubin, J. Lygeros, M. Quincampoix, S. Sastry, and N. Seube, “Impulse differentialinclusions: A viability approach to hybrid systems,” IEEE Transactions on AutomaticControl, vol. 47, no. 1, pp. 2–20, 2002.

[14] J. Aubin, Viability Theory. Boston:Birkhauser, 1991.

[15] M. Bardi and I. Capuzzo-Dolcetta, Optimal control and viscosity solutions ofHamilton-Jacobi-Bellman equations. Boston Birkhäuser, 2008.

[16] E. Barron and H. Ishii, “The Bellman equation for minimizing the maximum cost,”Nonlinear Analysis: Theory, Methods and Applications, vol. 13, no. 9, pp. 1067–1090,1989.

[17] E. Barron and R. Jensen, “Semicontinuous Viscosity Solutions For Hamilton Ja-cobi Equations With Convex Hamiltonians,” Communications in Partial DifferentialEquations, vol. 15, no. 12, pp. 1713–1742, 1990.

[18] T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory. New York: Aca-demic Press, 1982.

[19] A. M. Bayen, I. M. Mitchell, M. Oishi, and C. Tomlin, “Aircraft Autolander Safety Anal-ysis Through Optimal Control-Based Reach Set Computation,” Journal of Guidance,Control, and Dynamics, vol. 30, no. 1, pp. 68–77, 2007.

[20] A. Bemporad, F. Borrelli, and M. Morari, “Model Predictive Control Based on Lin-ear Programming - The Explicit Solution,” IEEE Transactions on Automatic Control,vol. 47, no. 12, pp. 1974 – 1985, 2002.

[21] A. Bemporad and M. Morari, “Robust Model Predictive Control: A Survey,” LectureNotes in Control and Information Sciences, Springer Berlin / Heidelberg, 1999.

[22] A. Ben-Tal and A. Nemirovski, “Robust convex optimization,” Mathematical Opera-tional Research, vol. 23, no. 4, pp. 769–805, 1998.

[23] ——, “Robust solutions to uncertain programs,” Operational Research Letters, vol. 25,no. 1, pp. 1–13, 1999.

[24] ——, “On tractable approximations of uncertain linear matrix inequalities affectedby interval uncertainty,” SIAM Journal on Optimization, vol. 12, no. 3, pp. 811–833,2002.

140


[25] D. Bertsimas, D. Pachamanova, and M. Sim, “Robust linear optimization under gen-eral norms,” Operations Research Letters, vol. 32, pp. 510–516, 2004.

[26] D. Bertsimas and M. Sim, “The Price of Robustness,” Operations Research, vol. 52,no. 1, pp. 35–53, 2004.

[27] ——, “Tractable Approximations to Robust Conic Optimization Problems,” Mathe-matical Programming, Series B, vol. 107, pp. 5–36, 2006.

[28] D. Bestle and M. Zeitz, “Canonical form observer design for non-linear time-invariantsystems,” International Journal of Control, vol. 38, no. 2, pp. 419–431, 1983.

[29] J. Birk and M. Zeitz, “Extended Luenberger observer for non-linear multivariable sys-tems,” International Journal of Control, vol. 47, no. 6, pp. 1823–1835, 1988.

[30] H. Blom, G. Bakker, B. K. Obbink, and M. Klompstra, “Free flight safety risk modelingand simulation,” Proc. 2nd International Conference on Research in Air Transporta-tion, pp. 1–10, 2006.

[31] H. Blom, J. Krystul, and G. Bakker, “A particle system for safety verification of freeflight in air traffic,” IEEE Conference on Decision and Control, pp. 1574–1579, 2007.

[32] M. Boccadoro, Y. Wardi, M. Egerstedt, and E. Verriest, “Optimal Control of SwitchingSurfaces in Hybrid Dynamical Systems,” Journal of Discrete Event Systems, vol. 15,no. 4, pp. 433–448, 2005.

[33] O. Bokanowski, N. Forcadel, and H. Zidani, “Reachability and minimal timesfor state constrained nonlinear problems without any controllability assumption,”2009. [Online]. Available: http://hal.archives-ouvertes.fr/docs/00/39/55/89/PDF/BFZ-Obstacle.pdf

[34] O. Bokanowski and H. Zidani, “Minimal Time Problems with Moving Targets and Ob-stacles,” IFAC World Congress, pp. 2589–2593, 2011.

[35] F. Borrelli, A. Bemporad, and M. Morari, Predictive Control for Linear and Hybrid Sys-tems. preprint, Cambridge, 2011.

[36] F. Bouffard and F. Galiana, “Stochastic security for operations planning with signifi-cant wind power generation,” IEEE Transactions on Power Systems, vol. 23, no. 2, pp.306 – 316, 2008.

[37] F. Bouffard, F. Galiana, and A. Conejo, “Market-clearing with stochastic security- parti: Formulation,” IEEE Transactions on Power Systems, vol. 20, no. 4, pp. 1818 – 1826,2005.

[38] ——, “Market-clearing with stochastic security- part ii: Case studies,” IEEE Transac-tions on Power Systems, vol. 20, no. 4, pp. 1818 – 1826, 2005.

141

http://hal.archives-ouvertes.fr/docs/00/39/55/89/PDF/BFZ-Obstacle.pdf

http://hal.archives-ouvertes.fr/docs/00/39/55/89/PDF/BFZ-Obstacle.pdf

[39] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge University Press,2004.

[40] R. Burkart, K. Margellos, and J. Lygeros, “Nonlinear control of wind turbines: An ap-proach based on switched linear systems and feedback linearization,” IEEE Confer-ence on Decision and Control, pp. 5485–5490, 2011.

[41] G. Calafiore, “Random Convex Programs,” SIAM Journal on Optimization, vol. 20,no. 6, pp. 3427–3464, 1998.

[42] G. Calafiore and M. Campi, “The scenario approach to robust control design,” IEEETransactions on Automatic Control, vol. 51, no. 5, pp. 742–753, 2006.

[43] G. Calafiore, F. Dabbene, and R. Tempo, “Randomized algorithms for probabilisticrobustness with real and complex structured uncertainty,” IEEE Transactions on Au-tomatic Control, vol. 45, no. 12, pp. 2218–2235, 2000.

[44] ——, “Research on probabilistic methods for control system design ,” Automatica,vol. 47, no. 7, pp. 1279 – 1293, 2011.

[45] G. Calafiore and B. T. Polyak, “Fast algorithms for exact and approximate feasibility ofrobust LMIs,” IEEE Transactions on Automatic Control, vol. 46, no. 11, pp. 1755–1759,2001.

[46] E. Camacho and C. Bordons, Nonlinear Model Predictive Control: An IntroductoryReview. Springer Berlin / Heidelberg, 2007, vol. 358.

[47] M. Campi and G. Calafiore, “Notes on the scenario design approach,” IEEE Transac-tions on Automatic Control, vol. 54, no. 2, pp. 382–385, 2009.

[48] M. Campi and S. Garatti, “The exact feasibility of randomized solutions of uncertainconvex programs,” SIAM Journal on Optimization, vol. 19, no. 3, pp. 1211–1230, 2008.

[49] ——, “A sampling-and-discarding approach to chance-constrained optimization:feasibility and optimality,” Journal of Optimization Theory and Applications, vol. 148,no. 2, pp. 257–280, 2011.

[50] M. Campi, S. Garatti, and M. Prandini, “The scenario approach for systems and con-trol design.” Annual Reviewes in Control, vol. 33, no. 2, pp. 149–157, 2009.

[51] P. Cardaliaguet, “A differential game with two players and one target,” SIAM Journalon Control and Optimization, vol. 34, no. 4, pp. 1441–1460, 1996.

[52] P. Cardaliaguet, M. Quincampoix, and P. Saint-Pierre, “Set valued numerical anal-ysis for optimal control and differential games,” in M.Bardi, T.Raghaven, andT.Papasarathy (eds.) Annals of the International Society of Dynamic Games, pp. 177–247, 1999.

142

[53] ——, “Pursuit differential games with state constraints,” SIAM Journal on Control andOptimization, vol. 39, no. 5, pp. 1615–1632, 2001.

[54] L. Castelli, L. Corolli, and G. Lulli, “Critical flights detected with Time Windows,”Transportation Research Record: Journal of the Transportation Research Board, vol.2214, no. 1, pp. 103–110, 2011.

[55] G. Chaloulos and J. Lygeros, “Effect of wind correlation on aircraft conflict probabil-ity,” AIAA Journal of Guidance, Control, and Dynamics, vol. 30, no. 6, pp. 1742–1752,2007.

[56] R. E. Cole, C. Richard, S. Kim, and D. Bailey, “An assessment of the 60 km rapid updatecycle (RUC) with near real-time aircraft reports,” NASA/A-1, MIT Lincoln Laboratory,Technical Report, 1998.

[57] P. Cousot and R. Cousot, “Constructive versions of Tarski’s fixed point theorems,” Pa-cific Journal of Mathematics, vol. 82, no. 1, pp. 43–57, 1979.

[58] E. Cross and I. Mitchell, “Level set methods for computing reachable sets of sys-tems with differential algebraic equation dynamics,” American Control Conference,pp. 2260–2265, 2008.

[59] F. Dabbene and R. Tempo, Probabilistic and Randomized Tools for Control Design.The Control Handbook, Second Edition (W. S. Levine, Editor), Taylor and Francis,London, 2010.

[60] B. Davey and H. Priestley, Introduction to Lattices and Order. Cambridge UniversityPress: 2nd edition, 2002.

[61] J. Deng, V. Becerra, and R. Stobart, “Input onstraints handling in an MPC/FeedbackLinearization scheme,” International Journal of Applied Mathematics and ComputerScience, vol. 19, no. 2, pp. 219–232, 2009.

[62] J. Ding and C. Tomlin, “Robust Reach-Avoid Controller Synthesis for Switched Non-linear Systems,” IEEE Conference on Decision and Control, pp. 6481 – 6486, 2010.

[63] R. Dorf and R. Bishop, Modern Control Systems. Pearson Prentice Hall, 2008.

[64] E. Ergueta, R. Seifried, R. Horowitz, and M. Tomizuka, “Extended Luenberger Ob-server for a MIMO Nonlinear Nonholonomic System,” Proceedings of the 17th WorldCongress, The International Federation of Automatic Control, Seoul, Korea, 2008.

[65] L. Evans, Partial Differential Equations. Providence, RI: American MathematicalSociety, 1998.

[66] L. Evans and P. Souganidis, “Differential games and representation formulas for solu-tions of Hamilton-Jacobi-Isaacs equations,” Indiana University of Mathematics Jour-nal, vol. 33, no. 5, pp. 773–797, 1984.

143

[67] I. J. Fialho and T. Georgiou, “Worst Case Analysis of Nonlinear Systems,” IEEE trans-actions on Automatic Control, vol. 44, no. 6, pp. 1180–1196, 1999.

[68] G. Franklin, J. Powell, and A. Emami-Naeini, Feedback Control of Dynamic Systems.Prentice-Hall, 2002.

[69] F. D. Galiana, F. Bouffard, J. M. Arroyo, and J. F. Restrepo, “Scheduling and pricingof coupled energy and primary, secondary, and tertiary reserves,” Proceeding of theIEEE, vol. 93, no. 11, pp. 1970–1983, 2005.

[70] Y. G. Gao, J. Lygeros, and M. Quincampoix, “On the Reachability Problem for Un-certain Hybrid Systems,” IEEE Transactions on Automatic Control, vol. 52, no. 9, pp.1572–1586, 2007.

[71] S. Garatti and M. Prandini, “A Simulation-Based Approach to the Approximation ofStochastic Hybrid Systems,” Proceedings of IFAC Conference on Analysis and Designof Hybrid Systems, 2012.

[72] L. E. Ghaoui and H. Lebret, “Robust solutions to uncertain semidefinite programs,”SIAM Journal on Optimization, vol. 9, no. 1, pp. 33–52, 1998.

[73] W. Glover and J. Lygeros, “A Stochastic Hybrid Model for Air Traffic Control Simula-tion,” Hybrid Systems: Computation and Control, Springer Verlag, no. 2993, pp. 372–386, 2004.

[74] M. Heymann, F. Lin, and G. Meyer, “Synthesis and viability of minimally interventivelegal controllers for hybrid systems,” Discrete Event Dyn.Syst.: Theory and Applicat.,vol. 8, no. 2, pp. 105–135, 1998.

[75] J. Hu, M. Prandini, and S. Sastry, “Aircraft conflict detection in presence of a spatiallycorrelated wind field,” IEEE Transactions on Intelligent Transportation Systems, vol. 6,no. 3, pp. 326–340, 2005.

[76] ILOG.SA., “CPLEX11.0 users manual,” Technical report, Gentilly, France, 2008.

[77] A. Isidori, Nonlinear Control Systems. Communications and Control EngineeringSerie, Springer Verlag, 1989.

[78] M. R. C. Jackson, “Sensitivity of Trajectory Prediction in Air Traffic Management andFlight Management Systems,” University of Minnesota, PhD thesis, 1997.

[79] M. Jackson, V. Sharma, C. Haissig, and M. Elgersma, “Airborne technology for dis-tributed air traffic management,” Decision and Control, 2005 and 2005 EuropeanControl Conference. CDC-ECC ’05. 44th IEEE Conference on, pp. 3947 – 3954, 2005.

[80] K. Johansson, M. Egerstedt, J. Lygeros, and S. Sastry, “Regularization of Zeno hybridautomata,” Systems and Control Letters, pp. 141–150, 1999.

144

[81] S. Kanev, B. D. Schutter, and M. Verhaegen, “An ellipsoid algorithm for probabilisticrobust controller design,” System and Control Letters, vol. 49, no. 5, pp. 365–375, 2003.

[82] H. K. Khalil, Nonlinear Systems. Macmillan. New York, 1992.

[83] I. Kitsios and J. Lygeros, “Final Glide-back Envelope Computation for ReusableLaunce Vehicle Using Reachability,” IEEE Conference on Decision and Control, pp.4059–4064, 2005.

[84] A. Krener and A. Isidori, “Linearization by output injection and nonlinear observers,”Systems and Control Letters, vol. 3, pp. 47–52, 1983.

[85] P. Kundur, Power System Stability and Control. McGraw-Hill Inc., 1994.

[86] A. B. Kurzhanski, I. M. Mitchell, and P. Varaiya, “Optimization Techniques for State-Constrained Control and Obstacle Problems,” Journal of Optimization Theory andApplications, vol. 128, no. 3, pp. 499–521, 2006.

[87] A. B. Kurzhanski and P. Varaiya, “Optimization Methods for Target Problems of Con-trol,” Proceedings of Mathematical Theory of Networks and Systems Conference, 2002.

[88] ——, “Ellipsoidal Techniques for Reachability Under State Constraints,” SIAM Jour-nal on Control and Optimization, vol. 45, no. 4, pp. 1369–1394, 2006.

[89] J. Lavaei and S. H. Low, “Zero Duality Gap in Optimal Power Flow Problem,” IEEETransactions on Power Systems, vol. 27, no. 1, pp. 92 – 107, 2012.

[90] A. Lecchini-Visintini, J. Lygeros, and J. Maciejowski, “Monte Carlo optimization forconflict resolution in air traffic control,” IEEE Transactions on Intelligent Transporta-tion Systems, vol. 7, no. 4, pp. 470–482, 2006.

[91] A. Lecchinni-Visintini, J. Lygeros, and J. Maciejowski, “Stochastic Optimization onContinuous Domains With Finite-Time Guarantees by Markov Chain Monte CarloMethods ,” IEEE Transactions on Automatic Control, vol. 55, no. 12, pp. 2858 – 2863,2010.

[92] D. Liberzon, Calculus of Variations and Optimal Control Theory. A Concise Introduc-tion. Princeton University Press, 2011.

[93] X. Lin, S. Janak, and C. Floudas, “A new robust optimization approach for schedulingunder uncertainty: I. Bounded uncertainty,” Computers and Chemical Engineering,vol. 28, no. 6-7, pp. 1069–1085, 2004.

[94] C. Livadas and N. Lynch, “Formal verification of safety-critical hybrid systems,” Hy-brid Systems: Computation and Control, no. 1386 in LNCS, pp. 253–272, 1998.

[95] J. Löfberg, “YALMIP: A toolbox for modeling and optimization in MATLAB,” IEEEInternational Symposium on Computer Aided Control Systems Design, pp. 284–289,2005.

145

[96] A. Lotz, “Implementation and Performance Evaluation of Time of Arrival Control forAir Traffic Management,” Diploma Thesis, ETH Zürich, 2009.

[97] J. Lygeros, Reachability, viability and invariance: An approach based on minimumcost optimal control. Department of Engineering, University of Cambridge: Techni-cal Report CUED/F-INFENG/TR.444, 2002.

[98] ——, “On reachability and minimum cost optimal control,” Automatica, vol. 40, no. 6,pp. 917–927, 2004.

[99] J. Lygeros, D. N. Godbole, and S. Sastry, “Verified hybrid controllers for automatedvehicles,” IEEE Transactions on Automatic Control, vol. 43, no. 4, pp. 522–539, 1998.

[100] J. Lygeros, K. Johansson, S. Simic, J. Zhang, and S. Sastry, “Dynamical properties ofhybrid automata,” IEEE Transactions on Automatic Control, vol. 48, no. 1, pp. 2–17,2003.

[101] J. Lygeros, C. Tomlin, and S. Sastry, “Controllers for reachability specifications for hy-brid systems,” Automatica, vol. 35, pp. 349–370, 1999.

[102] I. Lymperopoulos, Sequential Monte Carlo methods in air traffic management. ETHZürich: Ph.D. dissertation, 2010.

[103] I. Lymperopoulos and J. Lygeros, “Improved multi-aircraft ground trajectory predic-tion for air traffic control,” AIAA Journal of Guidance, Control, and Dynamics, vol. 33,no. 2, pp. 347–362, 2010.

[104] ——, “Sequential Monte Carlo methods for multi-aircraft trajectory prediction in airtraffic management,” International Journal of Adaptive Control and Signal Process-ing, vol. 24, no. 10, pp. 830–849, 2010.

[105] I. Lymperopoulos, J. Lygeros, A. Lecchini, W. Glover, and J. Maciejowski, “A StochasticHybrid Model for Air Traffic Management Processes,” University of Cambridge, De-partment of Engineering, Technical Report, vol. AUT07-15, 2007.

[106] Z. Ma, P. Caines, and R. Malhame, “On the Viscosity Solutions of Hybrid HJB Equa-tions Arising in Optimal Control of Loss Networks,” Communications in Informationand Systems, vol. 8, no. 1, pp. 17–38, 2008.

[107] J. Maciejowski, Predictive Control with Constraints. Prentice Hall, London, 2002.

[108] L. Magni and R. Scattolini, Robustness and Robust Design of MPC for NonlinearDiscrete-Time Systems. Springer Berlin / Heidelberg, 2007, vol. 358.

[109] O. Maler, A. Pnueli, and J. Sifakis, “On the synthesis of discrete controllers for timedsystems,” in Theoretical Aspects of Comp. Science ser.LNCS, vol. 900, pp. 229–242,1995.

146

[110] K. Margellos, P. Goulart, and J. Lygeros, “On the road between robust optimizationand the scenario approach for chance constrained optimization problems,” submit-ted to IEEE Transactions on Automatic Control, 2012.

[111] K. Margellos, T. Haring, P. Hokayem, M. Schubiger, J. Lygeros, and G. Andersson, “Arobust reserve scheduling technique for power systems with high wind penetration,”Proceeding of Probabilistic Methods Applied to Power Systems Conference, 2012.

[112] K. Margellos and J. Lygeros, “Air Traffic Management with Target Windows: An ap-proach using Reachability,” IEEE Conference on Decision and Control, pp. 145–150,2009.

[113] ——, “A Simulation Based MPC Technique for Feedback Linearizable Systems withInput Constraints,” IEEE Conference on Decision and Control, pp. 7539 – 7544, 2010.

[114] ——, “Hamilton-Jacobi Formulation for Reach-Avoid Problems with an Applicationto Air Traffic Management,” American Control Conference, pp. 3045 – 3050, 2010.

[115] ——, “Hamilton-Jacobi Formulation for Reach-Avoid Differential Games,” IEEETransactions on Automatic Control, vol. 56, no. 8, pp. 1849–1861, 2011.

[116] ——, “Computing Viability Sets for Hybrid Systems Using Optimal Control,” submit-ted to Nonlinear Analysis: Hybrid Systems, 2012.

[117] ——, “Revisiting the Viability Algorithm for Hybrid Systems Using Optimal Control,”Proceedings of IFAC Conference on Analysis and Design of Hybrid Systems, 2012.

[118] ——, “Toward 4D Trajectory Management in Air Traffic Control: A Study Based onMonte Carlo Simulation and Reachability Analysis,” submitted to IEEE Transactionson Control Systems Technology, 2012.

[119] K. Margellos, V. Rostampour, M. Vrakopoulou, M. Prandini, G. Andersson, andJ. Lygeros, “Stochastic unit commitment and reserve scheduling: A tractable formula-tion with probabilistic certificates,” submitted to European Control Conference, 2012.

[120] I. Mitchell, “A Toolbox of Level Set Methods,” Technical Report, University of BritishColumbia, 2007.

[121] ——, “Comparing Forward and Backward Reachability as tools for Safety Analysis,”in Hybrid Systems: Computation and Control, pp. 428–443, 2007.

[122] I. Mitchell, A. M. Bayen, and C. Tomlin, “Validating a Hamilton Jacobi approximationto hybrid reachable sets,” in M.Di.Benedetto and A.Sangiovanni-Vincentelli (eds.) Hy-brid Systems: Computation and Control Springer Verlag, pp. 418–432, 2001.

[123] ——, “A time-dependent Hamilton-Jacobi formulation of reachable sets for continu-ous dynamic games,” IEEE Transactions on Automatic Control, vol. 50, pp. 947–957,2005.

147

[124] I. Mitchell and C. Tomlin, “Level set methods for computations in hybrid systems,” inM.Di.Benedetto and A.Sangiovanni-Vincentelli (eds.) Hybrid Systems: Computationand Control Springer Verlag, pp. 310–323, 2000.

[125] I. M. Mitchell, Application of level set methods to control and reachability problems incontinuous and hybrid systems. Stanford University: Ph.D. dissertation, 2002.

[126] P. Mohajerin, M. Vrakopoulou, K. Margellos, J. Lygeros, and G. Andersson, “A robustpolicy for Automatic Generation Control cyber attack in two area power network,”IEEE Control and Decision Conference, Atlanta, Georgia, USA, pp. 5973–5978, 2010.

[127] ——, “Cyber Attack in a Two-Area Power System: Impact Identification using Reach-ability,” American Control Conference, Baltimore MD, USA, pp. 962–967, 2010.

[128] J. M. Morales, A. Conejo, and J. Perez-Ruiz, “Economic valuation of reserves in powersystems with high penetration of wind power,” IEEE Transactions on Power Systems,vol. 24, no. 2, pp. 900–910, 2009.

[129] A. Nemirovski and A. Shapiro, Scenario Approximation of Chance Constraints. Prob-abilistic and Randomized Methods for Design under Uncertainty, eds. G. Calafioreand F. Dabbene, Springer, 2005.

[130] A. Nerode, J. B. Remmel, and A. Yakhnis, “Controllers as fixed points of set-valuedoperators,” in Hybrid Systems: Computation and Control Springer Verlag, pp. 344–358, 1995.

[131] V. Nevistic, Constrained Control of Nonlinear Systems. Ph.D dissertation, ETHZürich, 1997.

[132] V. Nevistic and M. Morari, “Constrained control of feedback-linearizable systems,”Proceedings of the European Control Conference, Rome, Italy, pp. 1726–1731, 1996.

[133] ——, Model Predictive Control with Feedback Linearization. unpublished TechnicalReport, ETH Zürich, 1999.

[134] V. Nevistic and J. Primbs, “Constrained Nonlinear Optimal Control: A Converse HJBApproach,” Technical Memorandum CIT-CDS 96-021, California Institute of Technol-ogy, Pasadena, 1996.

[135] M. Oishi, “Recovery from Error in Flight Management Systems: Applications of Hy-brid Reachability,” Advanced Process Control Applications for Industry Workshop,2006.

[136] M. Oishi, I. Mitchell, C. Tomlin, and P. S. Pierre, “Computing Viable Sets and Reach-able Sets to Design Feedback Linearizing Control Laws Under Saturation,” Conferenceon Decision and Control, 2006.

148

[137] M. A. Ortega-Vazquez and D. S. Kirschen, “Optimizing the spinning reserve require-ments using a cost/benefit analysis,” IEEE Transactions on Power Systems, vol. 22,no. 1, pp. 24–33, 2007.

[138] S. Osher and J. Sethian, “Fronts propagating with curvature-dependent speed: Algo-rithms based on Hamilton-Jacobi formulations,” Journal of Computational Physics,vol. 79, pp. 12–49, 1988.

[139] R. Paielli and H. Erzberger, “Conflict probability estimation for free flight,” AIAA Jour-nal of Guidance, Control, and Dynamics, vol. 20, no. 3, pp. 588–596, 1997.

[140] ——, “Conflict probability estimation generalized to non-level flight,” Air Traffic Con-trol Quarterly, vol. 7, no. 3, pp. 195–222, 1999.

[141] D. Panagou, Viability Control for Nonholonomic Systems. Application to the Compen-sation of Disturbances for Underwater Robotic Vehicles. PhD dissertation, NTUA,Athens, 2012.

[142] D. Panagou, K. Margellos, S. Summers, J. Lygeros, and K. Kyriakopoulos, “A ViabilityApproach for the Stabilization of an Underactuated Underwater Vehicle in the Pres-ence of Current Disturbances,” IEEE Conference on Decision and Control, pp. 8612 –8617, 2009.

[143] G. Papaefthymiou and B. Klöckli, “MCMC for wind power simulation,” IEEE Transac-tions on Energy Conversion, vol. 23, no. 1, pp. 234–240, 2008.

[144] A. Papavasiliou, S. Oren, and R. O’Neill, “Reserve requirements for wind power in-tegration: A scenario-based stochastic programming framework,” IEEE Transactionson Power Systems, vol. 26, no. 4, pp. 2197 – 2206, 2011.

[145] G. Pappas, J. Lygeros, and D. Godbole, “Stabilization and Tracking of Feedback Lin-earizable Systems under Input Constraints,” Conference of Decision and Control, NewOrleans, 1995.

[146] M. Prandini and J. Hu, “Application of reachability analysis for stochastic hybrid sys-tems to aircraft conflict prediction,” IEEE Transactions on Automatic Control, vol. 54,no. 4, pp. 913–917, 2009.

[147] A. Prekopa, Stochastic Programming. Cluwer Academic Publishers, Dordrecht,Boston, 1995.

[148] Y. Rebours, D. Kirschen, M. Trotignon, and S. Rossignol, “Survey of frequency andvoltage control ancillary services-part i: Technical features,” IEEE Transactions onPower Systems, vol. 22, no. 1, pp. 350–357, 2007.

[149] ——, “Survey of frequency and voltage control ancillary services-part ii: Economicfeatures,” IEEE Transactions on Power Systems, vol. 22, no. 1, pp. 358–366, 2007.

149

[150] K. Roebenack, “Computation of the observer gains for extended Luenberger observerusing automatic differentiation,” Journal of Mathematical Control and Information,vol. 21, pp. 33–47, 2004.

[151] S. Sastry, Nonlinear Systems: Analysis, Stability and Control. New York: SpringerVerlag, 1999.

[152] J. A. Sethian, Level Set Methods: Evolving Interfaces in Geometry, Fluid Mechanics,Computer Vision, and Materials Science. New York: Cambridge University Press,1996.

[153] M. Shaikh and P. Caines, “On the Optimal Control of Hybrid Systems: Optimizationof Trajectories, Switching Times and Location Schedules,” Hybrid Systems: Computa-tion and Control, Eds. O. Maler, A. Pneuli. Springer Verlag, pp. 466–481, 2003.

[154] ——, “On the Hybrid Optimal Control Problem: Analysis and Algorithms,” IEEETransactions on Automatic Control, vol. 52, no. 9, pp. 1587–1603, 2007.

[155] A. Shapiro, “Stochastic programming approach to optimization under uncertainty ,”Mathematical Programming, Series B, vol. 112, pp. 183 – 183, 2008.

[156] B. Stott, J. Jardim, and O. Alsac, “DC power flow revisited,” IEEE Transactions on PowerSystems, vol. 24, no. 3, pp. 1290–1300, 2009.

[157] H. Sussmann, “Set-Valued Differentials and the Hybrid Maximum Principle.” IEEEConference on Decision and Control, pp. 558–563, 2000.

[158] Y. Suzuki and T. Hikihara, “Predicting voltage instability of power system via hybridsystem reachability analysis,” American Control Conference, pp. 4166–4171, 2007.

[159] A. Tarski, “A Lattice-Theoretical Fixpoint Theorem and its Applications.” Pacific Jour-nal of Mathematics, vol. 5, pp. 285 – 309, 1955.

[160] R. Tempo, E. W. Bai, and F. Dabbene, “Probabilistic robustness analysis: Explicitbounds for the minimum number of samples,” System and Control Letters, vol. 30,no. 5, pp. 237–242, 1997.

[161] H. W. Toi, “The synthesis of controllers for linear hybrid automata,” in Proc. IEEEConf. Decision Control, San Diego, CA,, pp. 4607–4613, 1997.

[162] C. Tomlin, Hybrid control of air traffic management systems. University of California,Berkeley: Ph.D. dissertation, Department of Electrical Engineering and ComputerSciences, 1998.

[163] C. Tomlin, J. Lygeros, and S. Sastry, “Aerodynamic envelope protection using hybridcontrol,” American Control Conference, pp. 1793–1796, 1998.

150

[164] ——, “A game theoretic approach to controller design for hybrid systems,” Proceed-ings of the IEEE, vol. 88, no. 7, pp. 949–969, 2000.

[165] C. Tomlin, G. Pappas, and S. Sastry, “Conflict resolution for Air Traffic Management:A Study in Multiagent Hybrid Systems,” IEEE Transactions on Automatic Control,vol. 43, no. 4, pp. 509–521, 1998.

[166] J. Tzanos, K. Margellos, and J. Lygeros, “Optimal Wind Turbine Placement via Ran-domized Optimization Techniques,” Power Systems Computation Conference (PSCC),pp. 1–8, 2011.

[167] P. P. Varaiya, “On the existence of solutions to a differential game,” SIAM Journal onControl and Optimization, vol. 5, no. 1, pp. 153–162, 1967.

[168] V. Venkatasubramanian, H. Schättler, and J. Zaborszky, “Voltage dynamics: Study ofa generator with voltage control, transmission, and matched MW load,” IEEE Trans-actions on Automatic Control, vol. 37, no. 11, pp. 1717–1733, 1992.

[169] M. Vidyasagar, A Theory of Learning and Generalization. London, U.K.: Springer-Verlag, 1997.

[170] ——, “Randomized algorithms for robust controller synthesis using statistical learn-ing theory,” Automatica, vol. 37, no. 10, pp. 1515–1528, 2001.

[171] M. Vidyasagar and V. Blondel, “Probabilistic solutions to some NP-hard matrix prob-lems,” Automatica, vol. 37, no. 9, pp. 1397–1405, 2001.

[172] A. Vladimirsky, “Static PDEs for time-dependent control problems,” Interfaces andFree Boundaries, vol. 8, no. 3, pp. 281–300, 2006.

[173] M. Vrakopoulou, K. Margellos, J. Lygeros, and G. Andersson, “A Probabilistic Frame-work for Reserve Scheduling and N-1 Security Assessment of Systems with High WindPower Penetration,” submitted to IEEE Transactions on Power Systems, 2012.

[174] ——, “A Probabilistic Framework for Security Constrained Reserve Scheduling ofNetworks with Wind Power Generation,” IEEE International Conference & Exhibition(ENERGYCON), pp. 452 – 457, 2012.

[175] ——, “Probabilistic Guarantees for the N-1 Security of Systems with Wind Power Gen-eration,” Proceeding of Probabilistic Methods Applied to Power Systems Conference,2012.

[176] X. Zhang, Nonlinear Control of Wind Turbines based on an Iterative MPC Techniqueand Feedback Linearization. Semester Project Thesis, ETH Zürich, 2012.

[177] H. Zheng, E. Lee, and A. Ames, “Beyond Zeno: Get on with it!” Proceedings of HybridSystems: Computation and Control (HSCC) LNCS 3927, pp. 1–15, 2006.

151

[178] R. D. Zimmerman, C. E. Murillo-Sanchez, and R. J. Thomas, “MATPOWER: Steady-state operations, planning, and analysis tools for power systems research and educa-tion,” IEEE Transactions on Power Systems, vol. 26, no. 1, pp. 12–19, 2011.

152

Curriculum Vitae

Kostas Margellosborn on December 5th, 1984 in Athens, Greece

2008 – 2012 ETH Zürich, SwitzerlandDoctorate StudentAutomatic Control Laboratory,Department of Information Technology and ElectricalEngineering.

2002 – 2008 University of Patras, GreeceDiploma in Electrical and Computer EngineeringGraduation with honors, major in Power Systems.

1999 – 2002 High School studies, Athens, Greece

153

rights / license: research collection in copyright - non ......giorgos, kostas, nikos, yannis for...

Documents