Decentralized Optimization, with application to

Multiple Aircraft Coordination

Decision Making Under Uncertainty MURI Review,

July 2002

Gökhan Inalhan, Dusan Stipanovic, Claire Tomlin

Hybrid Systems LaboratoryDepartment of Aeronautics and Astronautics

Stanford University

Motivating applications

(Source: Boeing X45-A)

(Source: Northrop Grumman-X47A)

(Source: NASA Ames)

Hybrid systems

• Continuous systems controlled by a discrete logic: embedded systems (autopilot logic)

• Coordinating processes: multi-vehicle systems interfacing continuous control with coordination protocols

• Continuous systems with a phased operation: (biological cell growth and division)

xç= f(x;u;d)

continuous systems(control)

discrete systems(computer science)

Verification: a mathematical proof that the system satisfies a property

Controller synthesis: the design of control laws to guarantee that the system satisfies the property

• Methods give definitive answers, unlike simulation• Often give surprising answers, trajectories which one

might not think to simulate• Reduces development time, cost of certification

Verification and Controller Synthesis

unsafe

initial

Verification: a mathematical proof that the system satisfies a property

Controller synthesis: the design of control laws to guarantee that the system satisfies the property

• Methods give definitive answers, unlike simulation• Often give surprising answers, trajectories which one

might not think to simulate• Reduces development time, cost of certification

Verification and Controller Synthesis

unsafe

initial

Verification: a mathematical proof that the system satisfies a property

Controller synthesis: the design of control laws to guarantee that the system satisfies the property

• Methods give definitive answers, unlike simulation• Often give surprising answers, trajectories which one

might not think to simulate• Reduces development time, cost of certification

Verification and Controller Synthesis

unsafe

initial

Verification: a mathematical proof that the system satisfies a property

Controller synthesis: the design of control laws to guarantee that the system satisfies the property

Safety Property can be encoded as a condition on the system’s reachable set of states

Verification and Controller Synthesis

unsafeunsafe initialization

initial

unsafe

safe, under appropriate control

Example: Aircraft Collision Avoidance

Two identical aircraft at fixed altitude & speed:

ud

uxv

uyvv

duy

x

dt

d

sin

cos

),,(xf

‘evader’ (control) ‘pursuer’ (disturbance)

x

y

uv

d

v

Continuous Reachable Set

x

ySolve:

Display: G(t) = fx : J (x;t) < 0g

à @t@J (x;t) =minf0;maxumindH(x; @x

@J (x;t);u;d)g

Collision Avoidance Filter

Simple demonstration– Pursuer: turn to head toward evader– Evader: turn to head right

pursuer

safety filter’s input modification

pursuer’s inputevader’s desired input

evader

evader’s actual input

unsafe setcollision set

Movies …

Blunder Zones for Closely Spaced Approaches

evader

EEM Maneuver 1: accelerateEEM Maneuver 2: turn 45 deg, accelerate

EEM Maneuver 3: turn 60 deg

Blunder Zone is shown by the yellow contour

Red Zone in the green tunnel is the intersection of the BZ with approach path.

The Red Zone corresponds to an assumed 2 second pilot delay. The Yellow Zone corresponds to an 8 second pilot delay

Implementation: Display design courtesy of

Chad Jennings, Andy Barrows, David Powell

Map View showing a blunder

The BZ calculations are performed in real time (40Hz) so that the contour is updated with each video frame.

Verified Mode Switching in Autopilots

Use in Cockpit Interface Verification• Controllable flight envelopes for landing and Take Off / Go

Around (TOGA) maneuvers may not be the same• Pilot’s cockpit display may not contain sufficient information to

distinguish whether TOGA can be initiated

flareflaps extendedminimum thrust

rolloutflaps extendedreverse thrust

slow TOGAflaps extended

maximum thrust

TOGAflaps retracted

maximum thrust

flareflaps extendedminimum thrust

rolloutflaps extendedreverse thrust

TOGAflaps retracted

maximum thrust

revised interface

existing interface

controllable flare envelope

controllable TOGA envelope

intersection

V1

V2

V3 V4

Communication Zone

Safety Assurance Zone

V1

V2

V3 V4

A More General Problem Structure

Neighborhood of ith vehicle

(Decomposed) Centralized Optimization

Fixed time horizon – complete global map

t à 2ts t à ts t t + T à 2ts t + T à ts t + T

Bargainingstart

fixed time horizon

Flight Plans published by aircraft 1

Another Example

Flight Plans published by aircraft 1

Receding horizon – incomplete global map

t à 2ts t à ts t t + T à 2ts t + T à ts t + T

moving time horizon

tsBargainingstart

• Constraints embed:local dynamics: coordinated turn and straight flight [hdi

]

input constraints: limited turn rate and velocity [gei]

global coordination constraints: minimum safety assurance [gsij] for all j within

neighborhood of i

Local Optimization with Constraints

Centralized Optimization

Decomposed Centralized Optimization

Decomposition I

Pareto optimalityNash equilibrium

Define Hamiltonian for each subsystem:

is a Nash equilibrium for the centralized optimization problem if:

where

Thus, none of the subsystems can improve its solution, with all other subsystems’ solutions remaining fixed.

Nash Equilibrium for Centralized Problem

xã= [xã1; :::;x

ãi ; :::x

ãm]

H ci (x;õ;ö) = fi(xi) +õTh(x) +öTg(x)

H ci (x

ã;õ;ö) ô H ci (x

ãi;õ;ö)

xãi = [xã1; :::;xi; :::xã

m]

Decomposed Centralized Optimization

Decentralized Optimization

Decomposition II

Nash equilibriumLocal optimal solutions

Define Hamiltonian for each subsystem:

is a Nash equilibrium for the decentralized optimization problem if:

Optimal solutions by each of the subsystems

Nash Equilibrium for Decentralized Problem

xã= [xã1; :::;x

ãi ; :::x

ãm]

H dci (xi;õi;öijf xjgi) = fi(xi) +õT

i hi(xijf xjgi) +öTi gi(xijf xjgi)

H dci (x

ãi ;õi;öijf x

ãjg) ô H dc

i (xi;õi;öijf xãjg)

Proposition: is a Nash equilibrium of the centralized problem if and only if it is a Nash equilibrium of the

decentralized problem

xã

Example: Nash Equilibrium at (0,0)

• Global contraction function from the local optimization structures

• For a particular solution, local optimization of the ith

vehicle only affects the portion of F tied to its own local optimization

Using Penalty function methods

• Eliminates cases in which a subsystem is artificially acting against a constraint dictated by another group

• Eliminates cases in which two subsystems act against each other with non-identical constraints

Cooperation Assumptions

Multiple solutions, or “threads”, exist within the system:

Vehicle #1 Vehicle #2 Vehicle #3 Vehicle #4Iteration #1

Iteration #2

Iteration #3

Iteration #4

Iteration #4

Nash Bargaining with Multiple Threads

Convergence Results

1. Global convergence to a (not necessarily feasible) Nash ‘solution’

2. If the gradients of the constraint functions are linearly independent (Linear Constraint Qualification Condition, LICQ), then global convergence to a feasible Nash solution

3. Pareto optimality for convex problems

[Inalhan, Stipanovic, Tomlin. Decentralized Optimization, with Application to Multiple Aircraft Coordination. CDC 2002, Submitted to JOTA]

V1

V2

V3 V4

4-Vehicle Example

PF i(xijf xjgi) = PF j(f xjgi;xi) 8j 2 N i

4-Vehicle Example

Flight Plans published by aircraft 1

Decentralized Initialization Procedure Heuristics– Multiple-Depots (Vehicles), Time-windows for access, Priority on

objectives and the vehicles– Iterative selection process carried at each vehicle– Best solution in the fleet is then selected from each vehicle’s

solution set

Applied to other problems of interest…

Non-cooperativeFull information

CooperativeFull information

Cooperativeincomplete information

Non-cooperativeNo information

Lack of information Bounded Irrationality

Spectrum of Approaches

Research Goals

• Design of provably correct and safe decentralized control protocols– Adapt to coordination– Allow for dynamic reconfiguration

• Treatment of information– Multi-scale provisioning of data based on inputs from various

sensing modalities– Urgency of the need for sensed data– Available bandwidth

• Verification algorithms– Used during design phase, to reduce the time spent during

the validation phase

Stanford DragonFly UAV

10 ft wingspan

12 ft wingspan

[Jang, Teo, Inalhan, and Tomlin, DASC 2001], [Jang and Tomlin, AIAA GNC 2002]