Quantum Annealing Applied to De-Conflicting Optimal Trajectories for Air TrafficManagement

Tobias Stollenwerk,1 Bryan O’Gorman,2, 3, 4 Davide Venturelli,3, 5 Salvatore Mandra,3, 4

Olga Rodionova,6 Hok K. Ng,6 Banavar Sridhar,6 Eleanor G. Rieffel,3 and Rupak Biswas3

1German Aerospace Center (DLR), Cologne, Germany 511472University of California, Berkeley, CA 94720

3Quantum Artificial Intelligence Laboratory (QuAIL), NASA Ames Research Center, Moffett Field, CA 940354Stinger Ghaffarian Technologies Inc., Greenbelt, MD 20770

5Research Institute for Advanced Computer Science, Universities Space Research Association (USRA), Mountain View, CA 940356NASA Ames Research Center, Moffett Field, CA 94035

We present the mapping of a class of simplified air traffic management (ATM) problems (strategicconflict resolution) to quadratic unconstrained boolean optimization (QUBO) problems. The mappingis performed through an original representation of the conflict-resolution problem in terms of a conflictgraph, where nodes of the graph represent flights and edges represent a potential conflict betweenflights. The representation allows a natural decomposition of a real world instance related to wind-optimal trajectories over the Atlantic ocean into smaller subproblems, that can be discretized and areamenable to be programmed in quantum annealers. In the study, we tested the new programmingtechniques and we benchmark the hardness of the instances using both classical solvers and theD-Wave 2X and D-Wave 2000Q quantum chip. The preliminary results show that for reasonablemodeling choices the most challenging subproblems which are programmable in the current devicesare solved to optimality with 99% of probability within a second of annealing time.

I. INTRODUCTION

One of the main functions of Air Traffic Control (ATC)is ensuring safe flight progress in the shared airspace.This in particular involves resolving potential conflictsbetween flights, where a conflict stands for a violation ofseparation norms established in the airspace.

There is an overall increase in air traffic over the lastdecades and this trend is believed to continue. As aresult, ATC workload is constantly increasing. Nowadaysthe flights are typically assigned the predefined routesfrom the air traffic network, which is becoming saturated.With the limited airspace available, novel approaches arenecessary to meet the increasing air traffic demand inthe coming decades. A promising approach to addressthis problem is to allow the aircraft to fly their wind-optimal trajectories instead of the routes [1]. However,this may result in the potential conflicts being detectedbetween such wind-optimal trajectories [2]. Thus, conflictresolution deserves a particular attention in this case.Conflict detection and resolution is a complex problemwhich has been studied for the decades [3, 4].

Quantum annealing is a promising computationalmethod which became increasingly important in recentyears. This development is driven also by first commer-cially available quantum annealing device by the companyD-Wave Systems. In addition to studying the fundamentalproperties of quantum annealing, it is imperative to findpossible real world application for this technology. Hardoperational planning problems are a promising candidatefor the latter [5–7].

In this work, we investigate the feasibility of applyingquantum annealing to solve the conflict resolution problemfor wind-optimal trajectories. To be amenable to a D-

Wave quantum annealer, the conflict-resolution problemhas to be formulated as a quadratic unconstrained binaryoptimization (QUBO) problem. For the main part ofthe paper, we restrict ourselves to a simplified version ofthe problem by considering departure delays only whileneglecting maneuvers. We present a detailed study of thestructure of this problem which provides insights beyondthe scope of quantum annealing.

In particular, we perform the following analyses

• Given the wind-optimal trajectories, we extract nat-ural subsets of the overall problem and study theirhardness. We found that the problems are hardin general and become harder as we increase themaximum allowed value for the departure delays.

• Restrictions to the configurations space are nec-essary for the reformulation of the problem as aQUBO. Therefore, we employ classical solvers toinvestigating the influence of discretization on thesolution quality. As a result, we found that finerdiscretization increases the solution quality and suf-ficiently large maximum allowed value for the de-parture delays is enough for an acceptable solution

• We demonstrate the mapping of the deconflictingproblem to a QUBO formulation for models with(see Appendix) and without maneuvers (in maintext). In the course of this, we investigate the suffi-cient penalty weights for the hard constraints in theproblem. Here we found that these penalty weightsare largely independent of the problem instances.

• We investigate the possibility to embeddeconfliction-derived QUBO instances ontothe D-Wave quantum annealer. More precisely,

arX

iv:1

711.

0488

9v1

[qu

ant-

ph]

13

Nov

201

7

2

we were able to embed and run smaller probleminstances and found that finer model discretizationsas well as larger problem sizes decrease the successprobability due to the limited precision of theD-Wave 2X machine.

The paper is organized as follows: We begin by formu-lating the conflict-resolution problem as a combinatorialoptimization problem and describing the preprocessingnecessary for this mapping in Section II. In Section IIIwe investigate the structure and hardness of probleminstances before we study the impact of discretizationon the solution quality in Section IV. Afterwards, wediscuss the mapping of the problem to a QUBO in Sec-tion V. We report on the embeddability of the QUBOinstances and their solution quality on a D-Wave 2X de-vice in Section VI. Finally, we conclude with discussionon improvements and future works. In the Appendix, wepresent more general mappings (including maneuvers) ofthe original deconflicting problem to QUBOs.

II. PROBLEM SPECIFICATION

The basic input of the conflict-resolution problem is aset of optimal flight trajectories (space-time paths). Suchtrajectories are the results of optimizations performed bythe flight operators.

In the present study, we consider wind-optimal trajec-tories. Such trajectories are obtained by minimizing thefuel cost over the routes with given origins and desti-nations and desired departure times in the presence offorecast winds. Because of the correlation between suchtrajectories arising from exploiting favorable winds, thesetrajectories are likely to conflict; that is, two or moreaircraft are likely to get dangerously close to each other iftheir optimal trajectories are followed without modifica-tion. The goal thus is to modify the trajectories to avoidsuch conflicts.

In theory, the configuration space consists of all phys-ically realistic trajectories; in practice, computationallimits constrain us to consider certain perturbations ofthe optimal trajectories. The simplest way to perturba trajectory is to delay the corresponding flight on theground prior to departure. These are the type of pertur-bations we mainly analyze in this work. We also considerlocal spatial modifications of the trajectories so that nonew potential conflicts are induced. Such local modi-fications can then parametrized as effective additionaldelay. Previous work [2] additionally considered a globalmodification of the trajectory geometrical shape.

A full accounting of the cost of such modificationswould include the cost of departure delays, the change infuel cost due to perturbing the trajectories, the relativeimportance of each flight, and many other factors. Asin previous work, we consider only the total, unweightedarrival delay, aggregated equally over all of the flights.

Formally, each optimal trajectory xi = (xi,t)τi,1t=τi,0

is

specified as a time-discretized path from the departurepoint xi,τi,0 at time τi,0 to the arrival point xi,τi,1 at timeτi,1. For each flight i, the geographical coordinates xi,t(as latitude, longitude, and altitude) are specified at everyunit of time (i.e. one minute) between τi,0 and τi,1.

For notational simplicity, suppose momentarily thateach trajectory xi is modified only by introducing delaysbetween time steps. Let δi,t be the accumulated delay offlight i at the time that it reaches the point xi,t, and letδ∗i,t be the maximum such delay at the point (given themodifications under consideration). Then, the total delayover Nf flights is

D =

Nf∑i=1

δi,τi,1 . (1)

A pair of flights (i, j) are in spatial conflict with eachother if any pair of points from their trajectories is inconflict. That is, a pair of trajectory points (xi,s, xj,t)conflict if their spatial and temporal separations are bothless than the respective mandatory separation standards∆x and ∆t:

‖xi,s − xj,t‖ < ∆x, (2a)

and

|(s+ δi,s)− (t+ δj,t)| < ∆t. (2b)

For the North Atlantic oceanic airspace, the separationstandard are set to be: 30 nautical miles for horizontalseparation ∆x and 3 minutes for temporal separation ∆t.Observe that the latter condition can be met for some(δi,s, δj,t) ∈ [0, δ∗i,s]× [0, δ∗j,t] if and only if

max{δ∗i,s, δ

∗j,t

}+ ∆t > |s− t|, (3)

in which case we call the pair of trajectory points poten-tially conflicting. Let us partition the set C of potentiallyconflicting pairs of trajectory points into disjoint sets, orclusters, Ck:

C =⋃k

Ck, (4)

such that if {(i, s), (j, t)} , {(i′, s′), (j′, t′)} ∈ Ckfor some k then i = i′ < j = j′ andfor all s′′ ∈ [min{s, s′},max{s, s′}] there ex-ists some t′′ ∈ [min{t, t′},max{t, t′}] such that{(i, s′′), (j, t′′)} ∈ Ck and vice versa. We will fur-ther refer to such clusters Ck simply as the conflicts.Figure 1 shows an example of two such conflict clusters.Thus every conflict k is associated with a pair of flightsIk = {i, j}. Let Ki = {k|i ∈ Ik} be the set of conflicts towhich flight i is associated, Nc the number of conflicts.

Having identified disjoint sets of conflicts, we relaxthe supposition that the trajectory modifications onlyintroduce delays between time steps. Instead, we considermodifications to the trajectories that introduce delays

3

local to particular conflicts. Specifically, the configuration

space consists of the departure delays d = (di)Nf

i=1 and theset of local maneuvers ak = (ak)k, where ak representssome parameterization of the local maneuvers used toavoid conflict k. Let di,k(d,ak) be the delay introducedto flight i at conflict k, as a function of the departuredelays and local maneuvers. With this notation, we canwrite the total delay as

D =

Nf∑i=1

(di +

∑k∈Ki

di,k

). (5)

This is the quantity we wish to minimize subject to avoid-ing all potential conflicts.

FIG. 1. Example of two potential conflicts between a pair oftransatlantic flights originating on the East Coast of the USA.

We focus on the case where conflicts will be avoidedonly by the introduction of extra delays, leaving for futurework the introduction of maneuvering choices.

Let

Di,k = di +∑

k′∈Ki,k

di,k′ (6)

be the accumulated delay of flight i by the time it reachesconflict k, where Ki,k = {k′ ∈ Ki|k′ < k}. We assumethat the set of conflicts Ki associated with flight i isindexed in temporal order, i.e. if k′ < k and k, k′ ∈ Ki,then flight i reaches conflict k′ before conflict k. Forsimplicity, we assume that no delay is introduced duringa conflict, so that δi,s = Di,k for all s associated withflight i in conflict k. The pairs of conflicting trajectorypoints associated with conflict k are given by

Tk = {(s, t)|{(i, s), (j, t)} ∈ Ck, i < j} . (7)

Thus the potential conflict is avoided only if

Dk = Di,k −Dj,k /∈ Bk (8)

where

Bk =⋃

(s,t)∈Tk

(−∆t + t− s,∆t + t− s) = [∆mink ,∆max

k ],

(9)

∆mink = 1−∆t + min

(s,t)∈Tk

{t− s}, (10)

∆maxk = ∆t − 1 + max

(s,t)∈Tk

{t− s}. (11)

In the remainder of this paper, we focus on the simpli-fied problem in which only departure delays are allowed.

In this case, the configuration space is simply d = (di)Nf

i=1,

the cost function (Eq. 5) transforms into D =∑Nf

i=1 di,and the constraints become di − dj /∈ Bk for all k.

III. INSTANCES

We test on realistic instances of the problem, using theprecalculated wind-optimal trajectories for transatlanticflights on July 29, 2012 [2]. This data consists of 984flights each of which has a constant cruising altitude andconstant speed. However, our methods can be generalizedto instances without these special properties.

To identify the instances of the conflict-resolution prob-lem we construct a conflict graph, whose vertices corre-spond to flights and which has an edge between a pair ofvertices if there is at least one potential conflict betweenthe corresponding flights. Note that the conflict graphfor a given set of trajectories depends on the parametersof the problem. In the case of only departure delays,whether or not a potential conflict, and thus an edge inthe conflict graph, exists between two flights is a functionof the maximum allowable departure delay dmax. Fora certain value of dmax, the conflict graph may containseveral connected components, which can be consideredas smaller, independent instances. Figure 2 shows thisdependence of the number of connected components (bothincluding and excluding trivial connected components, i.e.those containing a single vertex) on the maximum delaydmax, and Figure 3 shows the distribution of the sizesof the connected components for various values of dmax.As dmax increases, the conflict graph becomes denser; atsome point, the conflict graph saturates (though not nec-essarily as the complete graph), with every spatial conflictindicating a potential conflict. Interestingly, most of theconnected components are very small; for example, withdmax = 60 minutes, approximately 75% of the connectedcomponents contain no more than 10 flights.

In the remainder of this paper, we consider sets ofsmaller instances corresponding to the connected compo-nents of the conflict graph from the larger single instancefor various values of dmax, for the given flight set. LetIdmax

be the set of such instances for a particular value ofdmax, excluding trivial instances. We say that an instanceis trivial if there are no conflicts when all flights thereindepart without delay; in particular, this includes instancescontaining only a single flight.

As a part of our analysis, we also studied the proba-bility distribution of the degree of vertices in the conflictgraph. In other words, the number of flights for whicha given flight share a potential conflict with. Figure (4)shows the distribution of degrees of vertices in the con-flict graph for dmax = 60, which seem to approximatelycoincide with a power law, i.e. the number of verticeswith degree d is proportional to dα. This is consistentwith a so-called “small-world” model believed to be typ-

4

10 20 30 40 50 60dmax

50

100

150

200

250

num

ber 

of c

onne

cted

 com

pone

nts

including trivial connected componentsexcluding trivial connected components

FIG. 2. Number of connected components versus dmax.

2­4 4­8 8­1616­32

32­6464­128

128­256

256­512

Connected component size

0

5

10

15

20

25

30

35

40

Num

ber 

of c

onne

cted

 com

pone

nts

dmax = 6dmax = 12dmax = 36dmax = 48dmax = 60

FIG. 3. Histogram of the connected component size for variousvalues of the maximum delay time dmax.

ical of many real-world graphs [8], which are generatedby preferential attachment and resultingly contain a fewnumber of highly-connected hubs, as is the case with airtraffic. Figure (5) shows the dependence of this empiricalpower-law exponent α as a function of dmax. As dmax

increases, the exponent decreases. The larger the delay,the less the structure of the trajectories matters and theflatter the distribution of degrees in the conflict graph.

In many cases, generally hard problems are easy whenrestricted to tree-like instances [9, 10]. For example, if theconflict graph here is a tree, then the optimum could beeasily found by propagating the delays along the tree; onthe other hand, if the conflict graph is a complete graph,finding the optimum is much harder. The tree-width ofa graph formalizes this notion of tree-likeness, rangingfrom 1 for a tree to n− 1 for fully connected graph. We

0 10 20 30 40 50Degree

10 3

10 2

PD

F

FIG. 4. Histogram of the degrees of vertices in the conflictgraph for dmax = 60. The distribution of the degrees approxi-mately follows a power law, with the exponent depending ondmax.

10 20 30 40 50 60dmax

0.10

0.09

0.08

0.07

0.06

0.05

0.04

0.03

0.02

pow

er la

w e

xpon

ent f

or c

onne

ctiv

ity

FIG. 5. Empirical power-law exponent versus dmax. The errorbars indicate the error obtained from the linear regression.

examine the treewidth of the connected components as aproxy for the hardness of the instances they represent.

Figure 6 shows that the treewidth of a connected com-ponent scales approximately linearly with its size. Thissuggests that realistic instances of the deconflicting areindeed hard, and not restricted to easier (bounded tree-width) instances of the generally hard problem. Moreover,the correlation γ between the tree-width of a connectedcomponent and its size increases with dmax, as shown inFigure 7. The larger dmax, the more potential conflictsthere are; restricting dmax also restricts the number ofconflicts.

5

0 50 100 150 200 250Number of flights in connected components

0

10

20

30

40

50

60

Tre

ewid

th

dmax = 6dmax = 60

0 10 200

2

4

6

FIG. 6. The treewidths of connected components versustheir sizes for various values of dmax. The correlation is ap-proximately linear, with a slope γ that depends on dmax. Thelinear fit is representing the trend in the region with numberof flights greater than 50.

10 20 30 40 50 60dmax

0.075

0.100

0.125

0.150

0.175

0.200

0.225

tree

wid

th v

s. s

ize 

of c

onne

cted

 com

pone

nt s

lope

FIG. 7. Slope γ as a function of the maximum delay time.The error bars indicate the error obtained from the linearregression.

IV. DISCRETIZING THE CONFIGURATIONSPACE

To apply quantum annealing to the deconflicting prob-lem, we must encode the configuration space d in binary-valued variables. To do so, we must first discretize andbound the allowed values. Let ∆d be the resolution ofthe allowed delays and dmax = Nd∆d the maximum al-lowed delay, so that di ∈ {∆dl|l ∈ [0, 1, . . . , Nd]}. Thelarger the configuration space is, the more qubits areneeded to encode it, and so determining the effect of this

discretization on solution quality is crucial to the effec-tive use of quantum annealing. To do so, we solve theconflict-resolution problem with departure delays only forvarious delay resolutions and upper bounds and comparethe various optima to the continuous problem withoutrestrictions (other than non-negativity) on the delays.

We consider two sets of instances, I18 and I60. For I18,the exact optima are found by modeling the problem as aconstraint satisfaction problem [11]; the largest instancein I18 has 50 flights and 104 potential conflicts.

The instances in I60 are much larger and harder (thelargest instance in I60 has 257 flights and 4068 poten-tial conflicts); we solved them by mapping to QUBO (asdescribed in the next section) and then using the Isoener-getic Cluster Method (a rejection-free cluster algorithmfor spin glasses that greatly improves thermalization) [12],which has been shown to be one of the fastest classicalheuristic to optimize QUBO problems [13]. Because ICMis a classical method, the penalty weights can be set ar-bitrarily large, ensuring that the desired constraints aresatisfied. ICM is not guaranteed to return the globaloptimum in general. However, for the sizes of instancesto which we applied ICM the results are sufficiently wellconverged to conclude that the solution found is indeedglobally optimal with exceedingly high probability.

Figure 8 shows the minimum total delay of a probleminstance with 19 flights and 47 potential conflicts from I18

for various values of ∆d and dmax. With the exceptionof the small maximum delay dmax = 3, the total delay ofthe solutions is nearly independent of the maximum de-lay. The total delay is non-decreasing with respect to thecoarseness ∆d of the discretization for a fixed maximumdelay dmax, and non-increasing with respect to dmax for afixed ∆d. Since the original data is discretized in time inunits of 1 minute, ∆d = 1 yield the same result as a con-tinuous variable with the same upper bound. Above somethreshold value d0

max, further increasing the maximum de-lay does not decrease the minimum total delay. With oneexception, we found that for all the investigated probleminstances d0

max ≤ 6 minutes (see Figure 8). Therefore weconclude, that a moderate maximum delay is sufficienteven for larger problem instances. On the other hand,the delay discretization should be as fine as possible toobtain a high quality solutions.

Figure (9) shows the dependence of the total delaytime optimized by ICM on the delay discretization ∆d forvarious problem instances extracted from the connectedcomponents of the conflict graph. Results are for maxi-mum delay of 60 minutes. As expected, the total delaydecreases by decreasing ∆d. This is consistent with theidea that smaller ∆d allows a finer optimization of thedelays of the flights.

In Figure (10) we show the optimal delay time foundby ICM as a function of the number of the flights inthe connected components. Results are for a maximumdelay of 60 minutes. Unfortunately, ICM was unable tooptimize connected components with more than 12 flights.This can be explained by recalling that ICM works the

6

1 2 3 4 5 6 7 8 9

d

5

10

15

20

25

30

35T

otal

 del

aydmax = 3dmax = 6dmax = 9dmax = 18

1 2 3 4 6 7 9 26 47 58 104Nc

5019

117

54

32

Nf

6

6

18 3

3

2

2

2 3

3 10

3

6

9

12

15

18

num

ber 

of in

stan

ces

FIG. 8. Influence of discretization on the solution quality.Top: Minimum total delay of a problem instance from I18

with 19 flights and 47 conflicts for various values of ∆d anddmax. Bottom: Results for continuous delay variables withupper bounds dmax. We show the minimum upper bound dmax

necessary to obtain same result as that without bounding thedelay. We used various instances in I18. The color code showsthe number of instances with the same total delay.

1 4 12 20 30 60

d

300150140

8472604036302420161211

874320

Tot

al d

elay

11

11

1 11 8

11

71

1 71 1

1 61

111 431

17 17 17 17 17 170

3

6

9

12

15

num

ber 

of in

stan

ces

FIG. 9. Total delay in dependence of the discretization pa-rameter ∆d for 26 different problem instances from I60 withup to in 12 flights and 25 conflicts. The color code shows thenumber of instances with the same total delay. 17 of these26 instances had trivial solutions for all values of ∆d, i.e thetotal delay vanishes.

best for almost-planar problem while its performancequickly decreases for fully-connected problems. Indeed,as shown in Section III, the underlying graph of connectedcomponents look more and more like a fully-connectedgraph rather than a tree graph as the number of flightsinside the connected component increases.

2 4 6 8 10 12Number of flights in connected components

0

50

100

150

200

250

300

Tot

al d

elay

 

d = 1d = 4d = 12d = 20d = 30d = 60

FIG. 10. . Optimal total delay found by using the IsoenergeticCluster Method (ICM) at fixed time step ∆d as a function ofnumbers of flight within each connected component. ICM wasunable to find solutions for connected component with morethan 12 flights.

V. MAPPING TO QUBO

In this section, we describe how to map to QUBO fromthe conflict-resolution problem limited to only departuredelays; a more general mapping is given in the appendix.

A. Binary encoding

Having suitably discretized the configuration space, wemust then encode it into binary-valued variables. Thevalue of di is encoded in Nd + 1 variables di,0, . . . , di,Nd

∈{0, 1} using a one-hot encoding:

di,l =

{1, di = l,

0, di 6= l;di = ∆d

Nd∑l=0

ldi,l. (12)

To enforce this encoding, we add the penalty function

fencoding = λencoding

Nf∑i=1

(Nd∑l=0

di,l − 1

)2

, (13)

where λencoding is a penalty weight sufficiently large toensure that any cost minimizing state satisfies fencoding =

7

0. (Note that in practice, we could do away withthe bit di,0 by removing it from (12) and substituting∑Nd−1l=1

∑Nd

l′=l+1 di,ldi,l′ into fencoding.) In terms of thesebinary variables, the total delay contribution to the costfunction is

fdelay = ∆d

Nf∑i=1

Nd∑l=0

ldi,l, (14)

Lastly, actualized conflicts are penalized by

fconflict = λconflict

∑k

∑l,l′|∆d(l−l′)∈Dk

i,j∈Ik|i<j

di,ldj,l, (15)

where again λconflict is a sufficiently large penalty weight.The overall cost function to be minimized is

f = fencoding + fdelay + fconflict. (16)

B. Softening the constraints

In the QUBO formalism, there are no hard constraints;thus we use of penalty functions in the previous section.For sufficiently large penalty weights, the optimum willsatisfy the desired constraints. However, precision isa limited resource in quantum annealing; therefore, wewould like to determine the smallest sufficient penaltyweights.

0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00

conflict

0.00

0.25

0.50

0.75

1.00

1.25

1.50

1.75

2.00

enco

ding valid

invalidValidity Boundary

FIG. 11. Validity of exact solution to a QUBO extracted froma problem instance with Nf = 7 flights and Nc = 9 conflictsin dependence on the choice of the penalty weights, λencoding

and λconflict. Here, ∆t = 6 and dmax = 18. In order to obtainthe exact solutions, we used a Max-SAT solver [14] after wemapped the QUBO instances to Max-SAT instances.

For a given instance, we say that a pair of penaltyweights (λconflict, λencoding) is valid if the minimum of thetotal cost function satisfies both the conflict and encodingconstraints when using those weights. Figure 11 shows the

phase space of these penalty weights for a single instancewith 7 flights and 9 conflicts. The box-like boundarybetween valid and invalid penalty weights suggests thatthe validity of the two penalty weights is independent;this box-like boundary is found for all of our instanceswith up to 7 flights and 9 conflicts.

VI. QUANTUM ANNEALING

In this section we report on our efforts to solve probleminstances from the departure delay model from Section Vwith a D-Wave 2X quantum annealer. We restrictedourselves to instances with dmax = Dmax = 18 and ∆d ∈{3, 6, 9}.

A. Embedding

In order to make a QUBO amenable for a D-Wave 2Xquantum annealer, it has to obey certain hardware con-straints. For instance the connections between the binaryvariables are restricted to the so called Chimera graph [5].However, it is possible to map every QUBO to anotherQUBO which obeys the constraints of the Chimera archi-tecture while increasing the number of binary variablesused by a so called minor-embedding technique [15]

∆d 3 6 9

Number of flights Nf 13 19 50Number of conflicts Nc 27 47 104Number of logical qubits 91 76 150Average number of physical qubits 631 395 543

TABLE I. Parameters of the largest embeddable instances forthe D-Wave 2X

∆d 3 6 9

Number of flights Nf 19 50 64Number of conflicts Nc 47 104 261Number of logical qubits 133 200 192Average number of physical qubits 1235 1080 1121

TABLE II. Parameters of the largest embeddable instancesfor the D-Wave 2000Q

Of course the QUBO graph structure of the instances isnot suitable for direct calculation on the D-Wave machine.Therefore we used D-Wave’s heuristic embedding algo-rithm [16] to embed instances with up to Nf = 50 andNc = 104 on the D-Wave 2X as well as up to Nf = 64 andNc = 261 on the D-Wave 2000Q depending on discretiza-tion (cf. Tables I and II). We generated up to 5 differentembeddings for each QUBO instance, and selected theone that used the smallest number of physical qubits. Infigure 12 one can see the dependence of the number ofphysical qubits on the number of logical qubits.

8

0 25 50 75 100 125 150 175 200Number of logical qubits

0

250

500

750

1000

1250

1500N

umbe

r of

 phy

sica

l qub

itsd = 9d = 6d = 3

FIG. 12. Number of physical qubits versus the number oflogical qubits for embeddings into a D-Wave 2000Q of theQUBO instances corresponding to I18.

B. Success Probability

In order to investigate the performance of the D-Wavemachines 2X and 2000Q, we compared the annealing re-sults to the ones of an exact solver. We used an exactMax-SAT solver [14] after we mapped the QUBOs toMax-SAT. For each QUBO instance, we ran the anneal-ing process in between 104 and 106 times. The successprobability p is then given by the ratio of the number ofannealing solutions which are equal to the exact solutionand the number of total annealing runs. As a measure ofthe runtime of the machine, we used the time-to-solutionwith probability 99%.

T99 =ln(1− 0.99)

ln(1− p)TAnneal ,

where TAnneal is the annealing time which was set to 20µs.In figure 13 the dependence of the time to solution

T99 on the number of flights and the number of conflictsis shown. One can see, that the success probability de-creases for larger problem instances as well as for finerdiscretizations. We conjecture, that this is mainly due tothe limited precision in the specification of a QUBO on theD-Wave machines. In order to investigate the influenceof limited precision, we need a measure for the precisionneeded to represent a given QUBO instance. If the em-bedded QUBO instance is given by H =

∑ij Qijxixj

with xi ∈ {0, 1} the corresponding Ising model

H =∑i

hisi +∑ij

Jijsisj , si ∈ {−1, 1} ,

can be obtained by the transformation si = 2xi − 1. Ameasure for the precision needed is then given by themaximum coefficient ratio

Cmax = max

[maxi |hi|mini |hi|

,maxij |Jij |minij |Jij |

].

The larger this number is, the finer precision is needed forcorrectly representing the QUBO on a D-Wave machine.

2 × 100 3 × 100 4 × 100 6 × 100

Number of flights Nf

10 4

10 3

10 2

10 1

100

101

102

T 99 

/ sec

onds

d = 3d = 6d = 9

D2XD2000Q

100 2 × 100 3 × 100 4 × 100 6 × 100

Number of conflicts Nc

10 5

10 4

10 3

10 2

10 1

100

101

102

T 99 

/ sec

onds

FIG. 13. Median of the time to solution with 99 % withprobability T99 for QUBO instances in dependence of thenumber of flights Nf and the number of conflicts Nc. The errorbars indicate the 35% and 65% percentiles. We used 10000annealing runs for each instance, penalty weights λconflict =λencoding = 1 and 5 different embeddings. The ferromagneticcoupling between all physical qubits of the same logical qubitwas set to JF = −1 in absolute units for all the instances. Thesolid lines are results from the D-Wave 2X whereas the dashedlines are results from the D-Wave 2000Q. For these results, wedid not use gauges and used energy minimization to deal withbroken qubit chains.

1.2 1.0 0.8 0.6 0.4 0.2JF

0.0

0.1

0.2

0.3

Max

imum

 Suc

cess

 Pro

babi

lity

0.4 0.2

JoptF

0

25

50

FIG. 14. Maximum success probability on the D-Wave 2000Qfor a QUBO instance with Nf = 5, Nc = 5 and ∆d = 6 independence of JF , where JF is given in units of the largestcoefficient of the embedded Ising model. We used 5 differentembeddings, 100000 annealing runs and penalty weights λ =λconflict = λunique = 1 for each of the data points. The reddata point indicates the optimal value Jopt

F . The inset showsthe distribution of Jopt

F for all solvable instances.

9

The success probability also depends on the choice ofthe ferromagnetic intra-logical qubit coupling JF . Fig-ure 14 shows the dependence of the success probability onthis coupling for one particular problem instance. How-ever, the general behavior of this curve is instance indepen-dent. For very small JF , the qubit strings which representlogical qubits might be broken by coupling to outer logi-cal qubits and the success probability is suppressed. Onthe other hand, if JF is very large, the precision neededCmax will eventually surpass the machine precision andthe success probability will decrease. In between thesetwo extrema, there will be a sweet spot with an opti-mal Jopt

F which yield maximum success probability. We

determined the optimal coupling JoptF for the problem

instances by sweeping over values in between JF = −1.25to JF = −0.125 in units of the largest coefficient of theembedded Ising model. The inset in figure 14 shows thedistribution of the Jopt

F . Using the optimal couplings, theperformance is increased with respect to a constant valueJconstF = −1 in absolute units as one can see in figure 15.

2 3 4 5 10Number of flights Nf

10 5

10 4

10 3

10 2

10 1

100

101

102

T 99 

/ sec

onds

d = 9, JoptF

d = 6, JoptF

d = 3, JoptF

d = 9, , JconstF

d = 6, JconstF

d = 3, JconstF

1 2 3 4 5 10Number of conflicts Nc

10 5

10 3

10 1

101

T 99 

/ sec

onds

FIG. 15. Median of the time to solution with 99 % with proba-bility T99 for fixed and optimal JF . The error bars indicate the35% and 65% percentiles. We used up to 1000000 annealingruns on the D-Wave 2000Q for each instance, penalty weightsλconflict = λencoding = 1 and up to 5 different embeddings.The colored lines indicate results obtained with an optimalJF = Jopt

F which is instance dependent. The grey lines indicateresults obtained with a fixed JF = Jconst

f = −1 in absoluteunits.

Using the optimal coupling JoptF we can study the in-

fluence of the limited machine precision on the successprobability. Figure 16 shows the maximum success proba-

bility with optimal JF for all embeddable instances in I18.The influence of the limited machine precision can be seenin the decrease of the success probability with increasingprecision Cmax. The success probability vanishes aroundCmax ≈ 30 which corresponds to the machine precision ofthe D-Wave 2000Q of around ∼ 1/30. Since Cmax in gen-eral increases with the problem size as well as with finerdiscretizations, this explains the long time-to-solutionsfor large instances and fine discretizations in figures 13and 15.

1 5 10 30 100Cmax

0.0

0.2

0.4

0.6

0.8

1.0

Max

imum

 suc

cess

 pro

babi

lity 

 with

 opt

imal

 J FFIG. 16. Maximum success probability on the D-Wave 2000Qfor optimal JF for all embeddable instances in I18 against thecoefficient ratio Cmax.

VII. CONCLUSIONS

In this paper, we propose a novel QUBO mapping for asimplified version of the Air Traffic Management (ATM)conflict-resolution problem for wind-optimal trajectoriesinvolving minimum trajectory modifications. Althoughthese efforts are driven by making the problem amenableto quantum annealers, the techniques used may be ben-eficial also for the classical solution of the problem. Inour study, we considered the actual wind-optimal trajec-tories for transatlantic flights (NAT) on July 29, 2012.Given the large number of flights, the wind-optimal tra-jectories cannot be directly mapped in a QUBO model.To overcome this limitation, our modified version of theconflict-resolution problem assumes that the flight maneu-vers applied to avoid conflicts modify the wind-optimaltrajectories only locally, resulting in assigning “delays” tothe flights. Therefore, wind-optimal trajectories can be“hard encoded” in our QUBO formulation of the conflict-resolution problem with the flights delays being the onlyvariables to optimize. Nevertheless, as explained in Ap-pendix 2, our method is general enough to potentiallyinclude the effect of other maneuvers as well.

As part of our study, we also introduce a novel “pre-processing” algorithm to eliminate potential conflicts that,given a maximum delay, can never occur and clusteringadjacent conflicts. This novel approach does not onlyreduce the number of potential conflicts, but is also gives

10

an important indication of the underlying topology theconflict graph. Indeed, we have discovered that most ofthe flights have very few conflicts while there are fewflights that have conflicts in a non trivial way. The lattersets of flights represent the hardest part of the conflict-resolution problem to optimize. We want to emphasizethat the proposed pre-processing algorithm is generaland can be successfully applied to the existing conflict-resolution methods to improve both the speed and qualityof solutions.

We also present several different QUBO mapping in-cluding local and global trajectory deviations as well asincluding and excluding maneuvers. Due to the hard-ware limitations of the D-Wave machine we focus on amodel excluding maneuvers in order to keep the numberof variables small. Using D-Wave’s embedding algorithm,several smaller problem instances were embeddable ontothe D-Wave 2X as well as onto the D-Wave 2000Q. How-ever, the limiting factor for the success probability is notthe sizes of the chips but its limited precision. Thereforethe success probability is suppressed for finer discretiza-tions and larger problem sizes.

Finally, we have analyzed the performance of bothclassical and quantum heuristics in solving the QUBOmodel where only delays at the departure are allowed.Results show that it is already hard to find conflict-freesolution for a flight set that involve more than 12 flights.

This work represents the foundation for future work,including:

• Embed and solve QUBO instances for models thatbenefits from variable simplification (see section V),also including maneuvers on a quantum annealer.

• Improve performance of quantum annealing by alter-native embedding strategies and advanced annealingschedules available on newer D-Wave devices.

• Use best-available classical solvers that exploit theconflict graph description for classical solutions tothe problem. The studied problem is related to sev-eral works in multi-agent path planning [17, 18] andno-wait job-shop-scheduling problems [19] and thequantum annealing results could be benchmarkedwith the domain-specific solvers of these relatedproblems, after careful mapping.

VIII. ACKNOWLEDGEMENTS

This work has been funded by the NASA Aviation Sys-tems Division. D.V. was supported by NASA AcademicMission Services, contract number NNA16BD14C. Theauthors would like to acknowledge additional supportfrom the NASA Advanced Exploration Systems programand the NASA Ames Research Center. The views andconclusions contained herein are those of the authorsand should not be interpreted as necessarily represent-ing the official policies or endorsements, either expressed

or implied, of the U.S. Government. The U.S. Govern-ment is authorized to reproduce and distribute reprintsfor Governmental purpose notwithstanding any copyrightannotation thereon.

Appendix A: General QUBO mapping

In this section we describe a mapping to QUBO of amore general version of the deconflicting problem thanthat covered in the main text.

1. Alternative encodings

In the mappings describe both in the main text and theappendix, we use a one-hot encoding to encode a variable.This is best for the specific mappings we described, but invariants an alternative may be better. Say we have a vari-able x that we want to allow to have variables from finiteset W = {w1, w2, . . . , wm}. The one-hot encoding has mbits (xi)

mi=1 such that x =

∑mi=1 wixi and

∑mi=1 xi = 1.

While we focus on the case in which W = {0, 1, . . . ,m−1},our methods are not dependent on that being case, andin particular can address non-uniform sets of values, sayif via clever preprocessing it can be determined that sucha set would be sufficient. An alternative encoding wouldremove the requirement that exactly one of the bits is one.The variable x would still be encoded as x =

∑mi=1 wixi,

but without the one-hot constraint can take on valuesin {

∑i biwi|bi ∈ {0, 1}}. In particular, this encompasses

the unary encoding in which wi = 1 for all i and thusx ∈ [0,m], as well as the binary encoding wi = 2i−1 forwhich x ∈ [0, 2m − 1]. The latter has the advantage ofrequiring much fewer qubits, but at the cost of similarlyincreased precision. The former requires the same numberof qubits as the one-hot encoding we use, and even hasthe benefit of minimal precision, but does not allow forquadratic constraints that penalize certain pairs of valuesof variables, e.g. di − dj 6= Bk, without the use of ancil-lary bits. In models in which the bits xi only appear inthe sum

∑i xi, it is actually preferable to use the unary

encoding to improve the precision requirements. We stickto the one-hot encoding for simplicity, but in practice theunary encoding should be used when possible.

To make the expressions more concise, we define thegeneralized encoding penalty function

fencoding ({Xi}i) = λencoding

∑i

(∑x∈Xi

x− 1

)2

(A1)

that enforces the constraint that exactly one bit x is onefor each set of bits Xi.

11

2. Global trajectory modifications

Consider the case in which each trajectory can be mod-ified by a departure delay and some parameterized spatialtransformation, i.e. for each flight i there is a variable diand some parameter θi. For example, Rodionova et al. [2]consider a single angle θi that determines a sinusoidaltransformation of the trajectory. For the QUBO map-ping, we require that these variables be allowed to take onvalues from some finite set, so that are QUBO variablesare {di,α} and {θi,φ}, where di,α = 1 (di,α) indicates thatdi = α (di 6= α) and similarly for θi,φ. For every pairof flights i < j, we can efficiently (in time and spacepolynomial in the size of the input) compute whether thecorresponding trajectories conflict when modified accord-ing to di, dj , θi and θj . Let Bi,j be the set of valuesof (di,θi, dj ,θj) such that the the modified trajectoriesconflict. Lastly, let d(i,α),(j,β) = 1 indicate that di = αand dj = β, and similarly for θ(i,φ),(j,ψ). The overall costfunction is

fglobal

((di,α)i,α

(d(i,α),(j,β)

)i,j,α,β

(θ(i,φ),(j,ψ)

)i,j,φ,ψ

)=

fencoding + fconsistency + fdelay + fconflict, (A2)

where

fencoding

({{di,α}α ∪ {θi,φ}φ

}i

)(A3)

ensures that the values of di and θi are uniquely en-coded;

fconsistency =

λconsistency

[ ∑i<j,α,β

s(di,α, dj,β , d(i,α),(j,β)

)+

∑i<j,φ,ψ

s(θi,φ,θj,ψ,θ(i,φ),(j,ψ)

) ] (A4)

ensures consistency between the values of di,α, dj,β , andd(i,α),(j,β);

s(x, y, z) = 3z + xy − 2xz − 2yz (A5)

is a non-negative penalty function that is zero if and onlyif z = xy;

fdelay =∑i,α

αdi,α (A6)

is the cost function to be minimized; and

fconflict = λconflict

∑i<j

∑(α,φ,β,ψ)∈Bi,j

d(i,α),(j,β)θ(i,φ),(j,ψ)

(A7)penalize conflicts.

3. Local trajectory modifications

Alternatively, we can consider modifications to thetrajectory only near conflicts. We describe a few specialmodels and their mapping to QUBO, though many moresuch ways of doing so, and we leave a full accounting forfuture work.

a. Exclusive avoidance

Suppose for every conflict k and associated pair offlights i < j, there is a way for either flight to go aroundthe trajectory of the other, introducing some delay di,k toflight i or dj,k to flight j depending on which trajectory ischanged. Let ak = ai,k = 1 (ai,k = 0) indicate that flighti’s trajectory is changed (unchanged), and for conveniencelet aj,k = 1− ai,k, though only one (qu)bit will be usedper conflict. Adding in the departure delay, we have thetotal cost function

fexclusive

((di,α)i,α, (ak)k

)= fdelay + fencoding, (A8)

where

fdelay =∑i

[∑α

αdi,α +∑k∈Ki

di,kai,k

](A9)

and fencoding is as in (13). This assumes that the trajec-tory modifications don’t introduce potential conflicts withother flights; this assumption can be partially relaxed byadding penalty terms of the form ai,kaj,k′ or di,αaj,k asappropriate.

b. Flexible avoidance

In the exclusive avoidance model, it is required that oneor the other flight is delayed at each conflict. We can relaxthis by accounting for the fact that if the flights arrivingat a potential conflict are already relatively delayed, theconflict could be passively avoided (i.e. with no activemaneuver). Let Dk,γ = 1 (Dk,γ = 0) indicate that Dk = γ(Dk 6= γ), where Dk is the difference in the accumulateddelays at conflict k as defined in (8).

The total cost function is

fflexible

((di,α)i,α, (ai,k)k,i∈Ik , (Dk,γ)k,γ

)=

fencoding + fdelay + fconsistency + fconflict, (A10)

where the first term is

fencoding

({{di,α}α

}i∪{{Dk,γ}γ

}k

); (A11)

the consistency term is

fconsistency = λconsistency

∑k

(Di,k −Dj,k −

∑γ

γDk,γ

)2

(A12)

12

using the notational variables

Di,k =∑α

αdi,α +∑

k′∈Ki,k

di,k′ai,k′ ; (A13)

fdelay is as in (A9) but where ai,k and aj,k are separatebits; and

fconflict = λconflict

∑k

∑γ∈Bk

[Dk,γ (1− ai,k − aj,k) + 2ai,kaj,k]

(A14)If we want to allow both flights to be delayed at conflict

ai,k = aj,k = 1, we must introduce an ancillary bit akthat indicates whether at least one flight is delayed atconflict k, adding

λconsistency

∑k

[(ai,k + aj,k) (1− 2ak) + ai,kaj,k] (A15)

to fconsistency, and replacing fconflict with∑k

∑γ∈Bk

Dk,γ(1− ak). (A16)

c. Interstitial delays

In the interstitial-delay model, the local modificationsare not made at conflicts but between them, and conflictsare only avoided via accumulated delays. That is, the

delay di,k introduced to flight i before reaching conflict kbut after leaving the previous conflict κi,k = maxk′∈Ki,k

k′.Unlike in the flexible avoidance model, di,k is now avariable rather than a parameter, and we encode it usingbits di,k,δ.

finterstitial

((di,α)i,α, (Di,k,γ)i,k∈Ki,γ

)=

fencoding + fconsistency + fconflict + fdelay, (A17)

where

fencoding

({{di,α}α

}i∪⋃i

{{Di,k,γ}γ

}k∈Ki

), (A18)

fconsistency =∑i

∑k∈Ki

∑(γ,γ′)∈Bi,k

Di,k,γDi,κi,k,γ′ , (A19)

fconflict = λconflict

Nc∑k=1

∑(γ,γ′)∈Bk

Di,k,γDj,k,γ′ , (A20)

and

fdelay

∑i

∑γ

Di,maxKi,γ . (A21)

[1] H. K. Ng, B. Sridhar, and S. Grabbe, Optimizing Air-craft Trajectories with Multiple Cruise Altitudes in thePresence of Winds, Journal of Aerospace Information Sys-tems 11, 35 (2014), URL http://dx.doi.org/10.2514/

1.I010084.[2] O. Rodionova, D. Delahaye, B. Sridhar, and H. Ng., De-

conflicting wind-optimal aircraft trajectories in north at-lantic oceanic airspace, Proceedings of Advanced AircraftEfficiency in a Global Air Transport System (AEGATS’16)Conference (2016).

[3] J. K. Kuchar and L. C. Yang, A review of conflict detectionand resolution modeling methods, IEEE Transactions onintelligent transportation systems 1, 179 (2000).

[4] J. K. Kuchar and L. C. Yang, A comparison frameworkfor conflict detection and resolution multi agent modelingmethods in air traffic management, Intenational Journalof of Information Technology, Control and Automation(IJITCA) 2, 51 (2012).

[5] E. G. Rieffel, D. Venturelli, B. O’Gorman, M. B. Do,E. M. Prystay, and V. N. Smelyanskiy, A case study inprogramming a quantum annealer for hard operationalplanning problems, Quantum Information Processing 14,1 (2015), ISSN 1573-1332, URL http://dx.doi.org/10.

1007/s11128-014-0892-x.[6] E. Rieffel, D. Venturelli, M. Do, I. Hen, and J. Frank,

Parametrized families of hard planning problems fromphase transitions (2014), URL https://www.aaai.org/

ocs/index.php/AAAI/AAAI14/paper/view/8614/8697.

[7] D. Venturelli, D. J. J. Marchand, and G. Rojo, Quan-tum Annealing Implementation of Job-Shop Scheduling,arXiv:1506.08479 [quant-ph] (2015), arXiv: 1506.08479,URL http://arxiv.org/abs/1506.08479.

[8] R. Albert, H. Jeong, and A.-L. Barabasi, Internet: Diam-eter of the world-wide web, nature 401, 130 (1999).

[9] U. Bertele and F. Brioschi, Nonserial dynamic program-ming (Academic Press, 1972).

[10] R. Halin, S-functions for graphs, Journal of geometry 8,171 (1976).

[11] E. Hebrard, E. O’Mahony, and B. O’Sullivan, Con-straint Programming and Combinatorial Optimisation inNumberjack (Springer Berlin Heidelberg, Berlin, Heidel-berg, 2010), pp. 181–185, ISBN 978-3-642-13520-0, URLhttp://dx.doi.org/10.1007/978-3-642-13520-0_22.

[12] Z. Zhu, A. J. Ochoa, and H. G. Katzgraber, Efficientcluster algorithm for spin glasses in any space dimension,Physical review letters 115, 077201 (2015).

[13] S. Mandra, Z. Zhu, W. Wang, A. Perdomo-Ortiz, andH. G. Katzgraber, Strengths and weaknesses of weak-strong cluster problems: A detailed overview of state-of-the-art classical heuristics versus quantum approaches,Physical Review A 94, 022337 (2016).

[14] A. Kugel, Improved exact solver for the weighted max-satproblem, URL https://www.uni-ulm.de/fileadmin/

website_uni_ulm/iui.inst.190/Mitarbeiter/kuegel/

maxsat.pdf.

13

[15] V. Choi, [0804.4884] Minor-Embedding in AdiabaticQuantum Computation: I. The Parameter Setting Prob-lem, URL http://arxiv.org/abs/0804.4884.

[16] J. Cai, W. G. Macready, and A. Roy, A practical heuris-tic for finding graph minors, arXiv:1406.2741 [quant-ph](2014), arXiv: 1406.2741, URL http://arxiv.org/abs/

1406.2741.

[17] S. Akella and S. Hutchinson, in Robotics and Automa-tion, 2002. Proceedings. ICRA’02. IEEE InternationalConference on (IEEE, 2002), vol. 1, pp. 624–631.

[18] X. Wang, M. Kloetzer, C. Mahulea, and M. Silva, inDecision and Control (CDC), 2015 IEEE 54th AnnualConference on (IEEE, 2015), pp. 324–329.

[19] S. Sahni and Y. Cho, Complexity of scheduling shops withno wait in process, Mathematics of Operations Research4, 448 (1979).