Secure Communication for Mobile Agents in anAdversarial Environment

Sourabh BhattacharyaCoordinated Science Lab

University of IllinoisUrbana, IL 61801 USA

Email: sbhattac@illinois.edu

Tamer BasarElectrical and Computer Engineering

University of IllinoisUrbana, IL 61801 USA

Email: basar1@illinois.edu

Abstract—In this work, a mobility based perspective of jam-ming is presented, based on our ongoing work related tosecure communication in networks of autonomous vehicles. Theframework of differential game theory is extended from theclassical two-player scenario to multi-player games involving twoteams. Necessary conditions satisfied by the optimal trajectoriesare derived under assumptions of uniqueness and existence ofthe optimal trajectories. Finally, results are presented for severalsimulation scenarios involving different numbers of vehicles andtheir dynamics.Keywords: Differential Game Theory, Pursuit-EvasionGames, Jamming, Security.

I. INTRODUCTION

The decentralized nature of wireless ad hoc networks makesthem vulnerable to security threats. A prominent example ofsuch threats is jamming: a malicious attack whose objectiveis to disrupt the communication of the victim network inten-tionally, causing interference or collision at the receiver side.Jamming attack is a well-studied and active area of researchin wireless networks. In late 2009, there have been reports ofpredator drones being hacked [37], [38] resulting in intrudersgaining access to classified data being transmitted from anaircraft. Unauthorized intrusion of such kind has started arace between the engineers and the hackers; therefore, wehave been witnessing a surge of new smart systems aiming tosecure modern instrumentation and software from unwantedexogenous attacks [2].

Jamming is a malicious attack whose objective is to disruptthe communication of the victim network intentionally causinginterference or collision at the receiver side. Jamming attackis a well-studied and an active area of research in wirelessnetworks. Many defense strategies have been proposed byresearchers against jamming in wireless networks. In [30],Wu et al. propose two strategies to evade jamming. The firststrategy, channel surfing, is a form of spectral evasion thatinvolves legitimate wireless devices changing the channel thatthey are operating on. The second strategy, spatial retreats, is aform of spatial evasion whereby legitimate devices move awayfrom the jammer. In [28], Wood et al. present a distributedprotocol to map jammed region so that the network canavoid routing traffic through it. The solution proposed by

This work was supported in part by a grant from ARO MURI and AFOSR.

Cagalj et al. [6] uses different worm holes (wired wormholes, frequency-hopping pairs, and uncoordinated channelhopping) that lead out of the jammed region to report thealarm to the network operator. In [29], Wood et al. investigatehow to deliberately avoid jamming in IEEE 802.15.4 basedwireless networks. In [7], Lin Chen et al. propose a strategy tointroduce a special node in the network called the anti-jammerto drain the jammer’s energy. To achieve its goal, the anti-jammer configures the probability of transmitting bait packetsto attract the jammer to transmit.

For a static jammer and mobile nodes, the optimal strategyfor the nodes is to retreat away from the jammer after detectingjamming. Constraints in the motion of the agents as well as ofthe jammers might lead to situations in which the strategy ofsimple retreat ceases to be optimal. The scenarios presentedin this paper address the aforementioned problem. The overar-ching goal of our work is to analyze jamming in multi-agentsystems from the perspective of pursuit-evasion games so asto relate the resilience in the communication network to theirmobility models, in the presence of a mobile adversary in thecommunication channel. Pursuit-evasion games are a specialclass of zero-sum games [1] that model conflict scenariosbetween mobile players. Such dynamic games governed bydifferential equations can be analyzed using tools from differ-ential game theory [13] [11]. In [33], an elaborate referenceto prior work in two-player, as well as, multi-player pursuit-evasion games, relevant to this paper, is presented.

In this paper, we present a general framework and associatedtechniques, developed in [33], [34], [35] and [32], to addressdifferent scenarios of vehicular jamming in combat scenarios.We also shed some light on future problems relevant to jam-ming in mobile networks from the perspective of differentialgame theory. In Section II, we introduce the communicationand the dynamic model of the agents. In Section III, we presenta formal statement of the problem under consideration. In Sec-tion IV, we present a derivation of the necessary conditions thatmust be satisfied for optimality. In Section V, we present ourresults for various scenarios and provide a qualitative analysisof our results. In Section VI, we present our current researchand also the future problems that need to be addressed.

14th International Conference on Information FusionChicago, Illinois, USA, July 5-8, 2011

978-0-9824438-3-5 ©2011 ISIF 1985

II. COMMUNICATION AND MOBILITY MODELS

In this section, at first, we present the communication modelbetween a receiver and a transmitter in the presence of ajammer. Next, we present the dynamic models that governthe motion of the nodes as well as of the jammers.

A. Jammer and Communication Model

Consider a mobile node (receiver) receiving messages fromanother mobile node (transmitter) at some frequency. Bothcommunicating nodes are assumed to be lying on a plane.Consider a third node that is attempting to jam the communi-cation channel in between the transmitter and the receiver bysending a high power noise at the same frequency. This kindof jamming is referred to as trivial jamming. Two other typesof jamming are :

1) Periodic jamming: This refers to a periodic noise pulsebeing generated by the jammer irrespective of the pack-ets that are put on the network.

2) Intelligent jamming: In this mode of jamming a jammeris put in a promiscuous mode to destroy primarily thecontrol packets.

A variety of metrics can be used to compare the effectivenessof various jamming attacks. Some of these metrics are energyefficiency, low probability of detection, and strong denialof service [17] [16]. In this paper, we use the ratio of thejamming-power to the signal-power (JSR) as the metric. From[20], we have the following models for the JSR (ξ) at thereceiver’s antenna.

1) Rn model

ξ =PJT

GJRGRJPTGTRGRT

10n log10(DT RDJR

)

2) Ground Reflection Propagation

ξ =PJT

GJRGRJPTGTRGRT

(hJhT

)2(DTR

DJR

)4

3) Nicholson

ξ =PJT

GJRGRJPTGTRGRT

104 log10(DT RDJR

)

where PJTis the power of the jammer transmitting antenna,

PT is the power of the transmitter, GTR is the antenna gainfrom transmitter to receiver, GRT is the antenna gain fromreceiver to transmitter, GJR is the antenna gain from jammerto receiver, GRJ is the antenna gain from receiver to jammer,hJ is the height of the jammer antenna above the ground,hT is the height of the transmitter antenna above the ground,DTR is the Euclidean distance between the transmitter andthe receiver, and DJR is the Euclidean distance between thejammer and the transmitter. All the above models are based onthe propagation loss depending on the distance of the jammerand the transmitter from the receiver. In all the above modelsJSR is dependent on the ratio DT R

DJR.

For digital signals, the jammer’s goal is to raise the ratio toa level such that the bit error rate [21] is above a certainthreshold. For analog voice communication, the goal is to

reduce the articulation performance so that the signals are dif-ficult to understand. Hence we assume that the communicationchannel between a receiver and a transmitter is considered tobe jammed in the presence of a jammer if ξ ≥ ξtr where ξtr isa threshold determined by many factors including applicationscenario and communication hardware. If all the parametersexcept the mutual distances between the jammer, transmitterand receiver are kept constant, we can conclude the followingfrom all the above models: If the ratio DT R

DJR≥ η then the

communication channel between a transmitter and a receiveris considered to be jammed. Here η is a function of ξ, PJT

, PT ,GTR, GRT , GJR, GRJ and DTR. Hence if the transmitter isnot within a disc of radius ηDJR centered around the receiver,then the communication channel is considered to be jammed.We call this disc as the perception range. The perceptionrange for any node depends on the distance between thejammer and the node. For effective communication betweentwo nodes, each node should be able to transmit as well asreceive messages from the other node. Hence two nodes cancommunicate if they lie in each other’s perception range.

We will adopt the above jamming and communicationmodel, for the rest of the paper.

B. Dynamics of the nodes

We assume that there are m agents in the network in thepresence of a jammer. Let the dynamics associated with theith agent be given by the following equation:

xi = fi(xi,ui) (1)

where, xi ∈ Rni , ui ∈ Ui ' φ : [0, t] → Ai |φ(·) is measurable, where Ai ⊂ Rpi . fi : Rn ×Ai → R isuniformly continuous, bounded and Lipschitz continuous in xifor fixed ui. Consequently, given a fixed ui(·) and initial point,there exists a unique trajectory solving Equation (1) [39]. Letthe state of node i be denoted as xi ∈ Xi ⊂ Rni .

Let X′

i denote the state-space of the jammer. We assumethat the jammer has the following dynamics associated withitself:

x′i = f′

i (x′i,u′i) (2)

where x′i ∈ Rn′i , u′i ∈ U ′i ' φ : [0, t] → A′i |

φ(·) is measurable, where A′i ⊂ Rp′i . f ′i : Rn′

i × A′i → Ris uniformly continuous, bounded and Lipschitz continuous inx′i for fixed u′i.

Let X′ = X′1×· · ·×Xn×X′1×· · ·×X′m ⊂⊕

i Rni×⊕

i Rn′i

represent the entire state of the system, where,⊕

representsthe Cartesian product of the Euclidean spaces Rni .

We define the workspace [36] as the ambient space inwhich the agents operate. Since we are interested in vehicularnetworks, the ambient space of the nodes is either R2 or R3.Since all the agents reside in the same ambient space, weuse Ω to denote the workspace for all agents. Let xi denotethe coordinates of the ith agent in Ω. We assume that Ω isequipped with a distance metric ρ : Ω× Ω→ R.

As a simple example to highlight the difference between thestate space and the workspace, consider the following second

1986

order agent that moves on a straight line, with u as the controlinput.

x1 = x2

x2 = u

The state space for the system can be represented by thevector x = [x1 x2]T . The state space is two dimensionalbut the agent can only move on a straight line and hence theworkspace is Ω = R1. For any state x, x = x1. For two statesx∗, x∗∗ ∈ Ω, ρ(x∗, x∗∗) =| x∗1 − x∗∗1 |.

In the next section, we present a differential game formu-lation for the problem of maintaining connectivity among theagents in the presence of the jammer.

III. PROBLEM FORMULATION

In this section, the problem of maintaining network connec-tivity among the agents is formally stated. Due to the presenceof a jammer in the vicinity, the connectivity of the underlyingcommunication network depends on the position of the agentsrelative to the jammer. The network is modeled as a graph.Since the agents and the jammer are assumed to be mobile,the connectivity of the network evolves in time rendering thegraph to be dynamic. Moreover, the topology of the graphdepends on the state of the nodes. Therefore, the framework ofstate-dependent graphs [41] is considered. A state-dependentgraph is a mapping, gc, from the state space X, to the set ofall labeled graphs on n vertices, G(n), i.e.,

gc : X→ G(n)

It is assumed that the order of these graphs at all times is n,since the number of agents remains constant. Let E(gc(x))denote the edge-set of the graph under consideration.

Let d = ρ(xi, xj), where xi and xj are the coordinates ofthe nodes i and j in the workspace Ω equipped with a distancemetric ρ. Let Br[p] = y ∈ Ω | ρ(y, p) ≤ r. We define Sij ⊂Xi× Xj , to be the following set

Sij = (xi, xj) | x′k /∈ Bηd[xi] ∪Bηd[xj ],∀k ∈ [1, · · · ,m]

Nodes i and j belong to the edge set of gc(x) when thefollowing condition is satisfied:

ij ∈ E(gc(x)) if and only if (xi, xj) ∈ Sij (3)

The collection of edge states is denoted as

S = Siji,j∈[1,··· ,n],i 6=j with Sij ⊂ Xi × Xj

From [41], the state dependent graph is defined as follows:

Definition: Given the set system S , the map gc :X→ Gmwith an image consisting of graphs of order m, having anedge between vertex i and j iff (xi, xj) ∈ Sij , is defined asa state-dependent graph with respect to S .

Consider a situation in which the network is initiallyconnected in the presence of a jammers. The objective of the

jammers is to disconnect the communication network. Sincethe underlying communication network can be modeled asa graph, various notions of connectivity [40] can be used tospecify the objective of the team of jammers. In this work,the objective of the team of jammers is to partition the nodesof the graph into two or more components. Disconnectionrefers to a situation in which there are two agents in thenetwork and no path exists in the communication network forexchanging messages. Based on the objective of the jammers,we can formulate the following zero-sum differential gamebetween the jammers and the nodes in the network.

Problem: Consider a play in which the agents are initiallyconnected. The objective of the jammers is to minimize thetime in which it disconnects the communication network byjamming the communication channel between agents. Theobjective of agents is to maximize the time for which com-munication network remains connected. The game terminatesat the first instant at which the communication network getsdisconnected. Compute the optimal motion strategies for eachagent.

In the next section, we present the necessary conditions thatare satisfied by the optimal strategies of the agents.

IV. OPTIMAL STRATEGIES

In this section, we introduce the concept of optimal strate-gies for the vehicles.

Given the control histories of the vehicles,u1i (·)ni=1, u2

i (·)mi=1, the outcome of the game is denotedby π : X ×X′ × Un+m

i → R and is defined as the time oftermination of the game:

π(x0, u1i (·)ni=1, u2

i (·)mi=1) = tf (4)

where tf denotes the time of termination of the game whenthe players play (u1

i (·)ni=1, u2i (·)mi=1) starting from an

initial point x0 ∈ X × X′. The game terminates when thecommunication network gets disconnected. The objective ofthe jammers is to minimize the termination time and theobjective of the agents is to maximize it.

Since the objective function of the team of agents is inconflict with that of the team of jammers the problem canbe formulated as a multi-player zero sum team game. A play(u1∗

i (·)ni=1, u2∗i (·)mi=1) is said to be the optimal for the

players if it satisfies the following conditions:

u1∗i ni=1 = arg max

u1i n

i=1

π[x0, u1i (·)ni=1, u2∗

i (·)mi=1]

u2∗i mi=1 = arg min

u2i m

i=1

π[x0, u1∗i (·)ni=1, u2

i (·)mi=1]

The value of a game, denoted by the function J : X → R,can then be defined as follows:

J(x) = π[x0, u1∗i (·)ni=1, u2∗

i (·)mi=1] (5)

The value of the game is unique at a point X in thestate-space. An important property satisfied by optimal strate-gies is the Nash equilibrium property. A set of strategies

1987

(u1i ni=1, u

2i mi=1) is said to be in Nash equilibrium if no

unilateral deviation in strategy by a player can lead to abetter outcome for that player. This can also be expressedmathematically by the following condition:

π[x0, u1i ni=1, u

2i mi=1] ≤ π[x0, u1

i ni=1,u2j , u

2−j ],∀j ∈ [1,m]

π[x0,u1j , u

1−j , u

2i ni=1] ≤ π[x0, u1

i ni=1, u2i mi=1],∀j ∈ [1, n]

In the above expressions u1−j represents the controls of

all the agents except agent j. Similarly, u2−j represents the

controls of all the jammers except the jth jammer. From theabove definition, we can conclude that there is no motivationfor a player to deviate from its equilibrium strategy. In general,there may be multiple sets of strategies for the players thatare in Nash equlibrium. Assuming the existence of a value, asdefined in (5) and the existence of a unique Nash equilibrium,we can conclude that the Nash equilibrium concept of person-by-person optimality, is a necessary condition that needs tobe satisfied by the set of optimal strategies for the players,and furthermore, computing the set of strategies that are inNash equilibrium also provides the set of optimal strategies.In the following analysis, we assume that the aforementionedconditions are satisfied by the optimal strategies.

The following theorem provides a relation between theoptimal strategy of each player and the gradient of the valuefunction, ∇J .

Theorem 1: Assuming that J(x) is a smooth function of x,the optimal strategies (u1∗

i ni=1, u2∗i mi=1) satisfy the follow-

ing condition:

u1∗i = arg max

ui

∇J · f(x, u1i ni=1, u2∗

i mi=1), i ∈ [1, n]

u2∗i = arg min

u2∇J · f(x, u1∗

i ni=1, u2i mi=1), i = [1,m]

Proof: The proof essentially follows the two-player ver-sion as provided in [11].

Let us consider the game from a point x in the state spaceat time t. The outcome functional is provided by the followingexpression:

π(x(t), u1i (·)ni=1, u2

i (·)mi=1) =∫ t+h

t

dt+ J(x(t+ h))

Using Taylor series approximation of J we obtain the follow-ing relation:

J(x(t+ h))− J(x(t)) =J(x(t) + f(x(t), u1

i (t)ni=1, u2i (t)mi=1)h+ hε(h))− J(x(t))

where ε(h) is a vector with each entry belonging to o(h).Let δ = f(x,u)h+ hε(h). Then

=n∑i=1

∂J

∂xiδi + |δ|o(|δ|)

= h[n∑i=1

ni∑j=1

J(xi)j[fi(xi,u1

i )]j +m∑i=1

nJ∑j=1

J(x′i)j

[f ′i(x′i,u2i )]j

+α(h)]

where, limh→0 α(h) = 0 and J(xi)jrepresents the derivative

of J with respect to the jth element of xi.

π(x(t), u1i (·)ni=1, u2

i (·)mi=1) = J(x(t)) + h(1 +n∑i=1

ni∑j=1

J(xi)j[fi(xi,u1

i )]j +m∑i=1

nJ∑j=1

J(x′i)j

[f′

i (x′i,u2i )]j + α(h))

First, let us consider the controls of the team of jammers. Fromthe Nash property, we can conclude that if u1

j = u1∗j , ∀j ∈

[1, · · · , n] and u2−i = u2∗

−i then u2i = u2∗

i minimizes the lefthand side of the above equation. Therefore, we can concludethe following:

1) The optimal control satisfies the following condition:

u2∗i (t) = arg min

u2i

J(x′)j[f ′i(x′i,u

2i )]j (6)

In a similar manner the controls of the agents satisfy thefollowing condition:

u1∗i (t) = arg max

u1i

J(xi)j[fi(xi,u1

i )]j (7)

2) In the case, when ui = u∗i for all players in (6) weobtain the following relation:

π(x(t), σ1∗i ni=1, σ2∗

i mi=1)︸ ︷︷ ︸J(x(t))

= J(x(t)) + h(1 +

n∑i=1

ni∑j=1

J(xi)j[fi(xi,u∗i )]j +

m∑i=1

nJ∑j=1

J(x′i)j

[f ′i(x′i,u2∗i )]j + α(h))

⇒ h(1 +n∑i=1

ni∑j=1

J(xi)j[fi(xi,u∗i )]j +

m∑i=1

nJ∑j=1

J(xJ )j

[f ′i(x′i,u2∗i )]j + α(h)) = 0

Taking limh→0 on both sides leads to the following relation:

1 +n∑i=1

ni∑j=1

J(xi)j[fi(xi,u∗i )]j +

m∑i=1

nJ∑j=1

J(x′i)j

[f ′i(x′i,u2∗i )]j = 0 (8)

(6), (7) and (8) extend the Isaacs’ conditions that provide theoptimal controls for two-player zero-sum differential games tothe case of multi-player zero-sum team differential games.

The Hamiltonian of the system is given by the followingexpression:

H(x,∇J, u1∗i ni=1, u2∗

i mi=1) = 1 +n∑i=1

ni∑j=1

J(xi)j[fi(xi,u∗i )]j +

m∑i=1

nJ∑j=1

J(x′i)j

[f ′i(x′i,u2∗i )]j

which is the left side of (8). Hence, (8) can equivalently beexpressed as:

H(x,∇J, u1∗i ni=1, u2∗

i mi=1) = 0 (9)

1988

In addition to the above conditions, the value function alsosatisfies the retrogressive path equation which is given in thefollowing theorem.

Theorem 2: The value function follows the following par-tial differential equation (PDE) along the optimal trajectory,namely the retrogressive path equation (RPE)

˚(∇J) =∂H∂x

(10)

where˚denotes derivative with respect to retrograde time.The proof of the above theorem is an extension to the

two-player scenario presented in [11]. In the next section, wecharacterize the terminal manifold.

V. TERMINAL CONDITIONS

In Section III, we proposed a mapping from the state of thesystem to a graph G that encodes the underlying connectivityof the network of UAVs. The connectivity of G can beinvestigated by analyzing the Laplacian of the graph that isdefined as follows:• Laplacian of a graph (L(G)) : It is an n×n matrix with

entries given as follows:

1) aij =−1 if an edge exists between i and j0 if no edge exists between i and j

2) aii = −∑mk=1,k 6=i aik

The second-smallest eigenvalue of L(G), denoted asλ2(L(G)), is called the Fiedler value. It is also called thealgebraic connectivity of G. It has emerged as an importantparameter in many systems problems defined over networks.In [43], [42], [44], it has also been shown to be a measure ofthe stability and robustness of the networked dynamic system.Since this work deals with connectivity maintenance in thepresence of malicious intruders, λ2(L(G)) arises as a naturalparameter of interest for both players. The following theoremrelates the connectivity of G to its Fiedler value.

Lemma 1: [40]: A graph G is connected if and only ifλ2(L(G)) > 0.

Therefore, all disconnected graphs on m vertices belong tothe following set:

G = G|λ2(L(G)) = 0

Let F (λ,G) = det(L(G)−λIn) where, In denotes the identitymatrix of dimension n× n.

Theorem 3:

G = G | Fλ(λ,G)|λ=0 = 0

where Fλ denotes derivative with respect to λ.Proof: ⇒ The smallest eigenvalue of the Laplacian of

any graph is zero, i.e., λ1(L(G)) = 0 [40]. Let G be adisconnected graph. From Lemma 1, we can conclude thatλ2(L(G)) = 0. Therefore, λ = 0 is a repeated root of theequation F (λ,G) = 0⇒ Fλ(λ,G) |λ=0= 0.⇐ F (0, G) = 0 since λ = 0 is an eigenvalue of the

Laplacian matrix. In addition, Fλ(λ,G) |λ=0= 0. Therefore,λ = 0 is a multiple root of F (λ,G) with algebraic multiplicitygreater than 1. Hence λ2(L(G)) = 0⇒ G is disconnected.

In [32], we presented the following expression for F (λ,G)in terms of the minors of L(G).

Fλ(λ,G) |λ=0 = −n∑i=1

Mii

,where Mii is the minor of corresponding to the diagonalelement in the ith row. Substituting the above relationship inTheorem 3 leads to the following equation for the terminalmanifold in terms of the variables aijni,j=1:

n∑i=1

Mii = 0 (11)

Note that the variables aijni,j=1 can be related to the stateof the agents and the jammers. Therefore, the above equationcan be expressed in terms of the state variables.

In the next section, we present different scenarios involvingvariations in the number of players, as well as their dynamics.

VI. RESULTS

In [33], we investigate a scenario in which an aerial jammerintrudes the communication channel in a multiple UAV forma-tion. The UAVs are assumed to be flying at a constant altitudeand their kinematics are modeled as a unicycle. The analysisof the previous sections can be applied to this problem for thespecial case of n = 2 and m = 1. Figure 1 summarizes theentire control algorithm. The controller of each agent takesas input the state variables and runs the RPE to computethe control. This control is then fed into the plant of therespective UAV. The plant updates the state variables basedon the kinematic equations governing the UAV. Finally thesensors feed back the state variables into the controllers. Inthis case the sensors measure the position and the orientationof each UAV.

JAMMER

UAV

PLANT

PLANT

σ2* −J x +J y )xyφ j

jj j

j

y11

x φ1

x1= −1+ y +cos σ

2φ

1 1

y1

= − x +sinσ2 1

φ1

φ1= − +σ

2σ

1

1σ*

σ1* = sign (

φJ

j)

Jx1

Jy1

J = − J J = − J ,x1

σ2 y

1y1

σ2 x

1

Jφ1

= sign− (Jx1y1

−Jy1

−x1

Jφ1−J

=

x jy

jφ

j

x j = −1+ y +cos σ2 j

φ j

yj =− x +sinσ

2 j φ j

φ j = − +σ2

σ j

σ j*

σ j =−sign (Jφ1)

JJx j y j

J = − J , J = − J y jσ

2 x jσ

2 yj

xj

Jφ =

**

*

* *

* *

*

*

* *

* *

sin − cos

Jxjsin − cosφ Jyj

φ jj

Jx1

Jy1

φ φ1 1

σ2*

1

CONTROLLER

CONTROLLER

σ2*

σ j*

σ1*

SENSOR

SENSOR

Figure 1. The Control Loop for the System

Figure 2 shows the trajectories of the players along withtheir optimal controls for a specific terminal condition. Theposition of the players corresponding to the termination sit-uation is shown by a small circle in the plots showing the

1989

trajectories of the players. The trajectories of the players areshown for a small time interval just before termination. Fromthe expression of the optimal controls in [33], we can infer thatthe controls of the players are bang-bang. This is also verifiedfrom the simulation results. From the nature of the controlsand kinematics of the system, we can infer that the optimalpaths comprise arcs of circles and straight line trajectories asmotion primitives. Arcs of circles are generated when the UAVkeeps its angular velocity saturated at one extremum for a non-zero interval of time. Straight line segments are obtained dueto rapid switching between the extremum value of the controls(chattering). An instance of such a behavior is exhibited byUAV2 in Figure 2.

−5 0 5 10 15−5

0

5

10

15Trajectory

x(a)

y

0 5 10−2

−1

0

1

2

σ1

t(sec)(b)

0 5 10−2

−1

0

1

2

σj

t(sec)(c)

0 5 10−2

−1

0

1

2

σ2

t(sec)(d)

Figure 2. Figure shows the trjectories of the players in a small time intervaljust before termination. The value of η = 1. The player in red is the jammer.The players in green and blue are UAV1 and UAV2, respectively. Figure (b)shows the control of the UAV1. Figure (c) shows the control of the jammer.Figure (c) shows the control of the UAV2.

In [32], we analyze the previous problem for general mand n. Figure 3 shows a simulation of the trajectories of threeUAVs and one jammer from a terminal state. The figure onthe left shows the trajectories of the UAVs. The jammer tracesthe path shown on the extreme right. The figure on the rightshows the connectivity of the UAVs. The network of UAVs isinitally connected. At termination, the jammers disconnect thenetwork by isolating one of the UAVs.

Figure 4 shows a simulation of the trajectories of four UAVsand two jammers from a terminal state. The figure on the leftshows the trajectories of the UAVs. The two UAVs on theextreme right represent the jammers. The figure on the rightshows the connectivity of the UAVs. The network of UAVs isinitally connected. At termination, the jammers disconnect thenetwork into two disjoint components.

In [35], we extend our previous work to address the jam-ming problem in a mobile network containing heterogeneousvehicles. In combat scenarios, teams of vehicles are deployedhaving different communication and motion constraints. Sincewe are interested in real scenarios, we choose the vehiclesas well as the jammer to resemble the dynamics of terres-trial or aerial vehicles. We use the motion models of UAVs

−200

−150

−100

−50

0

50

−100−50

050

100

−500

50

Trajectories of the UAVs

−250 −200 −150 −100 −50 0 50

−100

−50

0

50

100

150

Initial and Final Connectivity among the UAVs

Figure 3. Trajectories of the UAVs and Jammer

−250

−200

−150

−100

−50

0

50

−100

−50

0

50

100

150

Trajectories of the Players

−200

−100

0

−100

−50

0

50

100

150

−200

20

Initial and Final Connectivity among the UAVs

Figure 4. Trajectories of the UAVs and Jammer

(Unmanned Air Vehicles) and AGVs (Autonomous GroundVehicles) to model the dynamics of the players. By neglectingthe detailed description of the real system that might renderthe complete solution to be numerical in nature, the dynamicalmodels are simplified to a level so as to capture the essentialkinematic constraints of the system.

We assume the following motion models for the vehicles:1) UAV: We use the five state model [45] for the UAV

that takes into account the course angle, the flight pathangle and the height of the UAV from the groundduring its flight. The differential equations governing thekinematics of the system are given below:

x = W cosψ cos θ, y = W sinψ cos θ

z = W sin θ, ψ =g

Wη tanφ, θ =

g cos θW

(η − 1)

where W represents velocity, ψ the heading angle, θ thepitch angle, g the gravitational acceleration, φ the rollangle and η the load factor of the UAV. The values ofW , φ and η, satisfying the constraints 0 < Wmin ≤W ≤ Wmax, |φ| ≤ φmax and 0 ≤ η ≤ ηmax, are thecontrols of the UAV. The configuration space of the UAVis X ' R3 × S2.

2) AGV: From [46], we model the AGV as a car-likerobot with 5-dimensional configuration space using thefollowing differential equations:

x = v cos θ cos ζ, y = v sin θ cos ζθ = v sin ζ, v = u1, ζ = u2

1990

Figure 5. UAV jamming a team of AGV’s. Figure (a) shows the snapshots forthe vehicles along their optimal trajectory. Figures (b) shows the perceptionrange for the vehicles and the intra-vehicular distance as a function time.γ = 0.4

Figure 6. UAV jamming a team of AGV’s. Figures on the left show theoptimal controls of the vehicles. Figures on the right show the variation ofthe components of ∇J associated with the vehicles that are used to computethe optimal controls.u1max = 0.1, u2max = 0.015,Wmin = 2,Wmax =11, ηmax = 1.008, φmax = 0.1

where, u1 and u2, satisfying | u1 |≤ u1 max and| u2 |≤ u2 max, denote, respectively the linear andangular accelerations of the vehicles. We also considerthe fact that the car has a bound on the steering angle,i.e. | ζ |≤ ζmax. As in [46], the reference point (x, y) ischosen as the mid-point of the rear wheels. For the sakeof simplicity, in this work we assume that the distancebetween both rear and front axles is unity. The statespace of the system is X ⊂ R3 × S2.

Figure 5 and 6 shows the simulation results for the casewhen a UAV Jamming a team of AGVs.

VII. CONCLUSION AND FUTURE WORK

In this work, we presented a mobility based perspectiveof jamming among autonomous vehicles. The framework ofdifferential game theory was extended from the classical two-

player zero sum pursuit-evasion game to the multi-player teamgame scenario involving two teams. Necessary conditions sat-isfied by the optimal trajectories were derived under assump-tions of uniqueness and existence of the optimal trajectories.Finally, results were presented for several simulation scenariosinvolving different numbers of vehicles and their dynamics.

Our current focus has been to extend the analysis for thecase when two teams jam each others communication channel.[47], [48] analyze the aforementioned scenarios. We haveformulated the jamming problem as a resource allocationproblem by incorporating limitations in power and energyavailable to each agent, which is a major concern in realsituations. Some of our future work are as follows:

1) Computation of Singular Surfaces: In this work, we havecomputed the trajectories based on the necessary condi-tions of optimality imposed by the Isaacs’ conditions.In order to complete the construction of the optimaltrajectories of the agents, we have to identify the singularsurfaces in the state space [1].

2) Decentralized Strategies: Due to the high computa-tional complexity of evaluating the optimal strategies,one might resort to sub-optimal solutions that can becomputed efficiently. In [51] [50], the authors considera decentralized approach to a team jamming problem.A multi-pursuer multi-evader game is decomposed intoseveral one-pursuer-one-evader and multi-pursuer one-evader games. Each team is corrupts the measurementsof the opposite team. The cost function is a penaltythat measures the energy cost or computation cost. Theauthors propose an algorithm that leads to a compu-tationally efficient sub-optimal strategy for each team.An intersting future research direction is to extendthe aforementioned techniques to the jamming problemconsidered in this paper.

3) Noisy models: In real systems, noise corrupts the actu-ation signal and the sensor measurements. The problemof noise regulation has been successfully addressed fora special class of systems by treating the noise as anadversary [49]. Unfortunately, there has been hardly anyprogress regarding pursuit-evasion games with playershaving noisy observation and actuation models. The non-linearities present in the dynamic model of the playersrender the problems inutile from the perspective ofmathematical analysis. One of our future efforts is toexplore this direction using numerical techniques. Analternate approach to this problem is to use iterativestrategies as suggested in [52].

REFERENCES

[1] T. Basar and G. J. Olsder. Dynamic Noncooperative Game Theory, 2ndEd. SIAM Series in Classics in Applied Mathematics, Philadelphia, 1999.

[2] T. Alpcan and T. Basar. Network Security: A Decision and GameTheoretic Approach Cambridge University Press, 2011

[3] S. Bhattacharya and T. Basar. Game-theoretic analysis of an aerialjamming attack on a UAV communication network. Technical report,Univeristy of Illinois at Urbana-Champaign, 2010.

1991

[4] J. V. Breakwell and P. Hagedorn. Further properties of non-zereo sumdifferential games. In Journal of Optimization Theory and Applications,volume 3, pages 207–219, 1969.

[5] J. V. Breakwell and P. Hagedorn. Point capture of two evaders in suc-cession. In Journal of Optimization Theory and Applications, volume 27,pages 89–97, January 1979.

[6] M. Cagalj, S. Capcun, and J. P. Hubaux. Wormhole-based anti-jammingtechniques in sensor networks. In IEEE transactions on Mobile Comput-ing, volume 6, pages 100–114, January 2007.

[7] L. Chen. On selfish and malicious behaviours in wireless networks - anon-cooperative game theoretic approach. PhD thesis, Ecole NationaleSuperieure des Telecommunications, Paris, October 2008.

[8] X. C. Ding, A. Rahmani, and M. Egerstedt. Optimal multi-uav convoyprotection. In Conference on Robot Communication and Configuration,volume 9, pages 1–6, April 2009.

[9] D.M.Stipanovic, S.Shankaran, and C. Tomlin. Strategies for agents inmulti-player pursuit-evasion games. In Eleventh International Symposiumon Dynamic Games and Applications, 2008.

[10] P. Hagedorn and J. V. Breakwell. A differential game of approach withtwo pursuers and one evader. In Journal of Optimization Theory andApplications, volume 18, pages 15–29, January 1976.

[11] R. Isaacs. Differential Games. Wiley, New York, 1965.[12] A. Y. Levchenkov and A. G. Pashkov. Differential game of optimal

approach of two inertial pursuers to a noninertial evader. In Journal ofOptimization Theory and Applications, volume 65, pages 501–518, June1990.

[13] J. Lewin. Differential Games: Theory and Methods for Solving GameProblems with Singular Surfaces. Springer-Verlag, London, 1994.

[14] A. A. Melikyan. Generalized Characteristics of First Order PDEs:Applications in Optimal Control and Differential Games. Applications ofMathematics, 2000.

[15] I. M. Mitchell and C. J. Tomlin. Overapproximating reachable setsby hamilton-jacobi projections. In Journal of Scientific Computing,volume 19, pages 323–346, 2003.

[16] G. Noubir and G. Lin. Low power denial of service attacks in datawireless lans and countermeasures. 2003.

[17] P. Papadimitratos and Z. J. Haas. Secure routing for mobile ad hocnetworks. January 2002.

[18] A. G. Pashkov and S. D. Terekhov. A differential game of approachwith two pursuers and one evader. In Journal of Optimization Theoryand Applications, volume 55, pages 303–311, July 1987.

[19] M. Pavone, K. Savla, and E. Frazzoli. Sharing the load: Mobile roboticnetworks in dynamic environments. IEEE Robotics and AutomationMagazine, 16:52–61, 2009.

[20] R. A. Poisel. Modern Communication Jamming Principles and Tech-niques. Artech, Massachussets, 2004.

[21] J. J. Proakis and M. Salehi. Digital Communications. McGraw-Hill,2007.

[22] T. Samad, J. S. Bay, and D. Godbole. Network-centric systems formilitary operations in urban terrain: The role of UAVs. Proceedings ofthe IEEE, 95(1):92–107, 2007.

[23] S. Shankaran, D. Stipanovic, and C. Tomlin. Collision avoidancestrategies for a three player game. In International Society of DynamicGames, Wroclaw, Poland, July 2008.

[24] D. M. Stipanovic, I. Hwang, and C. J. Tomlin. Computation of an over-approximation of the backward reachable set using subsystem level setfunctions. In Dynamics of Continuous, Discrete and Impulsive systems,Series A: Mathematical Analysis, volume 11, pages 399–411, 2004.

[25] D. M. Stipanovic, A. A. Melikyan, and N. V. Hovakimyan. Some suffi-cient conditions for multi-player pursuit evasion games with continuousand discrete observaitions. In Annals of the International Society ofDynamic Games, volume 10, pages 133–145, 2009.

[26] J. Tisdale, Z. Kim, and J. Hedrick. Autonomous UAV path planning andestimation. IEEE Robotics and Automation Magazine, 16:35–42, 2009.

[27] E. M. Vaisbord and V. I. Zhukovskiy. Introduction to multi-playerdifferential games and their applications. Gordon and Breach, New York,1988.

[28] A. D. Wood, J. A. Stankovic, and S. H. Son. Jam: a jammed-areamapping service for sensor networks. In Proceedings of 24th IEEE Real-Time Systems Symposium (RTSS 03), pages 286–297, December 2003.

[29] A. D. Wood, J. A. Stankovic, and G. Zhou. Deejam: Defeating energy-efficient jamming in IEEE 802.15.4 based wireless networks. In 4thAnnual IEEE Conference on Sensor, Mesh and Ad Hoc Communicationsand Networks (SECON 07), pages 60–69, 2007.

[30] X. Wu, T. Wood, W. Trappe, and Y. Zhang. Channel surfing and spatialretreats: defenses against wireless denial of service. In 3rd ACM workshopon Wireless security (WiSe 04), pages 80–89, 2004.

[31] V. I. Zhukovskiy and M. E. Salukvadze. The Vector Valued Maxmin,volume 193. Academic Press, San Diego, 1994.

[32] S. Bhattacharya and T. Basar. Differential game-theoretic approachfor spatial jamming attack in a UAV communication network. In 14thInternational Symposium on Dynamic Games and Applications, Banff,CA, June 2010.

[33] S. Bhattacharya and T. Basar. Game-theoretic analysis of an aerialjamming attack on a UAV communication network. In Proceedings ofthe American Control Conference, pages 818–823, Baltimore, MD, June2010.

[34] S. Bhattacharya and T. Basar. Graph-theoretic approach to connectivitymaintenance in mobile networks in the presence of a jammer. InProceedings of the IEEE Conference on Decision and Control, Atlanta,GA, Dec. 2010.

[35] S. Bhattacharya and T. Basar. Optimal strategies to evade jammingin heterogeneous mobile networks. In Proceedings of the Workshop onSearch and Pursuit-Evasion, Anchorage, AK, 2010.

[36] H. Choset, K. Lynch, S. Hutchinson, G. Kantor, W. Burgard, L. Kavraki,and S. Thrun. Principles of Robot Motion: Theory, Algorithms, andImplementations (Intelligent robotics and autonomous agents). The MITPress, Cambridge, MA, 2005.

[37] S. Gorman, Y. J. Dreazen, and A. Cole. In-surgents hack U.S. drones, December 2009.http://online.wsj.com/article/SB126102247889095011.html.

[38] D. McCullagh. Predator drones hacked inIraq operations, December 2009. Available athttp://news.cnet.com/301-1009_3-10417247-83.html.

[39] V. I. Arnold. Geometric Method in the Theory of Ordinary DifferentialEquations. Springer-Verlag, New York, 1983.

[40] N. Biggs. Algebraic Graph Theory. Cambridge University Press,Cambridge, U.K., 1993.

[41] M. Mesbahi. On state-dependent dynamic graphs and their controllabil-ity properties. In IEEE Transactions on Automatic Control, volume 50,pages 387–392, 2005.

[42] R. Olfati-Saber and R. M. Murray. Consensus problems in networks ofagents with switching topology and time delay. In IEEE Transactions onAutomatic Control, pages 1520–1533, 2004.

[43] J. A. Fax and R. M. Murray. Information flow and cooperative controlof vehicle formations. In IEEE Transactions on Automatic Control,volume 9, pages 1465–1474, 2004.

[44] A. J. H. Tanner and G. Pappas. Flocking in fixed and switching networks.In IEEE Transactions on Automatic Control, volume 5, pages 863–868,May 2007.

[45] P. B. Sujit and R. Beard. Multiple UAV path planning using anytimealgorithms. In Proceedings of American Control Conference, pages 2978–2983, St. Louis, Missouri, June 2009.

[46] J. P. Laumond, S. Sekhavat, and F. Lamiraux. Guidelines in Nonholo-nomic motion planning for Mobile Robots. Springer, 1998.

[47] S. Bhattacharya, A. Khanafer, and T. Basar. Power allocation in teamjamming games in wireless ad hoc networks. In Proceedings of the 4th In-ternational ICST Workshop on Game Theory in Communication Networks(Gamecomm), 2011. http://arxiv4.library.cornell.edu/abs/1101.6030v1.

[48] A. Khanafer, S. Bhattacharya, and T. Basar. Adaptive re-source allocation in jamming teams using game theory*. InProceedings of the 7th International Workshop on Resource Al-location and Cooperation in Wireless Networks(RAWNET), 2011.http://arxiv4.library.cornell.edu/abs/1101.6030v1.

[49] T. Basar and P. Bernhard, H-infinity Optimal Control and RelatedMinimax Design Problems: A Dynamic Game Approach. Boston, MA:Birkhauser, August 1995.

[50] M. Wei, G. Chen, J. B. Cruz, L. Hyanes, K. Pham and E. Blasch. Multi-pursuer multi-evader pursuit-evasion games with jamming confrontationIn Aerospace Computing, Information, and Communication, volume 4,pages 693–706, March 2007.

[51] D. Li, J. B. Cruz, G. Chen, C. Kwan and M-H. Chang. A heirarchicalapproach to multi-player pursuit-evasion differential games In Proceed-ings of the IEEE CDC-ECC, pages 5674 - 5679, Seville, Spain, Dec.2005.

[52] D. Li, J. B. Cruz and C. Schumacher. Stochastic multi-player pursuit-evasion differential games International Journal of Robust and NonlinearControl, volume 18, pages 218-247, May 2007.

1992