Collaborative Mission Planning for UAV Cluster toOptimize Relay Distance

Cagatay TanılRoketsan Missile Industries

Ankara, 06780 Turkeycagataytanil@gmail.com

Chirag WartyRF and Wireless Eng.

IEEE MemberLos Angeles, USA

chiragwarty@ieee.org

Esam ObiedatRF Systems Engineer

CommScope Inc+1 214 440 8176

Obiedat@gmail.com

Abstract—Unmanned Aerial Vehicles (UAVs) coordinated pathplanning and intercommunication for visual exploration of ageographical region has recently become crucial. MultipleUAVs cover larger area than a single UAV and eliminate blindspots. To improve the surveillance, survivability and qualityof the communication, we propose two algorithms for the routeplanning of UAV cluster operated in obstacle rich environment:(i) Multiple Population Genetic Algorithm (MPGA) (ii) RelaySelection Criteria (RSC). The main objective of MPGA is tominimize the total mission time while maintaining an optimaldistance for communication between the neighboring nodes.MPGA utilizes evolutionary speciation techniques with a novelFeasible Population Creation Method (FPCM) and enhancedInter-species Crossover Mechanism (ISCM) to obtain diversi-fied routes in remarkably short time. In obtaining collision-free optimum paths, UAVs are subjected to constraints such aslimited communication range, maximum maneuverability andfuel capacity. In addition to the path planning, RSC is developedfor selection of UAVs relay nodes that is based on the location ofthe relay relative to source and destination. It is crucial sincethe Bit Error Rate (BER) performance of the link significantlydepends on the location of the selected relay. In this paper, pathplanning and relay allocation algorithms are combined to havea seamless high quality monitoring of the region and to providesuperior Quality of Service (QoS) for audio-video applications.Also, simulations in different operation zones with a cluster ofup to six UAVs are performed to verify the feasibility of theproposed algorithms both in optimality and computation time.

TABLE OF CONTENTS

1 INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 PROBLEM STATEMENT . . . . . . . . . . . . . . . . . . . . . . . . . . 23 OPTIMUM RELAY DISTANCE . . . . . . . . . . . . . . . . . . . . 34 MULTIPLE POPULATION PATH PLANNING . . . . . . 55 SIMULATION RESULTS FOR UAV CLUSTER . . . . 76 CONCLUSION AND FUTURE WORK . . . . . . . . . . . . . 10

REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10BIOGRAPHY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

1. INTRODUCTIONThe cooperative routing of UAV cluster is a crucial problemin obstacle-rich operation zones. The task can be com-pleted more efficiently if a single UAV is replaced with agroup of UAVs. An effective routing not only improves thetarget acquisition but also prevents UAVs both from beingdetected by hostiles and violating the obstacle zones. Apath is simply composed of a set of waypoints that the UAVhas to pass sequentially. In the literature, there are somesolution methods for this problem which can be classified

978-1-4577-0557-1/12/$26.00 c©2012 IEEE.1 IEEEAC Paper #2059, Version 1, Updated 01/08/2012.

into two kinds: (i) traditional classic algorithms (ii) modernintelligent algorithms. Traditional methods include dynamicprogramming and optimal control theory, cell decomposition,road map and potential fields etc. All of these classical meth-ods intend to find a unique optimum solution if one exists.The main drawback of these methods is their spending toomuch time for large-scale iterations and high risk in trappinginto local minima (premature convergence) since they donot utilize heuristics. In order to improve the efficiency ofconventional methods, probabilistic algorithms have beendeveloped. They use probabilistic search methods to obtain anear-global optimum solution. Although heuristic algorithmsdo not guarantee to find a solution, if they do, are likely to domuch faster than classic methods [1].

The most explicit objectives in path planning are that finalpaths must be safe, flyable, and convenient for the in-flightcommunication within UAVs [2]. While the safety of UAVsis achieved by obstacle avoidance, flyable paths are generatedby considering kinematic constraints and maneuverability ofthe UAVs. Total flight time is also critical in some taskssince the uncertainty of the operation zone increases whenthe mission time increases. In addition to single path planningconstraints, there are additional requirements that come withthe multiple path planning (MPP). Generating non-conflictingmultiple paths for several UAVs that are flying in the sameenvironment is a challenging task [3]. The UAVs are requiredto be launched by same platform at different time, fly throughseparate paths while maintaining the communication range;and finally arrive on target at a simultaneous time. Thus, theremust be time synchronization and kinematic coordinationbetween the trajectories of UAVs.

In addition to the kinematic restrictions, there are also someconstraint arisen from line-of-sight communication withinUAV cluster. UAV operations have evolved to be a significantpart of the next generation surveillance and reconnaissanceeffort. This has led to deployment of UAV clusters beingdeployed to have an umbrella coverage. Communicatingwith each other using cooperative communication, the clusternodes links. These links experience environmental attenua-tion and interference. To mitigate the fading and multipathpropagation effects, time, frequency, and spatial adversitytechniques or a hybrid of them can be utilized. Among whichthe spatial diversity has been studied extensively in contextof point to multipoint communication by using intermediaterelays, increasing the system throughput and reliability [4].In a cooperative strategy when a node has information totransmit, it cooperates with other nodes in the vicinity totransmit its information to the destination thus forming avirtual antenna array [4].

The location optimization algorithm studies the location de-pendency of the relay in the distributed coding system thatcan be simply adjusted to yield level of performance required

1

at the destination. In this paper, we apply the optimum lo-cation distance that a UAV can occupy to obtain the requiredBER. The locations of the relay with respect to the source anddestination are used to find the optimum distance of the relaythat might result in the desired performance and link qualityrequirement at the destination. Also, the concept of dis-tributed encoding is applied for source transmitted messagesover multiple relay nodes. It uses a modified iterative Turboproduct decoding at the destination to decode the receiveddistributed TPC over multiple channels.

The next step is to develop an efficient routing algorithmwhich should also take the constraints from communicationinto account. Recently, as well as swarm intelligence algo-rithms, Genetic Algorithm (GA) has become very popular forsolving complex path planning problems effectively in intelli-gent transportation systems, aerospace, robotics, and militaryguidance and navigation systems. Based on the principle ofsurvival of the fittest, GA reaches an optimal or suboptimalsolution by performing inheritance, selection, crossover andmutation operators to improve the next generations; andconverges in an optimal solution in the end. The automaticgeneration of optimal alternative paths is indispensible forsurveillance by UAV clusters in dynamic environments. Wepropose an autonomous MPP algorithm which aims to obtainsuboptimal paths simultaneously by using multiple popu-lation genetic algorithm (MPGA) based on speciation andevolution of multiple populations collaterally. The diversityin the initial population is a key factor that determines theoverall performance of the algorithm. In order to maintainthe diversity and feasibility in the initial population, a novelfeasible population creation method (FPCM) - which is alsomentioned in the previous work [5-6] - is developed. FPCMis basically an intelligent random tree search method, whichgenerates initial population composed of feasible trajectoriesonly. It creates substantially diversified individuals in a veryshort time by its flexible mechanism. Starting with a feasiblepopulation implies that the GA does not need to spend timeto obtain feasible paths in the following generations and thetotal convergence time is consequently influenced positively.As a clustering method, a simple speciation technique basedon the similarity of individual paths is utilized. The simi-larity function checks whether there is obstacle between twopaths. Furthermore, the MPGA is strengthened by addingimproved inter-species crossover method (ISCM) that helpsdiscovering new species in the next generations in case theinitial population is not diverse enough. The followingsections describe the proposed MPP algorithm in detail. Inthe simulation section, the algorithm is tested in an obstacle-rich environment and has been shown to produce efficienttrajectories in a reasonable time.

2. PROBLEM STATEMENTThe route optimization problems for cooperative tasks areusually solved considering a set of optimal planning criteriaand constraints including the limitations imposed by theUAVs flight dynamics as well as minimization of detectionand maximization of the operation zone coverage. This sec-tion explains the constraints on the problem and the routinesin determining search space of the problem. To simulate theaviating environment of the UAVs, the components of thescenario as well as physical limitations of the UAV shouldbe investigated first. Assuming that the UAVs fly at a lowaltitude in surveillance region, the trajectories can be settledby using waypoints such that the UAVs can make deceptivemaneuvers and bypass the enemy threat, terrain obstacles,

FLYABLE

ZONE

Takeoff

Base

Landing

Base

Area of Interest

for Surveillance

Terrain Obstacle

Buffer Zone

Radar

Threat Zone

de

dt

SAM

UAV-1

2

34

Figure 1. Digital Avoidance Zone Map

and third parties considering the performance of each UAV.Furthermore, there is a cooperation constraint that checks thefeasibility of paths when all the UAVs are flying simultane-ously.

In Figure 1, the blue area represents the safe flyable regionwhere waypoints can be assigned. In other words, any of thepath segments created by linking the successive waypointsmust not cross the avoidance zone. The avoidance zones aredigital map composed of grids, the resolution of which shouldbe determined according to the maximum range of the UAVor to the required accuracy of the search algorithm [6]. Inthe operation zone, the main factors that shape the waypointconstraint envelope and other constraints are listed below:

Radar Threat

Suppose the distribution of enemy threats, including surface-to-air missile (SAM) sites and their radars, are known byus. In order to prevent detection by enemy radars, the flightpath of the UAV should be kept at a minimum distancefrom enemies. This minimum distance de is a function ofmaximum radar detection range, maneuverability of the radarplatform and the data age of the position information.

: Current node (waypoint)

ni

ni+2

ni+1

ni-1

ni

ls

θi ≤ θmax li ≤ lmin

rmin

rmin

Oi+1

Oi

Figure 2. Geometric Constraints on Waypoints

2

Terrain Threat

In order to prevent the risk of blind impact when flying atlow altitudes, the UAV must fly around the terrain obstacles.For this purpose, the boundaries of the obstacles are enlargedtoward to flyable zone to create a buffer zone. The amountof standoff avoidance from dt is based on the UAVs positionerrors due to navigation bias and weather (e.g. wind) com-pensation factor . Note that the position uncertainty increasesas the range from launch platform decreases, therefore, thestandoff limits must be variable in the operation zone.

Total Fuel

The UAVs carry a limited amount of fuel and they have toreach their destination before consuming all of it. It puts alimit on the total range and the number of waypoints whichcan be produced by auto router. Also, it determines theboundary of area of interest (AOI) in the operation zone. AOIis defined as the area in the map where the MPGA algorithmsearches the optimal paths.

Constraints on each waypoint

Considering the UAV performance, each waypoint calculatedby the path planner must satisfy two operation conditionswhich are minimum length of the route legs lmin and max-imum turn angle θmax which are illustrated in Figure 2.The distance between two successive waypoints (ni, ni+1)is called leg length li and it should be less than a thresholdvalue lmin. This value must be large enough to ensure theUAV can successfully complete the consecutive turns. Thisavoids the second turning is to begin when the first one has notcompleted. The maneuverability of the UAV is constrainedby its minimum turning radius rmin which depends on themaximum load factor. In long distance flights, it is notgenerally desirable that the UAV weaves and turns constantly.Therefore, the waypoint turn angle should be less than aspecified maximum value.

Avoiding UAVs collision

When generating multiple paths in the same task, it is impor-tant to check whether any of two UAVs are flying too closewhile pursuing their paths. Therefore, the planner has to testif two paths coincide both in space and time. Since checkingUAVs collision for all multiple path combinations duringalgorithm brings high computation cost, in this study, it ischecked only after MPGA obtains the suboptimal paths andstops. In case of constraint violation, the paths not satisfyingthe constraint are repaired by small modifications.

Optimum Communication Distance

During following their trajectories, UAVs should maintain thecommunication in between them. It would be impracticalto expect all the UAVs are continuously in contact duringmission since they may have to turn around obstacles whichcauses cut off in communication. In this study, the UAVsare considered to maintain communication-in-burst; that is,they transfer the data only in predetermined communicationpoints. The main aim here is to maximize the number ofpossible communication nodes in order that the UAVs areaware of each other and passing onboard data as much aspossible [7-12]. The details of communication constraint areexplained in the next section.

Figure 3. Single Relay Model

Figure 4. The simplified three terminals line networktopology

3. OPTIMUM RELAY DISTANCEThe cooperative technique can improve the overall systemcapacity by adjusting relay positions in the network comparedto original non-cooperative system. The cooperative schemeissued in which the relay forwards an incremental redundancyto the destination about the recovered message from thetransmission source. The destination uses the two parts ofthe code received via the direct path and the relay channelto conduct message decoding. This paper considers a singlerelay model, consisting of source s, relay r and destination das depicted in the Figure 3. All three terminals are operatingin a half duplex mode and any transmissions from sourceto destination require tools time slots. The relay decodesthe received message and encodes it before sending it to thedestination.

This model is more practical in real systems, when the relay isusually located between the two terminals and the separationdistances are relatively large. A model which returns the sim-ulation problem from three-dimensional to two-dimensionalproblem has been used in many other works, e.g. [13-15]. The received signals at the relay and the destinationduring the two time slots for the line model can be generallyexpressed as follows:

yd[2k− 1] =√Esαsd[2k− 1]xs[2k− 1] +nsd[2k− 1] (1)

yr[2k − 1] =

√Es

(1− λ)2αsr[2k − 1]xs[2k − 1] + nsr[2k − 1]

(2)

yd[2k] =

√Er

(λ)2αrd[2k]xs[2k] + nrd[2k] (3)

where yj denotes the received signal at node j while xi isthe transmitted signal from node i and k is the time slot.The channel between the two nodes i and j has AWGNnoise nij , and channel attenuation αij .Es and Er are thetransmit energy/bit for the source and relay, respectively.Using free space propagation on the line model and assumingfixed transmission energy per bit, the SNR values for thethree channels, i.e. γsd, γsr and γrd for the direct, inter-userand relay channel respectively, are related by the followingexpressions [16]:

3

Figure 5. Normalized energy/bit of the source and the relayusing the proposed power allocation

γsr =γsd

(1− λ)2(4)

γrd =γsdλ2

(5)

where 0≤λ≤1 indicates the position of the relay with respectto the destination when the distance between the source andthe destination is normalized to 1, with λ = 0 when the relayis located at the destination. Figure 5 displays how the valuesof SNR at the destination and the relay change when the relayis moved across the source-destination line for a fixed γsd.

Location Optimization Algorithm

A. Optimization Algorithm Requirements— Here we applyoptimization rules on the power optimized Distributed TurboProduct Code (DTPC) cooperative system using the DF andSDF relaying protocols presented in [17] to search for themaximum attainable performance using the line experimentalsetup. Our main target in this paper is to find the optimal relaylocation which minimizes the final BER at the destinationnode. The performance of the cooperative systems, canbe improved by relaying with optimal power allocations asfound on [4] for the DTPC system. Therefore, we assumethat a maximal overall transmit power from the source andthe relay is fixed and is equal to the same power required totransmit the complete TPC codeword from the source to thedestination in the non-cooperative scenario. Then, the overalltotal transmitting power is to be optimally shared between thesource and the relay, so that the power is efficiently utilizedto gain the maximum performance possible for the DTPC.Basically, we optimize the power allocation by minimizingbit-error probability at the destination. Optimal power allo-cation will not only give better performance but also savesenergy for the relay node which in many situations will bebattery operated which makes the power a scarce resource,not like the source or the destination, a typical examplewireless sensor networks.

Since the main target for power optimization is to reducethe probability of error at the destination, the target functionfor our optimization problem is therefore the BER after thedecoder. However, there is no exact expression availableto model the probability of error after the decoder, but oneway to characterize this unknown function is by the empiricalfunction given by:

BER = f(α, P ) (6)

where the parameter is used to represent the location of

Figure 6. The principle of sliding ball used in designing theoptimization algorithm

the relay that we want to optimize, P is the total transmitpower. This relation is monotonically decreasing functionwith respect to the power P, so to find the optimal locationwhere the relay will help maximizing the BER performancewe have to find the values of that result in the minimumBER. Thus, the optimization problem is reduced to a onedimensional problem with only one variable parameter α.

B. Optimization Algorithm—Simulation errors could lead theoptimization algorithm to a wrong solution. This limitation issolved using the sliding ball principle on a slope as in Figure6. If a ball is dropped from any peak of the slope it willslid and will exceed the lowest point on the curve and thentraverses more distance upward beyond the solution until itstops and reverses its direction of movement. This continuesuntil the ball reaches steady state at the lowest point on theslope. The numbers on the balls in Figure 6 indicates thepositions of the ball when it reverses the sliding direction,where the number 1 indicates the starting point. Note that if asliding ball runs over a small bulge, it will pass this bulge andwill continue sliding until it reaches anuphill. The requiredoptimization algorithm should be designed to minimize therun time and the complexity of the algorithm. We set theoptimization algorithm to work on bit error rate level closeto 103 bit errors/frame to have accurate results with lowernumber of repetitions (the number of transmitted and receivedframes for a single SNR and pair).

The algorithm calculates the step size based on the lengthsearch segment and the number of steps. For each step, thealgorithm compares the current bit error rate of the decodingresult with the previous step. If the BER at current step issmaller than it at the previous step, the algorithm continues tothe next simulation step. If the current decoding error rate islarger than rate at the last step, then the algorithm comparesthe previous step result with the result two steps back: if BERone step back is also larger than the BER at the two stepsback, then it sets new boundaries (search segment) and stepsize. Otherwise, if the result one step back is smaller than theresult two steps back, then it continues to the next simulationstep.

The optimization algorithm continues on steps until it reachestwo consecutive points on the upward direction of the curve(i.e. last two results of BER are larger than previous step) oruntil it reaches the boundary of the curve segment. In the twocases, a new search segment is determined from the lengthcovered by the two steps before the current step. The step sizeis calculated from the length of the search segment and the

4

required number of steps. The optimization algorithm used tofind the value for each signal to noise ratio value γsd acrossthe line model is shown in algorithm 1.

Algorithm 1 Relay Distance Optimization1: Input : DTPC simulator with inputs a and SNR and

output BER.2: for SNR = 0→ 2 do3: set search segment boundaries: end and start4: set accuracy threshold Th5: set number of steps to N6: calculate step size, Step = (αstart − αend) /N7: while (αstart − αend) > Th do8: set (αstart − αend)9: for j = 1→ N do10: repeat11: simulation inputs α and SNR12: until maximum number of frames13: the output is BER [j]14: if j > 2 then15: if BER [j] > BER [j − 1] − BER [j − 2]

then16: /*going uphill*/17: αstart = α+ Step18: αend = α+ Step19: Step = (αstart − αend) /N20: break21: end if22: end if23: if j < N then24: /*going downhill*/25: α = α− Step26: end if27: if j = N then28: /*reached the end*/29: αstart = α30: αend = α+ 2× Step31: Step = (αstart − αend) /N32: end if33: end for34: end while35: end for36: Output : Location where minimum BER is obtained.

We used the accuracy threshold Th as stopping criterion todetermine when the algorithm has approached to the solutionwith a predetermined accuracy level. The number of stepsNSteps is the number of sections that the search segmentis divided to. As noted from the algorithm, the step sizereduces every time when a new search segment is found. Thenew search segment is determined to be the last two sectionscoming before the current segment at which the condition tofind new search segment is satisfied.

4. MULTIPLE POPULATION PATH PLANNINGThe main goal of the MPP problem studied here is to findenergy-optimal paths for a cluster of autonomous UAVs ona two-dimensional horizontal plane. Each UAV must travelfrom an initial state to a final state without colliding with theobstacles while obeying the flight dynamics constraints. Inorder to solve this highly non-linear optimization problem,a multi-population genetic algorithm (MPGA) is introduced.Due to the demand in generation of more than one trajectory,MPGA is performed to discover sub-optimal paths that canbe cooperated for reconnaissance of a region. Different from

traditional GAs, MPGA utilizes parallel populations calledsubpopulations or species. Each subpopulation evolves fora few generations isolated before one or more individualsare exchanged between the subpopulations. Since MPGAsimulates the evolution of a species in a way more similarto nature than the single population GA does, it is morepowerful and effective even for single path planning problems[18].

Unlike the simple GAs, MPGA starts optimization with afeasible initial population rather than randomly generatedindividuals. FPCM is responsible to create initial populationincluding feasible individuals only. Once the initial pop-ulation is created, Evolutionary Speciation Method (ESM)clusters the initial population into different species which iscalled subpopulation. The subpopulation represents a groupof similar paths and each subpopulation evolves parallel usingoperators of simple genetic algorithm. After few generations,ISCM operator starts choosing individuals from different sub-populations and recombines them by performing crossover.The offspring generated by cross-species recombination areallocated to either an existing species or new species. MPGAcontinues until termination conditions are simultaneously sat-isfied for each subpopulation. Let the termination conditionof kth subpopulation at pth generation be denoted by τPk .The termination condition of the algorithm at pth generationτp is defined by

τp =

N∏k−1

τpk where N: total number of species (7)

Encoding Method

The encoding scheme determines the representation of thechromosome (path). For this problem, the chromosome isdefined as a sequence of waypoints called nodes (genes).Each node contains the information of x and y coordinatesin the horizontal plane. The first and the last nodes representtakeoff and land platform respectively which are constantgenes in every individual. For maintaining flexibility anddiversity in the search space, the number of intermediatenodes in the path is kept variable in the proposed method.Let the jth gene of the ith individual (chromosome) in apopulation p is defined by

pij =[xij yij

]such that pi1 = [xs ys] and pin = [xt yt]

(8)

where s denotes the starting point, t is the target point and nrepresents the index of last gene of the individual pi [5].

Evaluation Function

Fitness function is composed of two terms represented bytotal range and maneuver. Total range implies the flight timewhich should be minimized whereas total maneuver repre-sents the fuel consumption that must be minimized duringflight. These two terms are normalized by normalizationfactors of range ηr and maneuver ηm respectively as well asmultiplied by weightings µr and µm. The cost function f forthe ith path pi in the population is defined by

5

f(Pi) = µrηr

n∑j=2

[(xij − xij−1

)2+(yij − yij−1

)2]1/2+ µmηm

n∑j=2

∣∣atan2(yij − yij−1, x

ij − xij−1

)∣∣ (9)

where ηr = r−1max, ηm = p−1 , µr + µm = 1 and rmax is

the maximum range of the UAV. Weightings factors µr andµm are positive values which imply the importance amongthe two criteria: (i) total travel distance (ii) total maneuver[5-6]. Note that there is no additional penalty term in thecost function since the entire path candidates are generated asfeasible in the proposed algorithm.

Feasible Population Creation Method (FPCM)

In conventional GAs, initial population is generated ran-domly; however, there are some disadvantages to it. In ob-stacle rich operation zones, the randomly chosen paths mightintersect with the no-fly zones which means that they areinfeasible. Also, the resulting paths might violate the physicalconstraints such as maneuverability, maximum range. Start-ing GA with an infeasible population means the algorithmhas to spend much effort and time to obtain first feasiblecandidates. On the other hand, this paper proposes a novelmethod for feasible path generation within a reasonable shorttime instead of spending too much time to obtain feasiblepaths by mutation and creating operators randomly [5-6]. Thepseudo code of the main steps of this method is given:

Algorithm 2 Feasible Population Creation1: current node = starting point (xs, ys)2: for counter = 1→ population size do3: calculate first waypoint (x1, y1) as

x1 = xs + l1 cosϕ, y1 = ys + l1 sinϕwhere l1 = lmin, ϕ ∼ random [0, 2π]

4: generate a random number{r ∈ Z | 1 <= r <= max. allowable waypoints}

5: repeat6: create next possible set of nodes n as

ni ∈ n | θi = θmax − (i− 1)β, li = lminβ = βres, i = 1, 2, . . . , 2θmax/β + 1

7: check obstacle avoidance for each node ni8: eliminate nodes that intersect obstacles or

travelled distance + distance to target ≥ rmax9: if waypoint number <= r then10: select next waypoint {ni ∈ n | i = random}11: else12: calculate distance to target di for nodes and

select next waypoint as {ni ∈ n | di = min (d)}13: end if14: until until target is in line of sight15: end for

By starting with a feasible population already, the conver-gence of the GA is enhanced. However, the risk of startingwith a feasible population is the possibility of losing thediversity in the population. To achieve this problem, theproposed technique uses random factors (r, ϕ, i) in generat-ing feasible paths to keep the search space varied and large.Among them, the most vital parameter determines when therandom node assignment is stopped and deliberative selectionis started. Random selection helps to increase diversity inpaths whereas deliberative selection guides the nodes toward

the target. For instance, the larger the parameter , the longerand more tortuous (zigzag) paths we obtain which is requiredfor maintaining the diversity in the initial population. On theother hand, once shrinking the value , the shorter and thesmoother paths are obtained. It is useful for improving theglobal convergence of GA.

Evolutionary Speciation Method (ESM)

The next operation following the construction of initial pop-ulation is the evolutionary speciation which is basically aclustering analysis that forms species. Clustering is a crucialclassification technique in the proposed MPGA where a setof paths, composed of path segments in a two-dimensionalhorizontal plane, are grouped into clusters in such a way thatpaths in the same cluster are similar in some sense. For this, itis necessary to first establish a measure of similarity includingrules for assigning paths to specific species/subpopulation[19]. As a dissimilarity metric in the proposed method, acriterion is set to judge if two paths are the same whetherthere are obstacles between two paths. If there are obstaclesbetween them, they belong to different species; otherwise,they are the same. For the obstacle check, a simple point-in-polygon (PIP) algorithm is used. It checks if there are anypoint belonging to any obstacle are inside the closed regionsurrounded by the pair of paths. Since these two paths canintersect each other with several points, the PIP algorithmsmust run with self-intersecting polygons. After speciation,the population is composed of several subpopulations whichrepresent the clusters:

p = p1 ∪ p2 ∪ . . .∪ ps ∪ . . .∪ pt∪ . . .∪ pn (10)

Denoting set of obstacles in the operation zone by Γ, the setof interior points O is defined by

O = {Oi|Oi ∈ I(Γi), i = 1, 2, . . . , n(Γ)} (11)

where I(Γi) represents the interior region of the ith obstacleOi, is a random point that is in the interior or on the boundaryof the ith obstacle. For any pair of two feasible individuals psiand psi that belongs to the same subpopulation (cluster) ps, itcan be stated that

∀O /∈ I(psi∪psj

): i = 1, 2, . . . , n (ps)

j = 1, 2, . . . , n (ps) , i 6= j (12)

For any feasible pair of individuals psi and pti that belongs todifferent species ps and pt

∃O ∈ I(psi ∪ ptj

): i = 1, 2, . . . , n (ps)

j = 1, 2, . . . , n(pt), s = 1, 2, . . . , n(p),

t = 1, 2, . . . , n(p), s6=t (13)

Using FPCM mentioned in the previous section, brings anadvantage in clustering by reducing the computation load ofPIP algorithm substantially. Since FPCM generates feasiblepaths only, it is sufficient to check only one random point inan obstacle in order to detect whether the obstacle is betweentwo paths. However, the number of check points (grids)would exponentially increase in the speciation of infeasiblepaths [5].

One of the main problems in speciation is the unpredictablesize of the subpopulations that are obtained after clustering.Some of the subpopulations may have very limited number

6

of individuals whereas others have overproduced individuals.This may lead to premature convergence of some of thesubpopulation. In order to prevent it, proposed algorithmequalizes the size of the subpopulations after speciation. Theequalizing mechanism works by either (i) eliminating theworst paths from the overproduced subpopulations or (ii) ap-plying low-perturb mutation to reproduce the subpopulationswhich have inadequate number of individuals.

Interspecies Crossover Method (ISCM)

In interspecies crossover method, crossover operator is per-formed for the individual paths from different subpopulationsand new offspring are produced. By means of ISCM, thealgorithm improves the diversity and increases the probabilityof obtaining new species in the next generations [5]. Thecross-species interaction is carried out for all pairs of pop-ulations and controlled by inter-species crossover rate suchthat 0 ≤ λc ≤ 1 . This constant rate is usually kept very lowin order not to ruin the convergenc of the algorithm.

Algorithm 3 Interspecies Crossover1: select parents from all combinations of species

C (n (ps) , 2)2: for i = 1→ C (n (ps) , 2) do3: while infeasible offspring do4: apply crossover operator5: if counter > 100 then6: apply high-perturb mutation7: end if8: end while9: end for10: for i = 1→ the number of offspring do11: if offspring ∈ existing subpopulation ps then12: replace the worst path in ps by offspring13: else14: create new subpopulation15: end if16: end for17: if size of any new subpopulations¡existing then18: reproduce individuals in new subpopulation

by low-perturb mutation19: end if

In the proposed approach, in addition to the inter-speciescrossover, there is also inner-species crossover which per-forms between the individuals of the same subpopulation.In the proposed method, instead of arithmetic crossover,single point crossover technique is preferred. The arithmeticcrossover usually tends to produce offspring toward the inte-rior of the search region, so it might not work for the shortestpath problems since the optimum usually lies on or near theboundaries (e.g. tangent to obstacles) [18]. In the single pointcrossover method, the two paths are connected with a straightline at a random node selected on each path.

Mutation

The mutation operator randomly picks a node in the indi-vidual path and perturbs it by shifting another location inthe search space. Most of the mutation operators have ahigh probability of occurrence rate (ξ) in the early stageof evolutionary process to shift the infeasible paths to thefeasible regions. However, there is no need to it in theproposed algorithm since it creates feasible paths only [5].For different purposes in MPGA, there are mainly two typesof mutation is used: (i) low-perturb mutation (ii) high-perturbmutation. In low-perturb mutation (0.1 ≤ ξ ≤ 0.3), the

selected nodes is disturbed by a small amount. Especiallywhen the mutation is applied for the evolution of each species,it is undesirable to obtain an offspring which is a member ofanother species. In order not to exceed the species boundary,low perturb mutation is performed for fine tuning the paths.On the other hand, high-perturb mutation (ξ≤0.6) is moreconvenient when it is used in the inter-species crossoveralgorithm. If the crossover across species cannot find afeasible offspring in a certain number of iteration, high-perturb mutation is applied to obtain highly diversified newspecies.

Summary of the Proposed MPGA for Multiple Path Planning

By using the genetic operators mentioned in the previoussections, the pseudo code of overall algorithm is:

Algorithm 4 Multi-Population Genetic Algorithm1: create feasible initial population p using FPCM2: cluster p into subpopulations ps using ESM3: reproduce small-size subpopulations

by applying low-perturb mutation4: shrink excessive-size subpopulations

by eliminating worst paths5: repeat6: for s = 1→ the number of subpopulations do7: store elite paths in ps8: select parents for the genetic operators9: apply crossover10: apply low-perturb mutation11: create next generations such that

ps = offspring ∪ elites12: check terminating condition τs for ps13: end for14: if rand < λc (inter-species crossover rate) then15: apply interspecies crossover by ISCM16: end if17: if terminating conditions ∀ ps then18: set terminating condition to true19: end if20: until terminating condition

After obtaining all of the possible suboptimal trajectoriesin the obstacle environment by MPGA, another simple al-gorithm selects the best combination of paths, number ofwhich equals to the number of UAVs in the cluster so as tomaximize the communication quality. It simply scans all ofthe combination of UAV paths and checks the communicationrates. Then, it grades each of the combination according theirin-flight communication probabilities (which is explained inthe third section) and selects the one that has best grade. Forexample, if the number of species found after MPGA is runis n and the UAV cluster size is c, then the total numberof possible path clusters that are to be checked is the c-combination of n.

5. SIMULATION RESULTS FOR UAV CLUSTEROptimal Relay Location Results

All the EBCH encoded n-bits code words from the source andthe relay are BPSK modulated and sent to the destination.All the three channels are considered to be orthogonal andhave AWGN and the transmitted signals are considered todecay according to free space propagation model, where thepath loss exponent is 2. The two component codes used

7

Figure 7. Decode and forward optimal relay location

Figure 8. Optimal relay location selection using Soft DFprocessing scheme.

in the DTPC simulations have the same parameters, wheren = 64, k = 51 and δ = 6. The TPC decoding at thedestination is based on the DBD SISO decoder [10], wherechannel statistics are assumed to be available for the decodingprocess.

In Figure 7, the optimization simulation for a DF cooperativecoding is run for each step of the SNR of the source todestination. After about 30 iterations, the optimal locationfor the relay which results in the highest BER performanceis:

recorded when the step size drops below 1 × 105. For theDF cooperative scheme it is found that the optimal locationfor the relay for the highest performance is near 0.45 of thedistance between the source and the destination. When theSNR for the source to destination signal is lower than 0.4dB,the optimal performance is found to be when the relay iscloser to the source between 0.35 and 0.45 of the normalizedsource destination distance. As the source-destination SNRget higher beyond 1.4dB the optimal location for the highestperformance tend to gradually be closer to the destinationfrom 0.45 to 0.55 of the normalized distance.

The Figure 8 shows the results for optimizing the location ofthe relay on the line between the source and the destination in

Figure 9. BER performance of the DTPC system with DFand SDF relaying.

the cooperative coding with SDF scenario implemented at therelay. The SNR for the message transmitted directly from thesource to the destination is changed each time and for eachstep the optimal relay location is recorded after running theoptimization method aforementioned that will result on thelowest BER. Each time the optimization is run, it will stopwhen the step size becomes less than 1× 105.

The results for the SDF case show that the optimal locationfor a wide range of SNRsd between 0.2 and 1.3dB is about0.41 of the source-destination distance, i.e. closer to thesource. In Figure 7 when the same SNR drops below 0.2dBor becomes higher than 1.3dB the relay prefers a locationcloser to the midway between the source and destination tocontribute the highest BER performance possible.

Figure 9 shows the optimal BER performance for both SDFand DF cooperative relaying schemes. Here BER perfor-mance is plotted when the relay is located at the optimalrelay position. The location where the optimal performanceis obtained from the optimization simulation discussed abovefor both SDF and DF methods and then the correspondinghighest BER performance resulted at the optimal location isrecorded and then plotted against the signal to noise ratio ofthe source to destination link. The result depicted becauseis the highest possible BER performance since each point isthe result of an optimization process that looks for the lowestBER.

Simulation Results of Multiple Path Planning Using the Op-timal Relay Distance Results

In order to establish the efficiency of the proposed multiplepath planning algorithm, mid-level difficulty operation sce-nario are simulated in MATLAB 2009a on a Windows XPdesktop with an Intel Core 2 Duo 2.67 GHz CPU with a 4GBRAM. In this scenario, the UAV cluster size is set to 6. Also,the maximum range of the UAVs is given 200 km, averagespeeds of the UAVs are assumed constant, value of which canbe adjusted and set before flight. Minimum route leg lengthlmin and maximum maneuver angle θmax are taken 10 kmand ±π/2 respectively.

In the scenario as seen in Figure 10, the proposed algorithmattempts to find the optimal combination of trajectories thatthe six UAVs follow during surveillance. The proposedMPGA discovers totally 13 sup-optimal path species so as

8

1

2

3

4 5

6

Start (00m00s)

Final (28m14s)

UAV

Cluster

at 10m30s

at 13m00s

at 17m20s

at 20m00s

UAVs Optimal Trajectories

LOS Communication Lines

Figure 10. Simulation Results of Multiple Path PlanningScenario with a UAV Cluster Size of 6

to minimize the mission time and maximize the coverage ofthe operation zone. Within 13 alternative trajectories, thealgorithm selects the best 6 combination of paths such thatthe optimum relay distances found in the previous sectionbetween UAVs are tried to be satisfied as much as possible.That is, in-flight intercommunication within UAVs is to bemaximized. In Figure 10, the resultant paths and their instantlocations at certain times (at 10m30s, 13m00s, 17m20s, and20m00s) are given respectively.

Figure 11 also gives an idea about the communication ratesof the UAVs during flight. For each of the UAV in the cluster,the number of neighbor UAVs communicated is given fromtakeoff to land. According to the figure, one can realize thatthe UAVs flying through the outer of the region have a lowercommunication than those flying in the inner of the region.However, the inner UAVs bridges between the outer UAVsto maintain the communication between them. The overallcommunication performance of the UAVs over the calculatedpaths in highly cluttered environment is encouraging.

The parameters of the MPGA are given in Table 1. Accordingto the table, the genetic algorithm converged at the end ofthe 109th generation, resulting in 13 different species. Eachspecies represents a different route in the operational region.Discovering all possible passages within obstacles is crucialfor improving the surveillance.

During calculating the optimum paths, following assumptionsare taken:

1. Each UAV takes off from the same air base platform2. Each UAV lands in the same air base platform3. Each UAV takes of at the same time and simultaneouslyarrived in final platform. That is, the total flight times foreach UAV are equal.4. As the total flight distance increases, the speed of the UAV

Operation Time [min]

Nu

mber

of

Oth

er

UA

Vs B

ein

g C

om

mu

nic

ate

d UA

V-1

UA

V-2

UA

V-4

UA

V-3

UA

V-5

UA

V-6

0

2

4

6

0

2

4

6

0

2

4

6

0

2

4

6

0

2

4

6

0 5 10 15 20 25 28.140

2

4

6

Figure 11. For each of UAV in the cluster, the number ofother UAVs which is being communicated in-flight

can be increased to certain level

Table 1. Execution Times and Parameters of MultiplePopulation Genetic Algorithm (MPGA). The results are the

medians obtained over 20 runs.# MPGA Inputs Values Unit1 Total Convergence Time 21, 59 Seconds2 Number of Generations 109 -3 Number of Species 13 -4 Initial Population Size 50 -5 Subpopulation Size 8 -6 Inner-species Crossover Rate 60 %7 Inter-species Crossover Rate 20 %8 Low-perturb Mutation Rate 40 %9 High-perturb Mutation Rate 80 %

With the assumptions above, the trajectory lengths, averagespeeds, total number of waypoints for each UAV and thecorresponding common mission time are given in Table 2.

The results of the simulations show that overall performanceof the proposed multi path planning method in highly clut-tered environment is encouraging both in the execution timeand the optimality. Unlike the most of multi-path planning al-gorithms waste most of their computation time in clustering;in the proposed method, total time of clustering constitutesover less than 1% of the entire computation time. Another im-provement over the traditional GAs that starts with a randominfeasible population is that; although traditional GAs spendsmuch time to shift the random population to feasible regions,the proposed FPCM is able to produce 50 feasible individualsless than 5 seconds for the scenario. Furthermore, the totalconvergence time for both of the scenarios is considerablyshort considering the number and the accuracy of the resultantlocal optimum paths. Thus, this assures practicability of theproposed algorithm in mission planning systems.

9

Table 2. Mission Planning Scheme of UAV Cluster

UAV TrajectoryLength

AverageSpeed Flight Time Total # of

# (km) (km/h) (mm:ss) Waypoints1 129.99 343 14 128.37 318 15 125.97 277 26 140.71 269 28:14 28 149.33 274 2

11 160.89 300 2

6. CONCLUSION AND FUTURE WORKCollaborative path planning forms a crucial part of a missionwith multiple UAVs in order to increase the surveillance anddecrease the detection. One of the important aspects is tomake sure that the way points are reached simultaneouslyalong different routes which are in harmony with one an-other. The main problem in MPP algorithms is their highcomputation time and their need in unbounded amount ofmemory to converge a sub-optimal solution. The convergencetime of the algorithm exponentially grows as the numberof required paths increases. In order to enhance the speedof the GAs, this paper describes an improved MPGA forpath planning of a set of cooperating UAVs. This improvedtechnique includes a novel approach (FPCM) to the creationof feasible initial population which also reduces the clusteringtime drastically. The number of alternative routes in thefinal solution is increased by using inner and inter-speciescrossover routines. Also, the flexibility of the algorithmis strengthened by introducing variable occurrence rates formutation.

In this paper we also proposed an optimal relay selectionoptimization algorithm depending on the location of the relay.It can be observed from the simulation results that to havea superior link performance the selected relays should belocated at (0.35 ≤ λ < 0.55). The paper points out thepossibility for a relay to relocate and position itself to improvethe link quality. After comparing the results obtained byapplying the algorithm to general case large gains in linkperformance (BER) can be seen. This, however, may changewith the variation in coding technique and forwarding schemeapplied the relay. The results obtained from applying theproposed location optimization method showed effectivenessin allocating power between the transmitting nodes. Inapplications such as routing in military UAV clusters, optimalrelay selection can be vital element in providing the desiredlevel of quality of service.

The proposed approach is simulated with multiple UAVs forsimultaneous arrival at the destination. As seen in the simu-lation results, the algorithm satisfactorily achieved to find op-timal combination of paths that provides best reconnaissanceof the region while maximizing the in-flight communicationbetween UAVs. Although the precision of the final pathsplanned by the proposed MPP algorithm is considerablyhigh; the convergence speed of MPGA is kept relatively fastconsidering the previous studies. Thus, it is suitable both forthe high-precision and time-critical path planning problemsfor UAV operated in obstacle-rich zones. This work wasmotivated by autonomous offline multiple path planning forUAVs, but applicable to any two-dimensional path planningproblem involving obstacles. Since the scope of proposedmultiple path planner includes only two-dimensional turnsin horizontal plane, the pitch-up/down maneuver is not con-sidered. However, it is important to add the third verticaldimension for climbing constraint which can be a future study

in this research.

REFERENCES[1] Masehian, E., Sedighizadeh, D., Classic and Heuristic

Approaches in Robot Motion Planning A ChronologicalReview, World Academy of Science, Engineering andTechnology 29, pp.101106, 2007.

[2] Tsourdos, A., White, B., A., Shanmugavel, M., Cooper-ative Path Planning of Unmanned Aerial Vehicles, JohnWiley and Sons, Ltd., ISBN. 978-0-470-74129-0, 2011.

[3] Kothari, M., Postlethwaite, I., Gu, D., Multi-UAV PathPlanning in Obstacle Rich Environments Using Rapidly-exploring Random Trees, Proc. of the IEEE Conferenceon Decision and Control and Control Conference, pp.30693074, 2009.

[4] P. Olsson, J. Kvarnstrm, P. Doherty, O. Burdakov, K.Holmberg, Generating UAV Communication Networks-for Monitoring and Surveillance, Proc. of of the IEEEICARCV, pp.1070-1077, 2010

[5] C. Tanil, Cooperative Path Planning for MultipleUAVsAn Evolutionary Speciation Approach, Proc. of theAIAA Guidance, Navigation, and Control Conference,AIAA 2012-4903, pp. 113, 2012

[6] C. Tanil, Improved Heuristic and Evolutionary Meth-ods for Tactical UAV Mission Planning, Proc. of theIEEE Aerospace Conference, pp. 978-1-4577-0556-4.18, 2012.

[7] O. Burdakov, P. Doherty, K. Holmberg, P. Olsson, Op-timal Placement of UV-based Communications RelayNodes, Journal of Global Optimization, (48), 4, 511-531,2010.

[8] M. Tortonesi, C. Stefanelli, E. Benvegnu, K. Ford,N. Suri, M. Linderman, ”Multiple UAV Coordinationand Communications in Tactical Edge Networks”, IEEECommunication Magazine, vol. 50, pp. 4850, 2012.

[9] Randal W. Beard, Timothy W. McLain, ”Multiple UAVCooperative Search under Collision Avoidance and Lim-ited Range Communication Constraints”, IEEE Confer-ence on Decision and Control, 2003.

[10] Randal W. Beard, Timothy W. McLain, Derek B. Nel-son, Derek Kingston, and David Johanson, ”Decentral-ized Cooperative Aerial Surveillance Using Fixed-WingMiniature UAVs”, Proceedings of the IEEE, Vol. 94, No.7, July 2006.

[11] Olsson, P., Kvarnstrom, J., Doherty, P., Burdakov, O.,Holmberg, K., ”Generating UAV communication net-works for monitoring and surveillance,”Control Automa-tion Robotics and Vision (ICARCV), 2010 11th Interna-tional Conference on , vol., no., pp.1070-1077, 7-10 Dec.2010.

[12] Sujit, P.B., Beard, R., ”Cooperative Path Planning forMultiple UAVs Exploring an Unknown Region,” Amer-ican Control Conference, 2007. ACC ’07 , vol., no.,pp.347-352, 9-13 July 2007.

[13] E. Obiedat and L. Cao, Soft Information Relaying forDistributed Turbo Product Codes (SIR-DTPC), IEEESignal Processing Letters, vol. 17, pp. 363366, 2010.

[14] N. Le, A.R. Soleymani, and Y.R. Shayan, Distance-based-decoding of block turbo codes, CommunicationsLetters, IEEE, vol. 9, no. 11, pp. 10061008, Nov. 2005.

[15] J. Hagenauer, E. Offer, and L. Papke, Iterative decoding

10

of binary block and convolutional codes, InformationTheory, IEEE Transactions on, vol. 42, no. 2, pp. 429445,Mar 1996.

[16] E.A. Obiedat and Lei Cao, Power allocation for dis-tributed turbo product codes (dtpc), Global Telecommu-nications Conference (GLOBECOM 2010), 2010 IEEE,dec. 2010, pp. 1 6.

[17] E. Obiedat, W. Xiang, J. Leis, and Cao L., Soft In-cremental Re-dundancy for Distributed Turbo ProductCodes, in 7th Annual IEEE Consumer Communicationsand Networking Conference. IEEE, 2010.

[18] Goldberg, D., E., Genetic Algorithms in Search, Opti-mization, and Machine Learning, AddisonWesley, 1989.

[19] Li, M., Hu, J., Li, L., Multipath planning based on neu-ral network optimized with adaptive niche in unknownenvironment, Proc. of the IEEE Intelligent ComputationTechnology and Automation Conference, pp. 761764,2010.

[20] E. Obiedat, G. Chen, and L. Cao, Distributed TurboProduct Codes Over Multiple Relays, in 7th AnnualIEEE Consumer Communications and Networking Con-ference. IEEE, 2010.

BIOGRAPHY[

Cagatay Tanıl (M.Sc.) received B.Sc.and M.Sc. degrees in Mechanical En-gineering Department of Middle EastTechnical University (METU), Ankara,Turkey. He is a Senior Design Engineerin the department of Tactical Systems inRoketsan Missiles Industries Inc. He hasbeen involved in control systems, model-ing and simulation and mission planningin aerospace applications.

Chirag Warty (CSM, MEng) is cur-rently a IEEE Associate Member. Hehas been the associate editor for theIEEE Interface magazine since January2009. He currently holds several posi-tions on IEEE Study Groups and IEEEsocieties. He has been consultant forseveral projects, in India and in the US.He also leads several research teamsacross the globe. He has perused several

academic degrees in India and United States with a BSEEfrom University of Mississippi with specialization in RFand wireless, Masters of Engineering from University ofIllinois-Chicago. He also has several graduate certificates incommunications and signal processing from Cornell, UCLAand Stanford. His research interests include MIMO, softwaredefined radio, OFDM and 4G technologies including gametheory for terrestrial wireless and satellite clusters.

Dr. Esam Obiedat (Ph.D) currentlyholds a position of Senior systems engi-neer at CommScope /Andrew Corpora-tion. He received his Bachelors (2003)and Masters (2005) from Jordan Instituteof Science and Technology and his doc-torate from University of Mississippi in2010. His current work is in distributedantenna systems and area of interest in-clude Turbo coding and distributed cod-

ing systems and their implementation for MIMO environmentusing distributed antennas. Before joining Andrew Corp. hehas contributed extensively to Research and Developementdivisions at Ericsson, 3S Network

11