ieee transactions on mobile computing, vol., no

14
IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 1 Tula: Balancing Energy for Sensing and Communication in a Perpetual Mobile System Jacob Sorber, Aruna Balasubramanian, Mark D. Corner, Joshua Ennen, Carl Qualls Abstract—Due to advances in low power sensors, energy harvesting, and disruption tolerant networking, we can now build mobile systems that operate perpetually, sensing and streaming data directly to scientists. However, factors such as energy harvesting variability and unpredictable network connectivity, make building robust and perpetual systems difficult. In this paper, we present a system, Tula, that balances sensing with data delivery, to allow perpetual and robust operation across highly dynamic and mobile networks. This balance is especially important in unpredictable environments; sensing more data than can be delivered by the network is not useful, while gathering less underutilizes the system’s potential. Tula is decentralized, fair and automatically adapts across different mobility patterns. We evaluate Tula using mobility and energy traces from TurtleNet—a mobile sensor network we deployed to study Gopher tortoises—and publicly available traces from the UMass DieselNet testbed. Our evaluations show that Tula senses and delivers data at up to 85% of an optimal, oracular system that perfectly replicates data and has foreknowledge of future energy harvesting. We also demonstrate that Tula can be implemented on a small microcontroller with modest code, memory, and processing requirements. Index Terms—Support for adaptation, store and forward networks, network management, wireless sensor networks, mobile communication systems. 1 I NTRODUCTION D UE to three key innovations: small pro- grammable sensors; energy harvesting [20], [26]; and disruption tolerant networking, mobile systems are poised to answer many questions about a wide range of natural and manmade systems. Recent ef- forts focusing on zebras [32], whales [12], turtles [26], people [15], and vehicles [9] have shown that in-situ monitoring using embedded devices can provide un- precedented and transformational data. When these systems harvest energy from their environment and gather data in a robust manner, they can become perpetual and self managing, streaming data directly to scientists for decades. However, a number of external factors make build- ing robust perpetual systems difficult. Seasonality, habitat disruption, changes in social networks and mobility can drastically affect network connectivity and energy harvesting. Without basic parameters such as network connectivity and energy availability, it is impossible to tune power management and routing. J. Sorber is with Dartmouth College, Computer Science De- partment, 6211 Sudikoff Lab, Hanover, NH 03755. E-mail: ja- [email protected] A. Balasubramanian is with the University of Washington, Computer Science & Engineering Department, 185 Stevens Way, Seattle, WA 98195. Email: [email protected] M. D. Corner is with the University of Massachusetts Amherst, Computer Science Department, 140 Governors Drive, Amherst, MA 01003. Email: [email protected] J. Ennen is with Maryville College, Natural Sciences Divi- sion, 502 E. Lamar Alexander Parkway Maryville, TN 37804. Email: [email protected] C. Qualls is with the University of Southern Mississippi, Department of Biological Sciences, 118 College Drive #5018, Hattiesburg, MS 39406. Email: [email protected] A key premise of this problem domain is that: node mobility, unpredictable network connectivity and uncertain energy availability represent the greatest challenges for designing perpetual systems.. Early perpetual systems, like ZebraNet [32], focused on minimizing energy consumption rather than en- ergy awareness and adaptation. More recent systems either use local energy adaptation techniques with- out considering data delivery [19], [23], [26] or use adaptation techniques for purely static networks [11]. However, adapting to both energy and network vari- ations is considerably more difficult. In particular, a node needs to adapt and balance both its sensing and routing tasks. In a long-running system the goal is to gather as much data from nodes as the limited resources of network bandwidth and energy permit. Sensing more data than can be delivered by the net- work is not useful, while gathering less underutilizes the system’s potential. In addition, sparse networks that depend on node cooperation for routing data [2], [27], must balance the energy devoted to sensing and routing their own data, with energy used to route data for other nodes. In this paper, we present a system, Tula, that ad- dresses this challenge for mobile sensor networks. A Tula node uses a distributed algorithm to balance energy allocation across three tasks—sensing, routing the node’s own data and the routing data for other nodes. The Tula energy allocation ensures max-min fairness, which allows data collection from all nodes, including poorly connected nodes. The key insight in Tula is that sensing and routing are inherently dependent, and optimizing only one or the other in an energy-constrained environment is futile. Given the allocation for sensing and routing, Tula

Upload: others

Post on 03-Nov-2021

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 1

Tula: Balancing Energy for Sensing andCommunication in a Perpetual Mobile System

Jacob Sorber, Aruna Balasubramanian, Mark D. Corner, Joshua Ennen, Carl Qualls

Abstract—Due to advances in low power sensors, energy harvesting, and disruption tolerant networking, we can now buildmobile systems that operate perpetually, sensing and streaming data directly to scientists. However, factors such as energyharvesting variability and unpredictable network connectivity, make building robust and perpetual systems difficult. In this paper,we present a system, Tula, that balances sensing with data delivery, to allow perpetual and robust operation across highlydynamic and mobile networks. This balance is especially important in unpredictable environments; sensing more data than canbe delivered by the network is not useful, while gathering less underutilizes the system’s potential. Tula is decentralized, fairand automatically adapts across different mobility patterns. We evaluate Tula using mobility and energy traces from TurtleNet—amobile sensor network we deployed to study Gopher tortoises—and publicly available traces from the UMass DieselNet testbed.Our evaluations show that Tula senses and delivers data at up to 85% of an optimal, oracular system that perfectly replicates dataand has foreknowledge of future energy harvesting. We also demonstrate that Tula can be implemented on a small microcontrollerwith modest code, memory, and processing requirements.

Index Terms—Support for adaptation, store and forward networks, network management, wireless sensor networks, mobilecommunication systems.

F

1 INTRODUCTION

DUE to three key innovations: small pro-grammable sensors; energy harvesting [20], [26];

and disruption tolerant networking, mobile systemsare poised to answer many questions about a widerange of natural and manmade systems. Recent ef-forts focusing on zebras [32], whales [12], turtles [26],people [15], and vehicles [9] have shown that in-situmonitoring using embedded devices can provide un-precedented and transformational data. When thesesystems harvest energy from their environment andgather data in a robust manner, they can becomeperpetual and self managing, streaming data directlyto scientists for decades.

However, a number of external factors make build-ing robust perpetual systems difficult. Seasonality,habitat disruption, changes in social networks andmobility can drastically affect network connectivityand energy harvesting. Without basic parameters suchas network connectivity and energy availability, it isimpossible to tune power management and routing.

• J. Sorber is with Dartmouth College, Computer Science De-partment, 6211 Sudikoff Lab, Hanover, NH 03755. E-mail: [email protected]

• A. Balasubramanian is with the University of Washington, ComputerScience & Engineering Department, 185 Stevens Way, Seattle, WA98195. Email: [email protected]

• M. D. Corner is with the University of Massachusetts Amherst,Computer Science Department, 140 Governors Drive, Amherst, MA01003. Email: [email protected]

• J. Ennen is with Maryville College, Natural Sciences Divi-sion, 502 E. Lamar Alexander Parkway Maryville, TN 37804.Email: [email protected]

• C. Qualls is with the University of Southern Mississippi, Departmentof Biological Sciences, 118 College Drive #5018, Hattiesburg, MS39406. Email: [email protected]

A key premise of this problem domain is that: nodemobility, unpredictable network connectivity and uncertainenergy availability represent the greatest challenges fordesigning perpetual systems..

Early perpetual systems, like ZebraNet [32], focusedon minimizing energy consumption rather than en-ergy awareness and adaptation. More recent systemseither use local energy adaptation techniques with-out considering data delivery [19], [23], [26] or useadaptation techniques for purely static networks [11].However, adapting to both energy and network vari-ations is considerably more difficult. In particular, anode needs to adapt and balance both its sensing androuting tasks. In a long-running system the goal isto gather as much data from nodes as the limitedresources of network bandwidth and energy permit.Sensing more data than can be delivered by the net-work is not useful, while gathering less underutilizesthe system’s potential. In addition, sparse networksthat depend on node cooperation for routing data [2],[27], must balance the energy devoted to sensing androuting their own data, with energy used to route datafor other nodes.

In this paper, we present a system, Tula, that ad-dresses this challenge for mobile sensor networks.A Tula node uses a distributed algorithm to balanceenergy allocation across three tasks—sensing, routingthe node’s own data and the routing data for othernodes. The Tula energy allocation ensures max-minfairness, which allows data collection from all nodes,including poorly connected nodes. The key insightin Tula is that sensing and routing are inherentlydependent, and optimizing only one or the other inan energy-constrained environment is futile.

Given the allocation for sensing and routing, Tula

Page 2: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 2

Fig. 1. A solar-powered tracking device used inTurtleNet.

uses an adaptive sensing system to collect data anda DTN routing algorithm to deliver the data. We for-mulate the Tula allocation problem as a constraint op-timization problem (COP). Each Tula node measuresenergy consumption for sensing and communicationand gathers data about the environment through nodemeetings, to locally solve the COP on an embed-ded device. Tula is general, and automatically adaptsacross mobility patterns, from static to highly mobileenvironments.

Our design of Tula targets mobile sensing systemsbroadly, and we evaluate Tula in the context of twospecific systems. The primary example is TurtleNet,a mobile sensor network that we deployed to studyGopher tortoises. The TurtleNet deployment consistsof 17 tortoises and we have collected energy harvest-ing and mobility data. The deployment has been inoperation since August 2008. In our evaluation, weuse traces from TurtleNet, combined with an imple-mentation of Tula on TinyNodes [7]. We also evaluateTula on publicly available traces obtained from theUMass DieselNet project [4].

Our evaluations over both TurtleNet and DieselNetshow that Tula senses and delivers data within 75%of an optimal, oracular system that perfectly repli-cates data and has foreknowledge of future energyharvesting. The protocol is fair in terms of deliveryrates across nodes, and comes within 95% of theoptimal in terms of the max-min fairness objective.Tula not only works well for sparse mobile networks,but also for static mesh networks. Our evaluations ona synthetically generated mesh network shows thatTula adapts well to the static environment and sensesand delivers data within 85% of the optimal. Tula alsooutperforms systems that adapt only sensing or onlyrouting, collecting up to 27% more data and com-pletely avoiding energy failures and energy waste.Finally, we show that Tula can be implemented ona small microcontroller with modest code, memory,and processing requirements.

2 APPLICATIONS AND CHALLENGESDevices that operate perpetually using harvested en-ergy represent a new class of mobile system that

promises to enable a wide and largely unexploredrange of potential applications. This vision includessignificant advances for scientists studying mobility innature. In spite of decades-worth of study, the move-ments and behaviors of most animal species in thewild are completely unknown. Current methods liketrapping and manual radio telemetry are labor inten-sive, yield few data points, and significantly increasethe frequency of animal interactions with humans. Byusing small in-situ sensor devices to observe animallocation, movement, and environmental conditions,researchers will be able to collect more data at highertemporal densities with minimal impact on behavior.This shift promises to answer long-debated questionsabout habitat usage, population trends, and complexinteractions between different species, including hu-mans.

2.1 TurtleNetIn light of these potential benefits, we have deployedTurtleNet, a mobile network with the goal of over-coming many of the challenges faced by perpetualsensing systems. Our deployment consists of 17 track-ing devices attached to Gopher Tortoises (Gopheruspolyphemus), shown in Figure 1. Each device consistsof a Shockfish TinyNode [7], a solar panel, a battery,multiple sensors, and additional energy measurementhardware.

During operation, the devices record connectionopportunities with neighboring nodes and periodicsensor readings, including temperature, GPS coordi-nates, battery level, solar energy harvested and en-ergy consumption. Unlike traditional networks, thesenodes rarely have an end-to-end connection to oneof the two deployed GPRS-enabled base stations, anddevices must opportunistically deliver collected datausing mobile-to-mobile routing [2], [27]. When twomobile nodes are within communication range, calleda connection opportunity, they exchange data. Thisdata is stored and then forwarded during subsequentconnection opportunities until it is eventually deliv-ered to the sink. The network has been in operationsince August 2008.

2.2 ChallengesOn analyzing the deployment traces, we uncovereda number of key challenges. The primary difficultyin designing TurtleNet—and generally any untetheredmobile network—is the continuous variation of bothenergy harvesting and network connectivity due tomobility. Figure 2 shows how a node’s daily har-vested energy varied—experiencing both day-to-dayand seasonal changes. Note that within a 10-dayperiod, daily harvesting ranged from less than 0.1kJ tomore than 1.7kJ. In order to support perpetual operation,a device must adapt its behavior over time.

In addition to temporal variation, energy harvestingalso varies considerably across the network. Figure 3

Page 3: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 3

0.0

0.5

1.0

1.5

Dai

ly E

nerg

y (k

J/da

y)

Nov Jan Mar May Jul

TortoiseHibernation

Fig. 2. Daily solar energy is shownfor a TurtleNet node before and afterhibernation.

Nodes

Avg

. Ene

rgy

(kJ/

day)

02

46

810

Fig. 3. The average daily energyharvested by each TurtleNet nodeduring a 1-month trace.

●●

0 50 100 150 200 250 300

020

0060

0010

000

Connection Opportunities (1 month)

Avg

Ene

rgy

(J/d

ay)

Fig. 4. Harvested energy plot-ted against number of meetings foreach node. Energy-rich nodes arenot necessarily better connectedand vice versa.

shows the average daily energy harvested by all nodesin the network over a 1 month period of time, sortedto show the energy distribution. The figure showsthat there is significant variation in energy harvestingacross nodes. With diverse energy budgets in the network,each node needs to balance its available energy betweensensing and delivering data.

Recall that nodes rely on other well connectednodes to store and forward their data to the desti-nation. Unfortunately, there is very little correlationbetween how connected a node is and the energyit gathers, as shown in Figure 4. The well-connectedhubs that are best positioned to route data may nothave sufficient energy to support network demand.Routing decisions in perpetual networks must depend onnot only topology, but also the available energy.

Finally, in TurtleNet—and most other mobilesystems—connections between mobile nodes oftenexhibit patterns due to social habitats as shown inFigure 5. The figure shows that 40% of node meetingsrepeat more than 10 times a month. The meetingpatterns are not completely random and can be lever-aged to combat the network’s uncertainty. In otherwords, if two peers have a connection opportunity, wecan expect that the peers will have future connectionopportunities.

2.3 Design GoalsIn this paper, we describe Tula, a system that ad-dresses the above challenges by supporting perpetualoperation and ensuring fair and efficient data collec-tion. Specifically, our design goals are: (i) Perpetualoperation, (ii) Ability to operate in sparse, resourceconstrained environment, and (iii) Fairness.

Perpetual operation requires that energy spent sens-ing, storing, processing, and communicating must bematched with harvested energy. In addition, the sens-ing rates must match with the rate of data delivery,since sensing more data than can be delivered is notuseful.

Tula must operate in sparsely connected mobilenetworks and on energy and computationally con-

0 10 20 30 40 50 60

0.0

0.2

0.4

0.6

0.8

1.0

Frequency (Meetings/Month)

Pai

rwis

e Li

nks

(%)

Fig. 5. CDF of the pair-wise meeting frequency during1 month of TurtleNet operation. While some meetingsoccur too infrequently to be very useful, 50% of thenode pairs repeat at least 5 times.

strained platforms. Sparse connectivity requires thatnodes use DTN routing [2], [30] to route data usingnode mobility to overcome disruptions.

Finally, fairness in Tula is critical. Tula operates innetworks with significant variation in both energy andnetwork connectivity. Maximizing network through-put or minimizing delay, without enforcing fairness,may result in well connected and energy-rich nodescollecting and delivering their own data at a highrate, while starving nodes that are further away. Wellconnected nodes may also have their energy budgetsdepleted by high network demand.

Different models have been proposed for sharingresources among nodes and flows within a network.Wildlife tracking applications typically seek to char-acterize animal behavior and their interactions withthe environment by collecting as much data from asmany nodes as possible. This emphasis on sensing“breadth” rather than “depth” can be expressed usingmax-min fairness, which requires that a nodes’ send-ing rate be improved only after all lower rates havealready been maximized. In Tula we focus on achiev-ing max-min fairness, and other fairness models are

Page 4: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 4

Tula Rate Allocator

Hardware

Adaptive Sensing

Eon, PixieOS, Levels

DTN Routing

Rapid, Spray&Wait, etc

Rate Allocation

Buffer

Rate Allocation

EnergyConsumed

EnergyHarvested

Network Estimates

Sensing Estimates

Fig. 6. The Tula architecture.

left to future work (see Section 9).

3 TULA ARCHITECTURE

The Tula architecture, shown in Figure 6, consists ofthree main components: an Adaptive Sensing systemfor collecting sensor data, a DTN routing algorithmfor opportunistically delivering that data, and a RateAllocator that coordinates both sensing and routingactivities by appropriately allocating resources.

Existing adaptive sensing systems adjust applica-tion sensing rates alone to adapt to the changes ina device’s energy budget. A sensing system mayhave several sensors and application tasks. Existingadaptive systems, including Eon [26], PixieOS [23],and Levels [19], estimate or measure the energy costsof various application tasks and automatically adjustapplication behavior to match a device’s energy bud-get.

Existing DTN routing systems opportunisticallyroute network packets from source to destinationusing sporadic and uncertain device-to-device meet-ings. Many systems have been designed, includingRapid [2] and Spray and Wait [27], to effectivelydeliver data over intermittent links while respondingto changing network conditions.

Unfortunately, the adaptive sensing and DTN rout-ing systems are not designed to work together. Ex-isting adaptive sensing systems consider only localenergy constraints, ignoring the impact of sensed dataon the network. Similarly, DTN routing systems as-sume unlimited energy and consider only bandwidthrestrictions. Tula’s core function is to overcome thischallenge by combining the benefits of adaptive sens-ing and DTN routing into a single coordinated system.Our evaluation (Section 7) shows that Tula signifi-cantly outperforms systems that only adapt sensingor only adapt routing to changing energy conditions.

Rather than build a complete system from scratch,Tula abstracts the sensing and routing and controlsthese systems using an allocator that balances energyfor sensing with that of data delivery. The Tula energyallocation is most easily understood in terms of rate:

the number of packets, or bytes, that can be generatedby sensing and delivered by routing, over some timeperiod.

The allocator’s objective is to maximize the rate atwhich sensor data is collected and delivered, while en-suring that the allocated rates are fair to all nodes. Tothis end, Tula must appropriately adjust (i) the rate ofsensing, (ii) the routing rate for the node’s own data,and (iii) the maximum routing rate for each neighbor’sdata. Given the sensing rate, Tula leverages existingsensing mechanisms that adapt the local sensing taskaccording to available energy. Given a routing rate,Tula adapts existing DTN routing protocols to routedata within the given rate.

3.1 Adapting Sensing

Adaptive sensing systems change their sensing ratesaccording to energy conditions. Adapting the sensingrate is especially important in sensor nodes that havemultiple sensors and have high variability in harvestenergy.

Eon [26], an adaptive sensing system that we usein Tula, uses hardware support to measure energyconsumption and harvesting. Eon combines thesemeasurements with runtime information about theapplication in order to estimate the system’s idlepower draw and the energy cost of various programtasks. Finally, Eon uses this information to determinehow much energy is required to sense data at a givenrate.

This relationship is communicated to the rate alloca-tor, which uses the information to solve the allocationproblem (Section 4).

3.2 Adapting Routing

The Tula rate allocator only assigns the maximumrate at which a node can route data for each of itsneighbors. The actual routing decision involves othertasks including estimating routes, tracking acknowl-edgements, and adapting the route to changing net-work connectivity. In Tula, we use the Rapid [2] DTNrouting protocol and adapt it to an energy constrainedenvironment.

Rapid estimates a distance metric between eachnode and the sink, where the distance is the expecteddelivery delay for the next packet added to its buffer.This delay estimate takes into account the numberof packets currently in the buffer. So, as the bufferfills the expected delay also increases. Rapid thenreplicates data through multiple routes based on amarginal utility heuristic. Rapid estimates networkparameters including the expected delay and band-width by averaging over a sliding window; it thencommunicates the estimates to the allocator.

The routing rate assigned by the allocator is only anupper bound. The actual data that is routed throughthe path depends on the quality of the route. For

Page 5: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 5

d

n

u1 u2 u3 uk

Fig. 7. A simplified example to illustrate the Tula dis-tributed allocation algorithm. The algorithm is executedby node n, whose upstream neighbors are u1, u2, . . . uk

example, let nodes A and B be peers and let A’sallocator assign a maximum rate at which it can routedata for B. Because of changes in the network (dueto mobility, interference, etc.), B may send data at amuch lower rate through A; in turn A will reduceits rate allocation for B and balance the sensing ratesappropriately. In other words, the routing protocoladapts to changing network conditions, that in turnaffect the rate allocation. We discuss this mechanismin more detail in Section 5.1.

In addition to Rapid [2], we have also modified twoother protocols, Spray and Wait [27] and a Randomrouting protocol; however, our experiments have notshown significant differences in overall performance.So, throughout this paper we focus only on Rapid.

4 RATE ALLOCATIONWe first describe the rate allocation algorithm by mak-ing two simplifying assumptions: (i) A node routespackets through only one neighbor, and (ii) datais only forwarded, never replicated. Later, we relaxthese assumptions. The network, shown in Figure 7,illustrates this simplified scenario. The allocation al-gorithm is described with respect to the node, n, withupstream nodes u1, . . . , uk routing data through n,and a single downstream neighbor d, through whichdata is routed toward a sink or base station.

A node’s neighbor set is determined from its meet-ing history, and neighbors are designated as upstreamand downstream based on the routing protocol’s dis-tance estimate. Most DTN routing protocols use adistance metric to evaluate different routes. In Rapid,the distance is the expected delivery delay. When twopeers meet, the peer with a lower delivery delay isconsidered the downstream neighbor, and the onewith the higher delivery delay is the upstream neigh-bor. Also, since nodes are mobile, these upstream/-downstream relationships will change over time. Theimpact of mobility and the implications of relaxing thedefinition of upstream and downstream neighbors arediscussed in Sections 5.4 and 5.

Node n executes the Tula allocation algorithm inorder to determine its own sensing rate rn and the

rate at which it can route data for its neighbors,r1, r2, . . . , rk. The allocation problem is formulated asa Constraint Optimization Problem (COP) with the ob-jective of finding a max-min fair rate allocation, basedon a set of input variables that are either estimated lo-cally or exchanged between one-hop neighbors duringcontact opportunities. Estimates for individual vari-ables are produced using a fixed-size history window,which may be adapted to individual applications.Tula’s default window size is two weeks which wehave found to work well in practice. Determining theideal window size is left for future work.

The max-min fairness objective, as described inSection 2.3, allows us to collect as much data fromas many nodes as possible.

Locally measured values, shown in Table 2, includethe energy required to collect a packet of sensor data(Es), to receive (Er) a data packet, and to deliver (Ed)that packet to the sink. The delivery energy (Ed) in-cludes the energy required to discover neighbors, andprotocol-overhead. P is the power budget available tothe node for its tasks, not including the system’s idlepower draw. Also included in this idle power estimateis the power required to listen for network beacons.Tula’s performance depends on the use of low-powerradio listening. Otherwise the energy required forradio listening would negate the advantages of adapt-ing. All of these measurements can be obtained bya node’s hardware while the device is deployed. InTula, we use the low-level energy profiling capabilitiesalready provided by the Eon [26] runtime system(Section 3.1).

The network variables, shown in Table 1, are ex-changed either upstream or downstream through thenetwork whenever nodes meet. The direction for eachvariable with respect to n is shown in the table.

Network variables are only exchanged with imme-diate peers and are not flooded across the network.These variables are described along with the COPformulation in the following paragraphs.

Objective function: The objective of the rate al-location is to achieve max-min fairness in the datacollected across the nodes. A rate allocation is max-min fair if increasing any rate, ri, requires the reduc-tion of a lesser rate, rj , rj ≤ ri. This is also referredto as a lexicographically maximized rate assignment(Equation 1).

Objective: Lexicographically max. {r1, r2, . . . , rn} (1)

Energy conservation constraint: Perpetual operationrequires that a node’s harvested energy be sufficientfor all sensing and networking tasks. Equation 2ensures that n can sense and deliver its own data at arate of rn and receive and deliver data at a rate of rifrom each upstream neighbor ui without exceeding its

Page 6: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 6

Fi (down) The fraction of node ui’s data that are sentthrough n

Bi (down) The maximum rate at which ui can forwarddata to n

Oj (up) The rate at which n can route packetsthrough its downstream neighbor, j

TABLE 1List of inputs that are exchanged between n and its

neighbors to solve the COP. Variables marked (up) areexchanged from n’s upstream neighbors, and

variables marked (down) are exchanged with n’sdownstream neighbors.

power budget, P . All variables are estimated locallyusing hardware instrumentation.

k∑i=1

ri(Er + Ed) + rn(Es + Ed) < P (2)

Downstream constraint: The total data that n canroute is capped by its downstream neighbor, d. In thesame way that n assigns a maximum routing rate to itsupstream nodes, d likewise assigns a maximum datarate to n. Equation 3 ensures that n will never acceptor collect (by sensing) data at a rate higher than therate, O, at which it can deliver data. Node n receivesthe value for O from d each time they meet.

rn +

k∑i=1

ri < O (3)

Upstream constraint: The objective and the first twoconstraints alone will result in all upstream routingrates being assigned equal to the local sensing rate.This equal division of resources is fair; however,the system will be underutilized if some upstreamneighbors are unable—due to energy or bandwidthlimitations—to take advantage of the allocated rate. Toavoid this condition, each upstream node, ui providesnode n with an additional value: Bi is the maximumamount of data that an upstream node can send,given its energy limitations. An upstream node ui cancompute its value of Bi by solving the COP withoutthe downstream constraint (Equation 3)

ri ≤ Bi (∀ i ∈ [1, k]) (4)

The COP can be solved using a well-known pro-gressive filling algorithm [3]. The algorithm evenlyadds rate to each upstream link. As rates reach theirlimits, they are excluded from receiving additionalrate, and the process continues until either all peersare excluded or no residual energy is available. Thisalgorithm is fast, easy to implement, and amenable touse on low-power platforms, as we show in Section 6.

Es Energy required to sense a packets worth ofdata

Ed Energy to deliver a packet

Er Energy required to receive a packet

P Power available for sensing and routing

TABLE 2Variables that are estimated locally by n to solve the

COP

Objective

Lexicographically max.{r1F1

,r2F2

, . . . ,rn−1Fn−1

, rn

}(5)

ConstraintsEnergy conservationk∑

i=1

ri(Er + Ed) + rn(Es + Ed) < P (6)

Downstream

rn +

k∑i=1

ri ≤m∑j=1

Oj (7)

Upstreamri ≤ Bi (∀i ∈ [1, k]) (8)

Fig. 8. Energy allocation problem formulation solvedby node n. The goal is to estimate rn, the local sensingrate and ri, the rate at which n can route packets foreach of its neighbors ui

Similar optimization formulations have been usedto enforce max-min fairness in sensor networks andsupport perpetual operation [11], [22]. These worksassume that the routes are static and that networkconnectivity is predictable. However, in the targetenvironment that we consider in Tula, nodes aremobile, requiring dynamic routing. In addition, thenetwork is sparse and connectivity is unpredictable.Both these factors add significant complications to thestraightforward formulation, as we discuss in the nextsection.

5 INCORPORATING ROUTING

The simplified Tula rate allocation makes assumptionsthat do not hold in practice, especially when usingDTN routing to navigate a sparsely connected net-work. In any sparse network a node’s informationabout the network may be incomplete and is oftenoutdated. Consequently, each node’s local solution tothe COP is only a rough approximation of the globalsolution.

Page 7: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 7

(c)

(a) (b)

u1n

d1

(d)

f 1-f

Infrequent meetings

v2v1

u1 u2

n

d1 d2

n

n n'

u1

Fig. 9. Scenarios that complicate the simple Tulaallocation algorithm

Additionally, Tula must address the following spe-cific challenges: (i) Routing through multiple nodes:Nodes have multiple downstream paths, making itharder to determine the fraction of data to be for-warded through each neighbor, (ii) Replication: Mul-tiple copies of the packet exist in the network, makingthe straightforward delivery estimation difficult, (iii)Transitive routing: A node may be routing data forseveral of its upstream neighbors, making the fair-ness estimation complicated, and (iv) Routing throughan upstream neighbor: Although our formulation re-stricts routing only through downstream neighbors, insparse environments it is not always easy to differen-tiate between upstream and downstream neighbors.

Below, we describe how Tula handles these com-plications and present the resulting modified COP inFigure 8.

5.1 Routing through multiple nodes

DTN routing algorithms often rely on multiple down-stream nodes to route packets. When presentedwith multiple downstream options, as shown in Fig-ure 9(a), the routing algorithm on n determines whichpackets will be routed through d1 and which will gothrough d2. These routing decisions are limited, how-ever, by the routing rates allocated by the downstreamnodes. Therefore, the total data that n can route is nowthe sum of the rates allocated by each downstreamneighbor, d1, d2 . . . dm. We account for this limit in theCOP by replacing the maximum downstream rate,O, with the maximum downstream rates allocatedby m downstream neighbors, O1, O2 . . . Om. The newdownstream constraint incorporating these variablesis shown in Equation 7.

This change also impacts downstream nodes, il-lustrated in Figure 9(b), where n receives only afraction, f , of the packets routed by u1. The remaining

fraction, 1 − f , of the packets are routed throughanother node n′. Using the original COP, both n andn′ would allocate resources to u1 as though each wererouting all of its data—clearly defeating Tula’s effortsat fairness. Node n avoids this by allocating rate to u1,proportionally to f . To accomplish this, we introducea variable, Fi, which represents the fraction of all datarouted by a node ui through n. Equation 5 shows themodified objective function using Fi to allocate ratesfairly to fractional network flows. A node receivesFi values from its upstream neighbors, which keeptrack of these values by maintaining a limited routinghistory.

In addition to ensuring fairness, the Fi values alsoprovide a mechanism by which an upstream node’srouting protocol can express demand to a downstreamnode’s rate allocator. For example, if the routing pro-tocol on u1 diverts packets from a less promising n toa more promising n′, the fraction of data routed by u1

through n will decrease, signaling the rate allocator atn to reduce its allocation. Alternatively, if u1 wants toroute more packets through n, it will communicate anincreased F1 value to n. This signals n to increase itsallocation, so long as the increase does not violate thefairness model.

Another result of allowing multiple downstreamroutes is that allocation decisions at node, n, canindirectly effect nodes that are not on n’s upstreamor downstream path. Consequently, solving the opti-mization problem optimally would, in the worst case,require each node to have complete knowledge of thenetwork. Obtaining this global knowledge is costly instatic, fully connected networks, and impossible in asparse mobile network, like TurtleNet. Fortunately, inpractice, we have found that ignoring these indirecteffects does not noticeably degrade performance.

5.2 Replication

Network uncertainty can often be masked by repli-cating the same data over different paths. Replicationadds robustness to the network and has been shownto improve delivery rates in disruption-prone environ-ments [2], [27]. Replicating data, if not accounted for,will also unfairly skew rate allocations in much thesame way as routing through multiple downstreamnodes.

Consider the scenario illustrated in Figure 9(a). If nsends all of its data to d and a duplicate copy to d′, nmight be tempted to send Fn = 1.0 to both d1 and d2,since each downstream node routed all of its data. Inthis case, n receives twice its fair allocation.

Instead, a Tula node n incorporates the replicationrate in its estimation of the fraction Fn. To this end, ncomputes Fn as the fraction of its total network trans-missions including replicas. In the previous example,n sends Fn = 0.5 to both d1 and d2, assuming thatboth paths are favored equally.

Page 8: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 8

5.3 Transitive routing

In sparse mobile networks, a node can be severalhops away from the sink. In the TurtleNet testbed,for example, some devices were as much as 5 hopsaway from the base station. In order to deliver data,these nodes must route data transitively as shown inFigure 9(c). In the example, nodes v1 and v2 routethrough u1, while nodes u1 and u2 route throughn. Assuming energy and connectivity constraints areequal, and n only considers u1 and u2 when allocatingresources, n will assign similar rates to u1 and u2, eventhough u1 is routing 3 nodes’ data and u2 only routesfor itself.

Accounting for transitivity requires no change tothe COP. Rather the variable, Fi is extended to includeupstream traffic. Node u1 sends the total fraction oftraffic that it routes though n, including its own andall its upstream nodes. In the example, if v1 andv2 route all their data through u1, and u1 in turnroutes all its data through n, it will send F1 = 3 ton. The value Fi needs to only be communicated tothe immediate downstream neighbor. Nodes v1 andv2 communicate their respective values to u1, and u1

aggregates the values with its fraction to estimate F1.To sum up, the variable Fi represents the fraction

of total packets a node sends—including its own andothers’ packets and accounting for replication—to itsdownstream neighbor.

5.4 Routing through an upstream neighbor

Under most conditions, data packets are routed to-ward the sink through downstream nodes; however,at times it makes sense to route data to a nodethat is farther from the destination, as illustrated inFigure 9(d). In the example, nodes n and u1 meet eachother frequently, but each rarely meets a shared down-stream node d. According to the routing algorithm’sdistance metric, n is slightly closer to d than u1.Therefore, n is downstream from u1, and u1 will routeits data through n. Since u1 is nearly as likely to meet das n, node n can significantly increase the probabilityof delivering data in a timely manner by routingdata through u1 as well. Unfortunately, this is notpermitted by our current definition of upstream anddownstream nodes. We have observed this scenariooften in TurtleNet, and we expect it to occur in anynetwork with social groups and non-uniform mixing.

One solution is to relax our definition and allownodes to be both upstream and downstream peersof each other. This solution, however, suffers fromthe count-to-infinity problem, where a node unknow-ingly becomes its own downstream peer. This canbe solved by exchanging link information for allupstream paths, however, it significantly increasesTula’s complexity—requiring, in the worst case, thatall nodes maintain information about all other nodesin the network.

Therefore, we use a simpler heuristic in Tula, whichhas worked well in practice. We allow a node to onlyreplicate its own data to upstream peers, but disallowforwarding other nodes’ data. This simple heuristicavoids the count-to-infinity problem by ensuring thatdata is never routed back down the path from whichit came.

6 IMPLEMENTATION

We developed two implementations of Tula: aNesC [14] version that runs on a microcontrollerplatform and a trace-based simulator for repeatableexperimentation.

6.1 NesC implementationThe goal of the NesC implementation is two-fold.First, it demonstrates that Tula can be implementedin the memory and processor constrained microcon-troller platform. Second, it allows us to measure theenergy required for each component of Tula—sensing,routing data, solving the COP and exchanging metadata for the routing algorithm. We then instantiatethe simulator with real energy measurements. Weplan to deploy the full implementation of Tula in ourTurtleNet testbed.

The NesC implementation is a fully functioning im-plementation running on the ShockFish Tinynode [7].The implementation incorporates all of the designfeatures, including the energy/rate allocator, the Eonruntime platform and the Rapid DTN layer. We adaptthe Rapid implementation to run on a memory con-strained platform. Rapid normally exchanges meta-data about the delay of each packet. Instead, wereduce the meta-data and exchange only the per-nodedelay and meta-data about a short packet history. Werefer to this reduced version as RapidLite.

We implemented the allocator in 390 lines of NesCcode, and RapidLite in 1172 lines of NesC code. TheEon runtime computes the energy budget of a sensornode by keeping track of the harvested energy andthe energy spent for sensing and communication.

6.2 Trace-based simulatorSimulation based on real data collected in situ fromdeployed systems is the most practical method forconducting realistic, fair, and reproducible compar-isons between different approaches. Our simulatortakes mobility and harvested energy traces from avariety of sources, including traces from our TurtleNetdeployment and from UMass DieselNet [4].

The simulator periodically solves the Tula COP andperforms sensing and routing based on the rates setby the COP. Connection opportunities are simulatedaccording to the mobility traces. Nodes exchangesensed data and meta information during a connec-tion opportunity. Sensed data is routed based on theRapidLite algorithm. The simulator assigns energy to

Page 9: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 9

Harvested energy

Meeting times, Bandwidth

Meetings

A B 10:30A 200KBB D 11:30A 500KBE A 11:45A 230KB

...

Solar Energy

A 10:25A 1.8mJB 10:25A 0.2mJA 10:26A 1.2mJ

...

Simulator

Exchange inputs (Table 1) Solve COP and assign rates

Run RAPID, EonReconcile Energy

...

NesCImplementation

Measurements:Idle energy

Sensing energySend/Recv energy

Fig. 10. Mobile connectivity, and energy traces fromreal deployments and energy measurements from ourimplementation are used to replay previous deploy-ment scenarios using different approaches. This trace-based approach provides an evaluation of Tula that isboth realistic and reproducible.

Sensor Sense/Send ratio

GPS(Max) 2.0×104

GPS(Avg) 5.0×103

GPS(Min) 5.0×103

Accel. (ADXL330) 6.5×10−3

Mag.(HMC1053) 7.2×10−1

TABLE 3Energy to sense vs. send for common sensors, and

the XE1205 low-power radio

nodes according to a harvested energy trace. A nodeaccounts for energy consumption due to processing,sensing, and communication using measurements ob-tained from our implementation.

7 EVALUATION

Tula adapts sensing and routing rates to providemax-min fairness in the network. We compare theperformance of Tula with three different kinds ofapproaches: (i) Optimal, An optimal adaptive policybased on an oracle (ii) Static policies that set static sens-ing and routing rates, and (iii) Semi-adaptive policiesthat either adapt their sensing rate or routing rate, butnot both. Our evaluation compares these policies interms of network performance, energy management,and fairness.

7.1 MethodologyComparing perpetual systems running in the wild is achallenge—mobility, network conditions, and energyharvesting vary and are not reproducible—and thedisparity between real system behavior and simula-tion results is often unacceptably large. We provide arealistic and reproducible evaluation of Tula by takingmobility, connectivity, and energy traces from real

deployments and then replaying those deploymentsusing different approaches.

The example in Figure 10 illustrates this approach.Meetings (e.g. node A meets node B at 10:30AM andnetwork conditions allow the pair to send up to 200KBof data) and energy harvesting events (e.g. node Aharvests 1.8mJ of energy at 10:25AM) are simulated asthey occurred during the original deployment. Usingour NesC implementation we measure the idle energycosts of the system as well as the energy required forindividual tasks, like sensing, receiving, and transmit-ting data. These measurements allow us to accuratelyaccount for energy consumption throughout the sim-ulation.

In each experiment, nodes have 512kB of storage,250mAhr batteries, and the Tula allocator is run ev-ery 2 hours. Running the allocator more frequentlyincreases Tula’s responsiveness, but increases process-ing overhead. In our experience, varying this intervalresults in only minor changes to system performance.To evaluate alternate allocation policies, we replacethe Tula COP with an allocator that enforces a static,semi-adaptive or optimal allocation policy.

Nodes can be configured to use a variety of sensors.Table 3 shows the ratio of sensing and sending costof three different sensors: GPS, Accelerometer, andMagnetometer.

Trace collectionWe conduct the trace-based simulations using threetraces: TurtleNet, DieselNet and a synthetic meshtrace. The TurtleNet traces include 45 days of datafrom 17 tracking devices deployed on Gopher tor-toises. The data contains measured solar energy (fromfrequent on-board voltage and current samples), con-nection opportunities, and the bandwidth availableduring a connection opportunity. More informationabout the design of the TurtleNet hardware can befound in [26]. The DieselNet traces are publicly avail-able traces from the UMass DieselNet vehicular net-work [4] collected from 20 mobile nodes for 55 daysin 2008.

The DieselNet bus traces do not contain energyharvesting information, since, like most vehicularnetworks, the devices rely on the vehicle’s batteryfor power. Energy harvesting in vehicular networkspromises to reduce installation time and cost. In orderto add solar information to the bus data, we combinehistorical solar energy traces [1] with the DieselNetbus schedules to estimate the energy harvested ateach bus over time. This approach is reasonable onlybecause buses are either parked in the garage (i.e., noenergy harvested) or driving on open roads with aclear view of the sky.

Finally, in order to assess the applicability of Tulato fixed networks, we also simulate a synthetic meshnetwork configuration of 16 nodes arranged in a 4x4grid topology. We randomly assign energy traces fromour TurtleNet data to these mesh nodes.

Page 10: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 10

Avg

. Dea

dtim

e (%

)

05

1015

2025

Avg

. Was

ted

Ene

rgy

(kJ/

day)

0.0

0.1

0.2

0.3

0.4

0.5

Avg

. Del

iver

y R

ate

(pkt

s/da

y)

050

100

150

200

Conservative Median Mean Tula Optimal

Fig. 11. Comparison of three static allocation policies, Tula and Optimal, using the TurtleNet mobility trace. Thepolicies are compared across three metrics: battery dead time, energy wasted since the battery was full andcould not charge, and average delivery rate. Tula avoids dead time and wasted energy successfully, and deliverswithin 92% of the oracle-based optimal policy.

5 50 500 5000

02

46

812

Sense/Send ratio

Avg

. Dea

dtim

e (%

)

5 50 500 5000

020

040

060

0

Sense/Send ratio

Avg

. Was

ted

Ene

rgy

(J/d

ay)

5 50 500 5000

020

0060

00

Sense/Send ratioA

vg. D

eliv

ery

Rat

e (p

kts/

day)

Adaptive Sensing Adaptive Routing Tula Optimal

Fig. 12. Comparison of two semi-adaptive allocation polices, Tula and Optimal. The comparison is performed fordifferent sensor applications with varying sensing-to-routing energy ratio using TurtleNet mobility data.

Optimal rate allocation using an oracleIn order to determine the optimal max-min fair rateassignment, we formulate each experimental scenarioas a set of linear programs, which can be solved usinga general purpose LP solver.

Our LP formulation extends the approach used byFan et al. [11] and we have added support for tempo-ral changes in network connectivity, rate adaptation,and storage limitations, by breaking up the linearprogram into discrete one-hour time segments. Weuse the connectivity information from each trace todetermine how much data can be sent between eachpair of nodes during that hour.

The solver has complete knowledge of future en-ergy harvesting and connectivity for each time seg-ment. The solution from the LP formulation is themaximum max-min fair rate assignment that does notsacrifice node lifetimes or data deliveries. Achievingthis rate, in practice, is not feasible since it has globalknowledge of the entire network and does not haveto exchange information between peers. Furthermore,all connections in a segment are assumed to occursimultaneously, while in reality connections may beordered. As a result, this “optimal” result will some-

times be more optimistic than the true optimal. Still,it provides a useful reference by which to measuresystem performance.

7.2 Network Performance

While Tula adaptively allocates energy for both sens-ing and routing, there is a wide range of alternativeapproaches that could be employed. In this section,we compare the performance of Tula with two classesof allocation policies: Static and Semi-adaptive.

Static rate allocation policiesOne challenge in designing a static allocation policyis determining what sensing rate should be assignedto the nodes. Due to variation in energy harvesting,setting one sensing rate across all nodes will resultin some nodes dying and other nodes having surplusenergy. To conduct a fair comparison, we examine arange of behaviors. First, using the oracle-based opti-mal allocator, we determine the optimal rate allocationfor each node. In a real deployment scenario, rateassignments would have to be made based on thesystem designer’s best guess.

Page 11: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 11

We examine the performance of three static rates:conservative, a rate that is sustainable by 90% of thenodes in the network; the median rate, sustainable by50% of the nodes; and the mean rate, which can beachieved by only 25% of the nodes. For this experi-ment, nodes are configured to use the GPS sensor, andsimulated using the TurtleNet mobility trace.

The results of this comparison are shown in Fig-ure 11. We compare the performance with respectto three metrics: aggregate dead time, total wastedenergy, and delivery rate. The aggregate dead timeis the total time that nodes in the network have noenergy. The total wasted energy is the energy thatcould not be stored due to limited battery size, evenwhen solar energy was available for harvesting. Deadtime is typically a result of over-utilizing energy,while wasted energy is a result of under-utilizing theavailable energy.

Using the conservative rate, nodes are dead only3% of the time, however, on average nearly 500Jof energy—enough to collect nearly 200 sensorreadings—are wasted daily per node. The mean ratewastes much less energy, but on average nodes aredead for 25% of the time. The median rate providesan unsatisfying tradeoff between the two extremes,resulting in mediocre performance across all threemetrics.

The results show that in a network with widevariations in energy availability and connectivity, astatic scheme will perform poorly, regardless of therate that is assigned. In contrast, by adapting per-node sensing rates, Tula is able to completely avoiddead time and wasted energy, resulting in 11% moredata collected than using the mean rate, and within92% of the optimal result.

Semi-adaptive rate allocation policies

Next, we make a similar comparison with two alter-nate adaptive policies—a policy that adapts only therouting rate or only the sensing rates.

In the adaptive sensing policy, routing decisions aremade without any energy restrictions. However, thenode adapts its sensing rates according to the remain-ing energy. In contrast, in the adaptive routing policy,sensing rates are fixed, and routing decisions aremade adaptively using the energy that remains aftersensing. The adaptive routing policy requires a fixedsensing rate, and we set the rate to the conservative ratedescribed previously. Recall that the conservative rateis a rate sustainable by 90% of the nodes. Setting thestatic rate to other values results in similar or worsetrade-offs.

Unlike the static allocation, the performance of par-tially adaptive rate allocation depends on the sensor.For example, if nodes only obtain accelerometer read-ings, the sensing cost is low enough that adapting thesensing rate does not provide benefits. Alternatively, ifthe sensor application obtains GPS readings, adapting

5 10 50 500 5000

0.0

0.2

0.4

0.6

0.8

1.0

Sensor/Send Ratio

Del

iver

y R

ate

(% o

f Opt

imal

) ●

● Tula−TurtleNetTula−MeshTula−DieselNet

Fig. 13. Delivery rate of Tula normalized to the opti-mal delivery rate over three networks configurations:TurtleNet, a static 4x4 grid mesh network, and theDieselNet vehicular traces.

the sensing rate is important to ensure that a nodedoes not exhaust its battery. Accordingly, we comparethe performance of the different allocation policies fora range of sensors with varying energy requirements(as shown in Table 3).

Figure 12, illustrates the chief shortcoming of thesepartially adaptive approaches—their ability to adaptto changes is limited by the consumption of the statictasks. As the energy for sensing increases (left of thegraph), the dead time when using adaptive routingpolicy increases from 0 to 12%. On the other hand,when the energy for sensing is low, more packets aresensed and routed. As a result, the average dead timeof the adaptive sensing policy increases to 10%.

Both the adaptive sensing and adaptive routingpolicy waste between 200 J to 600 J (equivalent to15–40 GPS readings or ) daily depending on thepolicy and the sensor. In contrast, Tula optimizes bothsensing and routing and the policy incurs no deadtime and no wasted energy. In terms of delivery rate,Tula collects on average 30-50% more data than boththe semi-adaptive techniques.

Network performance over DieselNet and MeshFigure 13 shows the delivery rates achieved byTula for three different network configurations. Tulaachieves a delivery rate of within 75% of the Optimalpolicy over TurtleNet and DieselNet, even without fu-ture knowledge of the harvest energy or node meetingschedules.

On a static mesh network, Tula is able to sense anddeliver data within 85% of the Optimal policy for arange of sensors, showing that Tula can adapt well todifferent topologies. More importantly, in the absenceof mobility, the rate set by the Tula allocation policyconverges close to the optimal rate.

7.3 FairnessThe objective of the Tula allocation policy is to setrates such that data is sensed and delivered to the sink

Page 12: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 12

Avg

. Sen

sing

Rat

e (r

eadi

ngs/

day)

010

030

050

070

0OptimalTula

Fig. 14. TurtleNet traces: Averageper-node delivery rate

Avg

. Sen

sing

Rat

e (r

eadi

ngs/

day)

010

030

050

070

0

OptimalTula

Fig. 15. DieselNet traces: Averageper-node delivery rate

Avg

. Sen

sing

Rat

e (r

eadi

ngs/

day)

050

100

150

200

250

OptimalTula

Fig. 16. Mesh: Average per-nodedelivery rate

at a max-min fair rate. In this section, we evaluate thefairness of Tula using two metrics. We compare theper-node delivery rate of Tula with Optimal. Recallthat the optimal oracle-based rate allocator is alsodesigned to set max-min fair rates.

Figures 14, 15 and 16 show the per-node deliveryrate of Tula compared to optimal for the three net-work configurations: TurtleNet, DieselNet and Mesh,respectively. For all three networks, the per-nodedelivery rate of Tula is close to the optimal per-node rate. For example, in TurtleNet, nearly all ofthe nodes achieve a delivery rate within 75% of theoptimal. Similar performance is seen for both Meshand DieselNet.

7.4 OverheadFinally, we quantify the overhead of Tula using mea-surements from our TinyNode-based implementation.Current draw is measured using a NI-PCI 6251 DAQ,measuring the voltage drop across a low-tolerancecurrent-sense resistor. Simultaneously measuring thevoltage across the device’s battery allows us to com-pute the energy consumption. The measurements areshown in Table 4.

Apart from the core sensing and networking tasks,energy is incurred when periodically solving the TulaCOP and computing the device’s energy budget.However, both tasks only consume roughly as muchenergy as transmitting 2-3 packets over the device’sradio. In addition to energy costs, our implementationof Tula requires 1.5kB of RAM and 22kB of additionalprogram space in addition to the space requirementsof the Eon runtime system. These size requirementsare easily met by nearly all current microcontroller-based platforms.

8 RELATED WORKTula builds on a large body of previous work inseveral fields: challenged networking, rate allocationand fairness, mobile sensor networks, and adaptation.In many ways this synthesis is too large to cover here,so we provide the most relevant work.

8.1 Mobile sensor deploymentsPrevious mobile sensor deployments have sharedmany of the same goals as Tula. ZebraNet [32], for

Tula Energy/Time Overhead

Operation Energy Time

Solve COP 0.9− 2.3mJ 0.5− 1.35sCompute Energy Budget 1.4− 1.8mJ 0.8− 1.0ms

Memory Overhead

RAM overhead 1.5kBAdditional code size 22kB

TABLE 4Measurements of Tula overhead.

example, initially explored the use of in-situ sensingdevices for wildlife tracking. These first devices werelarge (>0.5kg) and masked energy variations withlarge batteries and solar panels—too large for mostanimals to carry; however, they set the stage for futuremobile sensing systems, like Tula.

Of course, mobile sensor systems are not limitedto wildlife tracking. The Pothole Patrol [9] projectused mobile sensors in vehicles to provide cities withvaluable road-quality information. Like most vehicle-based networks, these devices receive power from thevehicles.

8.2 Low power sensor networksEnergy scarcity is a first class design concern for wire-less sensor networks. Low-power hardware platformswith energy harvesting support [20], [26] as wellas algorithms for estimating and predicting energyharvesting and consumption [8], [17], [26] are cru-cial components of all perpetual systems. Addition-ally, a variety of energy-aware networking techniqueshave been proposed for use with low-power sensors,including energy-aware clustering [31], aggregation,and traffic shaping to extend device lifetimes [25].However, previous research on energy aware sensornetworks has either focused on static network topolo-gies, ignored networking concerns, or neglected thechallenges of perpetual operation.

8.3 Challenged networksA wealth of previous research has focused on buildingdisruption tolerant networks with sparse connectivity.

Page 13: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 13

Research in this area has provided a range of proto-cols [2], [16], [21], [27], which opportunistically for-ward and replicate packets to mobile peers. Most DTNsolutions have targeted vehicular [5] and personaldevice networks [15]. Whether tapping into a vehicle’sbattery or relying on user-facilitated recharges, previ-ous solutions assume a steady and unlimited energysupply, and neglect the challenge of energy scarcitywhich is central to any untethered system.

8.4 Fair network rate allocation

A variety of fairness policies have been proposed [3],[18], along with many techniques for enforcing thosepolicies in wireless networks [13], [29], [33]. Of theseapproaches, the most closely related work, which hasbeen concurrently with Tula, provides both central-ized and distributed algorithms for enforcing max-min fairness in networks that have rechargeable sen-sors [11], [22]. However, the authors assume that theroutes in the network are static, and that the energyprofile of a node is known in advance. Tula enforcesmax-min fairness in networks with unpredictable net-work connectivity and dynamically changing energyconstraints.

A myriad of techniques aim to improve perfor-mance and enable new applications by providing ad-ditional coordination across traditionally independentnetwork layers [6], [28], when legacy abstractions failto meet the needs of emerging systems and environ-ments. Tula is also a cross-layer approach providinga tight link between the application and network inorder to address the combined challenges of mobility,heterogeneity and perpetual operation.

9 DISCUSSION

The model we have used for sensing and routing isintentionally simple and straightforward and workswell in practice. However, Tula also has its limita-tions. For example, while using local optimizationsto approximate a global optimization improves effi-ciency, it may introduce additional prediction error.Also, using a fixed history window to determine up-stream/downstream relationships means that patho-logical mobility patterns exist, under which Tula’srouting will perform poorly (e.g., oscillation).

While potentially problematic for Tula, these con-ditions have not occurred in any of the real mobilitypatterns we have studied. We expect Tula’s patholog-ical cases to be rare in real mobile networks, althoughthe solutions may provide academically interestingopportunities for future work.

We have also focused on a simple network model—each node streams raw data to a sink. There areother models used by sensor network applicationsincluding aggregation [10] and querying [24]. Incor-porating these alternative models in Tula, requiresthe system to estimate the effect that aggregation

and querying have on network load. In the case ofquerying, network load and the flexibility with whichTula can adapt would also depend on the nature of thequery itself. Supporting aggregation, would requirethe system to estimate the amount of compressionachieved by aggregation at each hop in the network.

Finally, this work could also be extended to supportother notions of fairness (e.g., proportional fairness)and to allow timely or critical data to be given higherpriority in the network. We expect these extensions tobe straightforward; however, in the interest of brevityand simplicity, we leave them as future work.

10 CONCLUSIONSIn this paper, we present Tula, a system which bal-ances sensing with packet delivery for energy har-vesting mobile sensor networks. Tula represents afirst step in managing the resources of constrainednodes, balancing sensing and communication, whilemaintaining a cooperative system for delivering data.Our evaluation of Tula, using mobility and energytraces from our TurtleNet deployment, shows thatTula collects and delivers data within 75% of anoptimal oracular policy. In addition, we have shownthat Tula successfully enforces a max-min fairnesspolicy and is suitable for use on low power sensingplatforms. As the scale and complexity of mobilesensing systems increases, proven techniques for es-timating, predicting, and efficiently sharing networkand energy resources will continue to be an essentialkey to their success.

REFERENCES[1] http://weather.cs.umass.edu/.[2] A. Balasubramanian, B. N. Levine, and A. Venkataramani.

DTN Routing as a Resource Allocation Problem. In Proc. ACMSigcomm, August 2007.

[3] J.-Y. Boudec. Rate adaptation, congestion control and fairness:A tutorial, 2000.

[4] J. Burgess, B. Gallagher, D. Jensen, and B. N. Levine. MaxProp:Routing for Vehicle-Based Disruption-Tolerant Networks. InIEEE INFOCOM, April 2006.

[5] J. Burgess, B. Gallagher, D. Jensen, and B. N. Levine. MaxProp:Routing for Vehicle-Based Disruption-Tolerant Networks. InProc. IEEE INFOCOM, April 2006.

[6] M. Conti, G. Maselli, G. Turi, and S. Giordano. Cross-layeringin mobile ad hoc network design. Computer, 37(2):48–51, Feb2004.

[7] H. Dubois-Ferriere, R. Meier, L. Fabre, and P. Metrailler.TinyNode: A comprehensive platform for wireless sensornetwork applications. In Proceedings of the fifth internationalconference on Information processing in sensor networks (Poster),pages 358–365, Nashville, TN, USA, April 2006.

[8] A. Dunkels, F. Osterlind, N. Tsiftes, and Z. He. Software-basedon-line energy estimation for sensor nodes. In EmNets ’07:Proceedings of the 4th workshop on Embedded networked sensors,pages 28–32, New York, NY, USA, 2007. ACM.

[9] J. Eriksson, L. Girod, B. Hull, R. Newton, S. Madden, andH. Balakrishnan. The pothole patrol: using a mobile sen-sor network for road surface monitoring. In MobiSys ’08:Proceeding of the 6th international conference on Mobile systems,applications, and services, pages 29–39, New York, NY, USA,2008. ACM.

[10] K.-W. Fan, S. Liu, and P. Sinha. Structure-free data aggregationin sensor networks. IEEE Transactions on Mobile Computing,6(8):929–942, 2007.

Page 14: IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL., NO

IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. ?, NO. ?, ? 20XX 14

[11] K.-W. Fan, Z. Zheng, and P. Sinha. Steady and fair rate alloca-tion for rechargeable sensors in perpetual sensor networks. InSenSys ’08: Proceedings of the 6th ACM conference on Embeddednetwork sensor systems, pages 239–252, New York, NY, USA,2008. ACM.

[12] A. Fioravanti-Score, S. V. Mitchell, and J. M. Williamson. Useof Satellite Telemetry Technology to Enhance Research andEducation in the Protection of Loggerhead Sea Turtles. In 19thAnnual Symposium on Sea Turtle Biology and Conservation, 1999.

[13] V. Gambiroza, B. Sadeghi, and E. W. Knightly. End-to-endperformance and fairness in multihop wireless backhaul net-works. In MobiCom ’04: Proceedings of the 10th annual inter-national conference on Mobile computing and networking, pages287–301, New York, NY, USA, 2004. ACM.

[14] D. Gay, P. Levis, R. von Behren, M. Welsh, E. Brewer, andD. Culler. The nesc language: A holistic approach to net-worked embedded systems. In PLDI ’03: Proceedings of theACM SIGPLAN 2003 conference on Programming language designand implementation, pages 1–11, New York, NY, USA, 2003.ACM.

[15] P. Hui, A. Chaintreau, J. Scott, R. Gass, J. Crowcroft, andC. Diot. Pocket Switched Networks and Human Mobility inConference Environments. In Proc. ACM Workshop on Delay-Tolerant Networking, pages 244–251, Aug. 2005.

[16] S. Jain, K. Fall, and R. Patra. Routing in a Delay TolerantNetwork. In Proc. ACM SIGCOMM, pages 145–158, August2004.

[17] A. Kansal, J. Hsu, S. Zahedi, and M. B. Srivastava. Powermanagement in energy harvesting sensor networks. ACMTransactions on Embedded Computing Systems, May 2006.

[18] H. Kushner and P. Whiting. Convergence of proportional-fairsharing algorithms under general conditions. Wireless Commu-nications, IEEE Transactions on, 3(4):1250–1259, July 2004.

[19] A. Lachenmann, P. J. Marron, D. Minder, and K. Rothermel.Meeting lifetime goals with energy levels. In Proc. of the 5thACM Conference on Embedded Networked Sensor Systems, pages131–144, 2007.

[20] K. Lin, J. Hsu, S. Zahedi, D. C. Lee, J. Friedman, A. Kansal,V. Raghunathan, and M. B. Srivastava. Heliomote: Enablinglong-lived sensor networks through solar energy harvesting.In Proceedings of ACM Sensys, November 2005.

[21] A. Lindgren, A. Doria, and O. Scheln. Probabilistic Routingin Intermittently Connected Networks. In Proc. Workshop onService Assurance with Partial and Intermittent Resources, August2004.

[22] R.-S. Liu, P. Sinha, and C. E. Koksal. Joint energy managementand resource allocation in rechargeable sensor networks. InProceedings of IEEE Infocom, 2010.

[23] K. Lorincz, B.-r. Chen, J. Waterman, G. Werner-Allen, andM. Welsh. Resource aware programming in the pixie os. InSenSys ’08: Proceedings of the 6th ACM conference on Embeddednetwork sensor systems, pages 211–224, New York, NY, USA,2008. ACM.

[24] S. R. Madden, M. J. Franklin, J. M. Hellerstein, and W. Hong.Tinydb: an acquisitional query processing system for sensornetworks. ACM Trans. Database Syst., 30(1):122–173, 2005.

[25] C. Schurgers and M. Srivastava. Energy efficient routing inwireless sensor networks. Military Communications Conference,2001. MILCOM 2001. Communications for Network-Centric Op-erations: Creating the Information Force. IEEE, 1:357–361 vol.1,2001.

[26] J. Sorber, A. Kostadinov, M. Garber, M. Brennan, M. D. Corner,and E. D. Berger. Eon: A Language and Runtime System forPerpetual Systems. In Proc. ACM SenSys, Sydney, Australia,November 2007.

[27] T. Spyropoulos, K. Psounis, and C. Raghavendra. Sprayand Wait: An Efficient Routing Scheme for IntermittentlyConnected Mobile Networks. In Proc. ACM Workshop on Delay-Tolerant Networking, pages 252–259, Aug. 2005.

[28] V. Srivastava and M. Motani. Cross-layer design: a survey andthe road ahead. Communications Magazine, IEEE, 43(12):112–119, Dec. 2005.

[29] L. Tassiulas and S. Sarkar. Maxmin fair scheduling in wirelessnetworks. INFOCOM 2002. Twenty-First Annual Joint Conferenceof the IEEE Computer and Communications Societies. Proceedings.IEEE, 2:763–772 vol.2, 2002.

[30] A. Vahdat and D. Becker. Epidemic Routing for Partially-

Connected Ad Hoc Networks. Technical Report CS-2000-06,Duke University, July 2000.

[31] O. Younis and S. Fahmy. HEED: A hybrid, energy-efficient,distributed clustering approach for ad-hoc sensor networks.IEEE Transactions on Mobile Computing, 4(4), October 2004.

[32] P. Zhang, C. M. Sadler, S. A. Lyon, and M. Martonosi. Hard-ware design experiences in zebranet. In SenSys ’04: Proceedingsof the 2nd international conference on Embedded networked sensorsystems, pages 227–238, New York, NY, USA, 2004. ACM.

[33] J. Zhu, K.-L. Hung, and B. Bensaou. Tradeoff between networklifetime and fair rate allocation in wireless sensor networkswith multi-path routing. In MSWiM ’06: Proceedings of the 9thACM international symposium on Modeling analysis and simula-tion of wireless and mobile systems, pages 301–308, New York,NY, USA, 2006. ACM.

Jacob Sorber is a Postdoctoral associate at Dartmouth College,with research interests broadly in mobile computing, embedded sys-tems, and sensor networks. Current research topics include efficientand usable security and privacy for mobile health-related systems(mHealth), and energy management for RFID-scale devices. Heearned a Ph.D. from the University of Massachusetts Amherst.

Aruna Balasubramanian is a Postdoctoral fellow at University ofWashington. She received her Ph.D from the University of Mas-sachusetts Amherst. Her research interests are broadly in systemsand networking, with a focus on mobile and vehicular networks. Hercurrent research work involves architecting energy efficient and highperformance mobile systems with an emphasis on end-user drivendesign and evaluation. Her postdoctoral work is supported by theComputing Innovation Fellowship. She is the recipient of a MicrosoftGraduate Research Fellowship.

Mark D. Corner is an expert in the areas of mobile device operatingsystems, power management, networking, file systems, and security.He is a tenured Associate Professor in the Department of ComputerScience at the University of Massachusetts Amherst and has beenthere since 2003. At UMass he teaches the Operating Systemscourse and a course on Usability. He has authored dozens of publica-tions on mobile systems, holds two patents, and has been awardedmillions of dollars in federal and industrial research grants. He wasselected for DARPA’s prestigious Computer Science Study Panel in2009. He was the recipient of an NSF CAREER award in 2005, BestPaper awards at USENIX FAST 2007, ACM Multimedia 2005, andACM Mobicom 2002. Prof. Corner also serves on the editorial boardof IEEE Pervasive. He has chaired the mobile systems community’spremier conference MobiSys, and the premier workshop, HotMobile.Mark Corner is a co-founder, and VP of Engineering at Fiksu Inc.,which provides advertising optimization for mobile applications.

Joshua Ennen is a visiting professor of Biology at Maryville College,Maryville, Tennessee. He previously work as a Wildlife Biologist forU. S. Geological Survey. He obtained his Bachelor of Arts fromMaryville College, Maryville, Tennessee. His Master of Science wasobtained from Austin Peay State University, Clarksville, Tennessee.He received his Doctor of Philosophy from the University of South-ern Mississippi. His research has focused mainly on reproductiveecology, movement behavior, systematics, and population geneticsof chelonians including Gopherus polyphemus, Gopherus agassizii,Graptemys gibbonsi, Graptemys pearlensis, Graptemys flavimacu-lata, Graptemys oculifera, and Sternotherus minor peltifer.

Carl Qualls is an Associate Professor in the Department of Biolog-ical Sciences at the University of Southern Mississippi. He earnedhis PhD from the University of Sydney. Carl is a herpetologist withresearch interests in ecology, conservation biology, and life historyevolution. Current research topics include reproductive ecology,phylogeography, and conservation genetics of the gopher tortoise,and a study of translocation methods for endangered frogs. Papersfrom these and other projects appear in numerous biological andherpetological journals.