1

Real-Time Wireless Sensor-Actuator Networks forIndustrial Cyber-Physical Systems

Chenyang Lu, Senior Member, IEEE, Abusayeed Saifullah, Member, IEEE, Bo Li, Mo Sha, Member, IEEE,Humberto Gonzalez, Member, IEEE, Dolvara Gunatilaka, Chengjie Wu, Lanshun Nie, and Yixin Chen, Senior

Member, IEEE.

Abstract—With recent adoption of wireless sensor-actuatornetworks (WSANs) in industrial automation, industrial wirelesscontrol systems have emerged as a frontier of cyber-physicalsystems. Despite their success in industrial monitoring appli-cations, existing WSAN technologies face significant challengesin supporting control systems due to their lack of real-timeperformance and dynamic wireless conditions in industrial plants.This article reviews a series of recent advances in real-timeWSANs for industrial control systems: (1) real-time schedulingalgorithms and analyses for WSANs; (2) implementation andexperimentation of industrial WSAN protocols; (3) cyber-physicalco-design of wireless control systems that integrate wireless andcontrol designs; (4) a wireless cyber-physical simulator for co-design and evaluation of wireless control systems. This articleconcludes by highlighting research directions in industrial cyber-physical systems.

Index Terms—Wireless Sensor-Actuator Networks, Cyber-Physical Systems, Real-Time Systems.

I. INTRODUCTION

Wireless sensor-actuator networks (WSANs) are gainingrapid adoption in process industries due to their advantage inlowering deployment effort in harsh industrial environments.Industrial standard organizations such as ISA [1], HART [2],WINA [3] and ZigBee [4] have been actively pushing theapplication of wireless technologies in industrial automationand manufacturing. While early success of industrial WSANsfocused on monitoring applications, there is significant valuein exploring WSANs for process control applications to takefull advantage of wireless technology in industrial plants.

However, wireless control systems face unique challengesthat distinguish them from traditional control systems. First,it is challenging to meet the stringent latency requirements offeedback control in WSANs. IEEE 802.15.4 radios commonly

Chenyang Lu, Bo Li, Dolvara Gunatilaka, and Yixin Chen are with theDepartment of Computer Science & Engineering, Washington University in St.Louis, Saint Louis, MO, 63130 USA. (e-mail: lu@wustl.edu, b.li@wustl.edu,dgunatilaka@wustl.edu, ychen25@wustl.edu).

Abusayeed Saifullah is with Department of Computer Science, MissouriUniversity of Science & Technology, Rolla, MO, 65409 USA. (e-mail:saifullaha@mst.edu)

Mo Sha is with Department of Computer Science, State Univer-sity of New York at Binghamton, Binghamton, NY 13902 USA. (e-mail:msha@binghamton.edu)

Humberto Gonzalez is with Department of Electrical & System Engineer-ing, Washington University in St. Louis, Saint Louis, MO, 63130 USA. (e-mail: hgonzale@wustl.edu)

Chengjie Wu is with Yahoo!, 701 First Ave. Sunnyvale, CA, 94089 USA.(e-mail: wuchjie@gmail.com)

Lanshun Nie is with Department of Computer Science & Technology,Harbin Institute of Technology, Harbin, Heilongjiang, 150001 China. (e-mail:nls@hit.edu.cn)

Manuscript received Jun 05, 2015; revised xxx xxx, 2015.

used in WSANs have a maximum bandwidth of only 250kbps. Multi-hop communication over mesh networks furtherincreases communication delays. Furthermore, channel con-ditions in industrial environments can change dynamicallydue to external interference, moving obstacles and weatherconditions. As a result, WSANs demand real-time schedulingalgorithms and analysis techniques that are both effectiveand efficient. In contrast to wired industrial networks (e.g.,Control Area networks [5]) with well established real-timescheduling techniques, there have been limited results on real-time scheduling for WSANs. The lack of analytical methodsfor achieving real-time performance in WSANs hinders theiradoption in control systems. Wireless control demands a newreal-time scheduling theory for WSANs [6].

Second, design of wireless control systems must deal withinterdependencies between control and communication. Forexample, while it is well known in digital control that a lowsampling rate usually degrades control performance [7], a highsampling rate may increase resource contention in bandwidth-constrained WSANs leading to long communication delays,which again may lead to degraded control performance [8].The coupling between wireless communication and controltherefore motivates a cyber-physical co-design approach thatintegrates wireless networks and control designs.

This article provides a review of recent advances in real-time WSANs for industrial control systems, an increasinglyimportant class of cyber-physical systems (CPS) in the dawnof Industrial Internet [9] and Industry 4.0 [10].

Section II describes real-time scheduling for industrialWSANs. We first introduce WirelessHART, an industrial stan-dard for wireless sensing and actuation. We then present asuite of real-time scheduling algorithms, delay analyses andprotocol implementation to achieve real-time performance inWirelessHART networks. Section III introduces our recentefforts to design industrial wireless control systems following acyber-physical co-design approach. We first study the problemof selecting sampling rates to optimize control performance.We then investigate how to incorporate emergency alarms inwireless control systems. To support the design and evaluationof wireless control systems, we introduce the Wireless Cyber-Physical Simulator (WCPS) for holistic studies of wirelesscontrol systems. We conclude this article by highlightingpromising research directions.

II. REAL-TIME INDUSTRIAL WSANS

We first provide an overview of the WirelessHART standardand then describe our implementation and experimentation of a

2

WirelessHART protocol stack. Finally, we summarize our real-time scheduling algorithms and analyses for WirelessHART.

A. WirelessHART

A wireless control system comprises feedback control loopsconnecting sensors, controllers and actuators through a wire-less mesh network. Sensors measure variables of the plant andsend the measurements to a controller over the wireless meshnetwork. The controller then sends control commands to theactuators in order to control the physical processes. Industryplants pose harsh environments for wireless communicationdue to significant channel noise, physical obstacles, multi-pathfading, and interference from coexisting wireless devices [11].These harsh and dynamic environments make it difficult forWSANs to meet the stringent reliability and real-time require-ments of industrial control applications.

Fig. 1. Architecture of a WirelessHART network [2].

There exist multiple industrial WSAN standards, e.g., Wire-lessHART [2], ISA100 [1], WINA [3] and ZigBee [4]. In thisarticle we will focus on the WirelessHART standard, althoughthe design principles may be extended for the other standardtechnologies. WirelessHART devices are now adopted world-wide for industrial process management and control [12], [13].Unique features that make WirelessHART particularly suitablefor industrial process control are described below.

A WirelessHART network is managed by a centralizednetwork manager. The network manager is responsible for themanagement, scheduling, creating the routes, and optimizingthe network. Network devices include a Gateway, a set of fielddevices, and several access points as shown in Fig. 1. Thenetwork manager and the controllers are installed or connectedto the Gateway. The field devices are wireless sensors andactuators. Each field device is equipped with a half-duplexomnidirectional radio transceiver compliant with the IEEE802.15.4 standard. Multiple access points are wired to theGateway to provide redundant paths between the wirelessnetwork and the Gateway.

A WirelessHART network adopts various mechanisms toensure reliable communication in unreliable industrial envi-ronments. Time is globally synchronized and slotted using abuilt-in time synchronization protocol. Each time slot is 10ms, which is sufficient to send or receive one packet and an

corresponding acknowledgement. Transmissions are scheduledbased on a multi-channel TDMA (Time Division MultipleAccess) protocol. A time slot can be either dedicated or shared.In a dedicated slot, only one sender is allowed to transmit tothe receiver. In a shared slot, more than one sender can attemptto transmit to the same receiver. Since shared slots are assignedto multiple senders, collisions may occur within a shared slot.To reduce collisions, senders adopt Carrier Sense MultipleAccess with Collision Avoidance (CSMA/CA) to contend ina shared slot. CSMA/CA allows multiple nodes to share acommon channel through carrier sensing, where nodes attemptto avoid collisions by transmitting only when the channel isidle.

The network uses 16 channels defined in IEEE 802.15.4,and adopts channel hopping in every time slot. Channelhopping provides frequency diversity to mitigate interferencesand reduce multi-path fading effects. Any excessively noisychannel is blacklisted.

A WirelessHART network supports two types of routingapproaches: source routing and graph routing. Source routingprovides a single directed path for routing from a sourceto a destination device. Graph routing involves a routinggraph consisting of a directed list of paths between the twodevices, and is adopted for enhanced end-to-end reliability.In graph routing, packets from field devices are routed tothe Gateway through the uplink graph. To every field device,there is a downlink graph from the Gateway. The end-to-end communication between a source (sensor) and destination(actuator) happens in two phases. In the sensing phase, on onepath from the source to the Gateway in the uplink graph, thescheduler allocates a dedicated slot for each device startingfrom the source, followed by allocating a second dedicatedslot on the same path to handle a retransmission. Then, tooffset failure of both transmissions along a primary link, thescheduler allocates a third shared slot on a separate pathto handle another retry. Then, in the control phase, usingthe same method, the dedicated links and shared links arescheduled in the downlink graph of the destination.

B. Implementation and Experimentation

To study and evaluate WSAN protocols, we have developedan experimental WSAN testbed [14]. The system comprisesa network manager on a server and a network protocol stackimplementation on TinyOS 2.1.2 [15] and TelosB motes [16].Each mote is equipped with a TI MSP430 microcontrollerand a TI CC2420 radio compatible with the IEEE 802.15.4standard. Figure 2 shows the motes deployment in our campusbuildings. In an experiment, our motes can be designatedas access points and field devices, which form a multi-hopwireless mesh network running WSAN protocols. To supportexperimentation and measurements, all motes in our systemare physically connected to a wired backplane network that canbe used for managing wireless experiments and measurementswithout interfering with wireless communication.

Our network manager implements a route generator anda schedule generator. The route generator is responsible forgenerating source routes or graph routes based on the collected

3

Fig. 2. TelosB motes deployment in Bryan Hall and Jolley Hall of WashingtonUniversity in St.Louis [17].

Fig. 3. Time frame format of RT-MAC.

network topology, while the schedule generator is responsiblefor generating packet transmission schedules. Our protocolstack adopts the CC2420x radio driver [18] as the radio core,which is responsible for transmitting and receiving packets.A multi-channel TDMA MAC protocol, RT-MAC, is imple-mented on top of the radio core. As shown in Fig. 3, RT-MACdivides time into 10 ms slots based on the WirelessHARTstandard. The clocks of all the motes in the network aresynchronized using Flooding Time Synchronization Protocol(FTSP) [19] during a Sync window. The network then trans-mits packets based on recurring superframes (transmissionschedules). A 2 ms guard time is reserved in the beginningof each slot to accommodate the clock synchronization errorand channel switching delay. Both dedicated and shared slotsspecified in the WirelessHART standard are supported by RT-MAC. Only one sender is allowed to transmit and the packettransmission occurs immediately after the guard time in adedicated slot, while more than one sender can contend forthe channel in a shared slot.

We have performed a series of experiments on our WSANtestbed [14]. For instance, we have conducted a comparativestudy of the two alternative routing approaches adopted byWirelessHART, namely source routing and graph routing, andinvestigated the tradeoff among reliability, latency, and energyconsumption under the different routing approaches.

We first run our experiments in a clean environment wherewe blacklist the four 802.15.4 channels overlapping withour campus Wi-Fi network and run the experiments on theremaining 802.15.4 channels. We then repeat our experimentsin a noisy environment where we configure the network touse channels overlapping with our campus Wi-Fi network andin a stress-testing environment where we generate controlledWi-Fi interference. In our study, we use reliable links withPRR higher than 80%, where PRR stands for Packet ReceptionRatio, i.e., the fraction of transmissions that are successfulover a wireless link. We set up 8 data flows, and run our

TABLE IMINIMUM, MEDIAN, AND MAXIMUM PDR AMONG ALL FLOWS.

Environment Source Routing Graph RoutingMin Med Max Min Med Max

Clean 0.97 0.99 1 0.95 1 1Noisy 0.65 0.82 0.92 0.79 0.95 1

Stress Testing 0.29 0.70 0.87 0.54 0.85 0.99

experiments long enough such that each flow delivers at least500 packets from its source to its destination. We measure thepacket delivery rate (PDR) of a flow, defined as the percentageof packets that are successfully delivered to their destination.Table I shows the minimum, median and maximum PDRamong all the flows in the clean, noisy and stress-testingenvironments. Graph routing improves the median PDR by15.9%, and 21.4% compared to source routing in noisy andstress testing. Moreover, graph routing drastically improves theminimum PDR, with 35.5% and 63.5% increase in noisy andstress testing environments. This result shows graph routingcan effectively enhance the predictability of real-time perfor-mance, especially under significant interference. However, ourstudy also shows that route diversity incurs costs for latencyand energy consumption, with graph routing suffering from anaverage of 80% increase in end-to-end latency and consumingan average of 130% more energy over source routing.

Our study concludes that it is important to employ graphrouting algorithms specifically designed to optimize latencyand energy efficiency. This problem was recently explored in[20], which proposed a set of real-time and energy-efficientrouting algorithms for WirelessHART networks. Both sourcerouting and graph routing have been implemented in ourWSAN testbed and the Wireless Cyber-physical Simulator(WCPS) (to be discussed in III.B), which enable systematiccomparison between the alternative routing approaches.

C. Real-Time Scheduling

To save cost and enhance flexibility, it is desirable tosupport many control loops in a same network. The stabilityand control performance of these systems heavily dependson real-time communication over the shared wireless net-work. Feedback control loops in a WSAN therefore imposestrict requirements on reliability and real-time guarantees inwireless communication in order to avoid plant shutdownsand/or accidents. For example, in oil refineries, spilling ofoil tanks has to be avoided by controlling oil level in real-time. However, industrial plants pose a harsh environmentfor wireless communication due to unpredictable channelconditions, limited bandwidth, physical obstacles, multi-pathfading, and interference from coexisting wireless devices [11].With the adoption of industrial wireless standards such asWirelessHART [2], process monitoring and control applica-tions have seen the feasibility of achieving reliability and real-time wireless communication through spatial and spectrumdiversity. Real-time communication in these wireless networkspose new and important challenges. Unlike wired networks,there have been limited results on real-time scheduling theoryfor wireless networks. Real-time transmission scheduling and

4

analysis for WSANs require new methodologies to deal withunique wireless characteristics.

1) Overview of State of the Art: Real-time communica-tion in wireless networks has been explored in many earlierefforts [21]–[32], [32]–[38]. The survey in [39] provides acomprehensive review of these works. However, these are notsuitable for industrial applications that usually need multi-channel communication, multi-path routing and real-time per-formance analysis results. Earlier works [40]–[44] on real-timeperformance analysis for wireless sensor networks focused ondata collection through a routing tree [40], [43].

Real-time WSAN for control purposes has been studiedfor single hop networks in [45]–[49]. For multihop wirelessnetworks, a mathematical framework to model and analyzeschedules for WirelessHART networks has been proposedin [50]. Real-time scheduling for WirelessHART networks hasreceived attention in recent works [51]–[57]. These schedulingpolicies can be broadly classified into two categories: fixedpriority scheduling and dynamic priority scheduling [58]. Ina fixed priority scheduling policy, each data flow has a fixedpriority, and every transmission has the priority of its flow.Due to its simplicity and low scheduling overhead, fixedpriority scheduling is a common class of real-time schedulingpolicies in practice [58]. On the other hand, dynamic priorityscheduling policy refers to the policy where there is no fixedpriority i.e., priorities of transmissions during the schedulingchange dynamically according to some chosen criterion. Whilesuch policies have higher scheduling overhead, they providebetter schedulability since priorities are changed dynamicallyto enhance the schedulability.

In the following we summarize our recent results on real-time scheduling for industrial WSANs under fixed priority anddynamic priority scheduling, respectively.

2) Fixed Priority Scheduling: Delay analysis can be usedto determine whether real-time data flows can meet theirdeadlines. When used offline, the analysis helps users designa WSAN to meet the real-time performance requirements.For example, we used schedulability analysis to select thesampling rates of wireless control loops to optimize controlperformance under communication deadlines [59]. In [54], weused schedulability analysis to assign priorities to real-timeflows in order to meet their deadlines. When used online,the network manager can adjust the workloads in response tonetwork dynamics. Industrial environments can cause frequentchanges in network topologies and channel conditions. If aWSAN can no longer guarantee the deadlines for all the flowsbased on delay analysis, the network manager may stop oradjust the data rates of non-critical flows in order to maintainreal-time guarantees to the remaining flows. The WirelessCyber-Physical Simulator (WCPS) introduced in Section III-Bcan be used to assess the pessimism of delay analysis andwireless control performance under realistic settings.

For fixed priority scheduling in WirelessHART networks,we proposed a suite of delay analyses for real-time flows [52],[53]. These analyses determine upper bounds on the end-to-end communication delays and provide sufficient conditionsfor schedulability. Two factors contribute to the communica-tion delay of a control loop. A lower priority flow can be

delayed by higher priority flows (a) due to channel contention(when all channels are assigned to transmissions of higherpriority flows in a time slot), and (b) due to transmissionconflicts (when a transmission of the flow and a transmis-sion of a higher priority flow involve a common node). Akey insight underlying this analysis is to map the real-timetransmission scheduling in WirelessHART networks to real-time multiprocessor scheduling. By incorporating the uniquecharacteristics of WirelessHART networks into the state-of-the-art worst case response time analysis for multiprocessorscheduling, the analysis can efficiently compute a safe andtight upper bound of the end-to-end delay of every flow.

Priority assignment is an important problem in fixed-priorityscheduling as it has a significant impact on the schedulabil-ity of real-time flows. An ideal priority assignment shouldnot only enable real-time flows to meet their deadlines, butalso work synergistically with real-time schedulability teststo support effective network capacity planning and efficientonline admission control and adaptation. In [54] we studiedoptimal and near optimal fixed priority assignment using localsearch. A salient feature of this local search approach isthat it exploits the delay bounds provided by a schedulabilityanalysis to efficiently search for feasible priority assignmentsby discarding unnecessary branches in the search space.

3) Dynamic Priority Scheduling: Dynamic priorityscheduling of transmissions was studied in [55], [57]for WirelessHART networks of with tree topologies. Ourwork in [51], [56] studied dynamic priority scheduling oftransmissions for general WirelessHART networks. The workin [51] has shown that the real-time transmission schedulingfor WirelessHART networks is NP-hard. We observed thattransmission conflicts play a major role in the communicationdelays and the schedulability of the control loops, whichmakes the traditional real-time scheduling policies suchas Least Laxity First (LLF) less effective. We proposean optimal local search scheduling algorithm that exploitsthe necessary condition to effectively discard infeasiblebranches in the search space. We also proposed a fasterheuristic called Conflict-aware Least Laxity First (C-LLF)for dynamic priority scheduling. The algorithm identifiesthe critical time windows in which too many conflictingtransmissions have to be scheduled, thereby determining thecriticality of each transmission. Criticality of a transmissionis quantified by its conflict-aware laxity, which is its laxityafter discarding the (estimated) time slots that can be wastedthrough waiting to avoid transmission conflict. Transmissionsexhibiting the lowest conflict-aware laxity are assessed tobe more critical. C-LLF gives the highest priority to thetransmissions exhibiting lower conflict-aware laxity. ThusC-LLF integrates LLF and the degree of conflicts associatedwith a transmission, and outperforms traditional real-timescheduling policies.

Our work in [56] has provided a schedulability analysisfor earliest deadline first (EDF), a common dynamic priorityscheduling in WirelessHART networks.

We evaluated our schedulability analyses against experimen-tal results on our WSAN testbed [17] as well as in simulations.In our evaluation, both fixed-priority scheduling [54] and

5

dynamic priority scheduling [51] policies outperformed thetraditional real-time scheduling policies by significant margins.All experimental results and simulations showed that ouranalyses provide safe upper bounds of communication delaysin the network and enable effective schedulability tests [53].For tighter upper bounds under graph routing, we establishedprobabilistic delay bounds in [6] that represent upper boundswith probability ≥ 0.90. The worst-case and probabilisticbounds can be used in different application scenarios depend-ing on the level of predictability required. Our analysis hencecan be used for an effective schedulability test and admissioncontrol of real-time flows in WirelessHART networks.

III. WIRELESS CONTROL CO-DESIGN

Due to limited resources in a WSAN shared by multiplecontrol loops, it is critical to optimize the overall control per-formance. However, in a wireless control system, the controlperformance not only depends on the control algorithms, butalso relies on real-time communication over the shared wire-less network. The coupling between real-time communicationand control requires a cyber-physical co-design approach fora holistic optimization of control performance. This sectionsummarizes our recent efforts on cyber-physical co-designfor industrial wireless control systems. We first study theproblem of selecting sampling rates to optimize the controlperformance of multiple feedback control loops sharing aWSAN. To support cyber-physical co-design of wireless con-trol systems, we present Wireless Cyber-Physical Simulator(WCPS), an integrated simulation environment for holisticstudies of wireless control systems. We then investigate howto incorporate emergency alarms in wireless process controlsystems. We wrap up this section with a case study of wirelessprocess control for coupled water tanks using both regular andemergency control loops.

A. Rate Selection

In control systems, the impacts of sampling rate have beenstudied for robot control [60], disturbance rejection [61], andvarious control performance indices [62]–[66]. The choice ofsampling rates needs to balance control and communication.While a high sampling rate is desirable from a pure digitalcontrol perspective, it also has the undesirable effect of heaviernetwork load and longer communication delay, which can alsodegrade the control performance.

The control performance can be quantified as a functionof sampling rates using the formulation proposed by Setoet al. [62], which characterizes the performance differencebetween the continuous-time control system and that of itsdigital implementation.

In [59], [67] we apply these ideas to optimize the perfor-mance of a set of feedback control loops over a WSAN, wherethe control cost of the i-th control loop at the sampling ratefi is approximated as αi e−βi fi , where αi, βi > 0 are themagnitude and decay rate coefficients, respectively. Given the

sampling rates of all of the control loops {f1, f2, . . . , fn}, thetotal control cost of the system is defined by:

n∑i=1

wi αi e−βi fi , (1)

where wi is the weight of the i-th control loop. We then usethe total control cost as the performance index of the systemand as the objective of an optimization problem to determinethe best sampling rates of a multi-loop control system sharinga WirelessHART network under stability and delay constraints.In particular, the objective is to minimize the cost in (1)subject to two constraints. First, the delay bound Ri of thei-th loop, which can be computed based on the sampling ratein [52], [53], must be smaller than its predetermined deadlineDi. Second, each control loop must have a sampling rate fiwithin its minimum and maximum rates, denoted fmin

i andfmaxi respectively, to ensure stability. Thus, we mathematically

formulate this optimization problem as follows:

min{fi}ni=1

n∑i=1

wi αi e−βi fi ,

subject to: Ri(f1, . . . , fn) ≤ Di, ∀i ∈ {1, . . . , n},fmini ≤ fi ≤ fmax

i , ∀i ∈ {1, . . . , n}.

(2)

The resulting constrained optimization problem is challeng-ing since the delay Ri is non-differentiable, non-convex, andnot in closed-form. We explored four methods to solve thisproblem: (a) a subgradient-descent algorithm, (b) a greedyheuristic, (c) a penalty method using simulated annealing (SA),and (d) a convex relaxation method based on a new delaybound that is convex and smooth but more pessimistic thanprevious analyses [52], [53] using a different approach. No-tably, the convex relaxation greatly simplifies the optimizationproblem, since the overall problem becomes convex and dif-ferentiable by simplifying the delay bound analysis.

In [59], we evaluate the methods through simulations basedon the topology of our WSAN testbed [17]. The results demon-strate that SA achieves the lowest control cost but requiresthe longest execution time. Both the greedy heuristic and thesubgradient method lead to high control costs because theoptimization problem is highly nonlinear with a large numberof local extrema. For example, for 30 control loops, the greedyheuristic incurs a control cost up to 2.67 times that of SA. Incontrast, the convex relaxation approach using an interior pointmethod incurs a control cost no more than 35% higher thanthat of SA, while incurring a much shorter execution time. Theconvex relaxation approach therefore achieves the desirablebalance between control performance and run-time efficiency.This result demonstrates the significant advantage of cyber-physical co-design that integrates control optimization andschedulability analysis in wireless control systems.

B. WCPS

Existing wireless control system research often relies on lab-scale equipment and simulations. However, lab-scale equip-ment usually suffers from limited physical size, which cannotcapture delays and data loss in real WSANs. Simulation tools

6

for control systems often lack realistic models of WSANsthat exhibit complex and stochastic behavior in real-worldenvironments. The lack of tools that can capture both the cyberwireless network and physical aspects of control systems hasbeen a hurdle to wireless control research.

Early experimental work on wireless control [68]–[73]usually relied on a lab testbed where wireless sensors arewithin a single hop and experience no data loss due to thephysical proximity of the devices. The challenge in realisticexperimentation with wireless control systems motivated thedevelopment of simulation tools for wireless control systems.For example, NCSWT [74] and Gisso [75] are two simulatorsdesigned for wireless control systems. Truetime [7] is a wellestablished control system simulator capable of holistic studiesof CPU scheduling, communication, and control algorithms.Unfortunately, none of these simulators provides a realisticwireless radio model or a state-of-the-art WSAN protocolstack.

Wireless Cyber-Physical Simulator (WCPS) [76] is de-signed to provide a holistic and realistic simulation of wirelesscontrol systems. WCPS employs a federated architecture thatintegrates (a) Simulink for simulating the physical systemdynamics and controllers, and (b) TOSSIM for simulatingWSANs. Simulink is commonly used by control engineersto design and study control systems, while TOSSIM [77]has been widely used in the sensor network community tosimulate WSANs based on wireless link models that havebeen validated in diverse real-world environments [78]. WCPSprovides an open-source middleware to orchestrate simulationsin Simulink and in TOSSIM.

WCPS 2.0 implements the WirelessHART network protocolstack at the routing and MAC layers [79]. To support Wire-lessHART we also extended TOSSIM to simulate wirelesscommunication over multiple channels. We have implementedboth Source Routing and Graph Routing as specified in theWirelessHART standard. To our knowledge, WCPS 2.0 is thefirst simulator that supports WirelessHART protocols based ona realistic wireless link model.

Thanks to WCPS we have been able to develop case studiesthat simulate wireless control systems for civil infrastructureand process plants. In [80] we simulated two wireless struc-tural control systems. Wireless structural control systems offeran attractive approach to protect civil infrastructures fromnatural hazards such as earthquakes and other natural disasters.In the first simulation we studied a benchmark building model,where the wireless traces were collected in a multi-storybuilding. In the second simulation we studied the structuralmodel of the Cape Girardeau bridge over the MississippiRiver, where the wireless traces were collected from a similarbridge in South Korea. These case studies shed light on thelimitations of traditional structural control approaches underrealistic wireless conditions. They further allowed us to vali-date the advantages of cyber-physical co-design algorithms forwireless communication protocols. Based on our experiencewith WCPS, we have recently enhanced the wireless buildingcontrol study, transforming it into a benchmark that can beused by the structural control community to explore andevaluate different wireless control approaches [81], allowing

practitioners to easily generate and configure realistic nui-sances such as network induced delay, data loss, measurementnoise, and control constraints.

C. Emergency Communication

In [79] we considered the case of WirelessHART controlnetworks comprised of (a) regular flows, which are periodi-cally generated and typically stabilize a desired part of a phys-ical system, and (b) emergency flows, which are infrequent andtypically signal that an unsafe situation is about to occur. Bothregular and emergency flows have predetermined deadlines totransmit a packet. Since emergency flows carry informationrelated to unsafe situations, these flows have a higher criticalitythan regular flows. Therefore, we proposed the followingapproach to transmit packets through the network: (a) inregular mode, i.e., when there is no emergency, all regularflows should meet their deadlines, and (b) in emergency mode,i.e., when an emergency occurs, all existing emergency flowsshould meet their deadlines, while regular flows are deliveredon a best-effort basis.

Periodic Scheduling (PS) is a baseline scheduling approachthat reserves periodic slots dedicated to emergency packets inthe transmission schedule. Emergency packets can be transmit-ted only in the reserved slots. When there is no emergency,the reserved slots are left unused. We take a two-level priorityassignment approach for scheduling, where we first scheduleemergency flows to meet their deadlines, and then we schedulethe regular flows using the remaining capacity. Within eachgroup, flows are scheduled using a rate monotonic policy. Thisbasic approach is simple in design, yet it wastes (a potentiallylarge amount of) network bandwidth, reserving time slots foremergency flows even when there is no emergency.

To avoid wasting resources when no emergencies occur, wedesigned a Slot Stealing (SS) method that allows emergencyflows to “steal” slots from regular schedules when emergenciesarise, removing the need to allocate exclusive time slots foremergency flows. The “stealing” process is implemented byadding a random backoff and a Clear Channel Assessment(CCA) to the transmission of all regular flow packets, andallowing emergency packets to be transmitted immediately atthe beginning of the time slot. Hence, if an emergency packetis available, then it will take the slot, and any regular flowpacket trying to use that slot will fail the CCA, forcing itto wait for the next available time slot or drop the packet. Byallowing the overlapping emergency and regular schedules, SSis able to schedule the same number of flows in a shorter timeframe when compared to PS, while ensuring timely deliveryof emergency packets in case of an emergency.

We also explored two alternative approaches to send emer-gency signals. For systems that need to periodically monitorand control the emergency state, an emergency flow is acti-vated whenever emergencies occur, and the emergency flowthen periodically generates data until the emergency is over.For systems that do not need to periodically monitor andcontrol the emergency state, the system can adopt an event-based approach to communicate the emergency alarms, i.e., anemergency sensor only sends one alarm-start packet and one

7

Pump

u

Tank 1

Tank 2

v1b d

v12

v2b

Basin

LH1

L1

LH2

L2

LHb

LLb

Valve

Lb

Fig. 4. Diagram of the coupled water tank system. The water levels ofTank 1, Tank 2, and Basin are denoted L1, L2, and Lb, respectively. Thewater emergency levels are denoted LH

1 , LH2 , LL

b , and LHb . The water flows

between tanks are denoted u, v12, v1b, and v2b. The state of the valve isdenoted d ∈ {0, 1}, where d = 1 if the valve is open.

alarm-stop packet at the beginning and end of the emergency,respectively. While the event-based communication results inthe same transmission schedule, it significantly reduces thenumber of regular transmissions that are dropped or delayedby emergency transmissions, leading to better control perfor-mance. In [79] simulation results showed the combinationof slot stealing and event-based emergency communicationproduced the best results in terms of emergency handling andcontrol performance.

D. Emergency Control Case Study

We tested our emergency communication protocol on a setof two identical water tank systems sharing a common wirelessnetwork simulated using WCPS 2.0. A diagram of each ofwater tank systems is shown in Fig. 4. Our choice is based onthe simple, yet representative, dynamics of water tanks, theirhybrid dynamical nature (since the evolution of the systemchanges when the water tanks are either full or empty), andmore importantly, its similarity to systems commonly used inindustrial applications. Also, we simulated two identical watertank systems sharing a wireless network to study the effect ofone system’s emergency over the second system’s regular flow.

Following the software architecture in WCPS 2.0, the sensordata generated by Simulink is fed into the WSAN simulatedusing TOSSIM. TOSSIM then returns the packets delayed ordropped according to the behavior of the network, which arethen fed to the controller implemented in Simulink. Controllercommands are then fed again into TOSSIM, which delays ordrops the packets and sends the outputs to the actuators in thewater tanks, closing the loop.

For this study we collect wireless traces from 21 nodes in aWSAN testbed [17]. We then use the wireless traces as inputsto the TOSSIM simulator to generate realistic wireless char-acteristics in a simulated WSAN. The route in the simulatedWSAN is 6 hops. We used further adjusted Received SignalStrength (RSS) to test the system performance with differentwireless signal strengths.

On the control side, we used a PID controller to regulatethe water level of Tank 2, denoted L2, by actuating the

Pump taking water from the Basin, as shown in Fig. 4. Ifno emergencies occur, then the Valve stays closed. We alsodefined a discrete emergency controller setting the values ofthe Pump and Valve, with the objective of avoiding waterspillage. Hence, if any of the emergency level sensors, denotedLH1 , LH2 , LLb , and LHb , detect a dangerous water level, then anemergency signal is sent and the controller switches from thePID controller to the discrete emergency controller, switchingback when the emergency is cleared. We defined a systemfailure using two principles: (a) if the system cannot stabilizeat a regular configuration (i.e., within non-emergency waterlevels) in a fixed 100 second interval, and (b) if any of thewater levels exceed the height of its associated water tank

Our simulations show that the combination of the slot-stealing and event-based emergency communication is highlyeffective in avoiding system failures, since it produces atighter schedule that results in a faster update frequency ofthe regular control flows. It also reduces communication loadand mitigates the impact of emergency communication onregular control flows. Our case study demonstrates that evenfor a 6-hop lossy wireless network (with 5.8% median packetloss), successful system control can still be achieved througha combination of regular and emergency control.

IV. RESEARCH DIRECTIONS

A. Enhancing Scalability

A major limitation of current industrial WSANs is theirlimited scalability due to the centralized network architecture.In WirelessHART, when a node or link fails, the centralizednetwork manager must generate a new global TDMA schedule,which requires the network manager to collect the currenttopology of the network, create a new schedule, and distributethe schedule among all field devices. As WSANs are subjectto frequent changes in channel quality and link condition,global changes to the network schedule create excessive com-munication overhead in a large network. As a result, whilethe centralized architecture has proven to be sufficient forsmall-scale installations, it can become a significant limitationas WSANs start to be deployed over large geographic areas(e.g., thousands of devices over an oil field). A key researchdirection is to make WSANs scalable while achieving end-to-end real-time communication.

A promising approach to enhance scalability is through ahierarchical network architecture, where a large WSAN isdivided into multiple subnetworks. Each subnetwork employsits own manager to manage local operations, and a globalmanager coordinates with the subnetwork managers to managethe entire network in a hierarchical fashion. An advantageof this architecture is that it can meet industrial needs forboth network visibility and scalability. As a submanager maydeal with the wireless dynamics within its subnetwork, thehierarchical architecture can scale effectively. A challengein designing a hierarchical architecture is to deal with theinterdependencies among the subnetworks that share the wire-less spectrum and need to support flows traversing multiplesubnetworks subject to end-to-end deadlines.

8

B. Exploring White Spaces

Another limiting factor of today’s WSANs stems fromthe short communication range of the IEEE 802.15.4 radiosadopted by industrial standards such as WirelessHART. Toovercome the short communication range, many WSANs formmulti-hop mesh networks resulting in long communicationdelays and limited scalability. An opportunity to enhance thescalability of real-time WSANs arises from the opening ofa new spectrum resulting from the transition to digital TVbroadcasting globally and freeing up the VHF/UHF spectrum.White spaces refer to the allocated but locally unused TV spec-trum. Since TV transmissions are in lower frequencies, whitespace transmissions have excellent propagation characteristicsover long distance. They can easily penetrate walls and otherobstacles, and hence hold enormous potential for industrialapplications that need real-time communication over largegeographic areas. Thanks to its long communication range andwall-penetration capability, a white space network will havesmall hop counts and will drastically reduce communicationdelays and protocol complexities.

While white spaces are mostly being tapped into for wirelessbroadband access to date, they also open up the opportunityto support highly scalable real-time industrial applicationsthat have been challenging under existing WSAN technolo-gies [82]. The characteristics and the application demandsin industrial sensing and actuation pose unique challenges inadopting white spaces for industrial WSANs. Instead of high-throughput traffic, a WSAN should exploit the wide spectrumin white spaces to support low-data-rate communication fromnumerous field devices. Furthermore, a WSAN over whitespaces must achieve high degrees of energy efficiency thatis comparable to today’s WSAN technologies. It also needs tohandle changes and variations in spectrum availability in whitespaces while maintaining desired real-time performance.

C. Cyber-Physical Co-Design

While earlier work has shown the promise of cyber-physicalco-design in wireless control systems through point designsand case studies, there exist opportunities to develop a theoryand practice of cyber-physical co-design through a broaderexploration of the interaction between the network protocolstack and control design. For example, if a controller employsa state observer to estimate system states based on intermittentobservations, the controller can tolerate a certain degree of dataloss from the sensors. A WSAN can exploit the controller’sresiliency against sensor data loss by reducing the routeredundancy from sensors to controllers, while dedicating morenetwork resources to enhance the reliability of communicationto actuators. That is, the allocation of network resourcesshould be dependent on the control design. Conversely, anactuator may buffer the control inputs from a model predictivecontroller and use previously received control inputs whenthe network fails to deliver the new control inputs. Whenthe wireless condition deteriorates causing more data loss, thecontroller may increase its sampling rates or control horizon toincrease its tolerance to data loss to the actuators. That is, thecontroller should adapt to network conditions. Note that the

adaptation of network resources and controller sampling ratescan be designed using approximated mathematical models, orusing distributed extensions of the data-driven methods in [83].Therefore, a wireless control system will need to co-designwireless networks and control in an interdependent fashion. Toestablish a unified cyber-physical co-design approach to indus-trial wireless control, it will be crucial to develop interfacesbetween the WSAN and the control components to maintainoptimal control performance by adapting to changing networkand physical dynamics in the wireless control systems. Weneed to further develop a suite of algorithms and analyticaltechniques to assure the safety of the control system underdynamic conditions. Finally, the research community willneed long-term collaboration among computer science, controltheory and domain experts for this inherently interdisciplinaryresearch.

D. Cyber-Physical Testbed

The advancement and adoption of new WSAN technologieshave been hindered by the lack of testbeds under realisticindustrial settings. While cyber-physical simulators such asWCPS are useful for studying wireless control systems, theyrequire realistic industrial plant models and wireless traces col-lected from real-world industrial settings. Despite best effortson modeling techniques, simulations cannot replace physicalcomponents and networks with complex and sometimes un-expected dynamics. Unfortunately, full-scale industrial plantsare often difficult to replicate for research due to space andsafety constraints. A step beyond simulations might be hybridtesting technology that combines physical components andsimulations for hardware-in-the-loop experiments. For exam-ple, a physical WSAN testbed may be carefully integratedwith simulations of an industrial plant to study the impactof real wireless dynamics on control systems. Conversely,an industrial plant may be integrated with a WSAN simu-lator to evaluate potential wireless control designs without aphysical WSAN deployment. This would require extendingtools such as WCPS to support hardware-in-the-loop testingwhere both the WSAN and the plant can be replaced byphysical implementations. Close partnership between industryand researchers will be critical to develop realistic cyber-physical testbeds for industrial wireless control systems.

V. CONCLUSION

Real-time WSANs are poised to play a key role in industrialautomation in the era of Industry 4.0 [10] and IndustrialInternet [9]. Recent research and industrial developments havedemonstrated the promise of supporting real-time communi-cation and control over WSANs. This article reviews recentadvances in two related fronts: (a) real-time scheduling andanalytical techniques for achieving real-time performance inindustrial WSANs and (b) cyber-physical co-design of wirelesscontrol systems. We highlight the significant challenges andopportunities in cyber-physical systems research that crosscutwireless and control domains. Interdisciplinary collaborationand partnership among wireless and control researchers andindustry communities will be crucial in realizing the vision ofwireless industrial control.

9

ACKNOWLEDGMENT

This work was supported, in part, by US National ScienceFoundation through grants 1320921 (NeTS), 1035773 (CPS),1017701 (NeTS) and 1144552 (NeTS) and National NaturalScience Foundation of China (NSFC) through grant 61273038.

REFERENCES

[1] ISA100. [Online]. Available: https://www.isa.org/isa100/[2] WirelessHART. [Online]. Available: http://en.hartcomm.org/[3] Wireless Industrial Networking Alliance. [Online]. Available: http:

//www.wina.org[4] ZigBee Alliance. [Online]. Available: http://www.zigbee.org[5] R. I. Davis, A. Burns, R. J. Bril, and J. J. Lukkien, “Controller area

network (can) schedulability analysis: Refuted, revisited and revised,”Real-Time Systems, vol. 35, no. 3, pp. 239–272, 2007.

[6] A. Saifullah, D. Gunatilaka, P. Tiwari, M. Sha, C. Lu, B. Li, C. Wu, andY. Chen, “Schedulability analysis under graph routing for wirelesshartnetworks,” in RTSS, 2015.

[7] A. Cervin, D. Henriksson, B. Lincoln, J. Eker, and K.-E. Arzen, “HowDoes Control Timing Affect Performance? Analysis and Simulation ofTiming Using Jitterbug and TrueTime,” IEEE Control Systems, vol. 23,no. 3, pp. 16–30, 2003.

[8] F.-L. Lian, J. Moyne, and D. Tilbury, “Network design consideration fordistributed control systems,” IEEE TCST, vol. 10, no. 2, pp. 297–307,2002.

[9] P. Evans and M. Annunziata. (November 26, 2012) Industrial Internet:Pushing the Boundaries of Minds and Machines, General Electric(GE). [Online]. Available: http://www.industrialinternetconsortium.org/white-papers.htm

[10] H. Kagermann, W. Wahlster, and J. Helbig, “Recommendations forimplementing the strategic initiative industrie 4.0: Final report of theindustrie 4.0 working group,” Tech. Rep., April 2013.

[11] J. Song, A. K. Mok, D. Chen, and M. Nixon, “Challenges of wirelesscontrol in process industry,” in CRTES, 2006.

[12] http://www.industry.usa.siemens.com/automation/us/en/process-instrumentation-and-analytics/process-instrumentation/brochures/Documents/PIBR-0A940-1014-one-stop-shop.pdf.

[13] http://en.hartcomm.org/hcp/tech/applications/applications successmitsubishi chemical.html.

[14] M. Sha, D. Gunatilaka, C. Wu, and C. Lu, “Implementation and exper-imentation of industrial wireless sensor-actuator network protocols,” inEWSN, 2015.

[15] TinyOS. [Online]. Available: http://www.tinyos.net/[16] Memsic. TelosB Mote Platform. [Online]. Available: http://www.

memsic.com/userfiles/files/Datasheets/WSN/telosb datasheet.pdf[17] The WUSTL Wireless Sensor Network Testbed. [Online]. Available:

http://wsn.cse.wustl.edu/index.php?title=Testbed[18] CC2420x Radio Driver. [Online]. Available: https://github.com/tinyos/

tinyos-main/tree/master/tos/chips/cc2420x[19] M. Maroti, B. Kusy, G. Simon, and A. Ledeczi, “The flooding time

synchronization protocol,” in Proceedings of the 2nd InternationalConference on Embedded Networked Sensor Systems, 2004, pp. 39–49.

[20] C. Wu, “Real-time and Energy-efficient Routing for Industrial WirelessSensor Actuator Networks,” Ph.D. Thesis, Washington University in St.Louis, 2014.

[21] H. Li, P. Shenoy, and K. Ramamritham, “Scheduling messages withdeadlines in multi-hop real-time sensor networks,” in RTAS, 2005.

[22] X. Wang, X. Wang, X. Fu, G. Xing, and N. Jha, “Flow-based real-timecommunication in multichannel wireless sensor networks,” in EWSN,2009.

[23] V. Kanodia, C. Li, A. Sabharwal, B. Sadeghi, and E. Knightly, “Dis-tributed multi-hop scheduling and medium access with delay andthroughput constraints,” in MobiCom, 2001.

[24] C. Lu, B. M. Blum, T. F. Abdelzaher, J. A. Stankovic, and T. He, “RAP:A real-time communication architecture for large-scale wireless sensornetworks,” in RTAS, 2002.

[25] K. Karenos, V. Kalogeraki, and S. V. Krishnamurthy, “A rate controlframework for supporting multiple classes of traffic in sensor networks,”in RTSS, 2005.

[26] K. Karenos and V. Kalogeraki, “Real-time traffic management in sensornetworks,” in RTSS, 2006, pp. 422–434.

[27] T. He, B. M. Blum, Q. Cao, J. A. Stankovic, S. H. Son, and T. F.Abdelzaher, “Robust and timely communication over highly dynamicsensor networks,” Real-Time Systems, vol. 37, no. 3, pp. 261–289, 2007.

[28] G.-S. Ahn, A. T. Campbell, A. Veres, and L.-H. Sun, “SWAN: Servicedifferentiation in stateless wireless ad hoc networks,” in INFOCOM,vol. 2, 2002, pp. 457–466.

[29] T. He, J. A. Stankovic, C. Lu, and T. Abdelzaher, “SPEED: A statelessprotocol for real-time communication in sensor networks,” in ICDCS,2003.

[30] E. Felemban, C.-G. Lee, and E. Ekici, “MMSPEED: multipath Multi-SPEED protocol for QoS guarantee of reliability and timeliness inwireless sensor networks,” IEEE TMC, vol. 5, no. 6, pp. 738–754, 2006.

[31] B. D. Bui, R. Pellizzoni, M. Caccamo, C. F. Cheah, and A. Tzakis, “Softreal-time chains for multi-hop wireless ad-hoc networks,” in RTAS, 2007.

[32] Y. Gu, T. He, M. Lin, and J. Xu, “Spatiotemporal delay control forlow-duty-cycle sensor networks,” in RTSS, 2009.

[33] A. Koubaa, M. Alves, and E. Tovar, “i-GAME: An implicit GTSallocation mechanism in IEEE 802.15.4 for time-sensitive wirelesssensor networks,” in ECRTS, 2006.

[34] M. Caccamo, L. Y. Zhang, L. Sha, and G. Buttazzo, “An implicitprioritized access protocol for wireless sensor networks,” in RTSS, 2002.

[35] K. Liu, N. Abu-Ghazaleh, and K.-D. Kang, “JiTS: Just-in-time schedul-ing for real-time sensor data dissemination,” in PerCom, 2006.

[36] B. Li, D. Wang, F. Wang, and Y. Q. Ni, “High quality sensor placementfor shm systems: refocusing on application demands,” in INFOCOM,2010.

[37] R. Mangharam, A. Rowe, R. Rajkumar, and R. Suzuki, “Voice oversensor networks,” in RTSS, 2006.

[38] T. Facchinetti, L. Almeida, G. C. Buttazzo, and C. Marchini, “Real-timeresource reservation protocol for wireless mobile ad hoc networks,” inRTSS, 2004.

[39] J. A. Stankovic, T. F. Abdelzaher, C. Lu, L. Sha, and J. C. Hou, “Real-time communication and coordination in embedded sensor networks,”Proceedings of the IEEE, vol. 91, no. 7, pp. 1002–1022, 2003.

[40] O. Chipara, C. Lu, and G.-C. Roman, “Real-time query scheduling forwireless sensor networks,” in RTSS, 2007.

[41] T. F. Abdelzaher, S. Prabh, and R. Kiran, “On real-time capacity limitsof multihop wireless sensor networks,” in RTSS, 2004.

[42] P. Jurcık, R. Severino, A. Koubaa, M. Alves, and E. Tovar, “Real-time communications over cluster-tree sensor networks with mobile sinkbehaviour,” in RTCSA, 2008.

[43] J. B. Schmitt and U. Roedig, “Sensor network calculus — a frameworkfor worst case analysis,” in DCOSS, 2005.

[44] N. Pereira, B. Andersson, E. Tovar, and A. Rowe, “Static-priorityscheduling over wireless networks with multiple broadcast domains,”in RTSS, 2007.

[45] H.-J. Korber, H. Wattar, and G. Scholl, “Modular wireless real-timesensor/actuator network for factory automation applications,” IEEE TII,vol. 3, no. 2, pp. 111–119, 2007.

[46] X. Liu and A. Goldsmith, “Cross-layer design of distributed control overwireless network,” in Advances in Control, Communication Networks,and Transportation Systems, ser. Systems and Control: Foundations &Applications, E. H. Abed, Ed. Springer, 2005, ch. 8, pp. 111–136.

[47] ——, “Wireless network design for distributed control,” in CDC, 2004.[48] L. Xiao, M. Johansson, H. Hindi, S. Boyd, and A. Goldsmith, “Joint op-

timization of wireless communication and networked control systems,”in Switching and Learning in Feedback Systems, R. Murray-Smith andR. Shorten, Eds. Springer, 2005, pp. 248–272.

[49] T. W. Carley, M. A. Ba, R. Barua, and D. B. Stewart, “Contention-free periodic message scheduler medium access control in wirelesssensor/actuator networks.” in RTSS, 2003.

[50] R. Alur, A. D’Innocenzo, K. H. Johansson, G. J. Pappas, and G. Weiss,“Modeling and analysis of multi-hop control networks,” in RTAS, 2009.

[51] A. Saifullah, Y. Xu, C. Lu, and Y. Chen, “Real-time scheduling forWirelessHART networks,” in RTSS, 2010.

[52] ——, “End-to-end delay analysis for fixed priority scheduling in wire-lesshart networks,” in RTAS, 2011.

[53] ——, “End-to-end communication delay analysis in industrial wirelessnetworks,” IEEE TC, vol. 64, no. 5, pp. 1361–1374, 2013.

[54] ——, “Priority assignment for real-time flows in WirelessHART net-works,” in ECRTS, 2011.

[55] P. Soldati, H. Zhang, and M. Johansson, “Deadline-constrained trans-mission scheduling and data evacuation in wirelesshart networks,” inECC, 2009.

[56] C. Wu, M. Sha, D. Gunatilaka, A. Saifullah, C. Lu, and Y. Chen,“Analysis of EDF scheduling for wireless sensor-actuator networks,” inIWQoS, 2014.

[57] H. Zhang, F. Osterlind, P. Soldati, T. Voigt, and M. Johansson, “Rapidconvergecast on commodity hardware: Performance limits and optimalpolicies,” in SECON, 2010.

10

[58] G. C. Butazzo, Hard Real-Time Computing Systems. Springer, 2005,2nd edition.

[59] A. Saifullah, C. Wu, P. B. Tiwari, Y. Xu, Y. Fu, C. Lu, and Y. Chen,“Near optimal rate selection for wireless control systems,” ACM TECS,vol. 13, no. 4s, 2014.

[60] P. Khosla, “Choosing sampling rates for robot control,” in ICRA, vol. 4,1987, pp. 169–174.

[61] H. Hirata and J. Powell, “Sample rate effects on disturbance rejectionfor digital control systems,” in ACC, 1990, pp. 1137–1145.

[62] D. Seto, J. P. Lehoczky, L. Sha, and K. G. Shin, “Trade-off analysis ofreal-time control performance and schedulability,” Real-Time Systems,vol. 21, pp. 199–217, 2001.

[63] A. Horch and A. J. Isaksson, “Assessment of the sampling rate in controlsystems,” Control Engineering Practice, vol. 9, no. 5, p. 533?544.

[64] E. Bini and A. Cervin, “Delay-aware period assignment in controlsystems,” in RTSS, 2008.

[65] D. Henriksson and A. Cervin, “Optimal on-line sampling period assign-ment for real-time control tasks based on plant state information,” inCDC-ECC, 2005, pp. 4469–4474.

[66] J. Bai, E. P. Eyisi, F. Qiu, Y. Xue, and X. D. Koutsoukos, “Optimalcross-layer design of sampling rate adaptation and network schedulingfor wireless networked control systems,” in ICCPS, 2012.

[67] A. Saifullah, C. Wu, P. B. Tiwari, Y. Xu, Y. Fu, C. Lu, and Y. Chen,“Near optimal rate selection for wireless control systems,” in RTAS,2012.

[68] Y. Wang, A. R. Swartz, J. P. Lynch, K. H. Law, and C.-H. Loh, “Perfor-mance evaluation of decentralized wireless sensing and control in civilstructures,” in Proceedings of SPIE, Nondestructive Characterization forComposite Materials, Aerospace Engineering, Civil Infrastructure, andHomeland Security, vol. 6531, 2007.

[69] Y. Wang and K. H. Law, “Structural control with multi-subnet wirelesssensing feedback: experimental validation of time-delayed decentralizedH∞ control design,” Advances in Structural Engineering, vol. 14, no. 1,pp. 25–39, 2011.

[70] J. P. Lynch, Y. Wang, A. R. Swartz, K. Lu, and C. Loh, “Implemen-tation of a closed-loop structural control system using wireless sensornetworks,” Structural Control and Health Monitoring, vol. 15, no. 4, pp.518–539, 2008.

[71] P. Tabuada, “Event-triggered real-time scheduling of stabilizing controltasks,” IEEE TAC, vol. 52, no. 9, pp. 1680–1685, 2007.

[72] M. Pajic, S. Sundaram, J. Le Ny, G. J. Pappas, and R. Mangharam,“Closing the loop: A simple distributed method for control over wirelessnetworks,” in IPSN, 2012.

[73] A. Bemporad, S. Di Cairano, E. Henriksson, and K. H. Johansson,“Hybrid model predictive control based on wireless sensor feedback:An experimental study,” International Journal of Robust and NonlinearControl, vol. 20, no. 2, pp. 209–225, 2010.

[74] E. Eyisi, J. Bai, D. Riley, J. Weng, W. Yan, Y. Xue, X. Koutsoukos, andJ. Sztipanovits, “Ncswt: An integrated modeling and simulation tool fornetworked control systems,” Simulation Modelling Practice and Theory,vol. 27, pp. 90–111, 2012.

[75] B. Aminian, J. Araujo, M. Johansson, and K. H. Johansson, “GISOO:a virtual testbed for wireless cyber-physical systems,” in IECON, 2013.

[76] Wireless Cyber Physical Simulator. [Online]. Available: http://wcps.cse.wustl.edu

[77] P. Levis, N. Lee, M. Welsh, and D. Culler, “TOSSIM: Accurate andscalable simulation of entire TinyOS applications,” in Sensys, 2003.

[78] H. Lee, A. Cerpa, and P. Levis, “Improving wireless simulation throughnoise modeling,” in IPSN, 2007.

[79] B. Li, L. Nie, C. Wu, H. Gonzalez, and C. Lu, “Incorporating emergencyalarms in reliable wireless process control,” in ICCPS, 2015.

[80] B. Li, Z. Sun, K. Mechitov, G. Hackmann, C. Lu, S. J. Dyke, G. Agha,and B. F. Spencer Jr, “Realistic case studies of wireless structuralcontrol,” in ICCPS, 2013.

[81] Z. Sun, B. Li, S. J. Dyke, C. Lu, and L. Linderman, “Benchmark problemin active structural control with wireless sensor network,” StructuralControl and Health Monitoring, 2015.

[82] A. Saifullah, C. Lu, J. Liu, R. Chandra, and S. Sankar, “Challenge:Wireless sensor networking over white spaces,” Microsoft Research,Tech. Rep. MSR-TR-2014-6, 2014. [Online]. Available: http://research.microsoft.com/pubs/208183/whitespace vision.pdf

[83] S. Yin, H. Luo, and S. X. Ding, “Real-Time Implementation of Fault-Tolerant Control Systems With Performance Optimization,” IEEE TII,vol. 61, no. 5, pp. 2402–2411, 2014.

Chenyang Lu is the Fullgraf Professor in the De-partment of Computer Science and Engineering atWashington University in St. Louis. His researchinterests include real-time systems, wireless sensornetworks, cyber-physical systems and Internet ofThings. He is Editor-in-Chief of ACM Transactionson Sensor Networks, Area Editor of IEEE Internetof Things Journal and Associate Editor of the newACM Transactions on Cyber-Physical Systems andReal-Time Systems. He also chaired premier confer-ences such as IEEE Real-Time Systems Symposium

(RTSS), ACM/IEEE International Conference on Cyber-Physical Systems(ICCPS) and ACM Conference on Embedded Networked Sensor Systems(SenSys). He is the author and co-author of over 150 research papers withover 14,000 citations and an h-index of 52. He received the Ph.D. degreefrom University of Virginia in 2001, the M.S. degree from Chinese Academyof Sciences in 1997, and the B.S. degree from University of Science andTechnology of China in 1995.

Abusayeed Saifullah is an assistant professor inComputer Science at Missouri University of Scienceand Technology. He received his PhD in 2014 fromWashington University in St Louis. His researchprimarily concerns cyber-physical systems with con-tributions spanning real-time systems, embeddedsystems, wireless sensor networks, and parallel anddistributed computing. He received the Turner Dis-sertation Award 2014 of Washington University, theBest Paper Award at RTSS’14, and the Best StudentPaper Award at RTSS’11.

Bo Li is a PhD candidate in Department of Com-puter Science & Engineering at Washington Univer-sity in St. Louis. He received his Bachelor and Mas-ter degrees in Electrical Engineering from HarbinInstitute of Technology, China, in 2006 and 2008,respectively. Between 2008 and 2010, Bo was aresearch assistant at the Hong Kong PolytechnicUniversity. His research interests span broadly inwireless system research, e.g., Wireless ProcessControl, Wireless Structural Health Monitoring &Control, and Optimization.

Mo Sha is an Assistant Professor in the Departmentof Computer Science at Binghamton University-The State University of New York. He received hisPh.D. degree in Computer Science from WashingtonUniversity in St. Louis in 2014. Prior to his PhD,he received a M.Phil. degree from City Universityof Hong Kong in 2009 and a B.Eng. degree fromBeihang University in 2007. His research primarilyconcerns cyber-physical systems, wireless sensornetworks, embedded systems, as well as their appli-cations to smart energy, smart home, wireless health,

and industrial process automation.

11

Humberto Gonzalez (S’03-M’12) received the B.S.and M.S. degrees in electrical engineering fromUniversidad de Chile, Santiago, Chile, in 2005,and the Ph.D. degree in electrical engineering andcomputer sciences from the University of Californiaat Berkeley, CA, USA, in 2012. He is currently anAssistant Professor in the Department of Electrical& Systems Engineering at Washington Universityin St. Louis. His research focuses on the theoryand implementation of control algorithms for cyber-physical systems, with an emphasis on optimal con-

trol and numerical algorithms.

Dolvara Gunatilaka is a PhD student in the Depart-ment of Computer Science and Engineering at Wash-ington University in St. Louis. She received her M.S.degree in Computer Science from the same schoolin 2013. Prior to her Master degree, she receivedher B.S. Degree in Information and CommunicationTechnology from Mahidol University, Thailand in2010. Her research focuses on industrial wirelesssensor-actuator networks.

Chengjie Wu is a software engineer at Yahoo!. Hereceived the Ph.D. degree in Computer Science fromWashington University in St. Louis in 2014, the M.S.and B.S. degrees from Tsinghua University in 2008and 2006, respectively. He was a recipient of theMcDonnell International Scholarship and EmersonCorporate Fellowship awarded by Washington Uni-versity in St. Louis. He received the Best PaperNomination at the RTAS’12. His research interestsinclude real-time systems, cyber-physical systems,wireless sensor networks, and Internet of Things.

Lanshun Nie received his B.S., M.S. and Ph.D.degrees from the Harbin Institute of Technology,China. He was a visiting scholar of the CPS labora-tory at Washington University in St. Louis. Lanshunis currently an Associate Professor of ComputerScience at the School of Computer Science andTechnology, Harbin Institute of Technology, China.His research interests involve Cyber-Physical Sys-tems and Wireless Sensor Networks.

Yixin Chen is an Associate Professor of ComputerScience at the Washington University in St. Louis.His research interests include data mining, machinelearning, artificial intelligence, and optimization. Hereceived the Best Student Paper Runner-up Awardat the KDD’14, and Best Paper Awards at AAAI’10and ICTAI’05. He has won First Prizes in theInternational Planning Competitions (2004 & 2006).He received an Early Career Principal InvestigatorAward from the Department of Energy (2006) and aMicrosoft Research New Faculty Fellowship (2007).

He is an Associate Editor for ACM Transactions of Intelligent Systemsand Technology and serves on the Editorial Board of Journal of ArtificialIntelligence Research.