A Context-Driven IoT Middleware Architecture Jagannathan Venkatesh, Christine Chan, Alper Sinan Akyurek and Tajana Simunic Rosing

University of California, San Diego

Abstract—The Internet of Things (IoT) refers to an environment of

ubiquitous sensing and actuation from devices connected to the web

backend. IoT applications leverage contextual information about

entities in the system for reasoning and actuation. These context-aware

applications are difficult to scale to the large amount of heterogeneous

data in the IoT, as the current state-of-the-art is black-box, monolithic,

application-specific implementations. We propose a middleware

framework for context-aware applications that generates intermediate,

reusable context extracted from input by breaking down applications

into a set of functional units, or context engines. Leveraging existing

IoT ontologies, we can replace application-specific implementations

with a composition of context engines that use statistical learning to

generate output, improving context reuse and reducing computational

redundancy and complexity. We implement an IoT application using

our framework, extracting residential user activity from plug loads,

and demonstrate a reduction in computational complexity by 23% and

execution overhead by 69%.

I. INTRODUCTION

Sensor networks and ubiquitous sensing are evolving into the

Internet of Things (IoT) – the collection of sensing and actuation

backed by the existing and growing Internet infrastructure [1]. Pre-

IoT work in this area envisioned a level of compatibility and control

over the sensors in the systems [2] and applications that used a

manageable amount of raw sensor data. However, the number of

available sensing and actuation devices has grown rapidly in the last

few years [3], promising a truly pervasive sensing and actuating

environment. The main goal of the ubiquitous sensing in the

Internet of Things is to drive automated actuation. These context-

aware applications operate on dynamically changing raw sensor

data to extract context. However, as the number and heterogeneity

of sensing devices increases, the idea of applications operating on

their data in a scalable manner becomes infeasible. A middleware

infrastructure that classifies and maintains the context derived from

the data for these applications can alleviate scalability issues. This

layer serves to identify the relationship between objects in the

system, and leverage both raw data and high-level context about the

dependent objects to enable useful processing and automation.

Related work [4] [5] on middleware places the burden of

implementation on each individual application, but this is inefficient

when multiple applications need to process the same data in the

same way. Furthermore, reliance on application-specific code

reduces the potential for designing and reusing general-purpose

machine learning across context-aware applications. In this work,

we design a middleware infrastructure designed to overcome these

limitations: an ontology-driven framework that enables the use of

general statistical learning to drive output for the IoT.

Our framework is implemented as middleware through an

extensible set of context engines, translating heterogeneous raw

sensor data into high-level usable context. Each context engine also

generates intermediate context. which can be reused across

applications instead of performing redundant computation. The

middleware this reduces an application’s input set and facilitates

more scalable application development. We also automate context-

aware computing by leveraging general implementations of

statistical machine learning that can easily be reused for other

applications.

We provide an overview of related work in IoT, sensor

ontologies, and characteristics and requirements of context-aware

applications and middleware in Section II. We outline the context

engine implementation in Section III, including the algorithm to

learn and train a model for data translation and output generation.

Finally, in Section IV, we design an example context-aware IoT

application and compare application characteristics between our

infrastructure and a single-step optimization example. We

demonstrate 69% speedup in execution time enabled by our

framework due to better scalability and parallelism while obtaining

comparable accuracy.

II. RELATED WORK

Pervasive sensors gather raw data from a diverse combination of

data sources, including raw sensors and user-supplied or high-level

context processed from mobile and computing devices. Analog

sensor data is digitized and/or preprocessed before software can use

it as meaningful input. In the IoT, most data will go through several

levels of abstraction, combination, or distillation to produce a

description of the environment (and its users) with discrete,

semantic states. This higher level context is used for visualization

[4] or actuation [5][6]. Discretized context trades off intuitive

reasoning for raw data precision, and can be reused across

applications. Current context-aware applications are individual

deployments that rarely share infrastructure, code, or data natively.

Practically, this end-to-end development approach results in a

disorganized data space, necessitating the use of ontologies –

formal data representations – to maintain a unified, regulated data

representation [9]. The Web Ontology Language (OWL), an early

Internet ontology, is one such standard that provides description of

web relationships with several implementations: smart corporate

spaces [6], homes [10] and semantic services (OWL-S) [9]. An

important aspect of ontologies for the IoT is identifying the object

to which an application is related. The Context Modeling Language

(CML) [11] accomplishes this by relying on the Object-Role

Model, attributing all context data to a physical or virtual entity (the

object) and provides a particular form of information associated

with it (role).

Finally, pervasive sensing and computing in the IoT requires

learning/reasoning to appropriately transform input data into output

context. K-means clustering is a prevalent way to relate low-level

data into high-level context [4]. Reinforcement learning (RL)

allows IoT users who are already involved in sensing and actuation

to reinforce and guide the system towards better accuracy and

intuitive actuation [5]. Rashidi et al. [12] remove reliance on

labeled data by performing unsupervised learning over low-level

sensor data to extract patterns that represent high-level activities.

The above examples are application specific, but we propose a

framework and algorithms that can perform a similar level of data

translation and actuation in a domain-independent manner.

The state of the art in IoT middleware defines it as an interface

between sensing and actuation [13]. Ontologies can be used to

organize applications, but they remain multi-input, multi-output

(MIMO) black boxes transforming input to application-specific

output [10]. However, we and others [1] observe that the overall

process is made up of similar, application-agnostic data processing.

III. SYSTEM DESIGN

Our main contribution in this work is an alternate view of IoT

middleware: a hierarchy of multiple-input-single-output (MISO)

functional units (context engines) that improve scalability while

reducing data redundancy across applications with identical

functionality of the previous monolithic multi-input multi-output

(MIMO) units. We trade off compact, specific implementations by

exposing intermediate data that can benefit other applications,

exploiting the unique opportunity in IoT where reasoning and data

is often replicated among applications (see Figure 1.right).

Furthermore, the smaller, hierarchical functional units represent a

simpler data translation than before, and we can implement a

general machine learning algorithm to perform data transformation

and reduce application-specific code. We prove that this approach

decreases overall compute complexity and enables scalability, in

terms of reduced input processing for higher function orders (III.A).

Additionally, splitting single-step applications into small functional

units (each with fewer inputs, simpler logic) facilitates a

generalized data transformation through machine learning (III.B).

A. Context Engine System Architecture

We outline our middleware architecture leveraging ontologies.

We use them to specify the interfaces to each of the functional units

and also to drive data transformation. Each context variable is the

individual input or output data unit, and the variable’s data space

defines the domain/range of the variable, and subsequently, the

context engine. The MIMO, black box implementation of current

IoT applications is then broken down into several context engines,

each producing a context variable. These smaller MISO functional

units are composed to form the same outputs as the consolidated

application (Figure 1.right). While the monolithic applications can

themselves be modular and parallelized, the breakdown is hidden

from the rest of the system.

We opt to expose the internal composition, reducing the

application into smaller, less-complex functional units and

improves scalability and compute complexity. This approach raises

questions about the overhead, latency, and accuracy of breaking

down a possibly compact application into a composition of steps.

We validate our approach by proving that the overall computational

complexity of the architecture is actually reduced with a possible

marginal impact on output accuracy.

We aim to prove that dividing an N-input single context engine to

multiple context engines decreases the total computational

complexity. We start with a general representation of a context

engine: N inputs and a computational complexity order of α, for a

maximum computational overhead of 𝑁𝛼. We divide the single

engine into two stages: 𝑁

𝐴 multiple engines with an arbitrary number

of inputs A. The second stage takes the outputs of the first stage and

gives the final output. The total complexity overhead is now 𝑁

𝐴𝐴𝛼 +

(𝑁

𝐴)𝛼. We look for the conditions where the two-stage has a lower

complexity than the single engine, where A must be an integer and

the number of context engines must be an integer (𝑁

𝐴). This means

that A must be between 2 and 𝑁

2. The complexity of this staged

system is the multiplication of two terms. The minimum value for

the first term is achieved, when 𝐴 =𝑁

2 and results in 2𝛼−1. The

minimum value for the second term is achieved when 𝐴 = 2 and

results in 1 − 2−𝛼. This product provides a lower bound for the

result: 2𝛼−1(1 − 2−𝛼) = 2𝛼−1 −1

2, which satisfies our inequality

and provides another insight.;

(𝑁

𝐴)𝛼−1(1 −

1

𝐴𝛼) > 2𝛼−1 −1

2> 1 → 𝛼 > 𝑙𝑜𝑔23 ∼ 1.6 (1)

This result states that for any system of order greater than 1.6, i.e.

a nonlinear system, any arbitrary division of the single engine

results in a decrease in computational complexity, with the

minimum achieved for a system of 2-input engines. While in

general, context-aware applications do not necessarily fit perfectly

into a system of two-input engines, as we reduce inputs into each

CE and increase the path from initial input to output context, we

reduce the overall complexity of the system.

Figure 2. Breakdown of a single-step into lower-complexity equivalent reductions, with minimum complexity occurring with maximum division (right)

We now investigate the accuracy change between sequential and

consolidated applications. We consider a general functional

representation of data transformation from statistical learning:

generating a model for data transformation as a polynomial of

varying complexity and functional order, which can be solved to

best-fit through techniques such as regression [14]. By providing

the means to vary the inputs (ontology) and relationship (functional

Figure 1. (left) The current state-of-the-art: monolithic end-to-end applications. (right) Our implementation:. Functional units (context engines) are multi-in-single-output, and each context engine performs a general statistical learning and publishes intermediate context for reuse

order), complex relationships between the inputs and output can be

represented and trained. We now derive the error of a 2-stage, 2-

input sequential engine compared against the corresponding 4-

input, single-stage engine as a factor 𝛿, which creates 𝛿 ∗ 𝑖1 ∗ 𝑖2

truncation error, where 𝑖1, 𝑖2 are two inputs. We can also quantify

the impact of input signal noise on the sequential context engine

compared to the single-stage approach. We model each input with

zero-mean additive white Gaussian noise: 𝑥𝑖 + 𝑤𝑖, a common

expression of sensor noise [19]. The resulting noise coefficients

propagate through the application, compounding the truncation

error: 𝛿 ∗ 𝑖2 ∗ 𝑖4 + 𝛿 ∗ 𝑤2 ∗ 𝑖4 + 𝛿 ∗ 𝑤4 ∗ 𝑖2 + 𝛿 ∗ 𝑤2𝑤4. The first

term is the truncation error we previously quantified; the remaining

terms are the scaled Gaussian values due to noise. The significance

of the error terms is entirely dependent on the relationship between

the cross-product input terms 𝑖2and 𝑖4. However, from a system

design perspective, simply selecting highly correlated input terms

for context engines – the intuitive choice – mitigates truncation

error, minimizing the impact of the missing cross-coefficient.

B. Generalized Data Transformation

Our approach, a modular multi-stage context engine, results in

more functional units (FU) per application. An important

consequence is that each FU is a simpler translation of input data to

a single output. This enables us to use general data transformation

in place of application-specific code. Thus, a context-aware

application can be created by specifying the inputs and output of

each FU alone, and allowing the data transformation algorithm to

incur the overhead of generating and training a model based on

input and output observations. We can leverage ontologies that exist

in the current state-of-the-art middleware, as ontologies provide a

specification for each variable. From a data standpoint, they

regulate inputs and outputs of applications. We can exploit this

ontological information for machine learning algorithms that

generate results based on the input/output space of each FU.

Matrix-based stochastic learning models express potential data

dependencies as a system of equations. Over time, observed input

and output data is gathered until the coefficients can be trained and

a model generated. However, complex relationships can exist

among the input data for an IoT application, and a purely linear

model may not be sufficient [14]. Several works [15] [16]

implement learning by considering higher orders and time

correlation as well. We leverage TESLA, a learning model

originally designed for solar forecasting, as our data translation

algorithm. It provides efficient model generation: 𝑂(𝑛𝛼), where 𝑛

represents the number of inputs and 𝛼 represents the function order.

The generic Taylor expansion is as follows:

∑ 𝐶𝑖𝑥𝑖𝑛𝑖=0 (1st order), ∑ ∑ 𝐶𝑖𝑗𝑥𝑖𝑥𝑗

𝑖𝑗=0

𝑛𝑖=0 (2nd order) etc. (2)

where 𝐶𝑖𝑗 represents individual coefficients learned once

observations are determined, and 𝑥0 = 1 (a constant). The result is

the equation 𝐴 ∗ 𝑥 = 𝐵, where 𝐴 is the row matrix of input

observations; 𝑥 is the column vector of coefficients, and 𝐵 is the

column vector of output observations correlating with the

corresponding rows of A, and solved by least squares estimation.

A limitation of this model is that at least m independent

observations are required for training, where 𝑚 = 𝑛𝛼 for functional

order 𝛼, which is space-inefficient as the order grows. To use the

model, the equation is simply solved using the learned coefficients

and input context to produce the output context.

We select TESLA because of its general formulation, versatility,

and applicability to context processing, but other statistical learning

approaches can also be used: Bayesian Networks [7], Hidden

Markov Models, and Artificial Neural Networks can all leverage

input/output spaces to generate conditional probability models that

define output context [15].

IV. SYSTEM IMPLEMENTATION

For our case study, we investigate an application that is useful to

two IoT domains: the smart grid and smart environments. User

occupancy and activity is used in these domains for plug load

prediction and actuation, respectively. We focus into residential

spaces, which 1) have not seen the same automation as commercial

and industrial sectors, and 2) represent hundreds of millions of

consumers with different behavior and small individual energy

contribution but a significant grid share. Our middleware presents a

unique opportunity: to train and learn the behavior of different

residences using the same infrastructure. This information can be

used as the intermediate context for both domains. Our application

aims to identify active user presence in a room from plug loads

using the context engine middleware. Data Translation: In order to go from plug loads to user

activity, we break up the problem into two steps – 1) determining

appliance usage and 2) determining user activity based on 1). We

design a context engine for each, with the outputs, respectively:

{appliance-name: string; appliance-active: boolean} and {room-

name: string; user-active: boolean}. Context Engine Setup: Figure 5 illustrates both the sequential

context engine configuration (a) and the single-stage application (b)

to determine output context. From a set of n appliances and m

rooms, the first set of n context engines train and learn appliance

usage from the raw data. Each first-stage context engine is a single-

input, single-output (O(1)) engine consuming the raw power data

from its corresponding appliance as input, and trained on human-

observed binary output on appliance activity. The second stage

transforms the output of all the first-stage context engines (active

appliance usage) to a binary representation of whether or not a room

is being actively used. These second-stage context engines, with n

inputs, have a complexity of O(nα) for generating the output

context, where a is the functional order. The single-stage,

consolidated application represents the current state of the art,

taking in the same input to produce the same output. It will be used

for comparison of scalability, complexity, and accuracy. In this

case, the single-stage application has a complexity of O(nα).

Using 1Hz raw appliance power data from the REDD dataset

[17] as input, we train each context engine with input and output

traces. We use up to 172,800 samples (2 days’ worth), and test

(a) (b) Figure 3. (a) Sequential context engine and (b) consolidated application for room-level activity detection.

against 86,400 (1 day) samples.

Results: Complexity: the sequential context engine consists of n

second-order TESLA context engines for the first stage, each

consuming one input (appliance raw data) and producing binary

appliance usage context. The second stage has m second-order

context engines, one for each room, with n inputs, one from each

first stage. The result is n*O(1) complexity for the first stage and

m*O(n2) for the second stage, and an overall complexity of

O(m*n2). The consolidated context engine has exactly the same

configuration as the second stage alone and consequently, the same

complexity, at O(m*n2). However, even with the same complexity,

the sequential approach is advantageous in execution overhead. All

context engines iterate when a new input observation is recorded.

However, the second stage of the sequential context engine reacts

only to changes in binary input data, which is much less frequent

than the correspondingly subtle changes to the raw data. Table 1

below highlights the number of computations performed between

the single-stage and sequential context engines for the refrigerator:

The sequential application performs more individual computations

because there are more context engines, but it exhibits only 31% of

the latency of the single-stage context engine despite the additional

throughput. This is because of the nature of the computations: the

sequential context engine offloads the processing of raw data to the

low-overhead (O(1)) first stage. The single-stage context engine,

however, has no choice but to perform over 73000 O(n2)

computations. The sequential application requires only 23% of the

intensive O(n2) computations as the single-stage implementation,

and consequently, can complete the work 69% faster.

Table 1. Execution overhead for the context engines associated with the refrigerator

Refrigerator Number of computations

Total Latency 1st Stage 𝑶(𝟏) 2nd Stage 𝑶(𝒏𝟐)

Sequential 73428 17376 0.42 sec Single-Stage -- 73428 1.34 sec

Accuracy: While there is a reduction in compute overhead with our

approach, the accuracy of the sequential application and the single-

stage application must also be considered. We first investigate the

output prediction error for kitchen appliances in Table 2. Error is

calculated based on the percentage of instances where the model-

derived output differs from the expected output. These appliances

were trained with 10,000 unique observations culled from 2 days’

worth of data, and demonstrate generally very low error.

Table 2. First-stage error of appliance models for 2nd-order context engine

Appliance Output Prediction Error (%) Kitchen Outlets 2 32.8% Kitchen Outlets 3 1.4% Kitchen Outlets 4 0.6% Microwave 4.0% Oven 01 1.4% Oven 02 1.4% Refrigerator 0.8% Stove 0.0%

We now investigate the output accuracy of both implementations

of the same application. An important consideration is that the

sequential application contains more opportunities for training, as

each context engine can be trained individually. Conversely, the

consolidated application has only one context engine (see Figure

5(b)). If we train each context engine in both applications with the

same number of inputs, the sequential will consume and train on 9x

the data of the single-stage. Alternatively, we can train the

sequential application such that the total number of trained

observations matches the consolidated application - that is, provide

each of the 9 context engines with 1/9 observations compared to the

single-stage. We investigated both cases: training each context

engine with the same number of observations (row 1) and training

the overall application with the same number of observations (row

2). Table 3 compares the results of the two engines for the kitchen:

Table 3. Accuracy comparison between sequential and single-stage context engines

Application Type Output Prediction Error Sequential (same # obs/context engine) 2.4% Sequential (same total # observations) 34.3% Single-Stage 17.4%

The preprocessing in the first stage and the discretizing of raw

appliance data resulted in a significantly reduced error in the second

stage, with over 15% reduction in output error. However, when

relying upon the same total number of observations, the single-stage

application provided better accuracy, as the sequential application

undersampled and aggregated noise and error in the second stage.

V. CONCLUSION

In this work, we establish the motivation for and design a novel

middleware architecture for the Internet of Things. The current state

are implementations are ultimately monolithic and application-

specific [1]. This reduces the overall scalability and increases

compute complexity of applications. In contrast, we design a

modular framework that exploits common computational processes

between applications, exposing shareable intermediate context. We

increase scalability while reducing computational complexity by

decomposing an application into a sequence of functional units

driven by general data transformation. We implement this

transformation using a statistical learning technique that exploits

ontological data. We test our approach with residential activity

detection, demonstrating 69% improvement in execution overhead

and comparable accuracy while also exposing the intermediate

context variables for reuse. Overall, the result is a middleware

architecture for the IoT that leverages machine-learning intelligence

and improves scalability and complexity of context-aware designs.

REFERENCES [1] C. Perera, et al, "Context Aware Computing for the Internet of Things: A

Survey," IEEE Communications, Surveys, & Tutorials, pp. 414-454, 2013. [2] M. Friedewald and O. Raabe, "Ubiquitous computing: an overview of technology impacts," Telematics and Informatics, vol. 28, pp. 55-65, 2011. [3] J. Hammer and T. Yan, "Poster: A virtual Sensing Framework for Mobile

Phones," in Proceedings of MobiSys, 2014. [4] J.-H. Hong et al, "Conamsn: A context-aware messenger using dynamic bayesian

networks with wearable sensors," ESA, vol. 37, no. 6, p. 4680–4686, 2010. [5] S. K. Madhu et al, "An Ontology-based Framework for Context-Aware Adaptive E-Learning System," in ICCI 2013. [6] H. Chen, T. Finin and A. Joshi, "An Ontology for Context-Aware Pervasive

Environments," The Knowledge Eng. Review, vol. 18, no. 3, pp. 197-207, 2004. [7] S. Lee and K. C. Lee, "Context-prediction performance by a dynamic bayesian

network: Emphasis on location prediction in ubiquitous decision support

environment," Expert Systems with Applications, vol. 39, no. 5, p. 4908–4914, 2012. [8] M. Rudary et al, "Adaptive cognitive orthotics: combining reinforcement learning

and constraint-based temporal reasoning," in ICML 2004. [9] S. Staab and R. Studer, Handbook of Ontologies, Springer, 2010. [10] T. Gu, X. Wang, H. Pung and D. Zhang, "An Ontology-based Context Model in

Intelligent Environments," in CNDS 2004. [11] M. Nebeling et al, "XCML: providing context-aware language ext. for specification of multi-device web applications," WWW, vol. 15.4, pp. 447-81, 2012. [12] P. Rashidi et al, "Discovering activities to recognize and track in a smart

environment," IEEE TKDE, vol. 23, no. 4, pp. 527-539, 2011. [13] S. Bandyopadhyay et al, "A Survey of Middleware for Internet of Things,"

RTWMN ‘11, vol. 162, pp. 288-296, 2011. [14] B. O. Akyurek, A. S. Akyurek, J. Kleissl and T. Rosing, "TESLA: Taylor expanded solar analog forecasting," in IEEE SmartGridComm 14, 2014. [15] J. Gubbi et al, "Internet of Things (IoT): A vision, architectural elements, and

future directions," Future Generation Computer Systems vol 29.7 pp. 1645-60, 2013. [16] A. Katasonov et al, "Smart Semantic Middleware for the Internet of Things," in

ICINCO-ICSO, 2008. [17] J. Kolter and M. Johnson, "REDD: A public data set for energy disaggregation

research," in SustKDD '11, 2011.