DEVISING A SECURE DEPLOYMENTENVIRONMENT FOR THE INTERNET OF

THINGS

By

Bryan Pearson

A SENIOR RESEARCH PAPER PRESENTED TO THE DEPARTMENT OFMATHEMATICS AND COMPUTER SCIENCE OF STETSON UNIVERSITY INPARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF

BACHELOR OF SCIENCE

1

STETSON UNIVERSITY

2017

2

Contents

1 Introduction 7

2 Related Works 11

2.1 Current Shortcomings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2 Containerization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.3 Certi�cation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4 Additional Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3 Theory of Model 21

3.1 Certi�cates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.2 Containers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3.3 Accessibility and Other Considerations . . . . . . . . . . . . . . . . . . . . . 25

4 Methodology 27

4.1 List of Equipment and Services . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.2 Installation and Con�guration . . . . . . . . . . . . . . . . . . . . . . . . . . 29

5 Analysis 31

5.1 Insu�cient Authentication . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5.2 Lack of Transport Encryption . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.3 Insecure Network Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

5.4 Poor Physical Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3

6 Conclusion 35

4

List of Tables

1 Information about the four proposed attacks. . . . . . . . . . . . . . . . . . . 32

List of Figures

1 Raspberry Pi 3 Model B, designed by the Raspberry Pi Foundation. . . . . . 9

2 Comparison between virtual machines and containers. . . . . . . . . . . . . . 16

3 A certi�cate chain, consisting of three certi�cates, one intermediate CA, andone root CA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4 A new resin.io commit being pushed to a user's �eet. . . . . . . . . . . . . . 22

5 OWASP's top ten vulnerabilities for IoT devices, published in 2014. . . . . . 23

6 Docker uses multiple read-only image layers and a single read/write containerlayer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

5

Abstract

Devising a Secure Deployment Environment for the Internet of Things

By

Bryan Pearson

May 2018

Advisor: Daniel Plante

Department: Mathematics and Computer Science

The Internet of Things (IoT) is one of the largest emergent �elds of modern information

technology. IoT services have bled into almost every major industry today, and recent

e�orts have focused on perfecting the IoT deployment model. Fast, cheap, decentralized

control lies at the heart of IoT, presenting its own brand of unique challenges in terms of

security, scalability, and maintainability (e.g. automated updating) amongst developers.

Unfortunately, despite notable advances from the likes of Microsoft Azure, Resin.io, and

Amazon AWS, adequate research is still notably lacking. We propose IoTFlash: an ideal

IoT development environment that is scalable, certi�cate-based, and wholly containerized.

In addition to streamlining the delivery process, we aim for this infrastructure to support

fast, safe automated updates from the developer to allow for constant, seamless support of

the platform.

6

1 Introduction

Decentralized computing persists as one of the biggest trends in modern computer science.

Market analysts project that by 2020, the Internet of Things (IoT) will exceed 250 billion

dollars in technology funds, applications, and other solutions and services [1]. The IoT

ideology promises a landscape wherein technology can adopt to any and all user demands.

With notable commodity hardware like the Arduino and Raspberry Pi reaching the forefront

of hardware solutions, along with large-scale tech corporations such as Microsoft and Amazon

investing heavily into their cloud management platforms, implementing an IoT solution has

become as inexpensive and feasible as ever [2].

Unfortunately, major advancements in IoT have come at a big cost. Recent years have

revealed that a combination of carelessness and negligence in security practices have lead to

an uprise of vulnerable devices. For example, the Mirai botnet that was launched in October

2016 managed to easily capture over one hundred thousand smart devices, subsequently

launching a massive distributed denial-of-service (DDoS) attack on domain name service

(DNS) provider Dyn.com. The cyberattack exploited devices such as cameras and DVR

players which had failed to change their default credentials [3]. In addition, devices such as

St. Jude's implantable cardiac devices, Owlet's wi� baby monitors, and TRENDnet webcams

have been noted for their widescale security intrusions [4]. In 2015, researchers Charlie

Miller and Chris Valasek demonstrated a proof-of-concept by "carjacking" a Jeep Chrysler,

obtaining full control of the engine and electrical system; despite their non-malicious intent,

the two researchers starkly conveyed the inherent danger associated with smart devices [5].

Researchers and developers have certainly taken note of the inadequacy of IoT se-

curity, but as we hope to elucidate, further research is still necessary if we hope to address

the issue e�ectively. Thus, we propose IoTFlash: an architecture that secures devices for

developers at the outset, produces minimal overhead, scales with ease, and provides a sim-

ple interface for users. We follow in the vein of cloud management infrastructures such as

7

Resin.io, Apache Mesos, Amazon AWS, and Microsoft Azure which aim to streamline cloud

computing deployment; in addition, we hope to provide an extensible model for developers

to freely expand and use.

IoTFlash combines a variety of di�erent applications and services as a means to

complete its vision. The backbone of this project is Docker, a containerization platform

that allows developers to develop, deploy, and run their applications on any platform of

their choice. Docker, and speci�cally containerization, is highly desirable for many di�erent

reasons. First and foremost is application security. Docker takes several novel approaches

to guarantee security for developers, most notably its push for container isolation. Docker

utilizes namespaces as well as separate network stacks for each application. Unless whitelisted

by the developer, applications cannot "see" processes running outside of their scope. This

makes them extremely valuable in an IoT environment, where developers typically desire

limited device functionality. Moreover, Docker presents a simplistic and powerful command-

line interface (CLI) that allows complete control over applications, as well as 'clusters' of

devices (or nodes). The CLI is ubiquitous and identical for all Docker instances; components

such as Docker Swarm enable the ability to quickly scale up applications if needed, but the

CLI ensures a seamless process throughout the venture. As we discuss IoTFlash in further

detail, we will highlight more features of Docker that make it a suitable platform for us.

We construct our platform on top of HypriotOS, a lightweight operating system built

speci�cally to orchestrate Docker applications [6]. HypriotOS is derivative of Linux, but

lighter than Raspbian (the default OS for Raspberry Pi), and it comes prepackaged with the

newest version of the Docker client and daemon, making it the perfect operating system can-

didate for our purposes. Other *nix operating systems are applicable to our project, though

we take some key aspects of HypriotOS for granted during the implementation process, such

as assuming an open SSH port. Multi-platform functionality is a consideration for future

research but is left out of our model in the interest of time.

8

For the hardware setup, we use three Raspberry Pi 3 model B's, as well as a NET-

GEAR AC1200 wi� router model R6120. We will use the term 'RPi' to refer to the Raspberry

Pi 3, and the term 'router' will refer to the AC1200 wi� router. The RPi contains impressive

speci�cations for storage, computation, and functionality, and at an e�cient price of about

thirty-�ve dollars each. Meanwhile, the speci�cations of our router are less signi�cant, since

in the case of our implementation, the router is purely intended to serve as an initial wired

gateway between the host and the RPis. To ensure maximum security, IoTFlash expects an

ethernet connection prior to initialization, so that unwanted devices are not installed during

the initialization step. Afterwards, IoTFlash ignores the router unless the user wishes to add

more devices to their cluster. We will discuss the setup process more in the Methodology

section of the paper.

Figure 1: Raspberry Pi 3 Model B, designed by the Raspberry Pi Foundation.

Users will initialize IoTFlash via a Python script. This script will start by searching

for all applicable devices connected to the host through ethernet, then continue to set up

IoTFlash. We will discuss the script in depth later on.

We now make the following distinction to readers: an IoT platform is de�ned as a

9

service or environment that bears some signi�cance to the physical world and operates with

at least one clearly de�ned purpose. On the other hand, we de�ne an IoT environment

as a service or platform with identical sentiments as a typical IoT platform, except without

any singe de�ned purpose. As an example, we classify the Nest Thermostat as an IoT

platform, while we consider Amazon's AWS Greengrass as an IoT environment. We use the

term single-function entities to refer to platforms, and we use n-function entities to

refer to environments (where "n" designates a potentially large number). We feel obligated to

distinguish between platforms and environments in order to drive the underlying theme of our

project: IoTFlash is ultimately intended to serve as a deployment environment, designed with

no one speci�c application in mind. It can be described most succinctly as an "environment

designed for platforms." We intend to secure and streamline the deployment process, but

the details of the applications will always be left to the developer.

It is also worth mentioning the imminent con�ict of related terms such as IoT, fog

computing, edge computing, and decentralized computing. Throughout the paper, many of

our referenced works will appear to use the terms interchangeably. Although the details of

each term slightly di�er, depending on their contexts, in general we consider them synony-

mous with the following de�nition: a cluster of heterogeneous devices capable of working in

isolation and communicating with each other, overseen by at least one server. For all intents

and purposes, we will use the term IoT to refer to this concept.

The remainder of our paper is structured as follows. In Section 2, we present related

research on the relevant areas of interest, particularly in terms of current research short-

comings, as well as background information on certi�cates and containers. In Section 3, we

present the theory of our proposed model. In Section 4, we propose the methodology of our

proof of concept. Finally, in Section 5, we suggest analysis techniques by proposing four

di�erent attacks against our system to test its e�ectiveness.

10

2 Related Works

Poor security policies are not unique to the Internet of Things, yet are scarily rampant in

the culture of distributed computing. With devices such as the Linux-based Raspberry Pi

Zero entering the market at a measly �ve dollars per device, developers can build their IoT

platforms faster than ever before. The notion of IoT seems to imply a one-o� application in

which the developer quickly designs and disregards afterwards. For the most part, our ob-

servations unfortunately seem to con�rm these notions; many IoT developers pay little mind

to short- or long-term security implications of their applications, as long as it is accessible

to the user. We cannot hope to remedy this lazy mindset, nor is remediation the goal of

IoTFlash. Instead, we aim to resolve the innate shortcomings of developers by designing an

all-encompassing framework that protects against the largest known security exploits.

Jatinder Singh et al. [7] discuss twenty substantial bottlenecks of IoT security; we

derive many of our considerations from their list. We can summarize these considerations as

follows:

• Secure communications with devices and the cloud

• Su�cient data control or data management

• Encryption

• Identity management

• Malicious devices

• Certi�cation

Of course, by no means is this an exhaustive list of all security vulnerabilities worth

considering. However, we observe that the majority of security shortcomings fall under the

umbrella of one of these categories.

11

We now turn our attention to several notable instances of IoT security exploits, before

delving into other related works.

2.1 Current Shortcomings

The Mirai botnet, and the variants which derived from it, continue to pose as some of the

most prominent threats to IoT devices. This is because Mirai and its variants succeed by

exploiting default credentials in vulnerable services such as Telnet and SSH. Constantinos

Kolias et al. [8] discuss these botnets in further detail: they target webcams, routers, and

similar devices by scanning for open ports such as 23 or 2323 (Telnet), then use brute-

force tactics to obtain access to each device. Mirai gained access to around one-hundred

thousand devices, despite operating on only sixty-two di�erent combinations of usernames

and passwords. Kolias et al. explain four key elements of IoT that deem it a target for

attackers: they are "always-on," thus constantly exposing their attack surfaces; they often

o�er feeble protection, sacri�cing security for the sake of user-friendliness; they are poorly

maintained after the initial setup process, leaving older devices especially vulnerable; and

they typically o�er a minimal or non-interactive user interface, thus making it much more

di�cult to identify or mitigate infections. Again, these elements are nothing new in the

context of web development and similar disciplines, but current IoT trends have largely

foregone many best practices.

HP's Kristi Rawlington [9] echoes these sentiments, citing a study by HP that con-

cluded that about seventy percent of IoT devices are vulnerable to attack. HP's testbed

encompassed the ten most popular IoT devices, including smart TVs, webcams, home ther-

mostats, door locks, garage door openers, and others. Their �ndings were quite sobering for

developers and users alike. Eight out of ten devices raised privacy concerns by collecting

sensitive data such as name, address, and health information; moreover, eighty percent of

devices (including mobile and cloud counterparts) had insu�cient password complexity, sev-

12

enty percent failed to implement transport encryption, and sixty percent failed to encrypt

software updates.

The sheer breadth of IoT applications introduces many di�erent attack surfaces into

environments that would otherwise be considered safe. This matters especially in the context

of industries such as healthcare, where faulty devices can be much costlier than in terms of

dollars. Unfortunately, this is precisely what happened with the Abbott pacemakers [10].

The implantable heart rhythm regulators were found to contain a vulnerability that allowed

attackers to change commands or deplete the battery of the devices, thus posing a signi�cant

danger to wearers.

Similarly, security researcher Jonathan Zdziarski discovered several problems in the

design of the Owlet baby monitors [11]. In particular, Zdziarski found that the base station,

which linked the monitor to the manufacturer's servers, would create an ad-hoc wi� network

with no encryption and no authentication settings. The device would also be locked into

using port 80, allowing a quick scan of that port on the wi-� network to reveal the vulnerable

service to attackers. Perhaps most alarming was the lack of any software update mechanism,

allowing these vulnerabilities to persist for some users well after developers had addressed

them.

Finally, a related incident occurred in 2012 when hackers penetrated the live feeds

of Trendnet SecurView cameras [12]. Trendnet marketed its product for a variety of uses

including security cameras, baby monitors, and so forth. However, Trendnet failed to secure

the IP addresses of the cameras, allowing anybody with the addresses to compromise the

devices; additionally, Trendnet processed user credentials via the web and mobile interface

without encrypting them, leaving them exposed to attackers. In total, hackers were able to

successfully leak the "private" live feeds of around seven hundred SecurView cameras before

Trendnet �nally managed to deploy a software update.

All of these instances showcase the serious implications of shoddy security in our

13

devices. As developers steadily work to broaden the diversity of IoT devices in users' daily

lives, the importance of adequate security for these devices will only become more critical.

Society experienced a similar crisis in the mid-1990s, when the insecurity of personal com-

puters and software was rampant, companies tried their best to keep these vulnerabilities a

secret, and users had di�culties in even installing updates due to poor software implemen-

tations. On these fronts, the IoT security crisis might look awfully similar to the mid-1990s

crisis. However, as Bruce Schneier explains [13], the stakes are much higher now. Devices are

embedded much more closely into the daily lives of users, making their security all the more

critical. The nature of embedded systems�built from open-source, proprietary, or otherwise

cheap components for the sake of slim pro�t margins�exacerbates the problem for all parties.

IoT manufacturers will typically only focus on developing the next iteration of a product,

with little regard to previous releases; worse yet is how these products often come as bi-

naries instead of open-source, leaving third-parties or even ambitious users unable to patch

them. The result is that no entity has the incentive or ability to patch existing software, and

pressure is instead placed on developing a new product.

For now, the �rst step should be to raise awareness about existing systems bottlenecks.

This is precisely what has occurred in the past twenty years with personal computers: a

push for complete disclosure on any and all software and hardware vulnerabilities, allowing

users to make informed choices. However, the long-term solution will be to apply pressure

to embedded systems vendors to implement devices that support automatic updates, use

open-source software, and have a much more inherently secure infrastructure. Although

IoTFlash will work to provide a su�cient embedded system, the responsibility will still fall

to the developer to maintain proper security practices for any product they build on top of

IoTFlash.

14

2.2 Containerization

Although researchers have proposed many unique solutions for decentralized computing with

IoT devices, one element has remained almost universally constant: a containerized approach.

Recently, developers have widely embraced the advantages of containers over alternatives,

i.e. virtual machines [14]. The three most de�ning characteristics of containers are isolation,

portability, and lightweight.

Isolation: In an isolated environment, software does not have access to external

resources during operation. This means that the functionality, and hence the security, of

containerized software depends almost completely on the con�guration of the container itself.

Isolation improves security by narrowing the attack surface.

Portability: In a portable environment, the software is able to run on as many host

environments as possible. This is because the container itself comes bundled with all of the

dependencies needed to run the software, so no manual con�gurations are required from the

user. Limitations exist, however; in particular, cross-OS compatibility (especially Linux and

Windows) is complicated because containers run on top of their system kernels.

Lightweight: Since a container shares the same kernel as its host machine, it can

start up faster than a virtual machine and use less RAM to operate. After all of the de-

pendencies are added to a machine for the �rst time, a container becomes an abstraction of

the application layer, leading to a marginally small container size (typically on the order of

megabytes).

Containers and virtual machines are similar in that they both set out to virtualize an

environment and have similar isolation techniques, but whereas a virtual machine virtualizes

an entire operating system, a container only virtualizes the desired application. As a result,

containers are far more lightweight than virtual machines.

Docker is a container service platform that allows developers to implement a con-

tainerization solution. It is a popular candidate platform for many IoT researchers. Pahl

15

Figure 2: Comparison between virtual machines and containers.

Claus et al. proposes a platform-as-a-service (PaaS) architecture using Docker containers

along with a cluster of Raspberry Pi 1's [15]. Their use case for this architecture is a modern

ski resort with a diversity of variables like weather and people, as well as sample functions for

management of people and snow. In a massive, environmentally unpredictable environment

such as a ski resort, the location, status, and cooperation of each device is critical. Such an

extreme use case is not unlike how a conventional production-scale IoT architecture might

have to operate.

Research suggests that Docker shows great promise in meeting these needs. Bukhary

Ismail et al. conclude that Docker addresses the key concerns of an edge architecture: de-

ployment, termination, resource and service management, fault tolerance, and caching [16].

Roberto Morabito et al. make similar sentiments by implementing two practical cluster

frameworks with Docker: pair-oriented services between nodes and the server, and complete

edge management using Docker Swarm [17]. In both frameworks, Docker showcases im-

pressive robustness and performance, showing promise in terms of trade-o�s with resource

constraints, and providing �exible capabilities for developers with heterogeneous environ-

ments.

16

Performance and �exibility are not the only factors to consider with containerization.

We must carefully consider the security of such an environment. Docker also shows promise

on this front [18]. For one, Docker employs namespaces using process IDs, networking

information, and other methods for each running process such that they stay isolated from

one another. Additionally, containers include their own network stack, and they do not

have access to external sockets or interfaces unless directly speci�ed. Docker also prevents

unauthorized users from making sensitive changes to the environment; communicating with

the Docker daemon (which handles all Docker processes) requires elevated privileges, and

container permissions by default have reduced root privileges. Finally, and most importantly,

Docker enforces a strict separation of concerns model; network management takes place

outside of the container, and data volumes also exist externally.

Docker Swarm is an extension of Docker that handles cluster management capabilities

over to the user [19]. Swarm instances implement a virtual extensible local area network

(VXLAN) environment, meaning that a cluster of Docker nodes can communicate with

each other as if they existed on a LAN. The VXLAN complicates the security of Docker

by exposing an additional attack surface, but penetration attempts have proved largely

unsuccessful [20]. While researchers have demonstrated the feasibility of packet injection

into the VXLAN tunnel, this is only possible if the packet's source IP address is spoofed as

a fake Docker sender address; additionally, this method fails to prolong sessions, trivializing

the e�ectiveness of the attack. Furthermore, ARP attacks, and generally any layer 2 attacks,

proved impossible to exercise due to the limited privileges of the container, although this

may vary with each environment.

While Docker will serve as our candidate platform for containerization, it bears men-

tioning other containerization tools that may have additional potential. Kubernetes is an al-

ternative platform developed by Google in the Go programming language (similar to Docker).

It was designed with the initiative to orchestrate Docker containers without the burden of in-

17

teracting with the underlying infrastructure [21]. It therefore poses exclusively as a container

orchestration platform, aligning it more with Docker Swarm than Docker itself. Although

Kubernetes shows potential in initiatives such as stateful containers and data analytics, the

project is very much in early development. Comparisons between Docker Swarm and Ku-

bernetes reveal that Swarm is currently superior in several regards, notably its ease of use

(thanks to its universally applied command-line interface) and its response time for API

calls, which has shown to be �ve times faster than Kubernetes [22].

Another platform for consideration is Mesos, developed by Apache [23]. With seven

years of development behind it, Mesos is more dexterous than either Docker or Kubernetes,

supporting "workloads" rather than strictly containers. Workloads may consist of Java �les,

batch jobs, distributed data services, and of course orchestration of Docker microservices.

2.3 Certi�cation

Certi�cation is mysteriously absent in the context of many IoT studies. However, because

certi�cates protect network channels by encrypting communication between the client and

the server, its relevance is apparent to us.

Certi�cates are typically employed as the baseline for HTTPS (HTTP over SSL/TLS),

which encrypts web tra�c. Certi�cates are issued by a certi�cate authority (CA), which

receives certi�cate signing requests (CSR) from a domain. These requests are generated by

the domain using a private key. A 2048-bit RSA key is typically considered su�cient [24]. In

addition, users generate an adequate signature hashing algorithm for their certi�cate. Most

developers recommend using SHA256. Once a CSR is granted, the resultant certi�cate uses

SSL or TLS to protect the domain for a speci�ed validity period.

Major clients such as web browsers generally have a list of CAs that they trust. These

are referred to as root CAs. The CA of a domain is generally not a root CA; instead, its own

certi�cate is issued by yet another CA. Non-root CAs are called intermediate CAs. The full

18

stack of certi�cates traversing from the domain to the root CA is known as the certi�cate

chain.

Figure 3: A certi�cate chain, consisting of three certi�cates, one intermediate CA, and oneroot CA.

The implementation of a certi�cate model would serve as additional security for IoT-

Flash devices. While Docker protects the services running on the devices, containers protect

the devices themselves. In addition, in the event that developers need to communicate with

their devices without using Docker, certi�cates protect this communication from potential

man-in-the-middle (MiTM) attacks.

In the event that an attacker steals the private key of a certi�cate, it is imperative

that this certi�cate is immediately revoked and rotated with a new valid certi�cate. However,

certi�cate revocation poses some issues [25]. Many servers employ a certi�cate revocation

list (CRL), which houses all revoked certi�cates. For a client to truly verify the authenticity

of a certi�cate, it would have to traverse the full certi�cate chain of every certi�cate in the

CRL, which may take much longer than desired depending on the size of the list. This can

throttle the loading speed of webpages dramatically.

Online Certi�cate Status Protocol (OCSP) is an alternative solution to CRL that

19

takes a much faster, whitelisted approach; rather than verify against all revocations, OCSP

can simply check directly with the CA to report the status of a certi�cate. Additionally,

servers can employ "OCSP stapling," which tasks the server with obtaining the status of its

CA at various intervals. It then bundles the OCSP response with the certi�cate and passes

this information to the client.

2.4 Additional Studies

In the past couple years, research in IoT security has �ourished, although few tangible

solutions have come forward. Letian Sha et al. envision a comprehensive platform for

industrial-level IoT concerns, basing their implementation on the Technique for Order of

Preference by Similarity to Ideal Solution (TOPSIS), a multi-tier decision making method,

and Analytic Hierarchy Process (AHP), a method for complex decision analysis [26]. This

solution is elaborate, but e�ective, though it remains to be seen how transparently the

platform could scale down to meet less ambitious projects.

Hua Wang et al. published an editorial comprising various papers which discuss the

security and privacy of IoT [27]. The sentiments of these papers largely resonate with our

previous notions: simple penetration, via password guessing or �rmware miscon�guration, is

the single largest culprit of large-scale IoT attacks. Furthermore, traditional cryptographic

protocols are usually not applicable to IoT concerns; instead, IoT requires primitives and

protocols that are lightweight, data-driven, context-aware, and service-oriented.

Ruben Mayer et al. envision a "fog" platform that is far less dependent on the server

than previous studies. In their model, devices encompass RPis as well as drones and have the

capability to communicate with each other in the event of an emergency, such as a natural

disaster, where the lack of a server would normally hinder such communication [28]. This

scenario provides a fresh view on the capabilities of IoT.

20

3 Theory of Model

As the previous section makes abundantly clear, security is a complex and ever-broadening

issue that currently contains no single, easy solution for developers. IoT's limitations in

computation, storage, and communication complicate the issue even further, since currently

existing solutions are often optimized for more powerful hardware. Given this, we cannot rea-

sonably expect to design a proof of concept which addresses every major security bottleneck

for IoT devices.

Yet, still, projects do exist that address our concern. Resin.io, for example, o�ers

an innovative solution to streamline the deployment and development process for devices

by o�ering an open-source API, VPN connections for each device, and always-on containers

that are isolated from the application. [29]. Resin.io allows users to build containers using

Docker�les or packaged json �les, then retrieve provisioning keys from their server in order

to authenticate new devices. Communication between the server and devices occurs either

over the VPN or via HTTPS, ensuring an encrypted communication channel. Additionally,

the resin.io agent works with git to simplify the update process; users can update all of their

devices by issuing the command "git push resin master".

21

Figure 4: A new resin.io commit being pushed to a user's �eet.

Resin.io serves as a suitable model from which we plan to construct and compare

IoTFlash. But while resin.io is intended �rst and foremost as a marketable product for

developers, we propose IoTFlash purely as a proof-of-concept that invites further research

in order to develop and improve. Furthermore, no research has yet been conducted to

thoroughly assess the security of resin.io; such an assessment is conducive to their product's

development but otherwise irrelevant to our research.

We can now qualitatively gauge the e�ectiveness of IoTFlash by asking the following

question:

Does IoTFlash o�er conventional functionality for decentralized computing

while minimizing conventional security bottlenecks?

We use the term "conventional" to refer to common practices which are up to date

22

with the newest research and �ndings. Conventional functionality refers to features which

provide modern edge cloud expectations, e.g. Claus Pahl et al.'s expectations for envi-

ronmental sensors, communication between sensors, functions activated from environmental

factors, and ability to handle dynamic conditions i.e. device failure or migration. Conven-

tional attacks refer to the most common security vulnerabilities and the attacks associated

with them. In Section 2 we alluded to problems with insu�cient authentication, insecure

web interfaces, privacy concerns with data management, and others problems of interest.

Every few years, volunteers from the OWASP community collaborate to publish an

up-to-date list of the most common security vulnerabilities in web applications. Because of

the uprise of IoT devices, researchers have now started to compile a similar, but separate,

project known as the OWASP IoT Top Ten [30]. The list is organized depending on the

prominence of each attack. Therefore, it is imperative that IoTFlash works to tackle as

many of these vulnerabilities as possible.

Figure 5: OWASP's top ten vulnerabilities for IoT devices, published in 2014.

In the interest of staying concise, we narrow the focus of IoTFlash down to three key

elements:

• Certi�cates

23

• Containers

• Accessibility

The implementation and compatibility of each of these aspects will ultimately de-

termine the e�ectiveness of our project. By enforcing a certi�cate-based model, we ensure

that devices can only communicate with other authentic devices, and that this communi-

cation link stays encrypted via SSL or TLS. A containerized approach allows us to quickly

deploy and develop applications, manage clusters, and scale up or down as needed. Finally,

an accessible service addresses many user concerns related to predictability, reliability, and

e�ciency of the service.

3.1 Certi�cates

In an ideal certi�cate-based environment, only trusted entities may interact with the device.

In the traditional certi�cate model, a client (such as a web browser) can access the server

certi�cate and verify its authenticity. The most critical step is verifying the certi�cate

authority (CA), since an untrustworthy CAmay indicate that the client is not communicating

with the correct server, or that the device has been compromised. In our model, we act as

the CA and issue certi�cates to every device we plan to include in our cluster. Thus, only

the host has permission to communicate directly with the cluster.

Attacks such as SSLStrip exist to circumvent SSL/TLS security by forcing a victim

into an MiTM attack. The proposed solution to this is HTTPS Strict Transport Security

(HSTS) [32], which forces encryption over all communication. Although an IoT-oriented

solution has yet to arise, we can enforce similar practices manually by allowing devices to

only communicate through SSL or TLS.

24

3.2 Containers

Bukhary Ismail et al. provide su�cient context for the practicality of a Docker-based frame-

work. Docker uses a �le called a Docker�le to set up the dependencies of a container. These

dependencies act as the heavy read-only layer of a container, called the image layer. The

�nal layer, which typically executes the image, is a writable container layer. This means

that container updates only a�ect the container layer, which is small compared to the image

layer. If containers are running as services via Docker Swarm, the Swarm manager can del-

egate container requests by scheduling and �ltering through the most appropriate daemons

before sending the request. The Docker Shipyard project builds on top of Docker Swarm by

expanding daemon, container, and cluster management tools for the user. Finally, Docker

can easily clean up resources of containers that the user terminates. One caveat is that

data volumes which are attached to the container must be handled manually, since volumes

persist outside of the container or image layers.

Docker Swarm provides users with su�cient resource and service management. Swarm

supports service registry and discovery, allowing devices to come and go as needed without

compromising the entire service.

Docker also employs principles for safe fault tolerance. CRIU is an open source

project that allows users to checkpoint and restore Docker container states, which is useful

for recovery purposes. Additionally, Flocker is a Docker extension that manages the container

data volume; the Flocker dataset is portable, meaning any container can access it. Docker

can also export and import new daemons at new hosts as needed, although this does not

necessarily preserve memory and CPU instructions. Finally, the physically distributed nature

of Docker containers across a geographical site alleviates points of failure.

3.3 Accessibility and Other Considerations

We consider IoTFlash accessible if it can reasonably satisfy the following criteria:

25

Figure 6: Docker uses multiple read-only image layers and a single read/write containerlayer.

• The setup/build process can run successfully on various systems and produce identical

results on these systems.

• The update process is equivalent across various systems and a�ects all targeted devices.

• Software and hardware work together to maximize performance.

• Common points of failure (e.g. miscon�gured wi� information, device failure, etc.) are

addressed or mitigated.

• Documentation is available.

• All activity is logged.

If the setup process produces inconsistent results on di�erent machines, it becomes

di�cult to predict the long-term behavior of the environment. Additionally, it becomes

26

harder to mitigate potential issues that arise in the post-setup process, and it becomes

almost impossible to evaluate the potential security pitfalls that may leak. For these reasons,

a predictable setup process is crucial for us.

Similarly, the update process should behave on all machines equally. Moreover, an

update should target all devices in a cluster, ensuring devices remain up to date with each

other.

Performance is critical in an IoT setting, and the choice of hardware is just as impor-

tant as that of the software. It is important not just that hardware components are powerful

and that software components are e�cient; it is necessary that the combination of these

components produces an e�ective environment.

The last three points in our list relate to the same principle: service failure. It

is imperative that the program addresses common points of failure such as invalid user

commands, devices shutting o� while a script is running, etc. IoTFlash does not have to

remedy these problems per se, but we should do our best to supply as much information

about a speci�c incident as possible to the user. Similarly, it is important that we document

the speci�cations of IoTFlash to the best of our ability, especially in regards to common

points of failure. Finally, we need to log commands as verbosely as possible, both during

and after the setup process, so that less common points of failure can be traced.

4 Methodology

IoTFlash leverages various equipment and services to its advantage. What follows is a sum-

mary of these equipment/services and a thorough overview of the project's startup procedure.

4.1 List of Equipment and Services

Our implementation of IoTFlash will make use of the following equipment and services:

27

• Raspberry Pi 3 Model B (3)

• NETGEAR AC1200 wi� router model R6120

• Docker version 17.09.0-ce (client and server)

• OpenSSH version 7.6

• OpenSSL version 1.0.2g

• HypriotOS version 1.6.0

• Python version 3.5.2

Raspberry Pi 3 (RPi): Comparatively speaking, the RPi boasts some impressive

speci�cations: it contains a quad core 1.2 gigahertz Broadcom BCM2837 64-bit CPU, one

gigabyte of RAM, ARMV-A instruction set, four USB ports, wireless LAN and Bluetooth

4.0 support, a 40-pin extended general-purpose input/output (GPIO) set, and more [31].

Although prior RPi models will likely su�ce in terms of performance (depending on the needs

of the developer), we target model B primarily due to its built-in wi� support. Moreover,

Model B is quite a�ordable, at about thirty-�ve dollars each.

NETGEAR AC1200 (router): The speci�cations of our router are less signi�cant,

since in the case of our implementation, the router is purely intended to serve as an initial

wired gateway between the host and the RPis. In this case, we will use the router to

physically connect our three RPis for the setup script.

Docker : We will install Docker version 17.09.0-ce onto our host and nodes, which is

currently the newest stable release of the platform. Older Docker versions did not support

Docker Swarm and were less secure on a number of fronts. Because the daemon acts as a

separate entity from the client, we must take extra precautions to ensure that both versions

are up-to-date, and more importantly, that both versions are identical, otherwise we may

lose functionality.

28

OpenSSH : This is purely a suite of SSH-related utilities, including 'ssh' and 'sshpass'.

We will leverage these tools during the setup process in order to access our devices.

OpenSSL: OpenSSL is capable of generating .X509 certi�cates, which we can pass to

our devices via OpenSSH. We can also establish the CA of our system, the security key, and

the hashed signature algorithm, storing them in a secure location on our host.

HypriotOS : HypriotOS is a Unix-like operating system that comes from the Debian

family. It supports the ARM architecture just like Raspbian, meaning any processes that

can run on Raspbian also run on HypriotOS. However, HypriotOS holds two advantages

over Raspbian and several other alternatives: it is even more lightweight than Raspbian,

allowing it to potentially be deployed on less powerful hardware, and it comes prepackaged

with Docker, giving it the edge for us to use in our project.

Python: Finally, we use Python to execute our startup script. Python can easily call

external commands and scripts, and we can supply basic input/output control to our system.

We can also create, delete, and modify �les as needed.

4.2 Installation and Con�guration

For the sake of simplicity, we will assume the Raspberry Pi already comes with a fresh

installation of HypriotOS. If need be, the user can freely download and install the image

onto the micro SD card. We expect the user is running HypriotOS version 1.6.0 on all

devices. This version contains Docker 17.09.0-ce just like our host does, and it contains an

open SSH port, making it easy to streamline the setup process.

Along with the aforementioned case, we assume that the user at least has Python 3.*

installed, Docker 17.09.0-ce installed for the server and client, and support for SSH protocol

2. We plan to provide a pre-startup textual guide for users who need help following any of

these steps.

The user is then free to run the startup Python script. Once initiated, the following

29

procedure will occur:

1. IoTFlash will ensure that the Docker daemon and client APIs match each other, and

that they match or exceed the minimum required version.

2. The user should con�rm their desired network, which ideally is the ethernet network.

Additionally, users can con�rm if they want their devices connected to wi�. If so, users

can enter the wi� credentials. IoTFlash will verify these credentials.

3. The user will enter the desired cluster name and user name of devices. In our en-

vironment, host names will either be named 'cluster-name'-nodeX or 'cluster-name'-

masterX', where X represents a number. User names are named 'user-name'-masterX

or 'user-name'-slaveX.

4. IoTFlash scans the network for all potential devices.

5. IoTFlash reads these devices back to the user to con�rm the cluster. The user can

con�rm the cluster or blacklist speci�c devices, if they so choose.

6. IoTFlash will generate a secure CA for the user's host.

Once the cluster has been established, the following procedure occurs for each device:

1. IoTFlash will create a random SSH password for the device, hash and salt it, and store

it in a secure location on the user's host.

2. IoTFlash will generate a new certi�cate and copy the certi�cate over to the device.

3. IoTFlash will attempt SSH login using HypriotOS's default credentials 'pirate' and

'hypriot'. If this fails, the user can manually enter the credentials. If multiple failed

attempts occur, the script exits.

4. IoTFlash creates a new account on the device using the speci�ed username, as well as

the generated SSH password. It then restricts SSH access to only this new user.

30

5. IoTFlash add this device to Docker Swarm as either a worker or a manager, depending

on which is speci�ed.

6. IoTFlash scan for and turns o� all unnecessary ports.

7. IoTFlash attempt to connect the device to wi�.

8. Both the user and IoTFlash verify that the node has joined the cluster, all information

matches, and the device can connect to wi�.

Assuming the script ran successfully, the user will now have a cluster of certi�cate-

based containerized nodes. Every step is documented in a separate log �le, allowing users to

trace through these steps in the event that the script fails.

5 Analysis

In the previous section, we proposed a qualitative methodology for building a secure de-

ployment environment. We now propose a quantitative metric from which we can evaluate

IoTFlash. We will design four attacks as test cases for the security of the environment.

These attacks will exploit the following common vulnerabilities:

• Insu�cient authentication

• Lack of transport encryption

• Insecure network services

• Poor physical security

We plan to measure the e�ectiveness of IoTFlash by evaluating the following con-

ditional statement: if all four attacks fail to penetrate IoTFlash, then we will consider the

project a success; otherwise, we will address points of failure and potential methodologies to

improve our environment in order to mitigate the attacks.

31

Attack Type Attack Vectors Security Weakness Impacts• Weak passwords • Data loss

Insu�cient • Poorly protected • Common prevalence • Denial of accessAuthentication credentials • Easy detectability • Device

• Bad recovery mechanisms compromised

Lack of Transport • Data passing over • Common prevalence • Data lossEncryption network is visible • Easy detectability • Device compromised

• Data lossInsecure Network • Attack device directly • Uncommon prevalence • Denial of

Services • Bounce attacks o� device • Average detectability service• Attack facilitation

• USB or SD cards • DevicePoor Physical • Other storage means • Common prevalence compromised

Security • OS access • Average detectability • Data compromised• Stored data • Data corruption

Table 1: Information about the four proposed attacks.

5.1 Insu�cient Authentication

Insu�cient authentication/authorization is typically characterized by default passwords, lax

credentials requirements, or a failure to enforce these requirements. Passwords especially

need to avoid common uses such as '12456' or 'password' which exposes them to dictionary

attacks, and they should be of su�cient length in order to avoid brute force attacks. Addi-

tionally, passwords should never be stored in a system as plaintext; instead, they should be

hashed and salted.

If an attacker can break into a device by compromising the username, password, and

any other required credentials, he will obtain all user privileges, as well as full access to the

victims' data. The nature of IoT often perpetuates the risks of insu�cient authentication

since many user accounts are often left with elevated privileges by default.

Many tools exist to crack insu�cient authentication requirements. For our test case,

we plan to use hashcat, a popular password recovery tool, to gain access to both the end

devices and the server. We will combine brute force tactics with dictionary attacks in order

to carry out the attack. We will refer to the comprehensive wordlist 'rockyou.txt' and

32

potentially other lists for the dictionary portion of the attack.

5.2 Lack of Transport Encryption

Inadequate transport encryption is characterized by passing data over the network layer using

plaintext. In a live network environment, this can mean transporting sensitive information

such as passwords from one device to another without encrypting the information. This

leaves devices vulnerable to attackers who sni� network tra�c, engaging in what is known

as a man-in-the-middle (MiTM) attack. In order to mitigate MiTM attacks, users need to

encrypt their data using either SSL or TLS prior to transporting it across the network layer.

In a live IoT environment, because nodes often lack the storage and processing power

to single-handedly execute tasks, information is always moving from one device to the next.

Furthermore, even if some nodes enforce SSl/TLS encryption, it is imperative that all devices

enforce it, since compromising even a single device allows an attacker to prepare for di�erent

strategies.

In our example attack, we will use network sni�ng tools like ettercap and dsni�

to check for tra�c on our network. We will also use the advanced network sni�ng tool

bettercap, which is capable of more sophisticated tactics such as SSL stripping. We will

attempt to leverage more critical attacks such as device authentication using any data we

can obtain from the network.

5.3 Insecure Network Services

We de�ne network services as those running on a device which have open ports associated

with them, allowing external services to communicate with them. Common services include

SSH (port 22), Telnet (port 23), HTTP (port 80), and HTTPS (port 443). Every exposed

service introduces a new attack vector, since many services indirectly allow attackers to

execute DoS attacks, bu�er over�ows, fuzzing attacks, or authentication. It is critical that

33

devices minimize their attack surfaces as much as possible by closing all unnecessary ports,

or �ltering incoming tra�c via a �rewall.

Since a single IoT device typically only requires a few di�erent services to operate,

developers should con�dently keep only a few necessary ports open without compromising the

functionality of the system. Kevin Beaver reports that the most common ports vulnerable

to attack are FTP, SSH, Telnet, HTTP, SMTP, and various others [33]. As such, it is crucial

that we attempt to keep as many of these services closed as possible, and to prevent attacks

which may result in opening these services.

In our example attack, we will use nmap and Nessus to scan for open ports and

vulnerable services. Di�erent services and ports provide di�erent attack surfaces, so our

plan of action will change depending on what services are open. Any level of penetration

in an IoT environment is signi�cant, since even a single a�ected device can have dangerous

repercussions for an environment.

5.4 Poor Physical Security

Poor physical security is characterized by the ease at which an attacker can disassemble

or access a device once he obtains physical access to it. Preventing physical access to a

device requires hiding or storing the device in a secure environment, such as a locked room.

These actions depends entirely on the context of the user's environment as well as the

user himself, leaving this step completely out of our control. Physical security, however,

assumes that an attacker has already breached the physical access of the device, focusing

more on how easily an attacker can access, change, or delete data o� of a device once he has

physical access. Devices should always require su�cient authentication from the user prior

to loading any information, especially data which may be considered sensitive. If a device

contains information on an external storage medium, such as an SD card or USB drive, then

users should work to encrypt these devices so that attackers cannot access or modify them.

34

Additionally, devices should always verify updates to ensure that attackers cannot modify a

system via an external port.

In our test case, we will assume we already have physical access to all three RPis. We

will not assume, nor pursue, physical access to the server. Since all information on a RPi is

stored on an external micro SD card, we will attempt to crack and modify the SD card via

an SD card reader in order to leverage additional attacks.

6 Conclusion

In this paper, we have outlined our proposal for an IoT deployment environment that secures

against conventional attacks. IoTFlash draws inspiration from many projects and products,

notably resin.io, and incorporates the most e�ective elements of those projects into one

cohesive proof-of-concept. IoTFlash uses the Linux-based lightweight system HypriotOS,

which is built from the ground up to work with Docker. IoTFlash will use Python to handle

the setup process for a new cluster, or for adding new nodes to an existing cluster. Linux tools

such as 'ssh' and 'sshpass' will be used to initially communicate with fresh devices. We can

use 'OpenSSL' to generate secure certi�cates for the devices and encrypt their communication

between each other, allowing us to safely close the ssh ports.

We also propose four testable security attacks against IoTFlash in order to gauge how

e�ectively it would fare in a live environment. These attacks test weak points in our environ-

ment's authentication process, transport security, network services, and physical security.

In the next iteration of our paper, we will implement our project and report the

complete methodology of the environment. Additionally, we will create, test, and report

results from our four proposed attacks. Finally, we will include an appendix that includes

the most important code from the project, including the Python build script.

35

References

[1] Hunke, Nicolas, et al. "Winning in IoT: It's all about the Business Processes." Retrieved from

https://www.forbes.com. 2017.

[2] Chen, Deming et al. "Platform Choices and Design Demands for IoT Platforms: Cost, Power

and Performance Tradeo�s." IET Cyber-Physical Systems: Theory and Applications. 2016.

[3] Woolf, Nikky. "DDoS attack that disrupted internet was largest of its kind in history, experts

say." Retrieved from https://www.theguardian.com. 2016.

[4] "The 5 Worst Examples of IoT Hacking and Vulnerabilities in Recorded History." Retrieved

from https://iotforall.com. 2017.

[5] Greenberg, Andy. "Hackers Remotely Kill a Jeep on the Highway - With Me in It." Retrieved

from https://wired.com. 2015.

[6] "Getting started with Docker on your Raspberry Pi." Retrieved from https://blog.hypriot.com.

[7] Singh, Jatinder et al. "Twenty Security Considerations for Cloud-Supported Internet of

Things." Internet of Things Journal, IEEE. 2015.

[8] Kolias, Constantinos et al. "DDoS in the IoT: Mirai and Other Botnets." The IEEE Computer

Society. 2017.

[9] Rawlinson, Kristi. "HP Study Reveals 70 Percent of Internet of Things Devices Vulnerable to

Attack." Retrieved from www8.hp.com. 2014.

[10] Mezher, Michael. "Abbott Recalls 465,000 Pacemakers for Cybersecurity Patch." Retrieved

from http://www.raps.org. 2017.

[11] Thompson, Iain. "Wi-Fi Baby Heart Monitor May Have the Worst IoT Security of 2016."

Retrieved from https://www.theregister.co.uk. 2016.

36

[12] Adhikari, Richard. "WebcamMaker Takes FTC's Heat for Internet-of-Things Security Failure."

Retrieved from https://www.technewsworld.com. 2013.

[13] Schneier, Bruce. "The Internet of Things is Wildly Insecure�And Ofen Unpatchable." Retrieved

from https://www.schneier.com. 2014.

[14] Brown, Ravid. "Running Containers on Bare Metal vs. VMs: Performance and Bene�ts."

Retrieved from https://www.stratoscale.com. 2017.

[15] Pahl, Claus, et al. "A Container-based Edge Cloud PaaS Architecture based on Raspberry Pi

Clusters." 2016 IEEE 4th International Conference on Future Internet of Things and Cloud

Workshops.

[16] Ismail, Bukhary, et al. "Evaluation of Docker as Edge Computing Platform." Advanced Com-

puting Lab, MIMOS Berhad. 2016.

[17] Morabito, Roberto, et al. "Evaluating Performance of Containerized IoT Services for Clustered

Devices at the Network Edge." IEEE Internet of Things Journal, Vol. 4, NO. 4. 2017.

[18] "Docker Security." Retrieved from https://docs.docker.com.

[19] "Swarm Mode Overview." Retrieved from https://docs.docker.com.

[20] Brouwers, Marcel. "Security Considerations in Docker Swarm Networking." System and Net-

work Engineering, University of Amsterdam. 2017.

[21] Abdelrazik, Amr. "Docker vs. Kubernetes vs. Apache Mesos: Why What You Think You Know

is Probably Wrong." Retrieved from https://mesosphere.com. 2017.

[22] Parthasarathy, Akshai. "Kubernetes vs. Docker Swarm." Retrieved from

https://platform9.com. 2017.

[23] "Apache Mesos." Retrieved from http://mesos.apache.org/.

37

[24] Ristic, Ivan. "SSL and TLS Deployment Best Practices." Retrieved from https://github.com.

2017.

[25] Helme, Scott. "HTTPS Certi�cate Revocation is broken, and it's time for some new tools."

Retrieved from https://arstechnica.com. 2017.

[26] Sha, Letian et al. "IIoT-SIDefender: Detecting and defense against the sensitive information

leakage in industry IoT." Special Issue on Security and Privacy of IoT. 2017.

[27] Wang, Hua et al. "Editorial: Special Issue on Security and Privacy of IoT." Security and

Privacy of IoT. 2017.

[28] Mayer, Ruben et al. "The Fog Makes Sense: Enabling Social Sensing Services With Limited

Internet Connectivity." In Proceedings of The 2nd International Workshop on Social Sensing,

Pittsburgh, PA, USA, April 21 2017.

[29] "How does resin.io work?" Retrieved from https://resin.io.

[30] "Top IoT Vulnerabilities." From https://www.owasp.org. 2014.

[31] Bencho�, Brian. "Introducing the Raspberry Pi 3." Retrieved from https://hackaday.com. 2016

[32] Ndegwa, Amos. "What is HSTS?" Retrieved from https://blog.stackpath.com.

[33] Beaver, Kevin. Hacking For Dummies. 5th ed., For Dummies. 2015.

38