Project Title: Investigation of Airborne Network Vulnerabilities Project Number: 2102

Royal Military College of Canada Collège militaire royal du Canada

DESIGN OF A TIME-BASED INTRUSION DETECTION

ALGORITHM FOR THE MIL-STD-1553

16 January 2019

This document is a formal deliverable of

Project 2102

Submitted to: Directorate of Technical Airworthiness and Engineering Support 8 (DTAES-8) National Defence Headquarter 101 Colonel By Drive (455 blvd de la Carrière) Ottawa, Ontario K1A 0K2 Attention: Mr. Stephen Sterling

Prepared by:Capt. Blaine Losier

Reviewed by:Dr. Ron SmithDr. Vincent Roberge

Royal Military College of Canada PO Box 17000 Stn Forces Kingston, OntarioK7K 7B4

DRAFT dated March 25, 2017

TIME-BASED INTRUSION DETECTION

ON MIL-STD-1553

DETECTION D’INTRUSION EN

FONCTION DU TEMPS SUR

MIL-STD-1553

A Thesis Submitted to the Division of Graduate Studiesof the Royal Military College of Canada

by

Blaine Francis Losier, BEng

Captain

In Partial Fulfillment of the Requirements for the Degree ofMaster of Applied Science in Software Engineering

June, 2016c© This thesis may be used within the Department of National Defencebut copyright for open publication remains the property of the author.

This work is dedicated to all of my friends and family for their continuedencouragement and support.

i

Acknowledgments

This research is supported by the Directorate of Technical Airworthiness andEngineering Support (DTAES) - Air Warfare Engineering Section. This re-search is supervised at the Royal Military College of Canada (RMCC) by Dr.Ron Smith and Dr. Scott Knight of the Department of Electrical and Com-puter Engineering (ECE). The author’s attendance at RMCC is sponsored bythe CF-18 Weapon System Manager Detachment (WSM Det) - Mirabel.

ii

Abstract

In today’s modern era of technology, cyber warfare is an ever evolving threat.Cyber attackers are now not only focused on traditional PC networks, but arebeginning to target other types of networks as well, such as industrial controlsystems and vehicle platforms. In order to protect these sometimes safetycritical systems, we need to meet this threat and develop defensive techniquesto counter such attacks. This starts with being able to monitor and detect anattack on the system.

Classified research, recently conducted, has highlighted a number of vul-nerabilities in the MIL-STD-1553 protocol. This means there is a number ofpotential exploits which could be used to target a MIL-STD-1553 system ina cyber attack. The MIL-STD-1553 standard is utilized on-board numerousmilitary platforms, often integrating a variety of systems, some of which aresafety and mission critical. This makes protecting MIL-STD-1553 networksextremely important.

Communications in a MIL-STD-1553 network are controlled using a verystrict and deterministic schedule. This leads to the ability to apply time-basedintrusion detection techniques against it. This thesis will show that time-basedintrusion detection techniques can be used to build an intrusion detectionengine capable of detecting anomalous events in the timing properties of MIL-STD-1553 traffic, that are related to a cyber attack.

iii

Resume

***Insert French Translation of Abstract***

iv

Contents

Acknowledgments ii

Abstract iii

Resume iv

Contents v

List of Tables viii

List of Figures ix

List of Acronyms x

1 Introduction 11.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Statement of Aim . . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Research Approach . . . . . . . . . . . . . . . . . . . . . . . . . 21.4 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.5 Thesis Structure . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Background 52.1 MIL-STD-1553 . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 52.1.2 Network Communications . . . . . . . . . . . . . . . . . 62.1.3 Timing Requirements . . . . . . . . . . . . . . . . . . . 62.1.4 Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2 Intrusion Detection Systems . . . . . . . . . . . . . . . . . . . . 92.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2.2 Secure System Simplex Architecture . . . . . . . . . . . 9

v

Contents

2.2.3 SecureCore . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 Detecting Timing Anomalies in MIL-STD-1553 Traffic 133.1 Anomaly Detection Approach . . . . . . . . . . . . . . . . . . . 133.2 Development Environment . . . . . . . . . . . . . . . . . . . . . 133.3 Timing Profile Generation . . . . . . . . . . . . . . . . . . . . . 15

3.3.1 Timing Parameters . . . . . . . . . . . . . . . . . . . . . 153.3.2 Timing Property Extraction . . . . . . . . . . . . . . . . 163.3.3 Profile Building . . . . . . . . . . . . . . . . . . . . . . . 17

3.4 Detection Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 193.5 Summary of Timing Anomaly Detection . . . . . . . . . . . . . 22

4 Validation of Detection Engine 234.1 Test Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234.2 Simulation Environment Testing . . . . . . . . . . . . . . . . . 23

4.2.1 Test Bench Configuration . . . . . . . . . . . . . . . . . 234.2.2 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . 244.2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

Baseline . . . . . . . . . . . . . . . . . . . . . . . . . . . 25Scenario 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 25Scenario 2A . . . . . . . . . . . . . . . . . . . . . . . . . 26Results Summary . . . . . . . . . . . . . . . . . . . . . . 26

4.3 Hot Bench Testing . . . . . . . . . . . . . . . . . . . . . . . . . 274.3.1 Test Bench Configuration . . . . . . . . . . . . . . . . . 274.3.2 Experiment . . . . . . . . . . . . . . . . . . . . . . . . . 284.3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

Baseline . . . . . . . . . . . . . . . . . . . . . . . . . . . 29Scenario 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 29Scenario 2A . . . . . . . . . . . . . . . . . . . . . . . . . 30Scenario 2B . . . . . . . . . . . . . . . . . . . . . . . . . 30Scenario 3 . . . . . . . . . . . . . . . . . . . . . . . . . . 31Results Summary . . . . . . . . . . . . . . . . . . . . . . 32

4.4 Challenges and Limitations . . . . . . . . . . . . . . . . . . . . 324.4.1 Simulation Environment . . . . . . . . . . . . . . . . . . 324.4.2 Hot Bench Environment . . . . . . . . . . . . . . . . . . 334.4.3 Analyst Workstation and Software . . . . . . . . . . . . 334.4.4 The Real System . . . . . . . . . . . . . . . . . . . . . . 33

4.5 Summary of Validation . . . . . . . . . . . . . . . . . . . . . . . 34

5 Discussion and Conclusion 35

vi

Contents

5.1 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

6 Appendix 1 376.1 The Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

6.1.1 Scenario1 . . . . . . . . . . . . . . . . . . . . . . . . . . 376.1.2 Scenario2A . . . . . . . . . . . . . . . . . . . . . . . . . 376.1.3 Scenario2B . . . . . . . . . . . . . . . . . . . . . . . . . 376.1.4 Scenario3 . . . . . . . . . . . . . . . . . . . . . . . . . . 37

6.2 The Test Environments . . . . . . . . . . . . . . . . . . . . . . 376.2.1 Simulation Environment . . . . . . . . . . . . . . . . . . 376.2.2 Hot Bench . . . . . . . . . . . . . . . . . . . . . . . . . . 37

Bibliography 38

vii

List of Tables

4.1 Mux2 RTs Simulated by the Simulation Environment . . . . . . . 244.2 Avg %-Difference between Baseline and Scenario 1 Profiles on the

Simulation Environment . . . . . . . . . . . . . . . . . . . . . . . . 254.3 Avg %-Difference between Baseline and Scenario 2A Profiles on

the Simulation Environment . . . . . . . . . . . . . . . . . . . . . . 264.4 Simulation Environment Testing Detection Strengths . . . . . . . . 264.5 Configuration of RTs on Mux2 of the Hot Bench . . . . . . . . . . 284.6 Configuration of RTs on Mux1 of the Hot Bench . . . . . . . . . . 284.7 Avg %-Difference between Baseline and Scenario 1 Profiles on the

Hot Bench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.8 Avg %-Difference between Baseline and Scenario 2A Profiles on

the Hot Bench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.9 Avg %-Difference between Baseline and Scenario 2B Profiles on

the Hot Bench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 314.10 Avg %-Difference between Baseline and Scenario 3 Profiles on the

Hot Bench . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 324.11 Hot Bench Testing Detection Strengths . . . . . . . . . . . . . . . 32

viii

List of Figures

2.1 MIL-STD-1553 Network Architecture . . . . . . . . . . . . . . . . 52.2 MIL-STD-1553 Words . . . . . . . . . . . . . . . . . . . . . . . . . 72.3 Manchester Encoding . . . . . . . . . . . . . . . . . . . . . . . . . 72.4 MIL-STD-1553 Timing Requirements . . . . . . . . . . . . . . . . 82.5 Major and Minor Frames . . . . . . . . . . . . . . . . . . . . . . . 92.6 The GaIT Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

3.1 The Detection Process . . . . . . . . . . . . . . . . . . . . . . . . . 143.2 The Detection Engine Architecture . . . . . . . . . . . . . . . . . . 143.3 The Timing Property Extractor Module Architecture . . . . . . . 173.4 The Profile Generator Module Architecture . . . . . . . . . . . . . 183.5 KDE Profile of Bus Utilization of Mux2 from the Simulation En-

vironment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.6 Histogram Profile of Bus Utilization of Mux2 from the Simulation

Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.7 Histogram Profile of Bus Utilization of Mux2 from the Simulation

Environment with Best Fit Curve . . . . . . . . . . . . . . . . . . . 213.8 The Profile Comparator Module Architecture . . . . . . . . . . . . 21

4.1 Simulation Environment Test Bench Physical Configuration . . . . 244.2 Hot Bench Test Bench Physical Configuration . . . . . . . . . . . . 27

ix

List of Acronyms

1. API - Application Programming Interface2. ASCII - American Standard Code for Information Interchange3. BC - Bus Controller4. CPU - Central Processing Unit5. DTAES - Directorate of Technical Airworthiness and Engineering Sup-

port6. ECE - Electrical and Computer Engineering7. GaIT - Gaussian methods for Intrusion Detection using Timing profiles8. GE - General Electric9. IDE - Integrated Development Environment

10. IDS - Intrusion Detection System11. IP - Internet Protocol12. KDE - Kernel Density Estimation13. NSM - Network Security Monitoring14. PC - Personal Computer15. PCI - Peripheral Component Interconnect16. pdf - Probability Density Function17. RAM - Random Access Memory18. RMCC - Royal Military College of Canada19. RT - Remote Terminal20. S3A - Secure System Simplex Architecture21. SLTS - System Level Test Station22. WSM Det - Weapon System Manager Detachment

x

1 Introduction

1.1 Motivation

The realm of cyber warfare is evolving. It has grown to not only include tra-ditional Personal Computers (PCs)and Internet Protocol (IP) type networks,but also other platforms such as vehicles and industrial control systems. Inorder to combat this evolving cyber threat we must be able to first monitorand detect it.

Aircraft networks were once considered secure. Being isolated from othernetworks they were thought to be securely air gapped. However a numberof recent studies have shown otherwise. [1–7]. In their work, both Thanthryand Pendse [2] and Bhagavathula and Pendse [3] discuss air-gap jumping asa threat to cyber secuity aboard aircraft. The wide variety or radio andsatellite links made with a modern aircraft all offer potential vectors for air-gap jumping.

In order to protect against this threat Bhagavathula and Pendse [3] ad-vocate for the use of Network Security Monitoring (NSM) techniques. Inorder to effectively respond to potential cyber threats onboard the aircraft inreal-time, they suggest the use an anomaly-based Intrusion Detection System(IDS), supplemented by a signature-based system. Due to it’s inability to de-tect zero-day attacks, a signature-based IDS alone would not provide sufficientprotection to the safety critical aircraft networks.

Around the world, governments are becoming more aware of the growingcyber threat and beginning to invest more into cyber security. It was lessthan a year ago when Canada’s Public Safety Minister announced that $237million would be dedicated to improving cyber security in Canada over thenext five years [8]. Even more recently, the United States Congress announcedit’s plans to spend $200 million on penetration testing of all US militaryweapons platforms. During the initial stages of this project, “penetrationtesters were able to find the threats with merely “novice- and intermediate-level” techniques. No advanced hacking skills were needed to find system

1

Statement of Aim1.2.

vulnerabilities” [9]. The CybAIR RadBox [10, 11], developed by Thales Raytheon, is a recent

example of the military’s attempt to counter cyber threats against its radar systems. The CybAIR RadBox is meant to act as a firewall and IDS for radar systems that may be connected to various military networks. Little technical detail on the system is available, but it has been published that it works on the concept of anomaly detection, analyzing traffic patterns and behaviours being displayed by the radar for anything which may be considered unusual.

In military aviation, a recent research conducted by Stan and al. [12]revealed potential attacks that could compromise the integrity and availability of a MIL-STD-1553 communications bus. In their work, the authors present a comprehensive study of the threats faced by systems that use MIL-STD-1553, including possible attacks vectors and consequences. They also propose an approach to intrusion detection. Given the wide spread use of MIL-STD-1553 in military platforms, it is important to continue cyber security research intoMIL-STD-1553, particularly in defending it from a potential cyber threat.

1.2 Statement of Aim

The aim of this thesis is to develop an intrusion detection engine capable of detecting time-based anomalies in MIL-STD-1553 traffic, with the intention that the engine will be the first step towards the development of a MIL-STD- 1553 intrusion detection system to protect against vulnerabilities in the MIL- STD-1553 protocol. This thesis will utilize time-based intrusion detection techniques used in other types of systems and adapted for use in a MIL-STD- 1553 network in which communications are highly deterministic and follow astrict periodic schedule.

1.3 Research Approach

This research was conducted in three phases. Commencing with Phase one, a literature review was conducted examining existing documentation on the MIL-STD-1553 protocol, including recent classified research on a vulnerability analysis of MIL-STD-1553. Also examined in the literature review was docu- mentation on various IDS architectures and the detection techniques utilized by these architectures. In Phase two, a technique for detecting intrusions on a MIL-STD-1553 bus was chosen and a detection engine utilizing the selected technique was developed. The selected technique was chosen as having a likely

2

1.4. Contributions

high possibility of success based upon the research conducted in Phase one.Phase three consisted of testing the developed detection engine to validateits ability to successfully detect anomalies created by a cyber intrusion on aMIL-STD-1553 bus.

1.4 Contributions

Due to the work completed in this thesis, the following contributions weremade:

1. It was shown that despite its vulnerabilities, a MIL-STD-1553 systemcan be protected through NSM. This thesis demonstrates that NSMtechniques traditionally applied to other types of networks can also beadapted to successfully detect potential cyber intrusions on a MIL-STD-1553 bus.

2. It was demonstrated that time-based anomaly detection is an effectivemethod of detecting potential cyber intrusions on a MIL-STD-1553 bus.Due to the very deterministic timing of MIL-STD-1553 traffic it is dif-ficult to affect messages on the bus without altering at least one of thetraffic’s timing characteristics. It was shown that by monitoring a widearray of timing properties, the chance of detecting an intrusion alteringone of those properties is extremely high.

3. The work in this thesis provides a base, upon which an effective IDSfor MIL-STD-1553 can be built. This is an important achievement inmoving forward with protecting our various platforms which utilize MIL-STD-1553.

1.5 Thesis Structure

This thesis is divided into 5 chapters. Chapter 2 provides an overview of theMIL-STD-1553 protocol as well as the various IDS architectures and tech-niques considered for use in this thesis. Only the aspects of the MIL-STD-1553 protocol relevant to the work in this thesis are covered, with referencesprovided to more detailed descriptions of the protocol. Chapter 3 outlinesthe chosen detection technique and its implementation to create an intrusiondetection engine capable of detecting timing based anomalies on a MIL-STD-1553 bus. Chapter 4 provides details of the testing and validation processcarried out against the developed detection engine. A detailed description ofthe results for each of the test scenarios conducted is provided. Chapter 5

3

1.5. Thesis Structure

concludes this thesis, summarizing the work completed, contributions made,and potential for future work.

4

2 Background

2.1 MIL-STD-1553

2.1.1 Overview

MIL-STD-1553 [12–15] is a common networking standard utilized on boardmany modern military platforms to facilitate communications between varioussystems. It is was developed by the United States Department of Defense in1975 to be a reliable multiplexed networking standard.

A MIL-STD-1553 network uses a common bus to connect all devices to-gether. Each device is connected to the bus by a ‘stub’. For redundancy,multiple optional buses can also be used. The basic architecture of a MIL-STD-1553 network can be seen in figure 2.1.

Bus

Controller

Remote

Terminal (RT)

RT RT

Sub-Systems Sub-Systems Sub-Systems

Main Bus

Optional

Redundant Buses

Figure 2.1: MIL-STD-1553 Network Architecture

5

2.1. MIL-STD-1553

2.1.2 Network Communications

The individual devices which connect on a MIL-STD-1553 network are referredto as Remote Terminals (RTs). Each RT is assigned a unique address between1 and 31. A Bus Controller (BC) controls the communications between RTsfollowing a strict deterministic schedule. Communications on the bus occurin the form of “messages”, using a Command/Response style protocol.

Messages are comprised of a series of 20-bit words, always beginning witha command word. The 3 types of MIL-STD-1553 words are outlined below.Figure 2.2 breaks down the composition of each word type.

1. Command Word: The command word is used by the BC to issue com-mands to an RT. Valid commands may include a transmit or receive datacommand or a command from a list of valid mode codes. The commandword has an addressing field RT to indicate which RT and sub-systemthe command is for. It also has a fields to indicate if the targeted RT isto transmit or receive, and the number of data words. The same datawords field could also be used to contain a command in the form of amode code.

2. Data Word: The data word is used to transfer data across the network,and contains 16 bits of data.

3. Status Word: The status word is used by an RT to respond to commandsfrom the BC and indicate status and error information using variousflags. It also has an addressing field to indicate which RT and subsystemthe status is from.

Messages are encoded using the Manchester encoding scheme for transmis-sion on the bus. The data is XOR’d with a 1 MHz clock and the transitions ofthe resulting signal at mid-bit time are then used to determine its bit values.A bit value of ‘1’ is represented by a high to negative voltage transition, whilea transition from negative to high voltage represents a bit value of ‘0’. Thisprocess is illustrated in figure 2.3.

2.1.3 Timing Requirements

MIL-STD-1553 contains a number of strict timing requirements which the BCmust follow. These requirements are described below.

1. Response Time: Upon transmitting a command word to an RT, theBC expects to receive a response within 4 − 12µs. A response receivedbetween 12 − 14µs will generate a “Late Response” error. After 14µs,the BC will time out and generate a “No Response” error if a responsefrom the RT has not been received.

6

2.1. MIL-STD-1553

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Sync RT Address T/R Sub-Address / Mode Data Count / Mode Code P

Sync

Bit No.

Command

Word

Data

Word Data P

Status

Word Sync RT Address ME IN SR Reserved BC BS SF DC TF P

T/R: Transmit/Receive

P: Parity

ME: Message Error

IN: Instrumentation

SR: Service Request

BC: Broadcast Command Received

BS: Busy

SF: Subsystem Flag

DC: Dynamic Bus Control Acceptance

TF: Terminal Flag

Legend

Figure 2.2: MIL-STD-1553 Words

Bit Time

1 1 1 0 1 0 0 1 0

1 MHz Clock

Data

Encoded

Signal

Figure 2.3: Manchester Encoding

7

2.1. MIL-STD-1553

*

*

*

*

Command Word

Status Word Data Word BC to RT

Data Transfer

RT to BC Data Transfer

RT to RT Data Transfer

Mode Code Command

.... Command Word

Command Words (RT1&2)

Command Word

Data Word Data Word

Status Word Data Word .... Data Word Data Word

Status Word Data Word .... Data Word Data Word

Status Word

* Response Time

Time

Figure 2.4: MIL-STD-1553 Timing Requirements

2. Multiple Data Words: Data words within a message are to be trans-mitted consecutively and continuously without any gap in time betweenthem.

3. Inter-message Gap: The BC shall maintain a 4µs gap after the endof each message before beginning to transmit the next message in itsschedule.

The response time and multiple data words requirements within a messageare illustrated in figure 2.4, and the inter-message gap requirement betweenmessages is illustrated in figure 2.5.

2.1.4 Scheduling

Scheduling is conducted by the BC using “frames”, as shown in figure 2.5.Messages are first grouped together in minor frames. Minor frames are thengrouped into higher level major frames. Each major frame is then assigned afrequency at which it will transmit each of its minor frames one at a time.

Certain events such as a toggle switch being activated may also generateaperiodic traffic on the bus. The BC will handle aperiodic traffic by appendingit to the end of the next minor frame in the schedule which has space available.

8

2.2. Intrusion Detection Systems

Time

Major Frame

Minor Frame 1 .... Minor Frame 2 Minor Frame 3 Minor Frame N Minor Frame N-1

Minor Frame

# Message 1 # Message 2 # Message 3 # # Message M-1 .... # Inter-Msg Gap

Message M

Figure 2.5: Major and Minor Frames

2.2 Intrusion Detection Systems

2.2.1 Overview

Since MIL-STD-1553 traffic is controlled using a strict deterministic schedule,any malicious activity on the bus is likely to create a disruption in the traffictiming which would be detectable by a time-based intrusion detection system.Recent research by Yoon et al [16] draws upon similar concepts, examiningcode execution times on multicore processors in real-time embedded systems,and is based off research conducted by both Mohan et al [17] and Zimmer etal [18]. This thesis builds upon the techniques developed by Yoon et al [16]and adapts them for use in a MIL-STD-1553 network.

2.2.2 Secure System Simplex Architecture

Mohan et al [17] developed a time-based intrusion detection architecture calledSecure System Simplex Architecture (S3A). S3A relies on the concept of real-time control systems having a deterministic execution behavior. It assumesthat “the system consists of a set of periodic, real-time tasks with stringenttiming and deadline constraints managed by a real-time scheduler” [17]. S3Amonitors the side channels of a system’s Central Processing Unit (CPU) fortiming information and uses simple dynamic analysis to build a profile of itsexecution timings. The same timing information is then monitored at run-time and compared against the original profile for any significant variations.Any variations greater than a preset threshold value of 0.6 µs are flagged asa possible intrusion.

By monitoring CPU side channels, S3A makes the system difficult to at-tack without being detected. The side channel’s output would need to be

9

2.2. Intrusion Detection Systems

manipulated and masked to resemble the normal behavior of the system in or-der to avoid detection. There are numerous side channels available to monitorhowever S3A focuses particularly on the execution timing side channel. Someother side channels which Mohan et al [17] consider are very similar to someof the timing properties of MIL-STD-1553 that are profiled in the detectionalgorithm developed in this thesis. These side channels include:

1. Task activation periodicity, similar to RT periodicity; and2. Bus access times and durations, similar to bus utilization.

2.2.3 SecureCore

Using the S3A architecture as a base, Yoon et al [16] developed the SecureCorearchitecture to perform intrusion detection on multicore real-time embeddedsystems. Similar to S3A, SecureCore uses statistical learning to build a profileof execution timings which it then compares to timing information monitoredat run-time.

By using statistical learning methods in building its timing profiles, Se-cureCore is able to account for any slight variations in the monitored timingdata that may be caused by noise, processing delays, or control flow variations.The profiles are built by dividing each process running on the system into dis-tinct code blocks. A sample of the execution times of each code block is takenand used to generate a probability density function (pdf) using Kernel DensityEstimation (KDE). KDE allows an unknown pdf, f(e), to be estimated from aset of known samples, e1, e2, . . . , em [19]. In the case of SecureCore, the sam-ples of execution times of each code block. The pdf generated then representsthe likelihood that a specific execution time of a code block is legitimate. fk

in figure 2.6 represents a sample SecureCore profile.Equation 2.1 shows how the pdf is calculated, where Kh is the Kernel

function and h is the bandwidth. In the case of SecureCore, Yoon et al [16]elected to utilize the Gaussian Kernel shown in 2.2, where −∞ < x <∞. Themethod for selecting an appropriate Kernel to use appears to be arbitrary.There is no evidence in the literature which suggests a method for choosing aKernel. It is only stated by Parzen [19] that a valid Kernel function must benon-negative; it must have a mean of zero; and it must have an integral of one.The purpose of the bandwidth, h, is to act as a smoothing function for the pdf.The optimal bandwidth should be selected to create the minimal mean inte-grated squared error. There are a number of methods to determine the optimalbandwidth. Jones et al [20] give an overview of some of these methods, includ-ing least squares cross-validation, biased cross-validation, solve-the-equationplug-in approach, and smoothed bootstrap.

10

2.2. Intrusion Detection Systems

fh(e|e1, e2, . . . , em) =1

m

m∑i=1

Kh(e− ei) (2.1)

Kh(x) =1√2πh

e−x2/2h2(2.2)

After building a baseline profile, SecureCore then divides the e-axis ofthe resultant pdf, fk, into N − 1 bins of width u. The max and min val-ues of fk are defined as emax and emin respectively, and can be used to de-scribe each bin, i, as [emin + i · u, emin + (i+ 1) · u], where i = 0, . . . , N − 1,and u = (emax − emin)/(N − 1). To simplify this, the boundaries of each bincan be defined by bi

∗min and bi

∗max. Execution times, e∗, monitored at run time

are then associated with a specific bin and a simple estimation of the areaunder the curve in that bin is calculated as the probability that the executiontime is legitimate, P k(e∗). Equation 2.3 shows how P k(e∗) is estimated usingdiscrete integrations, where i∗ = |(e∗ − emin)/u|. The value of P k(e∗) is com-pared against a user-defined threshold value, θ, to determine if it is anomalousand should be flagged as a possible intrusion. Since Yoon et al [16] utilizedthe Gaussian kernel in building their baseline profiles they have thus coinedthis detection process as the “Gaussian methods for Intrusion Detection usingTiming profiles” (GaIT). An illustration of the GaIT process is shown in figure2.6.

P k(bi∗min ≤ e ≤ bi

∗max) ≈ fk(bi

∗min) · u (2.3)

There are some flaws in the GaIT process however. It does not take fulladvantage of the timing profile. Rather than rebuilding the profiles at run-time and comparing the execution times of a code block over time it onlycompares specific instances of run-time execution times against the baselineprofile. Further, by defining a threshold value, θ, it creates a cut-off line acrossthe profile which may have parts of the pdf below it. Any execution timeswhich fall into the bins representing these parts of the pdf will automaticallybe flagged as anomalous even though there may be some legitimate instancesof those values. These bins are represented in figure 2.6 by the asterisks alongthe e-axis.

11

2.2. Intrusion Detection Systems

Figure 2.6: The GaIT Process

12

3 Detecting Timing Anomaliesin MIL-STD-1553 Traffic

3.1 Anomaly Detection Approach

The approach used in this thesis to detect potential intrusions on the MIL-STD-1553 bus is derived from the GaIT proccess used in SecureCore [16].Rather than examining code blocks and profiling their execution timings, theMIL-STD-1553 traffic is examined and a select set of its timing propertiesprofiled. Like SecureCore, a baseline profile of the MIL-STD-1553 traffic onthe bus is taken. Profiles of the traffic at run time are then generated using asliding window approach and compared against the baseline profile. A percentdifference in the baseline and run time profiles over a user defined thresholdwill trigger an alert as an anomaly indicating a potential intrusion. Figure 3.1illustrates this process.

In order to implement this process a simple detection engine architecturewas created. The architecture utilizes reusable modules to conduct both thetiming property extraction and profile building steps in the process. Anothermodule then conducts the comparison of the baseline and runtime profiles todetermine if there are any anomalies which may indicate a potential intrusion.A diagram of the high level architecture can be seen in figure 3.2. The archi-tecture of each of the modules will be broken down further in the followingsections of this chapter.

3.2 Development Environment

To develop the detection software used in this thesis, a PC with a Periph-eral Component Interconnect (PCI) -Express Card slot was required in orderto support the R15-EC MIL-STD-1553 card used to read the traffic off theMIL-STD-1553 bus. In order to efficiently process the large amounts of data

13

3.2. Development Environment

Figure 3.1: The Detection Process

Figure 3.2: The Detection Engine Architecture

14

Timing Profile Generation3.3.

required the PC contained a 3.20 GHz hyper-threaded quad-core processor and 8 GB of Random Access Memory (RAM). The PC was also required to have General Electric (GE) BusTools installed in order to interface with the R15-EC card. The provided Application Programming Interface (API)for BusTools is written in C. As such the software developed for extracting the timing data from the MIL-STD-1553 traffic using the R15-EC card was also written in C using the Qt Creator Integrated Development Environment(IDE). Profiling the extracted timing data and comparing the profiles for de-tections was done using Microsoft Office’s Excel.

3.3 Timing Profile Generation

Both baseline and run time profiles are generated using the same technique. The timing parameters to be profiled are first extracted from the MIL-STD- 1553 traffic on the bus using the extraction software developed. The extracteddata is then used as input to generate the actual profiles.

3.3.1 Timing Parameters

The MIL-STD-1553 standard [13], outlines several timing properties which are present in MIL-STD-1553 traffic. For this thesis, five specific parameters were selected to be examined. Based on what is known from the classified results of the vulnerability assessment conducted by Stan et al. [12], it is believed that these five parameters provide the best likely hood of presenting a detectable anomaly that is the result of an intrusion on the bus. Whilean intrusion may not introduce an anomaly into all of the selected timing parameters, it is believed that at least one will be affected depending onthe nature of theintrusion. The properties selected to be examined are as follows:

1. Response Time - The time measured between when the BC transmitsa command word to an RT, and when it receives a response from that RT.

2. Inter-Message Gap - The time measured between when the BC receivesthe final status word of a message, and when it transmits the first com- mand word of the next message.

3. Data Throughput - The amount of data words transmitted across thebus during a 50 ms frame.

4. Bus Utilization - Measured as a percentage, the amount of time the bus is utilized for transmitting MIL-STD-1553 words during a 50 ms frame.

5. Periodicity - The mean time between the BC sending messages to aspecific RT, measured over a 50 ms frame.

15

3.3. Timing Profile Generation

With the exception of periodicity, all of the timing parameters can bemeasured with respect to all traffic on the bus, or with respect to traffic onlyto and from a specific RT. Because periodicity is specific to each RT it cannot be measured for all traffic on the channel. 50 ms was selected as the frametime for measuring data throughput, bus utilization, and periodicity as thatis the actual length of a minor frame in the BC schedule of the system utilizedas a test bench [21].

3.3.2 Timing Property Extraction

The Timing Property Extractor module of the Detection Engine Architectureis used to extract the desired timing properties from MIL-STD-1553 traffic.A breakdown of the architecture of the Timing Property Extraction moduleis shown in figure 3.3. The Traffic Capture component consists of the R15-ECcard in conjunction with the GE BusTools API [22] which is used to capturea recording of the traffic in BusTools’ bmdx file format. The bmdx file canthen be replayed using the card and the API used to extract the desiredtiming values from the played back traffic. During the replay of the recordedtraffic, the Traffic Filter component can filter the traffic so that only messagesaddressed to or from a specific RT are presented to the Property Extractorcomponent for processing. Each of the individual desired timing propertiesis then extracted by the Property Extractor Component using the processesdescribed below:

1. Response Time - The response time is simply extracted by using the APIto read the API BM MBUF.response1 field in each message recordedin the bmdx file.

2. Inter-Message Gap - The inter-message gap is calculated for each mes-sage by utilizing the BusTools InterMessageGap() function, automat-ically built into the API.

3. Data Throughput - Data throughput is calculated over a 50 ms time win-dow by summing the API BM MBUF.command1.wcount field fromeach message which occurs within the window.

4. Bus Utilization - Bus utilization is calculated over a 50 ms time windowby beginning with utilization variable initialized to 50 ms at the startof the window. Each time gap that occurs during the window suchas the response times of the RTs and the inter-message gaps are thensubtracted from the utilization variable. At the end of the time window,the remaining value of the utilization variable is compared against thebase 50 ms value to calculate a percentage which is the bus utilizationvalue.

16

3.3. Timing Profile Generation

Live 1553 Traffic

Traffic Capture (R15-EC)

Traffic Filter (Selected RT)

Property Extractor

Timing Data Files

Figure 3.3: The Timing Property Extractor Module Architecture

5. Periodicity - Periodicity is calculated over a 50 ms time window by takingthe mean of the differences between theAPI BM MBUF.time.microsecondsfields in each pair of consecutive messages within the window.

The extracted values are then written to a tab delimited American Stan-dard Code for Information Interchange (ASCII) file to be later imported toexcel for the profile building process.

3.3.3 Profile Building

The Profile Generator module of the Detection Engine Architecture is used tobuild the actual profiles required for the detection algorithm. A breakdown ofthe architecture of the Profile Generator module is shown in figure 3.4. Thetiming data files generated by the Property Extractor module and parsed into

17

3.3. Timing Profile Generation

Timing Data Files

Excel (Histogram Tool)

Timing Profiles

Figure 3.4: The Profile Generator Module Architecture

excel and the excel histogram tools is used to generate histograms of the data.This is done for each of the five desired timing properties. The histogramscreated, represent the profiles of their respective timing properties.

This is a simpler method of how SecureCore [16] operates using KDEand then dividing the generated pdf into bins. A histogram is essentially afrequency distribution already divided into bins. In fact both methods givevery similar results, as is evident in figures 3.5 and 3.6. Figure 3.5 representsa profile generated using KDE as it would be in SecureCore, while figure 3.6represents a profile generated using a histogram. Both profiles were developedusing the same set of data. Fitting a curve to the histogram from figure 3.6 asshown in figure 3.7 reveals an extremely similar profile to that seen in figure3.5.

The histogram profiles are all generated using a standard set of bins foreach of the selected timing properties being examined so as to allow for easycomparison. For fair comparison both runtime and baseline profiles are builtusing the same size data sets. For response times and inter-message gaps adata set of 20000 messages is used, and for data throughput, bus utilization,and periodicity, a data set of 1400 frames is used. The size of these datasets also represent the window size of the sliding window used. Through ex-

18

3.4. Detection Algorithm

Figure 3.5: KDE Profile of Bus Utilization of Mux2 from the SimulationEnvironment

perimentation, it was determined that these data set sizes are the minimumpoint at which the profiles stabilize. Less data points may produce inaccu-rate results as the profiles are not yet stable, and more data points do nothave a significant affect on the profiles and are simply redundant for profilingpurposes.

3.4 Detection Algorithm

Detection is completed by the Profile Comparator module. A breakdown ofthe architecture of the Profile Comparator module is shown in figure 3.8.Toperform detection using the histogram profiles, the baseline and runtime pro-files are compared bin to bin by the Profile Comparison component, whichcalculates a percent difference between the two profiles for each bin using thestandard percent difference formula shown in equation 3.1. An overall percentdifference for the profile is then calculated by taking the mean of the bin per-cent differences. The overall profile percent difference is then compared by theThreshold Detector component against a user defined threshold value to de-

19

3.4. Detection Algorithm

Figure 3.6: Histogram Profile of Bus Utilization of Mux2 from the SimulationEnvironment

termine if a potential intrusion alert is to be generated by the Alert Generatorcomponent.

BinPercentDiff =

∣∣∣∣baseline− runtimebaseline× 100

∣∣∣∣ (3.1)

In order to capture an unknown anomaly located potentially anywhere inthe recorded MIL-STD-1553 traffic, the runtime profiles are generated contin-uously across all of the recorded MIL-STD-1553 traffic data using a slidingwindow. The windows are generated consecutively throughout the data, iethe next window starts at the next message following the last message of theprevious window. Each runtime profile is then compared against the baselineprofile as described in the previous paragraph. If an anomaly is detected in aspecific window, the time stamps of the messages in that window can then beused to narrow down when during the recording that the anomaly occurredand that information can be used to further investigate the potential intrusion.

20

3.4. Detection Algorithm

Figure 3.7: Histogram Profile of Bus Utilization of Mux2 from the SimulationEnvironment with Best Fit Curve

Runtime Profiles Baseline Profiles

Profile Comparison

Threshold Detector

Alert Generator

Alerts

Figure 3.8: The Profile Comparator Module Architecture

21

3.5. Summary of Timing Anomaly Detection

3.5 Summary of Timing Anomaly Detection

There are five inherent timing properties from the MIL-STD-1553 Standard[13] which under normal circumstances are believed to behave in a fairly reg-ular manner. It is believed that an intrusion to the system would affect thetraffic on a MIL-STD-1553 bus in such a way as to create a detectable anomalyin the behaviour of at least one of the selected timing properties. Theseproperties include response times, inter-message gaps, data throughput, busutilization, and periodicity.

To detect anomalies in the timing properties the MIL-STD-1553 traffic ismonitored and the timing data extracted and used to build a baseline profile.This process then continues to build runtime profiles for comparison to thebaseline using a sliding window system. Differences between the baseline andruntime profiles above a user defined threshold value could indicate a potentialintrusion on the system.

22

4 Validation of DetectionEngine

4.1 Test Scenarios

To test the detection method described in Chapter 3, a number of test scenar- ios were recorded for which to test against. With the exception of the baseline scenario, the test scenarios utilized shall simply be referred to as, Scenario 1, Scenario 2A, Scenario 2B, and Scenario 3. A detailed description of each of the scenarios can be found in the classified Appendix 1.

4.2 Simulation Environment Testing

4.2.1 Test Bench Configuration

Initial testing was conducted using the simulation environment. The simula- tion environment is able to emulate the behavior of the two BCs of the test system. It is also able to simulate the behavior of the various RTs normally found on the bus. The simulation environment was configured to utilize the QPCX card and the DDC devices to simulate the MIL-STD-1553 traffic from Mux2 of the test system onto a hardware bus. An analyst workstation was connected into the hardware bus to monitor and record the traffic for analysis using the tools developed in chapter 3. A test platform workstation was also connected into the bus for the purposes of executing the classified test scenar- ios on the bus. A diagram of the physical configuration of the hardware bus can be seen in figure 4.1. A list of the RTs simulated on Mux2 can be seen in table 4.1. A more detailed table, with descriptions of the RTs can be found in the classified Appendix 1.

23

4.2. Simulation Environment Testing

Simulator with

QPCX Card

DDC Device Test

Workstation

Analyst

Workstation

Figure 4.1: Simulation Environment Test Bench Physical Configuration

RT No. MIL-STD-1553 A/B?

5 A10 A18 B19 B20 A25 B

Table 4.1: Mux2 RTs Simulated by the Simulation Environment

4.2.2 Experiment

A baseline scenario of 20,000 messages (approx. 30 sec) of normal traffic onMux2 was first taken. Two recordings of the same length were then takenwhere Scenario 1 was executed during the recording. In the first recordingScenario 1 was run for 10 seconds (approx. 7000 messages or 1/3 of therecording); and in the second recording it was run for the entire length of therecording. Both of the scenarios were then profiled for each of the selectedtiming properties using the techniques described in chapter 3, with the excep-tion of the Response Time profiles as explained in section 4.4.1. The generatedprofiles were then compared against the baseline. The same experiment wasthen repeated using Scenario2A, targeting RT18. The baseline recordings andscenarios had to be kept short in length due to performance limitations of thesimulation environment described in section 4.4.1.

24

4.2. Simulation Environment Testing

DetectionThreshold

Inter-MsgGap

DataThroughput

BusUtilization

Mux 210 sec

30% 233.5% 127.3% 14.8%

Table 4.2: Avg %-Difference between Baseline and Scenario 1 Profiles on theSimulation Environment

4.2.3 Results

The initial results from the simulation environment testing were very positiveand yielded successful detections for both of the scenarios tested, promptingfurther tests using the hot bench.

Baseline

Baseline recordings were taken from the simulation environment for all of Mux2 of the test system, as well as baseline recordings filtered for traffic to/fromRT18 on Mux 2. The baselines taken from the simulation environment werevery stable. Each of the baseline recordings were profiled from two differentpoints during the recording and the profiles compared. On Mux 2 the profilesvaried between 7.9 and 28.8 percent. For RT18 the profiles varied between1.2 and 24.2 percent. Based on this data the detection thresholds used toestablish a positive detection in the test scenarios were 30% for Mux 2, and30% for RT18.

Scenario 1

Scenario 1, being an extremely noisy type of attack targeting an entire Mux,yielded very good results from most of the timing properties. In particular,Scenario 1 was found to be extremely well detectable in the Inter-MessageGap property. Only the 10 sec scenario was able to be profiled. The dataduring the longer 30 sec scenario was corrupted before the end of the scenariosuch that it could no longer be read by the profiling software. However, evenwith the limited data, significant results were able to be obtained. Table 4.2outlines the results and shows the average percent difference found betweenthe baseline and scenario profiles. The cells highlighted in green represent apositive detection.

25

4.2. Simulation Environment Testing

DetectionThreshold

Inter-MsgGap

DataThroughput

BusUtilization

Periodicity

Mux 2 RT 1810 sec

30% 69.9% 23.4% 4.8% 50.7%

Mux 2 RT 1830 sec

30% 98.1% 48.6% 8.8% 74.9%

Table 4.3: Avg %-Difference between Baseline and Scenario 2A Profiles onthe Simulation Environment

Scenario 1 Scenario 2A

Inter-Msg Gap

Data Throughput

Bus Utilization

Periodicity N/A

Table 4.4: Simulation Environment Testing Detection Strengths

Scenario 2A

Scenario 2A, is a more subtle type of attack than Scenario 1 and targetsonly a specific RT. However it still yielded good results with RT18 on Mux2, especially regarding the response time and periodicity profiles. Table 4.3outlines the results and shows the average percent difference found betweenthe baseline and scenario profiles. The cells highlighted in green represent apositive detection.

Results Summary

The initial testing on the Simulation Environment went well and both of thescenarios tested were able to be detected through the technique described inChapter 3. Each scenario gave a strong detection result from at least 2 of theselected timing properties profiled, with evidence from the other propertiesto also support the detections. This validated the detection techniques devel-oped for further testing with all scenarios on the Hot Bench, as described insection 4.3 Table 4.4 gives an overview of the results achieved. Green repre-sents a strong detection, while yellow represents a weaker detection, and redrepresents no detection.

26

4.3. Hot Bench Testing

Hot Bench Test

Workstation

Analyst

Workstation

Figure 4.2: Hot Bench Test Bench Physical Configuration

4.3 Hot Bench Testing

4.3.1 Test Bench Configuration

After successful testing on the simulation environment, validation and verifi-cation testing was conducted using the hot bench. The hot bench is a mockup of the real life system emulated by the simulation environment. It utilizesreal hardware BCs, and can be configured with a combination of real hard-ware and simulated RTs. Tests were conducted first on Mux2 of the hot benchto verify the results received from the simulation environment testing. Thetest scenarios were then repeated on Mux1 of the hot bench for validationpurposes. An analyst workstation was connected into the bus being tested tomonitor and record the traffic for analysis using the tools developed in Chap-ter 3. A test platform workstation was also connected into the bus for thepurposes of executing the classified test scenarios on the bus. A diagram ofthe physical configuration of the test bench can be seen in figure 4.2. A listof the real hardware and simulated RTs on both buses can be seen in tables4.5 and 4.6. More detailed tables, with descriptions of the RTs can be foundin the classified Appendix 1.

27

4.3. Hot Bench Testing

RT No. MIL-STD-1553 A/B? Hardware or Simulated?

5 A Hardware7 A Off10 A Simulated18 B Hardware19 B Hardware20 A Simulated25 B Hardware

Table 4.5: Configuration of RTs on Mux2 of the Hot Bench

RT No. MIL-STD-1553 A/B? H/W or Simulated?

1 A Simulated4 A Hardware6 B Hardware10 B Simulated14 A Simulated15 A Simulated17 ? Simulated21 B Simulated24 ? Simulated28 ? Simulated30 A Simulated

Table 4.6: Configuration of RTs on Mux1 of the Hot Bench

4.3.2 Experiment

A baseline scenario of 20 minutes of normal traffic on Mux2 was first taken.Three sets of profiles were then taken from the data for each of the selectedtiming properties. The first set was calculated from all of the traffic seenon Mux2. The second set was calculated only from traffic to/from RT18 onMux2. The third set was calculated from only traffic to/from RT5 on Mux2.In all three cases, a 20,000 message window of the baseline recording was takenand profiled at 1:30, 10:00, and 18:00 into the recording. The three profileswere then compared to confirm the stability of the baseline profile.

A 20 minute recording of traffic on Mux2 was then taken for each of thescenarios described in Appendix 1. During each recording the scenario ofinterest, with the exception of Scenario 3 was executed at different pointsin the recording for durations of 5 seconds, 30 seconds, and 3 minutes. For

28

4.3. Hot Bench Testing

Scenario2A, RT18 was targeted. For Scenario 2B, RT18 SA6 was targeted.Due to the nature of Scneario 3 it was only profiled over a single durationas it’s affects on traffic are not continuous as are the other scenarios. ForScenario 3, RT5 SA4, and RT18 SA6 were targeted. The location of thescenarios being executed in the recording was recorded. Using the slidingwindow methodology described in chapter 3 each recording was profiled foreach of the selected timing properties and compared against their respectivebaseline profiles for anomalies.

4.3.3 Results

The results from the hot bench testing were very positive and yielded success-ful detections for all of the scenarios tested.

Baseline

Baseline recordings were taken from the hot bench for all of Mux 1and Mux2 of the test system, as well as baseline recordings filtered for traffic to/fromRT18 and RT5 on Mux 2 and RT 4 on Mux 1. The baselines taken from thehot bench were fairly stable for each Mux. Filtering the recordings and profilesto traffic to/from a specific RT yielded even more stable profiles. Each of thebaseline recordings were profiled from multiple points during the recordingand the profiles compared. On Mux 1 the profiles varied between 10 and 38percent. On Mux 2 the profiles varied between 3 and 20 percent. For RT18the profiles varied between 0.1 an 16 percent. For RT5 the profiles variedbetween 0.1 and 19 percent. And for RT4 the profiles varied between 1 and26 percent. Based on this data the detection thresholds used to establish apositive detection in the test scenarios were 40% for Mux 1, 25% for Mux 2,20% for RT18, 25% for RT5, and 30% for RT4.

Scenario 1

Scenario 1, being an extremely noisy type of attack targeting an entire Mux,yielded very good results from most of the timing properties on both Mux 1and Mux 2, for both shorter and longer durations of the intrusion running. Inparticular, Scenario 1 was found to be extremely well detectable in the Inter-Message Gap property. Only the 5sec and 30sec scenarios from Mux 2, andthe 5 sec scenario from Mux 1 were able to be profiled. The data after thispoint was corrupted by the scenario such that it could no longer be read bythe profiling software. However, even with the limited data, significant resultswere able to be obtained. Table 4.7 outlines the results and shows the average

29

4.3. Hot Bench Testing

DetectionThreshold

ResponseTime

Inter-MsgGap

DataThroughput

BusUtilization

Mux 15 sec

40% 21.8% 67.1% 90.8% 57.6%

Mux 25 sec

25% 74.0% 210.0% 16.8% 26.4%

Mux 230 sec

25% 55.3% 186.7% 24.2% 30.4%

Table 4.7: Avg %-Difference between Baseline and Scenario 1 Profiles on theHot Bench

percent difference found between the baseline and scenario profiles. The cellshighlighted in green represent a positive detection.

Scenario 2A

Scenario 2A, is a more subtle type of attack than Scenario 1 and targets onlya specific RT. However it still yielded good results with RT18 on Mux 2,especially regarding the response time and periodicity profiles. The resultsfrom RT4 on Mux 1 were much weaker, but still provided some detection. AsRT18 is much busier than RT4 in the bus schedule, it appears that the levelof traffic on a bus or to a particular RT has an effect on the detectability ofintrusions against that RT or Bus. Again only the 5sec and 30sec scenariosfrom both Mux 1 and Mux 2 were able to be profiled. The data after thispoint was corrupted by the scenario such that it could no longer be read bythe profiling software. Table 4.8 outlines the results and shows the averagepercent difference found between the baseline and scenario profiles. The cellshighlighted in green represent a positive detection.

Scenario 2B

Scenario 2B, is very similiar to Scenario 2A however it is even more subtle,and only targets a specific subaddress of an RT. In this case RT18 SA6 andRT4 SA1. The detection results were very similar to those seen in Scenario2A.. Again only the 5sec and 30sec scenarios from RT4 on Mux 1 were able tobe profiled. The data after this point was corrupted by the scenario such thatit could no longer be read by the profiling software. All of the 5sec, 30sec, and3min scenarios were able to be profiled however for RT18 on Mux 2. Table 4.9outlines the results and shows the average percent difference found between

30

4.3. Hot Bench Testing

DetectionThreshold

ResponseTime

Inter-MsgGap

DataThroughput

BusUtilization

Periodicity

Mux 1 RT 45 sec

30% 19.2% 14.3% 21.8% 9.1% 35.7%

Mux 1 RT 430 sec

30% 17.8% 14.3% 18.2% 7.5% 33.6%

Mux 2 RT 185 sec

20% 64.5% 33.7% 8.1% 8.3% 390.2%

Mux 2 RT 1830 sec

20% 80.4% 39.6% 13.0% 8.6% 309.9%

Table 4.8: Avg %-Difference between Baseline and Scenario 2A Profiles onthe Hot Bench

DetectionThreshold

ResponseTime

Inter-MsgGap

DataThroughput

BusUtilization

Periodicity

Mux 1 RT 45 sec

30% 20.9% 3.9% 25.8% 10.7% 35.9%

Mux 1 RT 430 sec

30% 25.5% 3.9% 18.0% 7.4% 27.8%

Mux 2 RT 185 sec

20% 60.4% 31.1% 11.8% 7.8% 272.9%

Mux 2 RT 1830 sec

20% 93.6% 36.4% 15.4% 10.0% 220.8%

Mux 2 RT 183 min

20% 116.2% 40.4% 24.5% 22.8% 92.6%

Table 4.9: Avg %-Difference between Baseline and Scenario 2B Profiles on theHot Bench

the baseline and scenario profiles. The cells highlighted in green represent apositive detection.

Scenario 3

Scenario 3 is another targeted attack which is even more subtle than Senario2A and 2B. Scenario was tested against RT18 and RT5 both on Mux 2. Despitebeing extremely subtle, Scenario 3 was still able to be detected with goodresults, especially in the response time and periodicity properties. Table 4.10outlines the results and shows the average percent difference found between

31

4.4. Challenges and Limitations

DetectionThreshold

ResponseTime

Inter-MsgGap

DataThroughput

BusUtilization

Periodicity

Mux 2 RT 5 25% 66.7% 373.8% 12.9% 10.2% 66.0%

Mux 2 RT 18 20% 55.1% 20.4% 16.8% 27.1% 98.2%

Table 4.10: Avg %-Difference between Baseline and Scenario 3 Profiles on theHot Bench

Scenario 1 Scenario 2A Scenario 2B Scenario 3

Response Time

Inter-Msg Gap

Data Throughput

Bus Utilization

Periodicity N/A

Table 4.11: Hot Bench Testing Detection Strengths

the baseline and scenario profiles. The cells highlighted in green represent apositive detection.

Results Summary

All of the scenarios tested were able to be detected through the techniquedescribed in Chapter 3. Each scenario gave a strong detection result from atleast 2 of the selected timing properties profiled, with evidence from the otherproperties to also support the detections. Table 4.11 gives an overview of theresults achieved. Green represents a strong detection, while yellow representsa weaker detection, and red represents no detection.

4.4 Challenges and Limitations

4.4.1 Simulation Environment

Due to the nature of the simulation environment and its software, it has someinherent limitations. Not all of the RTs from the real system are available asmodels in the simulation environment. Also, because the RT models whichare available are simulated they do not exactly replicate the timing behaviorsof the actual RTs. In particular, the simulated RTs are unable to accuratelyexhibit the response time behavior of RTs in the real system. The simulatedRTs have a fixed static response time of 6µs. This makes any response time

32

4.4. Challenges and Limitations

data gathered from the simulation environment invalid for the purposes ofperforming anomaly detection. Also, because of performance limitations thescenarios tested on the simulation environment had to be kept short as runningthem for any prolonged amount of time, longer than approximately 30 sec,would cause the simulation environment to crash.

4.4.2 Hot Bench Environment

The hot bench is a heavily used resource and time available on it for ex-perimentation and data collection was limited. The hot bench is normallyconfigured in a classified configuration which is representative of the real sys-tem it models. However in order to collect data for this thesis to be able totake out of the hot bench facility, it was required to remove components andalter the software load to an unclassified configuration that is close to but notcompletely representative of the real system.

The availability of hardware RTs was also a limitation. Not all of thehardware RTs were available for use at the hot bench facility at the time oftesting, simply due to a lack of available resources. This means that someRTs used in the hot bench testing were still simulated.

4.4.3 Analyst Workstation and Software

Performance issues with the BusTools API and RC-15EC were encounteredwhen attempting to process large volumes of traffic data through the timingproperty extraction software. When attempting to playback large size record-ings containing greater than 10,000 messages, the API would hang up andfreeze. To overcome this, the traffic was recorded in a set of consecutive filescontaining 10,000 messages each. The files were then read through the tim-ing extraction software in sequence using a loop. Further performance issueswere then encountered where the API would hang up and freeze after readingapproximately 20 files. The solution to this was to create a Batch script torepeatedly run the timing property extraction software, feeding it only onerecording file each time.

4.4.4 The Real System

Due to resource availability and security/sensitivity issues, testing was unableto be completed using a real system. Hot bench testing however is consideredan accurate representation of the behavior of the real system. In order toperform real system testing, additional hardware would also be required to beable to monitor and record traffic on the MIL-STD-1553 bus.

33

4.5. Summary of Validation

4.5 Summary of Validation

Initial testing using the simulation environment provided positive results. Abaseline profile was first established for each timing property. Each of thescenarios were then profiled and created a significant difference in the profilesof several of the timing properties.

Further testing on the hot bench confirmed the results obtained during ini-tial testing. The baseline profiles obtained from the hot bench were extremelystable throughout the traffic recordings. All of the scenarios generated sig-nificant anomalies in the profiles similar to those seen during the hot benchtesting, validating the detection engine described in chapter 3.

34

5 Discussion and Conclusion

5.1 Discussion

The threat of cyber attacks against military platforms is ever increasing. In-cluded as part of those platforms is the MIL-STD-1553 bus. Classified researchhas identified vulnerabilities in MIL-STD-1553 which need to be protected. Inorder to protect the MIL-STD-1553 bus, we must first be able to monitor anddetect potential intrusions.

This thesis proposed that this could be accomplished by using time-basedanomaly detection techniques. These types of techniques were previously suc-cessful in detecting intrusions on other types of real-time systems. And thisthesis has shown that they can be successfully used to detect intrusions onMIL-STD-1553 as well. Because MIL-STD-1553 traffic is highly deterministicand strictly scheduled it lends itself very well to time-based anomaly detection.

5.2 Future Work

The work conducted in this thesis is only the beginning of developing a fullintrusion detection solution for MIL-STD-1553. It is important to now con-tinue and further develop this research. Future work based off the researchfrom this theis includes:

1. Finding a solution to the performance issues with BusTools that weredescribed in section 4.4.3;

2. Automate the Profile Generator and Profile Comparators modules insoftware as opposed to an excel spreadsheet;

3. Further develop the detection engine to become part of a full IntrusionDetection Software Suite for MIL-STD-1553;

4. Begin gathering baseline data from real MIL-STD-1553 systems to beused for the deployment of such an Intrusion Detection System men-tioned in the previous item; and,

35

5.3. Conclusion

5. Conduct further validation testing of the detection engine developed inthis thesis against a real MIL-STD-1553 system.

5.3 Conclusion

The aim of this thesis was to develop an intrusion detection engine capable ofdetecting time-based anomalies in MIL-STD-1553 traffic, with the intentionthat the engine will be the first step towards the development of a MIL-STD-1553 intrusion detection system to protect against vulnerabilities in theMIL-STD-1553 protocol.

This aim was successfully achieved by adapting time-based intrusion detec-tion techniques used in other types of real-time systems a functional detectionengine was able to be created. Testing of this engine gave very positive re-sults and has shown that time-based anomaly detection is a highly effectivemeans of detecting potential intrusions on the MIL-STD-1553 bus. Using thisknowledge, a more complete MIL-STD-1553 Intrusion Detection System couldbe developed to monitor and protect any MIL-STD-1553 network, includingthose on board several of today’s military aircraft and other vehicles.

36

6 Appendix 1

6.1 The Scenarios

6.1.1 Scenario1

6.1.2 Scenario2A

6.1.3 Scenario2B

6.1.4 Scenario3

6.2 The Test Environments

6.2.1 Simulation Environment

6.2.2 Hot Bench

37

Bibliography

[1] G. Ingersoll. US NAVY: Hackers ’jumping the air gap’ would ’disrupt theworld balance of power’. Business Insider, November 2013.

[2] Nagaraja Thanthry and Ravi Pendse. Aviation data networks: Secu-rity issues and network architecture. IEEE Aerosp. Electron. Syst. Mag.,20(6):3–8, 2005.

[3] Muhammad Sabeeh Ali, Ravi Bhagavathula, and Ravi Pendse. Airplanedata networks and security issues. In Digital Avionics Systems Confer-ence, 2004. DASC 04. The 23rd, volume 2, pages 8–E. IEEE, 2004.

[4] Peter Skaves. Information for cyber security issues related to aircraftsystems Rev-A. In Digital Avionics Systems Conference (DASC), 2013IEEE/AIAA 32nd, pages 4A1–1. IEEE, 2013.

[5] Pierluigi Paganini. Cyber threats against the avia-tion industry. http://resources.infosecinstitute.com/

cyber-threats-aviation-industry/, April 2014.

[6] Kim Zetter. FAA: Boeing’s new 787 may be vulnerable to hackerattack. http://archive.wired.com/politics/security/news/2008/

01/dreamliner_security, January 2008.

[7] Guy Norris. Boeing, FAA cut 777 cyber vulnerabil-ity. http://aviationweek.com/commercial-aviation/

boeing-faa-cut-777-cyber-vulnerability, March 2014.

[8] Susana Mas. Steven blaney announces new fundingfor cyber security. http://www.cbc.ca/news/politics/

steven-blaney-announces-new-funding-for-cyber-security-1.

3163391, July 2015.

[9] Aliya Sternstein. Congress demands $200 million antivirus scan of con-nected weapons. http://www.nextgov.com/cybersecurity/2015/11/

congress-demands-200-million-antivirus-scan-connected-weapons/

123615/?oref=ng-channeltopstory, November 2015.

38

Bibliography

[10] CybAIR RadBox. http://www.thalesraytheon.com/fileadmin/tmpl/

Products/pdf/Cyber_Security/fiche_RadBox_EXE_low.pdf, 2013.

[11] Berenice Baker. Clear skies – how anti-hacking technology can pro-tect military radar. http://www.airforce-technology.com/features/

feature-clear-skies-anti-hacking-technology-protect-military-radar/, August 2013.

[12] O. Stan, Y. Elovici, A. Shabtai, G. Shugol, R. Tikochinski, and S. Kur. ”Protecting Military Avionics Platforms from Attacks on MIL-STD-1553 Communication Bus.” arXiv:1707.05032v1 [cs.CR]. 17 Jul, 2017.

[13] MIL-STD-1553: Aircraft Internal Time Division Command/ResponseMultiplex Data Bus. United States Department of Defense, WashingtonD.C., September 1978.

[14] Richard A Schuh. An overview of the 1553 bus with testing and simu-lation considerations. In Instrumentation and Measurement Technology Conference, 1988. IMTC-88. Conference Record., 5th IEEE, pages 20–25.IEEE, 1988.

[15] MIL-STD-1553 designer’s guide. http://www.

milstd1553.com/resources-2/desginers-guide/

review-and-rationale-of-mil-std-1553-a-and-b/

4-0-general-requirements/.

[16] Man-Ki Yoon, Swati Mohan, Jaesik Choi, Jung-Eun Kim, and Lui Sha.SecureCore: A multicore-based intrusion detection architecture for real-time embedded systems. In Real-Time and Embedded Technology and Applications Symposium (RTAS), 2013 IEEE 19th, pages 21–32. IEEE,2013.

[17] Sibin Mohan, Stanley Bak, Emiliano Betti, Heechul Yun, Lui Sha, andMarco Caccamo. S3A: Secure system simplex architecture for enhancedsecurity and robustness of cyber-physical systems. In Proceedings of the2nd ACM international conference on High confidence networked systems,pages 65–74. ACM, 2013.

[18] Christopher Zimmer, Balasubramanya Bhat, Frank Mueller, and SibinMohan. Time-based intrusion detection in cyber-physical systems. InProceedings of the 1st ACM/IEEE International Conference on Cyber-Physical Systems, pages 109–118. ACM, 2010.

[19] Emanuel Parzen. On estimation of a probability density function andmode. The annals of mathematical statistics, pages 1065–1076, 1962.

39

Bibliography

[20] M Chris Jones, James S Marron, and Simon J Sheather. A brief surveyof bandwidth selection for density estimation. Journal of the AmericanStatistical Association, 91(433):401–407, 1996.

[21] Martin Thibault. C19C B1 mission computer (MC) timing and load anal-ysis. Technical Report CF18-PMV-AVS-MC-OFP/REP-C19CB1, CAEMilitary Support Programs, Saint-Laurent, QC, May 2008.

[22] GE Intelligent Platforms, Goleta, CA. BusTools/1553-API Software Ref-erence Manual, 5.3 edition, June 2013.

40