23/4/21

Tru-Alarm: Trustworthiness Analysis of Sensor Network in Cyber Physical Systems

Lu-An Tang, Xiao Yu, Sangkyum Kim, Jiawei Han, Chih-Chieh Hung, Wen-Chih Peng

23/4/21 2DAIS UIUC

IntroductionIntroduction

A cyber-physical system (CPS) is a system featuring a tight combination of, and coordination between, the system's computational and physical resources – Definition by NSF

CPS = sensor network + data mining moduleTraffic monitoring system

healthcare system

battlefield surveillance, etc

Major Problem: Data reliability, especially the trustworthiness of anomalies/outliers/alarms

23/4/21 3DAIS UIUC

Motivation Example: Motion DetectorMotivation Example: Motion Detector

Battle Network: Deploy sensor network to detect hostile object and actions

Problem: Sensors are easily damage or influenced by irrelevant activities – generate false alarms

s2=4

s3=3

s8=90

s1=2

s4=66 s5=64

s7=99

s6=65

s15=93

s13=3

s14=3

s10=2 s11=4

s12=2

s16=9

s17=6

s9=3

23/4/21 4DAIS UIUC

Problem DefinitionProblem Definition

Given a CPS dataset including both alarming and normal data records, find out the trustworthy alarms – We focus on the trustworthiness tasks for alarming records

Formal Definition:

Let R = {r(s1, t1), r(s1, t2), . . . r(sm, tn) } be a CPS

dataset, be the set of alarm records, given a trustworthy threshold δt, the Tru-Alarm’s task is to find

out the trustworthy alarms ra(s, t) with τ(ra) > δt

aR R

23/4/21 5DAIS UIUC

Challenges in Trustworthiness Analysis Challenges in Trustworthiness Analysis

Huge size: A typical CPS contains hundreds of sensors and millions of data records

Unreliable Data: Buonadonna et.al: 51% of the data are faulty; Szewzyk et.al: 60% of the data are faulty in a deployment in green lake

No/Rare Training Sets: it is costly to manually label the large dataset

Conflicts of Sensors: Well deployed sensor network has reasonable redundancies (k-coverage). Conflicts among reliable and unreliable sensors

23/4/21 6DAIS UIUC

Two TasksTwo Tasks

The position of intrusion object is important for alarm trustworthiness analysis

However, it is hard to get the accurate positions from noisy data with lots of false alarms

Observation: Mutual Enhancement

More trustworthiness the alarms are, easier to estimate object locations

More accurate the object positions are, better trustworthiness analysis for alarms

23/4/21 7DAIS UIUC

Main FrameworkMain Framework

Object-Alarm Trustworthiness Analysis Framework

Estimate intrusion object locations from noisy data (Not accurate: many false positives: ghost objects that do not really exist)

Use such objects to verify the alarms – find out the false ones

Refine the object locations with trustable alarms

23/4/21 8DAIS UIUC

Estimate object locations Estimate object locations

Some state-of-art works in object tracking

Commonly used approach: sampling the object positions in the ranges of alarming sensors

23/4/21 9DAIS UIUC

Build up links of objects and alarmsBuild up links of objects and alarms

Construct a bipartite graph of object (positions) and sensor (records)

For each sensor s: the monitored objects Os

For each object o: the monitoring sensors So

Two kinds of links: Alarm-object Link

Normal_record-obj link

23/4/21 10DAIS UIUC

Trustworthiness Inference in the GraphTrustworthiness Inference in the Graph

The weight of link between object o and alarm ra(si,t):

how likely the alarm is caused by such object – the conditional trustworthiness τ(ra(si,t)|o)

Estimate τ(ra(si,t)|o): it is determined by the coherence

of other sensors’ readings in the same monitoring sensor set of So

23/4/21 11DAIS UIUC

Compute object trustworthiness Compute object trustworthiness

A low τ(ra|o) indicates two possibilities:

ra is a false alarm

ra is a true alarm, but it is not caused by object o

In either case, object o is not likely to be a real one; a real object should cause alarms for all its monitoring sensors

o’s trustworthiness τ(o) is the average of all its conditional alarm trustworthiness

23/4/21 12DAIS UIUC

Compute alarm trustworthinessCompute alarm trustworthiness

Even there is only one real object that causes the alarm, such alarm is still meaningful

If an alarm has different conditional trustworthiness with different objects, we will take the maximum one as τ(ra)

23/4/21 13DAIS UIUC

Running Example IRunning Example I

23/4/21 14DAIS UIUC

Running Example IIRunning Example II

23/4/21 15DAIS UIUC

Experiment SetupExperiment Setup

Synthetic a battle field with hundreds of sensors

Objects (i.e., tanks and soliders) move across the battlefield

Random false alarms added in

23/4/21 16DAIS UIUC

Precision and Recall with kNN methodsPrecision and Recall with kNN methods

23/4/21 17DAIS UIUC

Thank You Very Much!Thank You Very Much!

23/4/21 18DAIS UIUC

MoreMore

Introduction

Related works

Trustworthiness Inference

Experiments

23/4/21 19DAIS UIUC

CPS Sensors for Motion DetectionCPS Sensors for Motion Detection

The CPSs are deployed in different scenarios with various types of sensors

In the scenario of motion detection, several types of sensors are used

We use common sensors in this study, however, the method also works for other types of sensors

23/4/21 20DAIS UIUC

Related Works: Spatial SimilarityRelated Works: Spatial Similarity

Assumption: The sensors that are spatially close to each other should report the similar readings (Krishnamachari et. al 2004)

kNN ApproachSetup a neighbor threshold k

Judge the alarm trustworthiness by neighboring information

Suppose an alarm sensor s has l alarming neighbors in its kNN, if l/k > δt, the alarm is trustworthiness, else it is not

23/4/21 21DAIS UIUC

Problem of Spatial Similarity based ApproachProblem of Spatial Similarity based Approach

The edge sensor’s alarms may be ignored

Hard to determine k, k is indeed different by sensors

s2=4

s3=3

s8=90

s1=2

s4=66 s5=64

s7=99

s6=65

s15=93

s13=3

s14=3

s10=2 s11=4

s12=2

s16=9

s17=6

s9=3

S5 is an edge sensor, its 1st and 2nd NN s2 and s3 are normal sensors

23/4/21 22DAIS UIUC

Related Works: Temporal SimilarityRelated Works: Temporal Similarity

Assumption: The sensors that reports alarms in the same time are likely to report together in the future (Xiao et. al 2007)

Train a correlation model from historical data, test the alarms by such model

Problem:The noisy data are in a large portion (30% -- 50%)

The damaged sensors are likely to report false alarms for a long time

Some unreliable sensors and false alarms may have such strong correlations with real alarms

23/4/21 23DAIS UIUC

MoreMore

Introduction

Related works

Trustworthiness Inference Details

Experiments

23/4/21 24DAIS UIUC

Estimate coherence of two sensor records IEstimate coherence of two sensor records I

To estimate the coherence score between two sensors’ records rj and ri, the system should take count in both

their reading differences and positions

ri= f(dist(si, o), Ω(o)),where

Ω(o) is object o’s signal strength

Acoustic sensor:

ri= α*Ω(o)/dist(si, o)2+β

Estimate Ωi(o) by ri : Ωi(o) = f-1(dist(sj, o), rj)

rj’= f(dist(sj, o), Ωi(o)), -- the expect value of rj from ri

23/4/21 25DAIS UIUC

Estimate coherence of two sensor records IIEstimate coherence of two sensor records II

Coherence coh(ra(si, t), r(sj, t)) is judged by the

difference of the expected reading and real value

σ is the standard deviation of monitoring sensor set So

If si’ reading is the same as expected value, the

coherence score reaches the maximum of 1; if the difference is larger than σ, the score is set to 0

23/4/21 26DAIS UIUC

Tru-Alarm AlgorithmTru-Alarm Algorithm

Initialize the trustworthiness for each object and alarm

For each object o, first retrieves its related data records from the object-alarm graph, and computes the conditional alarm trustworthiness

The object’s trustworthiness is then computed as the average of its conditional alarm trustworthiness

The system groups the conditional alarm trustworthiness by alarm and select the max one as τ(ra)

23/4/21 27DAIS UIUC

MoreMore

Introduction

Related works

Efficient Trustworthiness Inference

Experiments

23/4/21 28DAIS UIUC

Efficient Trustworthiness AnalysisEfficient Trustworthiness Analysis

The time complexity of Tru-Alarm is linear in the number of objects

The efficiency will be a problem when there are a large number of objects generated by the sampling algorithm

Most objects turn out to be low trustworthy: In the running example, there are 10 objects but only one is trustworthy

Can we prune the untrustworthy objects in advance?

23/4/21 29DAIS UIUC

Upperbound of Upperbound of ττ((oo))

Let o be an object, So be its monitoring set and Rao be

the set of related alarms. τ(o)’s upper-bound τ(o) = |Rao|/|So|

23/4/21 30DAIS UIUC

Improved Tru-alarm AlgorithmImproved Tru-alarm Algorithm

Initialize the trustworthiness for each object and alarm

For each object o, first compute its upper-bound, if it is less than δt, then prune it

Retrieves o’s related data records from the object-alarm graph, and computes the conditional alarm trustworthiness

The object’s trustworthiness is then computed as the average of its conditional alarm trustworthiness

Groups the conditional alarm trustworthiness by alarm and select the max one as τ(ra)

23/4/21 31DAIS UIUC

Time Cost Time Cost