sensor and graph mining

INTEL 04 C. Faloutsos 1

School of Computer ScienceCarnegie Mellon

Sensor and Graph Mining

Christos Faloutsos

Carnegie Mellon University & IBMwww.cs.cmu.edu/~christos



Joint work with

• Anthony Brockwell (CMU/Stat)

• Deepayan Chakrabarti (CMU)

• Spiros Papadimitriou (CMU)

• Chenxi Wang (CMU)• Yang Wang (CMU)



Outline

• Introduction - motivation

• Problem #1: Stream Mining– Motivation– Main idea– Experimental results

• Problem #2: Graphs & Virus propagation

• Conclusions



Introduction• Sensor devices

– Temperature, weather measurements– Road traffic data– Geological observations– Patient physiological data

• Embedded devices– Network routers– Intelligent (active) disks



Introduction• Limited resources

– Memory– Bandwidth– Power– CPU

• Remote environments– No human intervention



Introduction – problem dfn• Given a emi-infinite stream of values (time

series) x1, x2, …, xt, …

• Find patterns, forecasts, outliers…



Introduction

Periodicity? (daily)

Periodicity? (twice daily)

“Noise”??

• E.g.,



Introduction

Periodicity? (daily)

Periodicity? (twice daily)

“Noise”??

• Can we capture these patterns– automatically– with limited resources?



Related workStatistics: Time series forecasting

• Main problem:

“[…] The first step in the analysis of any time series is to plot the data [and inspect the graph]” [Brockwell 91]

• Typically:• Resource intensive

• Cannot update online

• AR(I)MA and seasonal variants• ARFIMA, GARCH, …



Related workDatabases: Continuous Queries

• Typically, different focus:– “Compression”– Not generative models

• Largely orthogonal problem…– Gilbert, Guha, Indyk et al. (STOC 2002)– Garofalakis, Gibbons (SIGMOD 2002)– Chen, Dong, Han et al. (VLDB 2002); Bulut, Singh (ICDE 2003)– Gehrke, Korn, et al. (SIGMOD 2001), Dobra, Garofalakis, Gehrke

et al. (SIGMOD 2002)– Guha, Koudas (ICDE 2003) Datar, Gionis, Indyk et al. (SODA

2002)– Madden+ [SIGMOD02], [SIGMOD03]



Goals

• Adapt and handle arbitrary periodic components

• No human intervention/tuning

Also:

• Single pass over the data

• Limited memory (logarithmic)

• Constant-time update



Outline




• Conclusions



Wavelets“Straight” signal

t

I1

t

I2

t

I3

t

I4

t

I5

t

I6

t

I7

t

I8

time

t

xt



WaveletsIntroduction – Haar

t

W1,1

t

W1,2

t

W1,3

t

W1,4

t

W2,1

t

W2,2

t

W3,1

t

V4,1

time

frequ

ency

t

xt



Wavelets

• So?

• Wavelets compress many real signals well…– Image compression and processing– Vision; Astronomy, seismology, …

• Wavelet coefficients can be updated as new points arrive [Kotidis+]



WaveletsCorrelations

t

W1,1

t

W1,2

t

W1,3

t

W1,4

t

W2,1

t

W2,2

t

W3,1

t

V4,1

time

frequ

ency

xt

t

=



WaveletsCorrelations

t

W1,1

t

W1,2

t

W1,3

t

W1,4

t

W2,1

t

W2,2

t

W3,1

t

V4,1

time

frequ

ency

xt

t



Main ideaCorrelations

• Wavelets are good…

• …we can do even better– One number…– …and the fact that they are

equal/correlated



Proposed method

Wl,tWl,t-1Wl,t-2Wl,t l,1Wl,t-1 l,2Wl,t-2 …

Wl’,t’-1Wl’,t’-2Wl’,t’

Wl’,t’ l’,1Wl’,t’-1 l’,2Wl’,t’-2 …

Small windows suffice… (k~4)



More details…

• Update of wavelet coefficients

• Update of linear models

• Feature selection– Not all correlations are significant– Throw away the insignificant ones– very important!!

[see paper]

(incremental)

(incremental; RLS)

(single-pass)



Complexity• Model update

Space: OlgN + mk2 OlgNTime: Ok2 O1

Where– N: number of points (so far)– k: number of regression coefficients; fixed– m: number of linear models; OlgN

[see paper]

SKIP



Outline




• Conclusions



Setup

• First half used for model estimation

• Models applied forward to forecast entire second half

• AR, Seasonal AR (SAR): R– Simplest possible estimation – no maximum

likelihood estimation (MLE), etc.

• … vs. Python scripts



ResultsSynthetic data – Triangle pulse

• Triangle pulse• AR captures wrong trend (or none)• Seasonal AR (SAR) estimation fails



ResultsSynthetic data – Mix

• Mix (sine + square pulse)• AR captures wrong trend (or none)• Seasonal AR estimation fails



ResultsReal data – Automobile

• Automobile traffic– Daily periodicity with rush-hour peaks– Bursty “noise” at smaller time scales

(filtered)





• AR fails to capture any trend (average)• Seasonal AR estimation fails





• AWSOM spots periodicities, automatically





• Generation with identified noise



ResultsReal data – Sunspot

• Sunspot intensity – Slightly time-varying “period”• AR captures wrong trend (average)• Seasonal ARIMA

– Captures immediate wrong downward trend– Requires human to determine seasonal component period (fixed)



ResultsReal data – Sunspot

• Sunspot intensity – Slightly time-varying “period”

Estimation: 40 minutes (R) vs. 9 seconds (Python)



Variance

• Variance (log-power) vs. scale:– “Noise” diagnostic (if decreasing linear…)

– Can use to estimate noise parameters

~ 1 hour

SKIP

~Hurst exponent



Running time

stream size (N)

tim

e (

t)



Space requirements

Equal total number of model parameters



Conclusion

Adapt and handle arbitrary periodic components

No human intervention/tuning

Single pass over the dataLimited memory (logarithmic)Constant-time update



Conclusion

Adapt and handle arbitrary periodic components

No human intervention/tuning

Single pass over the dataLimited memory (logarithmic)Constant-time update

no human

limitedresources



Outline

• Introduction - motivation• Problem #1: Streams• Problem #2: Graphs & Virus propagation

– Motivation & problem definition– Related work– Main idea– Experiments

• Conclusions



Introduction

Internet Map [lumeta.com]

Food Web [Martinez ’91]

Protein Interactions [genomebiology.com]

Friendship Network [Moody ’01]

► Graphs are ubiquitious



Introduction

• What can we do with graph analysis?– Immunization;– Information

Dissemination– network value of a

customer [Domingos+] “Needle exchange” networks of drug users

[Weeks et al. 2002]

“bridges”



Problem definition

• Q1: How does a virus spread across an arbitrary network?

• Q2: will it create an epidemic?

• (in a sensor setting, with a ‘gossip’ protocol, will a rumor/query spread?)



Framework

• Susceptible-Infected-Susceptible (SIS) model – Cured nodes immediately become susceptible

Susceptible/

healthy

Infected &

infectious

Infected by neighbor

Cured internally



The model

• (virus) Birth rate β : probability than an infected neighbor attacks

• (virus) Death rate δ : probability that an infected node heals

Infected

Healthy

NN1

N3

N2Prob. β

Prob. β

Prob. δ



Epidemic threshold

Defined as the value of , such that

if / < an epidemic can not happen

Thus,

• given a graph

• compute its epidemic threshold



Epidemic threshold

What should depend on?

• avg. degree? and/or highest degree?

• and/or variance of degree?

• and/or determinant of the adjacency matrix?



Basic Homogeneous Model

Homogeneous graphs [Kephart-White ’91, ’93]

• Epidemic threshold = 1/<k>• Homogeneous connectivity <k>, ie, all

nodes have ~same degree unrealistic



Power-law Networks

• Model for Barabási-Albert networks– [Pastor-Satorras &

Vespignani, ’01, ’02]

– Epidemic threshold = <k> / <k2>

– for BA type networks, with only γ = 3 (γ = slope of power-law exponent)



Epidemic threshold

• Homogeneous graphs: 1/<k>• BA (=3) <k> / <k2>

• more complicated graphs ?

• arbitrary, REAL graphs ?

• how many parameters??



Epidemic threshold

• [Theorem] We have no epidemic, if

β/δ <τ = 1/ λ1,A



Epidemic threshold

• [Theorem] We have no epidemic, if

β/δ <τ = 1/ λ1,A

largest eigenvalueof adj. matrix A

attack prob.

recovery prob.epidemic threshold

Proof: [Wang+03]



Epidemic threshold for various networks

• sanity checks / older results:

• Homogeneous networks– λ1,A = <k>; τ = 1/<k>

– where <k> = average degree– This is the same result as of Kephart & White !





• Star networks– λ1,A = sqrt(d); τ = 1/ sqrt(d)

– where d = the degree of the central node





• Infinite, power-law networks– λ1,A = ∞; τ = 0 : *any* virus has a chance!

[Barabasi et al]

• Finite power-law networks– τ = 1/ λ1,A



Outline

• Introduction - motivation• Problem #1: Streams• Problem #2: Graphs & Virus propagation

– Motivation & problem definition– Related work– Main idea– Experiments

• Conclusions



Experiments

• 2 graphs– Star network: one “hub” + 99 “spokes”– “Oregon” Internet AS graph:

• 10,900 nodes, 31180 edges

• topology.eecs.umich.edu/data.html

• More in our paper: [SRDS ’03]



β/δ > τ (above threshold)

β/δ = τ (at the threshold)

β/δ < τ (below threshold)

Experiments (Star)



Experiments (Oregon)

β/δ > τ (above threshold)

β/δ = τ (at the threshold)

β/δ < τ (below threshold)



Our prediction vs. previous prediction

• our predictions are more accurate

Oregon Star

PL3PL3

OurOur

Nu

mb

er o

f in

fect

ed n

odes

β/δ β/δ



Conclusions

We found an epidemic threshold

√ that applies to any network topology

√ and it depends only on one parameter of the graph



Overall conclusions

• Automatic stream mining: AWSOM

• graphs and virus propagation: eigenvalue



Ongoing / related work

• Streams– how to find hidden variables on multiple

streams [w/ Spiros and Jimeng Sun]– ‘network tomography’ [w/ Airoldi +]

• Graphs– graph partitioning [w/ Deepay+]– important subgraphs [w/ Tomkins + McCurley]– graph generators [RMAT, w/ Deepay]



Thank you!

Contact info:christos @ cs.cmu.edu

spapadim @ cs.cmu.edu

deepay @ cs.cmu.edu



Main References

• Spiros Papadimitriou, Anthony Brockwell and Christos Faloutsos Adaptive, Hands-Off Stream Mining VLDB 2003, Berlin, Germany, Sept. 2003.

• [Wang+03] Yang Wang, Deepayan Chakrabarti, Chenxi Wang and Christos Faloutsos: Epidemic Spreading in Real Networks: an Eigenvalue Viewpoint, SRDS 2003, Florence, Italy.



Additional References

• Connection Subgraphs, C. Faloutsos, K. McCurley, A. Tomkins, SIAM-DM 2004 workshop on link analysis

• RMAT: A recursive graph generator, D. Chakrabarti, Y. Zhan, C. Faloutsos, SIAM-DM 2004

• iFilter: Network tomography using particle filters, Edoardo Airoldi, Christos Faloutsos (submitted)

sensor and graph mining

Documents