school of computer science carnegie mellon uiuc 04c. faloutsos1 advanced data mining tools: fractals...
TRANSCRIPT
UIUC 04 C. Faloutsos 1
School of Computer ScienceCarnegie Mellon
Advanced Data Mining Tools: Fractals and Power Laws for Graphs,
Streams and Traditional Data
Christos Faloutsos
Carnegie Mellon University
UIUC 04 C. Faloutsos 2
School of Computer ScienceCarnegie Mellon
THANK YOU!
• Prof. Jiawei Han
• Prof. Kevin Chang
UIUC 04 C. Faloutsos 3
School of Computer ScienceCarnegie Mellon
Overview
• Goals/ motivation: find patterns in large datasets:– (A) Sensor data– (B) network/graph data
• Solutions: self-similarity and power laws
• Discussion
UIUC 04 C. Faloutsos 4
School of Computer ScienceCarnegie Mellon
Applications of sensors/streams
• ‘Smart house’: monitoring temperature, humidity etc
• Financial, sales, economic series
UIUC 04 C. Faloutsos 5
School of Computer ScienceCarnegie Mellon
Motivation - Applications• Medical: ECGs +; blood pressure etc
monitoring
• Scientific data: seismological; astronomical; environment / anti-pollution; meteorological
Sunspot Data
0
50
100
150
200
250
300
UIUC 04 C. Faloutsos 7
School of Computer ScienceCarnegie Mellon
Motivation - Applications (cont’d)
• Computer systems
– web servers (buffering, prefetching)
– network traffic monitoring
– ...
http://repository.cs.vt.edu/lbl-conn-7.tar.Z
UIUC 04 C. Faloutsos 8
School of Computer ScienceCarnegie Mellon
Problem definition
• Given: one or more sequences x1 , x2 , … , xt , …; (y1, y2, … , yt, …)
• Find – patterns; clusters; outliers; forecasts;
UIUC 04 C. Faloutsos 9
School of Computer ScienceCarnegie Mellon
Problem #1
• Find patterns, in large datasets
time
# bytes
UIUC 04 C. Faloutsos 10
School of Computer ScienceCarnegie Mellon
Problem #1
• Find patterns, in large datasets
time
# bytes
Poisson indep., ident. distr
UIUC 04 C. Faloutsos 11
School of Computer ScienceCarnegie Mellon
Problem #1
• Find patterns, in large datasets
time
# bytes
Poisson indep., ident. distr
UIUC 04 C. Faloutsos 12
School of Computer ScienceCarnegie Mellon
Problem #1
• Find patterns, in large datasets
time
# bytes
Poisson indep., ident. distr
Q: Then, how to generatesuch bursty traffic?
UIUC 04 C. Faloutsos 13
School of Computer ScienceCarnegie Mellon
Overview
• Goals/ motivation: find patterns in large datasets:– (A) Sensor data
– (B) network/graph data
• Solutions: self-similarity and power laws• Discussion
UIUC 04 C. Faloutsos 14
School of Computer ScienceCarnegie Mellon
Problem #2 - network and graph mining• How does the Internet look like?• How does the web look like?• What constitutes a ‘normal’ social
network?• What is the ‘market value’ of a
customer? • which gene/species affects the others
the most?
UIUC 04 C. Faloutsos 15
School of Computer ScienceCarnegie Mellon
Problem#2Given a graph:
• which node to market-to / defend / immunize first?
• Are there un-natural sub-graphs? (eg., criminals’ rings)?
[from Lumeta: ISPs 6/1999]
UIUC 04 C. Faloutsos 16
School of Computer ScienceCarnegie Mellon
Solutions
• New tools: power laws, self-similarity and ‘fractals’ work, where traditional assumptions fail
• Let’s see the details:
UIUC 04 C. Faloutsos 17
School of Computer ScienceCarnegie Mellon
Overview
• Goals/ motivation: find patterns in large datasets:– (A) Sensor data– (B) network/graph data
• Solutions: self-similarity and power laws
• Discussion
UIUC 04 C. Faloutsos 18
School of Computer ScienceCarnegie Mellon
What is a fractal?
= self-similar point set, e.g., Sierpinski triangle:
...zero area: (3/4)^inf
infinite length!
(4/3)^inf
Q: What is its dimensionality??
UIUC 04 C. Faloutsos 19
School of Computer ScienceCarnegie Mellon
What is a fractal?
= self-similar point set, e.g., Sierpinski triangle:
...zero area: (3/4)^inf
infinite length!
(4/3)^inf
Q: What is its dimensionality??A: log3 / log2 = 1.58 (!?!)
UIUC 04 C. Faloutsos 21
School of Computer ScienceCarnegie Mellon
Intrinsic (‘fractal’) dimension
• Q: fractal dimension of a line?
• A: nn ( <= r ) ~ r^1(‘power law’: y=x^a)
• Q: fd of a plane?• A: nn ( <= r ) ~ r^2
fd== slope of (log(nn) vs.. log(r) )
UIUC 04 C. Faloutsos 22
School of Computer ScienceCarnegie Mellon
Sierpinsky triangle
log( r )
log(#pairs within <=r )
1.58
== ‘correlation integral’
= CDF of pairwise distances
UIUC 04 C. Faloutsos 23
School of Computer ScienceCarnegie Mellon
Observations: Fractals <-> power laws
Closely related:
• fractals <=>
• self-similarity <=>
• scale-free <=>
• power laws ( y= xa ; F=K r-2)
• (vs y=e-ax or y=xa+b)log( r )
log(#pairs within <=r )
1.58
UIUC 04 C. Faloutsos 24
School of Computer ScienceCarnegie Mellon
Outline
• Problems
• Self-similarity and power laws
• Solutions to posed problems
• Discussion
UIUC 04 C. Faloutsos 25
School of Computer ScienceCarnegie Mellon
time
#bytes
Solution #1: traffic
• disk traces: self-similar: (also: [Leland+94])• How to generate such traffic?
UIUC 04 C. Faloutsos 26
School of Computer ScienceCarnegie Mellon
Solution #1: traffic
• disk traces (80-20 ‘law’) – ‘multifractals’
time
#bytes
20% 80%
UIUC 04 C. Faloutsos 28
School of Computer ScienceCarnegie Mellon
80-20 / multifractals20
• p ; (1-p) in general
• yes, there are dependencies
80
UIUC 04 C. Faloutsos 29
School of Computer ScienceCarnegie Mellon
Overview
• Goals/ motivation: find patterns in large datasets:– (A) Sensor data
– (B) network/graph data
• Solutions: self-similarity and power laws– sensor/traffic data
– network/graph data
• Discussion
UIUC 04 C. Faloutsos 30
School of Computer ScienceCarnegie Mellon
Problem #2 - topology
How does the Internet look like? Any rules?
UIUC 04 C. Faloutsos 31
School of Computer ScienceCarnegie Mellon
Patterns?
• avg degree is, say 3.3• pick a node at random
- what is the degree you expect it to have?
degree
count
avg: 3.3
UIUC 04 C. Faloutsos 32
School of Computer ScienceCarnegie Mellon
Patterns?
• avg degree is, say 3.3• pick a node at random
- what is the degree you expect it to have?
• A: 1!!
degree
count
avg: 3.3
UIUC 04 C. Faloutsos 33
School of Computer ScienceCarnegie Mellon
Patterns?
• avg degree is, say 3.3• pick a node at random
- what is the degree you expect it to have?
• A: 1!!• A’: very skewed distr.• Corollary: the mean is
meaningless!• (and std -> infinity (!))
degree
count
avg: 3.3
UIUC 04 C. Faloutsos 34
School of Computer ScienceCarnegie Mellon
Solution#2: Rank exponent R• A1: Power law in the degree distribution
[SIGCOMM99]
internet domains
log(rank)
log(degree)
-0.82
att.com
ibm.com
UIUC 04 C. Faloutsos 35
School of Computer ScienceCarnegie Mellon
Solution#2’: Eigen Exponent E
• A2: power law in the eigenvalues of the adjacency matrix
E = -0.48
Exponent = slope
Eigenvalue
Rank of decreasing eigenvalue
May 2001
UIUC 04 C. Faloutsos 37
School of Computer ScienceCarnegie Mellon
Power laws - discussion
• do they hold, over time?
• do they hold on other graphs/domains?
UIUC 04 C. Faloutsos 38
School of Computer ScienceCarnegie Mellon
Power laws - discussion
• do they hold, over time?
• Yes! for multiple years [Siganos+]
• do they hold on other graphs/domains?
• Yes!– web sites and links [Tomkins+], [Barabasi+]– peer-to-peer graphs (gnutella-style)– who-trusts-whom (epinions.com)
UIUC 04 C. Faloutsos 39
School of Computer ScienceCarnegie Mellon
Time Evolution: rank R
-1
-0.9
-0.8
-0.7
-0.6
-0.50 200 400 600 800
Instances in time: Nov'97 and on
Ra
nk
ex
po
ne
nt
• The rank exponent has not changed! [Siganos+]
Domainlevel
log(rank)
log(degree)
-0.82
att.com
ibm.com
UIUC 04 C. Faloutsos 40
School of Computer ScienceCarnegie Mellon
The Peer-to-Peer Topology
• Number of immediate peers (= degree), follows a power-law
[Jovanovic+]
degree
count
UIUC 04 C. Faloutsos 41
School of Computer ScienceCarnegie Mellon
epinions.com
• who-trusts-whom [Richardson + Domingos, KDD 2001]
(out) degree
count
UIUC 04 C. Faloutsos 42
School of Computer ScienceCarnegie Mellon
Outline
• problems
• Fractals
• Solutions
• Discussion – what else can they solve? – how frequent are fractals?
UIUC 04 C. Faloutsos 43
School of Computer ScienceCarnegie Mellon
What else can they solve?
• separability [KDD’02]• forecasting [CIKM’02]• dimensionality reduction [SBBD’00]• non-linear axis scaling [KDD’02]• disk trace modeling [PEVA’02]• selectivity of spatial queries [PODS’94,
VLDB’95, ICDE’00]• ...
UIUC 04 C. Faloutsos 44
School of Computer ScienceCarnegie Mellon
Problem #3 - spatial d.m.
Galaxies (Sloan Digital Sky Survey w/ B. Nichol) - ‘spiral’ and ‘elliptical’
galaxies
- patterns? (not Gaussian; not uniform)
-attraction/repulsion?
- separability??
UIUC 04 C. Faloutsos 45
School of Computer ScienceCarnegie Mellon
Solution#3: spatial d.m.
log(r)
log(#pairs within <=r )
spi-spi
spi-ell
ell-ell
- 1.8 slope
- plateau!
- repulsion!
CORRELATION INTEGRAL!
UIUC 04 C. Faloutsos 46
School of Computer ScienceCarnegie Mellon
Solution#3: spatial d.m.
log(r)
log(#pairs within <=r )
spi-spi
spi-ell
ell-ell
- 1.8 slope
- plateau!
- repulsion!
[w/ Seeger, Traina, Traina, SIGMOD00]
UIUC 04 C. Faloutsos 47
School of Computer ScienceCarnegie Mellon
spatial d.m.
r1r2
r1
r2
Heuristic on choosing # of clusters
UIUC 04 C. Faloutsos 48
School of Computer ScienceCarnegie Mellon
Solution#3: spatial d.m.
log(r)
log(#pairs within <=r )
spi-spi
spi-ell
ell-ell
- 1.8 slope
- plateau!
- repulsion!
UIUC 04 C. Faloutsos 49
School of Computer ScienceCarnegie Mellon
Problem#4: dim. reduction
• given attributes x1, ... xn
– possibly, non-linearly correlated
• drop the useless ones mpg
cc
UIUC 04 C. Faloutsos 50
School of Computer ScienceCarnegie Mellon
Problem#4: dim. reduction
• given attributes x1, ... xn
– possibly, non-linearly correlated
• drop the useless ones
(Q: why? A: to avoid the ‘dimensionality curse’)Solution: keep on dropping attributes, until
the f.d. changes! [SBBD’00]
mpg
cc
UIUC 04 C. Faloutsos 51
School of Computer ScienceCarnegie Mellon
Outline
• problems
• Fractals
• Solutions
• Discussion – what else can they solve? – how frequent are fractals?
UIUC 04 C. Faloutsos 52
School of Computer ScienceCarnegie Mellon
Fractals & power laws:
appear in numerous settings:
• medical
• geographical / geological
• social
• computer-system related
• <and many-many more! see [Mandelbrot]>
UIUC 04 C. Faloutsos 53
School of Computer ScienceCarnegie Mellon
Fractals: Brain scans
• brain-scans
octree levels
Log(#octants)
2.63 = fd
UIUC 04 C. Faloutsos 54
School of Computer ScienceCarnegie Mellon
More fractals
• periphery of malignant tumors: ~1.5
• benign: ~1.3
• [Burdet+]
UIUC 04 C. Faloutsos 55
School of Computer ScienceCarnegie Mellon
More fractals:
• cardiovascular system: 3 (!) lungs: ~2.9
UIUC 04 C. Faloutsos 56
School of Computer ScienceCarnegie Mellon
Fractals & power laws:
appear in numerous settings:
• medical
• geographical / geological
• social
• computer-system related
UIUC 04 C. Faloutsos 57
School of Computer ScienceCarnegie Mellon
More fractals:
• Coastlines: 1.2-1.58
1 1.1
1.3
UIUC 04 C. Faloutsos 59
School of Computer ScienceCarnegie Mellon
Cross-roads of Montgomery county:
•any rules?
GIS points
UIUC 04 C. Faloutsos 60
School of Computer ScienceCarnegie Mellon
GIS
A: self-similarity:• intrinsic dim. = 1.51
log( r )
log(#pairs(within <= r))
1.51
UIUC 04 C. Faloutsos 61
School of Computer ScienceCarnegie Mellon
Examples:LB county
• Long Beach county of CA (road end-points)
1.7
log(r)
log(#pairs)
UIUC 04 C. Faloutsos 62
School of Computer ScienceCarnegie Mellon
More power laws: areas – Korcak’s law
Scandinavian lakes
Any pattern?
UIUC 04 C. Faloutsos 63
School of Computer ScienceCarnegie Mellon
More power laws: areas – Korcak’s law
Scandinavian lakes area vs complementary cumulative count (log-log axes)
log(count( >= area))
log(area)
UIUC 04 C. Faloutsos 64
School of Computer ScienceCarnegie Mellon
More power laws: Korcak
Japan islands;
area vs cumulative count (log-log axes) log(area)
log(count( >= area))
UIUC 04 C. Faloutsos 65
School of Computer ScienceCarnegie Mellon
More power laws
• Energy of earthquakes (Gutenberg-Richter law) [simscience.org]
log(count)
Magnitude = log(energy)day
Energy released
UIUC 04 C. Faloutsos 66
School of Computer ScienceCarnegie Mellon
Fractals & power laws:
appear in numerous settings:
• medical
• geographical / geological
• social
• computer-system related
UIUC 04 C. Faloutsos 67
School of Computer ScienceCarnegie Mellon
A famous power law: Zipf’s law
• Bible - rank vs. frequency (log-log)
log(rank)
log(freq)
“a”
“the”
“Rank/frequency plot”
UIUC 04 C. Faloutsos 68
School of Computer ScienceCarnegie Mellon
TELCO data
# of service units
count ofcustomers
‘best customer’
UIUC 04 C. Faloutsos 69
School of Computer ScienceCarnegie Mellon
SALES data – store#96
# units sold
count of products
“aspirin”
UIUC 04 C. Faloutsos 70
School of Computer ScienceCarnegie Mellon
Olympic medals (Sidney 2000):
y = -0.9676x + 2.3054
R2 = 0.9458
0
0.5
1
1.5
2
2.5
0 0.5 1 1.5 2
log( rank)
log(#medals)
UIUC 04 C. Faloutsos 71
School of Computer ScienceCarnegie Mellon
Olympic medals (Athens04):
log( rank)
log(#medals)
0
0.5
1
1.5
2
2.5
0 0.5 1 1.5 2
athens
sidney
UIUC 04 C. Faloutsos 72
School of Computer ScienceCarnegie Mellon
Even more power laws:
• Income distribution (Pareto’s law)• size of firms
• publication counts (Lotka’s law)
UIUC 04 C. Faloutsos 73
School of Computer ScienceCarnegie Mellon
Even more power laws:
library science (Lotka’s law of publication count); and citation counts: (citeseer.nj.nec.com 6/2001)
log(#citations)
log(count)
Ullman
UIUC 04 C. Faloutsos 74
School of Computer ScienceCarnegie Mellon
Even more power laws:
• web hit counts [w/ A. Montgomery]
Web Site Traffic
log(freq)
log(count)
Zipf“yahoo.com”
UIUC 04 C. Faloutsos 75
School of Computer ScienceCarnegie Mellon
Fractals & power laws:
appear in numerous settings:
• medical
• geographical / geological
• social
• computer-system related
UIUC 04 C. Faloutsos 76
School of Computer ScienceCarnegie Mellon
Power laws, cont’d
• In- and out-degree distribution of web sites [Barabasi], [IBM-CLEVER]
log indegree
- log(freq)
from [Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan, Andrew Tomkins ]
UIUC 04 C. Faloutsos 77
School of Computer ScienceCarnegie Mellon
Power laws, cont’d
• In- and out-degree distribution of web sites [Barabasi], [IBM-CLEVER]
• length of file transfers [Bestavros+]
• duration of UNIX jobs [Harchol-Balter]
UIUC 04 C. Faloutsos 78
School of Computer ScienceCarnegie Mellon
Conclusions
• Fascinating problems in Data Mining: find patterns in– sensors/streams – graphs/networks
UIUC 04 C. Faloutsos 79
School of Computer ScienceCarnegie Mellon
Conclusions - cont’d
New tools for Data Mining: self-similarity & power laws: appear in many cases
Bad news:
lead to skewed distributions
(no Gaussian, Poisson,
uniformity, independence,
mean, variance)
Good news:• ‘correlation integral’
for separability• rank/frequency plots• 80-20 (multifractals)• (Hurst exponent, • strange attractors,• renormalization theory, • ++)
UIUC 04 C. Faloutsos 80
School of Computer ScienceCarnegie Mellon
Resources
• Manfred Schroeder “Chaos, Fractals and Power Laws”, 1991
• Jiawei Han and Micheline Kamber “Data Mining: Concepts and Techniques”, 2001
UIUC 04 C. Faloutsos 81
School of Computer ScienceCarnegie Mellon
References
– [ieeeTN94] W. E. Leland, M.S. Taqqu, W. Willinger, D.V. Wilson, On the Self-Similar Nature of Ethernet Traffic, IEEE Transactions on Networking, 2, 1, pp 1-15, Feb. 1994.
– [pods94] Christos Faloutsos and Ibrahim Kamel, Beyond Uniformity and Independence: Analysis of R-trees Using the Concept of Fractal Dimension, PODS, Minneapolis, MN, May 24-26, 1994, pp. 4-13
UIUC 04 C. Faloutsos 82
School of Computer ScienceCarnegie Mellon
References
– [vldb95] Alberto Belussi and Christos Faloutsos, Estimating the Selectivity of Spatial Queries Using the `Correlation' Fractal Dimension Proc. of VLDB, p. 299-310, 1995
– [vldb96] Christos Faloutsos, Yossi Matias and Avi Silberschatz, Modeling Skewed Distributions Using Multifractals and the `80-20 Law’ Conf. on Very Large Data Bases (VLDB), Bombay, India, Sept. 1996.
UIUC 04 C. Faloutsos 83
School of Computer ScienceCarnegie Mellon
References
– [vldb96] Christos Faloutsos and Volker Gaede Analysis of the Z-Ordering Method Using the Hausdorff Fractal Dimension VLD, Bombay, India, Sept. 1996
– [sigcomm99] Michalis Faloutsos, Petros Faloutsos and Christos Faloutsos, What does the Internet look like? Empirical Laws of the Internet Topology, SIGCOMM 1999
UIUC 04 C. Faloutsos 84
School of Computer ScienceCarnegie Mellon
References
– [icde99] Guido Proietti and Christos Faloutsos, I/O complexity for range queries on region data stored using an R-tree International Conference on Data Engineering (ICDE), Sydney, Australia, March 23-26, 1999
– [sigmod2000] Christos Faloutsos, Bernhard Seeger, Agma J. M. Traina and Caetano Traina Jr., Spatial Join Selectivity Using Power Laws, SIGMOD 2000
UIUC 04 C. Faloutsos 85
School of Computer ScienceCarnegie Mellon
References
– [kdd2001] Agma J. M. Traina, Caetano Traina Jr., Spiros Papadimitriou and Christos Faloutsos: Tri-plots: Scalable Tools for Multidimensional Data Mining, KDD 2001, San Francisco, CA.