in spaces of curves algorithms and data structuresadriemel/wogml_talk.pdf · anne driemel:...

Anne Driemel: Algorithms and data structures for spaces of curves

Algorithms and Data Structures

in Spaces of Curves

Anne Driemel

TU Eindhoven, the Netherlands


Spaces of Curves

A curve is a continuous map

f : [0, 1]→ Rd

f(0)

f(t)

f(1)


Spaces of Curves

A curve is a continuous map

Example:time series data

f : [0, 1]→ Rd

f(0)

f(t)

f(1)


Machine Learning

Step (1): feature extraction

Step (2): learning the distribution in feature space

Hyndman, Wang, Laptev: Large-Scale Unusual Time Series Detection,ICDM Workshops 2015


– results depend on the choice of features

– features have to be manually designed

– features capture certain aspects only

– features are not always independent of one another

Disadvantages of working in feature space:

Machine Learning


– results depend on the choice of features

– features have to be manually designed

– features capture certain aspects only

– features are not always independent of one another

Question: Is it possible to bypass the feature spaceand learn the distribution of curves directly?

Disadvantages of working in feature space:

Machine Learning


Overview of this talk

Machine

learning in

spaces of

curves



Clustering

Median and Depth

Machine

learning in

spaces of

curves

Density estimate

Distribution of curves



Clustering

Median and Depth

Classification ofcurves

Range searching

Machine

learning in

spaces of

curves

NN-suche

Density estimate


Decision trees



Clustering

Median and Depth


Range searching

Dimension reduction

Machine

learning in

spaces of

curves

Geometric approximation

Representation

NN-suche

Density estimate

Overfitting


Decision trees



Clustering

Median and Depth


Range searching

Dimension reduction

Machine

learning in

spaces of

curves


Representation

NN-suche

Density estimate

Overfitting

Distribution of curvesComplexity

Time

Error rate

Space

Sample size

Decision trees


Density estimate

Density estimate using sophisticated histograms (KDE)

Only works in low-dimensional space

Modes

[Scott Multivariate Density Estimation 1950].


Density estimate for curves?

What is the typicalshape of a hurricanetrajectory?


Definition of the median

Let (X, d) be a metric space on a set X withdistance function d : X2 → R≥0

The median of a set P ⊆ X is defined as

argminc∈X

∑p∈P

d(c, p)

c


Overfitting of the median

A

B



A

B

c



A1, . . . , An/2

B1, . . . , Bn/2

c


Smoothing of the median

Definition: Let Xd` be the set of polygonal curves in

Rd that have exactly ` edges.

The `-Median of a set P ⊆ Xdm is defined as

argminc∈Xd

`

∑p∈P

d(c, p)


Clustering of curves

argminC⊆Xd

`|C|=k

∑p∈P

d(nn(p, C), p)

argminC⊆Xd

`|C|=k

maxp∈P

d(nn(p, C), p)

The (k, `)-center-clustering is

The (k, `)-median-clustering is

Let C ⊆ Xd` be a k-set of centers

Let nn(p, C) be the closest center to a point p


Clustering in discrete metric spaces

k-median, randomised:– (1 + ε)-approximation in O(n2(k/ε)

O(1)

) time– Algorithm of [Kumar, Sabharwal, Sen ’10]– Doubling spaces: [Ackermann, Blomer, Sohler ’10]

k-center:– 2-approximation in O(kn) time [Gonzales ’85]

k-median:– (1 +

√3 + ε)-approximation [Li and Svensson ’13]

– Lower bound for the approximation factor ∼ 1.73

Most variants of clustering are NP-hard or APX-hard


Distance measure for curves

A

B

What is the minimal distance with which two peoplecan traverse both entire curves?

δ

The Frechet distance measures the similarity of curves


Clustering of curves

1-center (no smoothing):

– O(m1m2 log(m1 +m2))[Alt and Godau ’95] [Buchin et al. ’13]

– Lower bound based on SETH Ω((m1 +m2)2−δ) (∀δ > 0)[Bringmann ’14] [Bringmann and Mulzer ’15]

– weak Frechet distance: O(mn) time and space[Har-Peled, Raichel ’11]

Given n curves in Rd with m edges each

– discrete Frechet distance: O(mn) time and space[Ahn et al. ’15]

Single distance computation:

Given two curves m1 and m2 edges


Theorem:We can compute an O(1)-approximation for the(k, `)-center-clustering and (k, `)-median-clusteringin O(nm) time. For d = 1 we can compute a(1 + ε)-approximation in the same time.

Our results for clustering

Driemel, Krivosija und Sohler:Clustering time series under the Frechet distanceSODA 2016


Theorem:We can compute an O(1)-approximation for the(k, `)-center-clustering and (k, `)-median-clusteringin O(nm) time. For d = 1 we can compute a(1 + ε)-approximation in the same time.

Sampling-Property:For d = 1 one can derive a (1 + ε)-approximation of the(1, `)-median based on a sample of size mε`.

Our results for clustering

Driemel, Krivosija und Sohler:Clustering time series under the Frechet distanceSODA 2016



Clustering

Median and Depth


Range searching

Dimension reduction

Machine

learning in

spaces of

curves


Representation

NN-suche

Density estimate

Overfitting


Time

Error rate

Space

Sample size

Decision trees


Classification

A central task in machine learning is the approximationof a function

The input is usually a set of examples

(p, `) | p ∈ X, ` ∈ L

f : X → L

X = instancesL = labels


Classification of curves

de Vries, van Someren Machine learning for vessel trajectories using compression,alignments and domain knowledge Expert Syst. Appl. 2012

Cargo ship

Tanker

Law enforcement

Tug


Nearest-neighbor searching

x1

x2

label 1label 2



x1

x2

label 1label 2

q

Query



x1

x2

label 1label 2

q The label of the exampleinstance closest to q

Answer:Query



x1

x2

label 1label 2

q

(1) Partition the space intohomogeneous areas

(2) Data structure forpoint location

The label of the exampleinstance closest to q

Answer:

f : X → L

Query



x1

x2

label 1label 2

q

(1) Partition the space intohomogeneous areas

(2) Data structure forpoint location

The label of the exampleinstance closest to q

Answer:

f : X → L

Voronoi diagram

Θ(ndd/2e)

Query



(a) if d(p,q) ≤ d1 then Pr [h(p) = h(q)] ≥ p1(b) if d(p,q) ≥ d2 then Pr [h(p) = h(q)] ≤ p2

A set of hash functions H is called (d1, d2, p1, p2)-locality-sensitive if it holds for all p,q ∈ Rd:

d1

d2

p


Driemel und Silvestri: Locality-sensitive-hashing for curvesSoCG 2017

Space Query time ApproximationO(mn) O

(m2n

)1 –

O(|U |√m(m

√mn)2

)O(mO(1) log n

)O(logm+ log log n) [Indyk02]

O(n log n+mn) O(m log n) O(m) new

O(24mdn log n+mn) O(24mdm log n)O(1) new

O(22kmk−1n log n+mn)O(22kmk log n) O(m/k) new

Our results for NN-searching

We show LSH-families for the discrete Frechet distanceinput: n curves from Xd

m.



Clustering

Median and Depth


Range searching

Dimension reduction

Machine

learning in

spaces of

curves


Representation

NN-suche

Density estimate

Overfitting


Time

Error rate

Space

Sample size

Decision trees


Range searching for curves

q

Which hurricate trajectories aresimilar to the query curve q?



A range query (q, r) among a set of curves S ⊆ Xdm

should return the set

p ∈ S | d(p, q) ≤ r

The range is defined as the metric ball of radius rcentered at q ∈ Xd

` .



Only known data structure for this type of queries:

For any parameter value 1 ≤ s ≤√n

– approximation ≈ 6.24– query time in O(spolylog n)– space in O((ns )2 polylog n)

Queries: Only supports query curves from X21

[de Berg, Cook and Gudmundsson ’13]

Output: Count of the curves in the range


Our results for range searching

Theorem:

Assume there exists a data structure for Frechet rangequeries among an n-set from Xd

m with– Space in ≤ S(n)– Query time in ≤ Q(n) +O(k) (k is output size)

Then it holds

S(n) = Ω

((n

Q(n)

)2)

Driemel und Afshani, On the complexity of range searchingamong curves (Manuscript).



Our proof uses a volume argument of [Afshani ’13]

Input:Slabs in R2

Output:Slabs that contain q

q

Q = [0, 1]2




Input:Slabs in R2


Q = [0, 1]2

Input construction withoutput size k = t




Input:Slabs in R2


Q = [0, 1]2

S(n) ≥ tvmax


vmax




Input:Slabs in R2


v ≤ t3

n2·π

tn

Q = [0, 1]2

πt


S(n) ≥ tvmax

≥(nt

)2




Input:Slabs in R2


Q = [0, 1]2


S(n) ≥ tvmax

≥(nt

)2=(

nQ(n)

)2

v ≤ t3

n2·π



q q

Dualization of range queries

p p

input space query space



What is the locus of queries that output this curve?

Restrict queries to X21 (Single edges)



1

2

3

4

5

6

r





α

β

`

Represent query lines in the dual space (β, α)



α

β

`

p

xc

ycr

Represent query lines in the dual space (β, α)

Lines stabbing the disk at (xc, yc) with radius r:

α ≤ xc sin(β) + yc cos(β) + r

α ≥ xc sin(β) + yc cos(β)− r


β

α

q

q

Dualization of range queries for curves



A


β

α

q

q




A B


β

α


Q

Q



A B



t

nt

Sketching the input construction for the lower bound:

– Fix the input curves at their endpoints



t

nt

xi


– A query line outputs an input curve pij iff the queryline intersects the vertical grid edge at (i, j)

yj




t

nt



xi


yj



t

nt




???



β

α

Two input curves in the query space

Q



β

α

Two input curves in the query space


Lower bounds for range searching

Theorem:

Assume there exists a data structure for Frechet rangequeries in Xd

` among an n-set from Xdm with

– Space in ≤ S(n)– Query time in ≤ Q(n) +O(k) (k is output size)

Then it holds

Driemel und Afshani, On the complexity of range searchingamong curves (Manuscript)

S(n) = Ω

((n

Q(n)

)2)


Lower bounds for range searching

Assume there exists a data structure for Frechet rangequeries in Xd


– Space in ≤ S(n)– Query time in ≤ Q(n) +O(k) (k is output size)

Then it holds


S(n) = Ω

((n

Q(n)

)2

·(

log(n/Q(n))

`2

)`−2)

Theorem (Extension):


Data structures

Theorem (discrete Frechet):

We can build a data structure for discrete Frechet rangequeries in Xd


– Space in O(n(log log n)m−1)

– Query time in O(n1−1/d logO(m) n · `O(d) + k)( where k is the output size)

assuming ` ∈ O(polylog n).

We can match the lower bound with a multi-levelpartition tree that has the correct number of levels:



Data structures



We can build a data structure for continuous Frechetrange queries in X2

` among an n-set from X2m with

– Space in O(n(log log n)O(m2))

– Query time in O(√n logO(m) n+ k)

( where k is the output size)assuming ` ∈ O(polylog n).

Theorem (continuous Frechet):


Data structure for discrete Frechet

Sketch of the data structure

(input curves)




P1

Multi-level partition tree

P1 P2 P3 . . .

(input curves)


P2




P1 P2 P3 . . .

(input curves)




P3


P1 P2 P3 . . .

(input curves)




P4


P1 P2 P3 . . .

. . .

(input curves)


q1

q2

q3q4

q5


Sketch of the query algorithm

(query curve)


q1

q2

q3q4

q5



q1 1q2 1q3 0q4 1q5 0

(query curve)


q1 1 0 0 1q2 1 0 0 0q3 0 1 0 0q4 1 0 1 0q5 0 0 0 1




q1 1 0 0 1q2 1 0 0 0q3 0 1 0 0q4 1 0 1 0q5 0 0 0 1



P1


q1 1 0 0 1q2 1 0 0 0q3 0 1 0 0q4 1 0 1 0q5 0 0 0 1



P2


q1 1 0 0 1q2 1 0 0 0q3 0 1 0 0q4 1 0 1 0q5 0 0 0 1



P3


q1 1 0 0 1q2 1 0 0 0q3 0 1 0 0q4 1 0 1 0q5 0 0 0 1



P4


q1 1 0 0 1q2 1 0 0 0q3 0 1 0 0q4 1 0 1 0q5 0 0 0 1



feasible alignment


Data structures

Theorem (discrete Frechet):

We can build a data structure for discrete Frechet rangequeries in Xd


– Space in O(n · (log log n)m−1)

– Query time in O(n1−1/d logO(m) n · `O(d) + k)( where k is the output size)

assuming ` in O(polylog n).




Data structures



We can build a data structure for continuous Frechetrange queries in X2

` among an n-set from X2m with

– Space in O(n · (log log n)O(m2))

– Query time in O(√n logO(m) n+ k)

( where k is the output size)assuming ` in O(polylog n).

Theorem (continuous Frechet):



Clustering

Median and Depth


Range searching

Dimension reduction

Machine

learning in

spaces of

curves


Representation

NN-suche

Density estimate

Overfitting


Time

Error rate

Space

Sample size

Decision trees


Summary

We talked about:

(3) Machine learning for curves

(1) Clustering for curves

Challenges:

– Curse of dimensionality

– Risk of overfitting

– Complexity of the distance measure

(2) Data structuring for curves


Some open problems

– Clustering

– NN-Searching

Practical algorithms for clustering of curvesMore robust definitions of median and depth

Randomized curve simplificationLSH for continuous Frechet distance

– Range-Searching

DS for high-space/ low-query-time regime

Continuous Frechet distance in 3D (seems much harder)

in spaces of curves algorithms and data structuresadriemel/wogml_talk.pdf · anne driemel:...

Documents