phd defense - aarhus universitetgerth/advising/slides/jesper-sindahl... · 2015. 10. 2. · phd...

AARHUSUNIVERSITY

PhD Defense

Jesper Sindahl Nielsen

Implicit Data Structures,Sorting, and Text Indexing

Overview

1/24


Overview

1/24


1. Algorithms and Data Structures(a) Abstract description(b) Example

2. Implicit Data Structures(a) Finger Search(b) Priority Queues

3. Integer Sorting4. Text Indexing5. State and University Library

Algorithms

2/24


Algorithms

2/24


Algorithm

Algorithms

2/24


Algorithminput

Algorithms

2/24


Algorithminput output

Algorithms

2/24



x + yAddition

Algorithms

2/24



x=5y=7 x + y

Addition

Algorithms

2/24



12x=5y=7 x + y

Addition

Algorithms

2/24



12x=5y=7 x + y

Addition

Sort L

Algorithms

2/24



12x=5y=7 x + y

Addition

L=5,1,3,2,4,6 Sort L

Algorithms

2/24



12x=5y=7 x + y

Addition

L=1,2,3,4,5,6L=5,1,3,2,4,6 Sort L

We want to design fast algorithms

Data Structures

3/24


Data Structures

3/24


Data

Data Structures

3/24


DataData

Structure

Data Structures

3/24


DataData

Structure

Data organized insome clever manner

Data Structures

3/24


Data BuildData

Structure


Data Structures

3/24


Data BuildData

Structure


Mom,375874Dad,278393Brother,681958Advisor,359617Arnold,123456Sylvester,987654Jean-Claude,456321Wesley,789123

Data Structures

3/24


Data BuildData

Structure



create sorted phonebook

Data Structures

3/24


Data BuildData

Structure




Advisor,359617Arnold,123456Brother,681958Dad,278393Jean,456321Mom,375874Sylvester,987654Wesley,789123

Data Structures

3/24


Data BuildData

Structure


DataStructure

input




Data Structures

3/24


Data BuildData

Structure


DataStructure

Queryinput




Data Structures

3/24


Data BuildData

Structure


DataStructure

Queryinput

answer to query on data




Data Structures

3/24


Data BuildData

Structure


DataStructure

Queryinput





Arnold

Data Structures

3/24


Data BuildData

Structure


DataStructure

Queryinput





find phone number

Arnold

Data Structures

3/24


Data BuildData

Structure


DataStructure

Queryinput





find phone number

Arnold 123456

Data Structures

3/24


Data BuildData

Structure


DataStructure

Queryinput





find phone number

Mom 375874

Arnold 123456

Data Structures

3/24


Data BuildData

Structure


DataStructure

Queryinput





find phone number

Mom 375874

Arnold 123456

efficient when data is organized

Data Structures

3/24


Data BuildData

Structure




Advisor,359617Arnold,123456Brother,681958Dad,278393Jean,456321Mom,375874Sylvester,987654Wesley,789123Data

StructureUpdate

inputUpd. DataStructure

Data Structures

3/24


Data BuildData

Structure





StructureUpdate


Dad,357951

Data Structures

3/24


Data BuildData

Structure





StructureUpdate


Dad,357951


Change phone number

Data Structures

3/24


Data BuildData

Structure





StructureUpdate


Dad,357951


Change phone number

We want space efficient structures,fast queries, and fast updates

Outline

4/24





Implicit Data Structures

5/24



5/24


Primary goal: space efficiency


5/24



Data Structure has n comparable elements, laid out in an array of size n.


5/24



0 1 2


n− 1


5/24



0 1 2


n− 1

Only the array and n is maintained between operations.


5/24



0 1 2


n− 1


Strict implicit model: Nothing but elements and n stored


5/24



0 1 2


n− 1



Weak implicit model: Allow a constant number of extra memory cells


5/24



0 1 2


n− 1




We work with the strict implicit model


5/24



0 1 2


n− 1





Fundamental trick: pair encoding bits.


5/24



0 1 2


n− 1





Fundamental trick: pair encoding bits. x yx<y ⇒1

x>y ⇒0


5/24



0 1 2


n− 1






x>y ⇒0Requires distinct elements.


5/24



0 1 2


n− 1






x>y ⇒0Requires distinct elements.

Procedures can use a constant number of working memory/registers

Dictionary and Fingersearch

6/24



6/24


Maintain a (dynamic) set of n keys, while supporting:


6/24


Insert(k)



6/24


Insert(k)

Delete(k)



6/24


Insert(k)

Delete(k)

Find(k)



6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)



6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Successor(k)



6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Time: O(log n)

Successor(k)



6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)

Only in dynamic structures

Time: O(log n)

Successor(k)



6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)


Time: O(log n)

Successor(k)


Special element: the finger element


6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)


Time: O(log n)

Successor(k)



a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10


6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)


Time: O(log n)

Successor(k)



a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

n=11


6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)


Time: O(log n)

Successor(k)



a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

finger

n=11


6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)


Time: O(log n)

Successor(k)



a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

finger

n=11

Find(m)


6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)


Time: O(log n)

Successor(k)



a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

finger

n=11

Find(m)

Distance: t = 3


6/24


Insert(k)

Delete(k)

Find(k)

Predecessor(k)


Time: O(log n)

Successor(k)



a c d g j ` m o q s z0 1 2 3 4 5 6 7 8 9 10

finger

n=11

Find(m)

Distance: t = 3

Time: O(log t) forFind, Predecessor, and Successor

Implicit Finger Search Idea

7/24



7/24


· · ·

220

221

222

22`

· · · · · · · · ·


7/24


· · ·

220

221

222

D0

22`

· · · · · · · · ·


7/24


· · ·

220

221

222

D0 D1

22`

· · · · · · · · ·


7/24


· · ·

· · ·

220

221

222

D0 D1 D2 D`

22`

· · · · · · · · ·


7/24


· · ·

· · ·

220

221

222

D0 D1 D2 D`

22`

· · ·

Elements in sorted order

· · · · · ·


7/24


· · ·

· · ·

220

221

222

D0 D1 D2 D`

f

22`

· · ·


· · · · · ·


7/24


· · ·

· · ·

220

221

222

D0 D1 D2 D`

f

22`

· · ·


· · ·

· · · · · ·

· · ·


7/24


· · ·

· · ·

220

221

222

D0 D1 D2 D`

f

22`

· · ·


· · ·

· · · · · ·

· · ·

Operation Time

Insert

Delete

Find O(log t)

Change-Finger

O(logn)

O(logn)

O(nε)


7/24


· · ·

· · ·

220

221

222

D0 D1 D2 D`

f

22`

· · ·


· · ·

· · · · · ·

· · ·

Operation Time

Insert

Delete

Find O(log t)

Change-Finger

O(logn)

O(logn)

O(nε)

optimal trade-off

Priority Queues

8/24


Priority Queues

8/24


Maintain a set of n keys and support:

• FindMin: Return the minimum element• ExtractMin: Remove the minimum element from the set• Insert(k): Add the key k to the set

Priority Queues

8/24




Insert ExtractMin Moves StrictIdenticalElements

Amortized bounds are marked with ?

Priority Queues

8/24




Williams



log nlog n log n yes yes

Priority Queues

8/24




Williams

Carlsson et al.




log n1 log n no yes

Priority Queues

8/24




Williams

Carlsson et al.




log n1 log n no yes

log n1 log n no yesEdelkamp et al.

Priority Queues

8/24




Williams

Carlsson et al.

Harvey & Zatloukal




log n1 log n no yes


? log n? 1 ? log n yes yes

Priority Queues

8/24




Williams

Franceschini & Munro

Carlsson et al.

Harvey & Zatloukal




log n1 log n no yes



? log n? log n ? 1 yes no

Priority Queues

8/24




Williams


Carlsson et al.

Harvey & Zatloukal




log n1 log n no yes




? log n? 1 ? 1 yes yesNew

Priority Queues

8/24




Williams


Carlsson et al.

Harvey & Zatloukal




log n1 log n no yes




? log n? 1 ? 1 yes yes

log n1 log n yes no

New

New

Outline

9/24


Integer Sorting

10/24


Comparison based sorting: only allowed operation ≤

Integer Sorting

10/24



A B

Integer Sorting

10/24



A B

conclude A > B

Integer Sorting

10/24



A B

conclude A > B

To sort n gold pieces we needΘ(n log n) scalings/comparisons

Integer Sorting

10/24



A B

conclude A > B


A B

287315

Integer Sorting

10/24



A B

conclude A > B


A B

287315

Question: Can we sort faster ifwe have access to a digital scale?

Integer Sorting

10/24



A B

conclude A > B


A B

287315


Answer: Yes.

Integer Sorting

10/24



A B

conclude A > B


A B

287315


Answer: Yes.

Question: How much faster?

Integer Sorting

10/24



A B

conclude A > B


A B

287315


Answer: Yes.

Depends on accuracy of the scale

Question: How much faster?

Integer Sorting

11/24


Integer Sorting

11/24


Time Space NotesAlgorithm

w is the number of bits per word.

Integer Sorting

11/24



Radix sort (1887?) O(n · wlogn

) O(n) optimal for w = O(log n)


Integer Sorting

11/24





vEB sort (1975) O(n log w) O(2w) Only O(n) space with Y-fast trie


Integer Sorting

11/24






Kirkpatrick &Reisch (1984)

O(n log wlogn

) Only asymptotically better thanvEB when w = O(log n)


Integer Sorting

11/24







O(n log wlogn


Signature sort(1995)

O(n) O(n) Only for w = Ω(log2+ε n)


Integer Sorting

11/24







O(n log wlogn




Han & Thorup(2002) O(n

√log(w/ log n)) Randomized with mulitplicationO(n)


Integer Sorting

11/24







O(n log wlogn





√log(w/ log n)) Randomized with mulitplication

Thorup (2002) O(n log log n) O(n) Randomized, AC0

O(n)


Integer Sorting

11/24







O(n log wlogn





√log(w/ log n)) Randomized with mulitplication

Thorup (2002) O(n log log n) O(n) Randomized, AC0

O(n)

Han (2004) O(n log log n) O(n) Deterministic, with multiplication


Integer Sorting

12/24


Time NotesAlgorithm


) Optimal for w = O(log n)

Signature sort (1995) O(n) Only for w = Ω(log2+ε n)

Han & Thorup (2002) O(n√log(w/ log n)

)Randomized with mulitplication

Time per element

w

log n

O(1)

O(√log log n)

log2 n log log n log2+ε n

NEW1

2

3

1

2

3

Overview

13/24





Text Indexing

14/24


Text Indexing

14/24


Most of us daily use services like Google, Bing, etc.

Text Indexing

14/24



We want documents that contain both terms

Text Indexing

14/24



We want documents that contain both terms

Find all emails sent to madalgo that are not seminars

Text Indexing - Two Patterns

15/24



15/24


Problem definition:


15/24


Problem definition:

Given documents D = d1, d2, . . . , dD


15/24


Problem definition:


preprocess and store D such that


15/24


Problem definition:



when given two text strings P1, P2 (online)


15/24


Problem definition:




we can report all di where both P1 and P2 occur


15/24


Problem definition:




we can report all di where both P1 and P2 occur(The two-pattern problem)


15/24


Problem definition:





we can report all di where both P1 occurs but P2

does not occur

(The two-pattern problem)


15/24


Problem definition:





we can report all di where both P1 occurs but P2

does not occur

(The two-pattern problem)

(The forbidden pattern problem)


16/24


Space Query Time AuthorsProblem


16/24



Two-Pattern (2P) n2−α logc n nα + t Ferragina et al. ’03


16/24




Two-Pattern n logn√nt log n log2 n + t Cohen and Porat ’10


16/24




Two-Pattern n logn√nt log n log2 n + t

Two-Pattern n√nt log n log n + t

Cohen and Porat ’10

Hon et al. ’10


16/24







Hon et al. ’10

Forbidden Pattern (FP) n3/2/ logn√n + t Fischer et al. ’12


16/24







Hon et al. ’10


Forbidden Pattern Hon et al. ’12n√nt log n log2 n + t


16/24







Hon et al. ’10



Both 2P and FP n√nt log n logε n + t New


16/24







Hon et al. ’10



Both 2P and FP n√nt log n logε n + t New

Is√n query time really necessary?


17/24



17/24


Build suffix tree on concatenation of the documents


17/24



S = d1$d2$ · · · $dD$


17/24



S = d1$d2$ · · · $dD$

Annotate each leaf with thedocument id the suffx starts in

4 5 4 7 6 4 8 9 9 1 2 3


17/24



S = d1$d2$ · · · $dD$


4 5 4 7 6 4 8 9 9 1 2 3Query: P1 P2and


17/24



S = d1$d2$ · · · $dD$


4 5 4 7 6 4 8 9 9 1 2 3Query: P1 P2and

P1P2


17/24



S = d1$d2$ · · · $dD$


4 5 4 7 6 4 8 9 9 1 2 3Query: P1 P2and

A

P1P2


17/24



S = d1$d2$ · · · $dD$


4 5 4 7 6 4 8 9 9 1 2 3Query: P1 P2and

A B

P1P2


17/24



S = d1$d2$ · · · $dD$


4 5 4 7 6 4 8 9 9 1 2 3Query: P1 P2and

A B

Report: A B∩

P1P2


17/24



S = d1$d2$ · · · $dD$


4 5 4 7 6 4 8 9 9 1 2 3Query: P1 P2and

A B

Report: A B∩

P1P2

= 4


17/24



S = d1$d2$ · · · $dD$


4 5 4 7 6 4 8 9 9 1 2 3Query: P1 P2and

A B

Report: A B∩

P1P2

= 4set intersection is usually hard


18/24



18/24


Lower Bound NotesProblem


18/24



Both 2P and FP ω is matrix multiplication constant (new)P (n) + nQ(n) = Ω(nω)


18/24




Two-Pattern P (n) + n1+εQ(n) = Ω(n4/3−o(1)) Kopelowitz et al (3SUM)


18/24




Two-Pattern P (n) + n1+εQ(n) = Ω(n4/3−o(1))

Two-Pattern P (n) + n3/4+εQ(n) = Ω(n4/3−ε)

Kopelowitz et al (3SUM)



18/24



Both 2P and FP ω is matrix multiplication constant (new)

Both 2P and FP Pointer Machine Model (new)

P (n) + nQ(n) = Ω(nω)

S(n)Q(n) = Ω(n2−o(1))






18/24





Semi group model (new)

P (n) + nQ(n) = Ω(nω)

S(n)Q(n) = Ω(n2−o(1))

S(n)Q(n)2 = Ω(n2−o(1))Both 2P and FP






18/24





Semi group model (new)

P (n) + nQ(n) = Ω(nω)

S(n)Q(n) = Ω(n2−o(1))

S(n)Q(n)2 = Ω(n2−o(1))Both 2P and FP





Q(n) = (nt)12−α ⇒

Both 2P and FPS(n) = Ω(n

1+6α1+2α

−o(1))

Pointer Machine Modelt is the output size (new)

Text Indexing - How to prove the lower bounds?

19/24


We use two different techniques:

1) Reduction from Matrix Multiplication

2) Find a hard input for a Pointer Machine Data Structure

1) Given two matrices, produce a set of documentsand queries such that the product of the twomatrices can be found


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

dA1 = 1#2#3#

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

dA1 = 1#2#3#

dA2 = 3#

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

123

4 5 6 Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6#

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


× =T F TT F FT T T

F F TF T TT F T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24

P1 = 1 P2 = 4Jesper Sindahl Nielsen

Query:

× =T F TT F FT T T

F F TF T TT F T

T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24

P1 = 1 P2 = 4Jesper Sindahl Nielsen

Query:

× =T F TT F FT T T

F F TF T TT F T

T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24

1 5P1 = 1 P2 =Jesper Sindahl Nielsen

Query:

× =T F TT F FT T T

F F TF T TT F T

T F

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T


20/24


Query:

× =T F TT F FT T T

F F TF T TT F T

T F

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T

i ∈ d` ⇔ Ai,` = Tj + 3 ∈ d` ⇔ B`,j = T


20/24


Query:

× =T F TT F FT T T

F F TF T TT F T

T F TF F TT

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T

i ∈ d` ⇔ Ai,` = Tj + 3 ∈ d` ⇔ B`,j = T


20/24


Query:

× =T F TT F FT T T

F F TF T TT F T

T F TF F TT

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T

i ∈ d` ⇔ Ai,` = Tj + 3 ∈ d` ⇔ B`,j = T


20/24

3 5P1 = P2 =Jesper Sindahl Nielsen

Query:

× =T F TT F FT T T

F F TF T TT F T

T F TF F TT T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T

i ∈ d` ⇔ Ai,` = Tj + 3 ∈ d` ⇔ B`,j = T


20/24

3 5P1 = P2 =Jesper Sindahl Nielsen

Query:

× =T F TT F FT T T

F F TF T TT F T

T F TF F TT T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T

i ∈ d` ⇔ Ai,` = Tj + 3 ∈ d` ⇔ B`,j = T


20/24


3 6P1 = P2 =Query:

× =T F TT F FT T T

F F TF T TT F T

T F TF F TT T T

A B C

dA1 = 1#2#3#

dA2 = 3#

dA3 = 1#3#

dB1 = 6#

dB2 = 5#6#

dB3 = 4#6#

d1 = 1#2#3#6#

d2 = 3#5#6#

d3 = 1#3#4#6# D

123

Ci,j = T ⇔ ∃` s.t.

Ai,` = B`,j = T

i ∈ d` ⇔ Ai,` = Tj + 3 ∈ d` ⇔ B`,j = T


20/24


3 6P1 = P2 =Query:

Overview

21/24





State and University Library

22/24



22/24


”Old Days”


22/24


”Old Days”

LOTS of data

A nuisance in a collection

23/24



23/24


Danish Radio collection 2 hour chunks

Before one runs out another is started


23/24



Before one runs out another is started⇒ overlap


23/24




Annoying for customers


23/24





Task: find overlap and eliminate it


23/24





Task: find overlap and eliminate it

Use Cross Correlation.

24/24

Thank You


phd defense - aarhus universitetgerth/advising/slides/jesper-sindahl... · 2015. 10. 2. · phd...

Documents