a space- and time- e ffi cient hash table hierarchically indexed by bloom filters author: heeyeol yu...
TRANSCRIPT
![Page 1: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/1.jpg)
A SPACE- AND TIME-EffiCIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS
Author: Heeyeol Yu and Rabi Mahapatra
Publisher/Conf.: Parallel and Distributed Processing, 2008. IPDPS 2008. IEEE International Symposium on
Speaker: Han-Jhen Guo
Date: 2009.05.20
1
![Page 2: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/2.jpg)
OUTLINE
The Proposed Scheme Concept Insertion Query Modification
Deletion Insertion
Performance Simulation for IP Lookup Time Complexity Memory Consumption
2
![Page 3: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/3.jpg)
THE PROPOSED SCHEME- CONCEPT
Basic configuration of hierarchical indexing tree of 0- and 1-tree
3
![Page 4: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/4.jpg)
THE PROPOSED SCHEME- CONCEPT
Hierarchically Indexed Hash Table (HIHT) Assumption
hash functions are perfectly random n keys → s = log2(n) layers (SRAM modules)layers (SRAM modules) at same level i, each BF use mi-bit vector and ki hash
functions The BFs in the hierarchical indexing tree (HIT)
used for indexes to a key table is hierarchically partitioned to make indexes
4
![Page 5: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/5.jpg)
THE PROPOSED SCHEME- INSERTION
(eg. insert key = 011, rule = E, addr(key)addr(key) = = 44)
5
1-3. set all k1 m1-bit vectors with hi(key) in parallel (0 <= i <= k1-1)
1-1. base addr = 100 * m1 * k1
key table
rule table
![Page 6: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/6.jpg)
THE PROPOSED SCHEME- INSERTION
(eg. insert key = 011, rule = E, addr(key)addr(key) = = 44)
6
2-3. set all k2 m2-bit vectors with hi(key) in parallel (0 <= i <= k2-1)
key table
rule table
2-1. base addr = 100 * m2 * k2
![Page 7: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/7.jpg)
THE PROPOSED SCHEME- INSERTION
(eg. insert key = 011, rule = E, addr(key)addr(key) = = 44)
7
3-3. set all k3 m3-bit vectors with hi(key) in parallel (0 <= i <= k3-1)
key table
rule table
3-1. base addr = 100 * m3 * k3
![Page 8: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/8.jpg)
THE PROPOSED SCHEME- INSERTION
(eg. insert key = 011, rule = E, addr(key)addr(key) = = 44)
8
011
E
key table
rule table
![Page 9: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/9.jpg)
THE PROPOSED SCHEME- QUERY
No f-positive (eg. search key = 01101011)
9A B C D E F G H
key table
rule table
000 001 010 110 011 100 101 111
calculate hi(key) in parallel (0 <= i <= k1-1 )
count 1’s = k1 → hit;A = 1
![Page 10: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/10.jpg)
THE PROPOSED SCHEME- QUERY
No f-positive (eg. search key = 01101011)
10
000 001 010 110 011 100 101 111
A B C D E F G H
key table
rule table
calculate hi(key) in parallel (0 <= i <= k2-1 )
count 1’s = k2 → hit;A = 10
![Page 11: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/11.jpg)
THE PROPOSED SCHEME- QUERY
No f-positive (eg. search key = 01101011)
11
000 001 010 110 011 100 101 111
A B C D E F G H
key table
rule table
calculate hi(key) in parallel (0 <= i <= k3-1 )
count 1’s = k3 → hit;A = 100
![Page 12: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/12.jpg)
THE PROPOSED SCHEME- QUERY
No f-positive (eg. search key = 01101011)
12
000 001 010 110 011 100 101 111
A B C D E F G H
key table
rule table
A = 100
key is matched
![Page 13: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/13.jpg)
THE PROPOSED SCHEME- QUERY
No f-positive (eg. search key = 01101011)
13
000 001 010 110 011 100 101 111
A B C D E F G H
key table
rule table
fetch relative rule
![Page 14: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/14.jpg)
THE PROPOSED SCHEME- QUERY
F-positive (eg. search key = 01101011)
14A B C D E F G H
key table
rule table
000 001 010 110 011 100 101 111
![Page 15: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/15.jpg)
THE PROPOSED SCHEME- QUERY
No f-positive (eg. search key = 01101011)
15
000 001 010 110 011 100 101 111
A B C D E F G H
key table
rule table
A = 100
key is matched
A = 010
key is unmatched
![Page 16: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/16.jpg)
THE PROPOSED SCHEME- QUERY
No f-positive (eg. search key = 01101011)
16
000 001 010 110 011 100 101 111
A B C D E F G H
key table
rule table
fetch relative rule
![Page 17: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/17.jpg)
THE PROPOSED SCHEME- MODIFICATION
Dual HITs valid bit array (VBA)
on-chip; indicate whether the key exists in the key table
next on-chip; indicate the address for inserting next new
key free address stack (FAS)
off-chip; store addresses of empty spaces
17
![Page 18: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/18.jpg)
THE PROPOSED SCHEME- DELETION
No f-positive (eg. delete key = 011)
18
000
001
010
110
011
100
101
111
A B C D E F G H
FAS
1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0
next
0 VBA
100
100
![Page 19: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/19.jpg)
THE PROPOSED SCHEME- DELETION
F-positive (eg. delete key = 001)
19
000
001
010
110
100
101
111
A B C D F G H
100
FAS
1 1 1 1 0 1 1 1 0 0 0 0 0 0 0 0
100
next
0 VBA
001
001
f-positive is checkedf-positive is checked by unmatched key
![Page 20: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/20.jpg)
THE PROPOSED SCHEME- INSERTION
(eg. insert key = 101, r = G)
20
000
010
110
100
111
A C D F H
001
100
FAS
1 0 1 1 0 1 0 1 0 0 0 0 0 0 0 0
110
next
VBA1
101
G
001
![Page 21: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/21.jpg)
Performance- Simulation for IP Lookup
Architecture
21
![Page 22: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/22.jpg)
PERFORMANCE- SIMULATION FOR IP LOOKUP
Related work - Prefix collapsing (eg. collapse stride = 3)
22
![Page 23: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/23.jpg)
PERFORMANCE- TIME COMPLEXITY
Average access time to a table or a linked list as a function of SS rate
23
![Page 24: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/24.jpg)
PERFORMANCE- MEMORY COMSUMPTION
On-chip memory size with various hash schemes for 6 BGP tables in 40Gbps and 160Gbps routers
24
table# of
prefixes
B1AS650
0023345
1
B2AS644
723530
7
B3AS122
118129
5
B4AS126
5417045
9
B5AS545
978133
B6AS330
368739
![Page 25: A SPACE- AND TIME- E ffi CIENT HASH TABLE HIERARCHICALLY INDEXED BY BLOOM FILTERS Author: Heeyeol Yu and Rabi Mahapatra Publisher/Conf.: Parallel and Distributed](https://reader036.vdocuments.net/reader036/viewer/2022062803/56649f575503460f94c7caa7/html5/thumbnails/25.jpg)
25