Keyword Search on External Memory Data Graphs
Bhavana Bharat Dalvi, Meghana Kshirsagar, S. SudarshanPVLDB 2008
Reported by: Yiqi Lu
Background: Graph Model Direct graph model for data
Background: Answer Tree Model Answer Tree
Keyword Query
Background: score function A function of the node score and edge score of
answer tree Several score models have been proposed.
Background: keyword search Input: keywords, data graph Output: top-k answer trees Algorithm:
first looking up an inverted keyword index to get the node-
ids of nodes Second
a graph search algorithm is run to find out trees connecting the keyword nodes found above. The algorithm finds rooted answer trees, which should be generated in ranked order.
Example: backward expanding search For each keyword term ki
First find the set of nodes Si that contain keyword ki
Run Dijkstra SP algorithm which provides an interface to incrementally retrieve the next nearest node Traverses the graph To find a common vertex from which a forward path
exists to at least one node in each set Si Then the answer tree’s root is the common vertex and
the keywords are leaves
Background: external memory search Run search algorithm on an external memory
graph representation which clusters nodes into disk pages
Naïve migration will lead to poor performance keyword search algorithms designed for in-
memory search access a lot of nodes, and such node accesses lead to a lot of expensive random IO when data is disk resident.
Background： 2-level graph
Clustering parameters are chosen such that supernode graph fits into the available amount of memory
Background: 2-phase search algorithm
This algorithm lack consideration of time locality
multi-granular graph structure This paper proposes a multi-granular graph
structure to exploit information present in lower-level nodes that are cache-resident at the time a query is executed
MG graph a hybrid graph
A supernode is present either in expanded form (all its innernodes along with their adjacency lists are present in the cache)
Or unexpanded form (its innernodes are not in the cache)
several types of edges
several types of edges
•Supernode answer•Pure answer
ITERATIVE EXPANSION SEARCH Explore phase: Run an in-memory search
algorithm on the current state of the multi-granular graph (the multi-granular graph is entirely in memory)
Expand phase: Expand the supernodes found in top-n results of the (a) and add them to input graph to produce an expanded multi-granular graph
ITERATIVE EXPANSION SEARCH
ITERATIVE EXPANSION SEARCH the stopping criterion：
The algorithm stops at the iteration where all top-k results are pure.
node-budget heuristic: Stop search when
ITERATIVE EXPANSION SEARCH A assumption: the part of graph relevant to
the query fits in cache May fail in some cases
Query has many keywords or algorithm explores a large number of nodes
Have to evict some supernodes from the cache based on a cache replacement policy some parts of the multi-granular graph may shrink
after an iteration Such shrinkage can unfortunately cause a problem
of cycles in evaluation
ITERATIVE EXPANSION SEARCH do not shrink the logical multi-granular graph,
but instead provide a “virtual memory view” of an ever-expanding multi-granular graph. maintain a list, Top-n-SupernodeList, of all
supernodes found in the top-n results of all previous iterations.
Any node present in Top-n-SupernodeList but not in cache is transparently read into cache whenever it is accessed.
INCREMENTAL EXPANSION SEARCH Iterative Expansion algorithm restart
search when supernodes are expanded This can lead to significantly increased CPU time
Incremental expansion algorithm updates the state of the search algorithm
Take BES as example
Heuristics to improve performance stop-expansion-on-full-cache
Intra-supernode-weight heuristic We define the intra-supernode weight of a
supernode as the average of all innernode → innernode edges within that supernode.
Experiment Search Algorithms Compared
Iterative Expanding search Incremental Expanding (Backward) Search with
different heuristics the in-memory Backward Expanding search the Sparse algorithm from “Efficient IR-Style
keyword search in relational databases” A naive approach to external memory search
would be to run in-memory algorithms in virtual memory.
we have implemented this approach on the supernode graph infrastructure, treating each supernode as a page
Data sets DBLP 2003 IMDB 2003 Cluster using EBFS technique Default supernode size is 100 innernodes
corresponding to an average of 7KB on DBLP and 6.8KB on IMDB
Supernode contents were stored sequentially in a single file, with an index for random access within the file to retrieve a specified supernode.
Data sets
Clustering result
Cache Management 3GB RAM, and a 2.4GHz Intel Core 2
processor, and ran Fedora Core 6 All results are taken on a cold cache.
Force linux kernel to drop page cache, inode cache and dentry cache
By excuting sync(flush dirty pages back to disk) then excuting echo 3 > /proc/sys/vm/drop_caches
Queries
Experimental Results first implemented Incremental search without
any of the heuristics did not perform well, and gave poor results, taking
unreasonably long times for many queries. results for this case not presented
two versions of Incremental expansion, one with and one without the intra-supernode-weight heuristic
the intra-supernode-weight heuristic reduces the number of cache misses drasti cally without significantly reducing answer quality.
Comparison With Alternatives
