processing xml view queries including user-defined foreign functions on relational databases

Dec. 13, 2002Dec. 13, 2002 WISE2002WISE2002

Processing XML View Queries Processing XML View Queries Including User-defined Foreign Including User-defined Foreign

Functions on Relational DatabasesFunctions on Relational Databases

Yoshiharu Ishikawa Yoshiharu Ishikawa Jun KawadaJun Kawada

Hiroyuki KitagawaHiroyuki KitagawaUniversity of TsukubaUniversity of Tsukuba

{ishikawa,kitagawa}@is.tsukuba.ac.jp{ishikawa,kitagawa}@is.tsukuba.ac.jp

Presentation OverviewPresentation Overview

BackgroundBackground XML ViewsXML Views Support for foreign functionsSupport for foreign functions

Overview of Our ApproachOverview of Our Approach The XPERANTO ApproachThe XPERANTO Approach Extension to the XPERANTO ApproachExtension to the XPERANTO Approach Query Processing ArchitectureQuery Processing Architecture Experimental EvaluationExperimental Evaluation Conclusions and Future WorkConclusions and Future Work

XML ViewsXML Views XML: content-description language on the WeXML: content-description language on the We

bb XML Views over RDBsXML Views over RDBs

Constructing Constructing virtual XML viewsvirtual XML views over RDBs over RDBs Data items are stored in RDBsData items are stored in RDBs Selecting and transforming data items into appropriate Selecting and transforming data items into appropriate

XML formatsXML formats XML views are constructed using middleware techXML views are constructed using middleware tech

nologiesnologies Effective use of the data management and query Effective use of the data management and query

processing facilities of the underlying RDBMSsprocessing facilities of the underlying RDBMSs XPERANTO (IBM) [3,7,8], SilkRoute (AT&T) [4,5]XPERANTO (IBM) [3,7,8], SilkRoute (AT&T) [4,5]

Virtual XML Views and MiddlewareVirtual XML Views and Middleware

Middleware provides Middleware provides virtual XML virtual XML view view facilityfacility

A user query is specified by an A user query is specified by an XXML query languageML query language (e.g., XQuer (e.g., XQuery) toward an XML viewy) toward an XML view

Middleware creates a query plan Middleware creates a query plan and issues an and issues an SQL query SQL query to RDBto RDBMSMS

Middleware transforms an SQL Middleware transforms an SQL query result into the final XML forquery result into the final XML formatmat adds XML tagsadds XML tags may perform remaining query tasksmay perform remaining query tasks

Middleware

RDBMS

VirtualXMLView

UserQuery

QueryResult

SQLQuery

SQL QueryResult

Example of Relational DatabaseExample of Relational Databasecitycity

cidcid cnamecname populationpopulation

C0100C0100 AA 1616

C0101C0101 BB 1313

C0103C0103 DD 44

...... ...... ......

locationlocation

cidcid xx yy

C0100C0100 100100 400400

C0100C0100 100100 200200

...... ...... ......

C0101C0101 110110 250250

C0101C0101 150150 200200

...... ...... ......

facilityfacility

fidfid fnamefname cidcid

I0015I0015 E MallE Mall C0100C0100

I0016I0016 F ParkF Park C0100C0100

I0017I0017 G StudiumG Studium C0100C0100

I0018I0018 H LibraryH Library C0101C0101

I0019I0019 I MallI Mall C0102C0102

I0020I0020 J ParkJ Park C0103C0103

...... ...... ......

Fig. 2Fig. 2

Example of XML ViewExample of XML View<cities><cities> <city id="C0100"><city id="C0100"> <cname>A</cname><cname>A</cname> <population>16</population><population>16</population> <location><location> <coord><x>100</x><y>400</y></coord><coord><x>100</x><y>400</y></coord> <coord><x>100</x><y>200</y></coord><coord><x>100</x><y>200</y></coord> ...... </location></location> <facilities><facilities> <facility id="I0015"><fname>E Mall</fname></facility><facility id="I0015"><fname>E Mall</fname></facility> <facility id="I0016"><fname>F Park</fname></facility><facility id="I0016"><fname>F Park</fname></facility> ...... </facilities></facilities> </city></city> ......</cities></cities> Fig. 1Fig. 1


BackgroundBackground XML ViewsXML Views Support for foreign functionsSupport for foreign functions

Overview of Our ApproachOverview of Our Approach The XPERANTO ApproachThe XPERANTO Approach Extension to the XPERANTO ApproachExtension to the XPERANTO Approach Query Processing ArchitectureQuery Processing Architecture Experimental EvaluationExperimental Evaluation Conclusions and Future WorkConclusions and Future Work

User Query with Foreign FunctionUser Query with Foreign Function

User query may contain User query may contain foreign functionsforeign functions Example in XQueryExample in XQuery

<result> {<result> { for $city in view("cities")/cities/cityfor $city in view("cities")/cities/city where where isWider($city/location, 10000, "km")isWider($city/location, 10000, "km") and $city/population >= 10and $city/population >= 10 return <city-info>return <city-info> $city/cname$city/cname $city/facility$city/facility </city-info></city-info>} </result>} </result> Fig. 3Fig. 3

User-defined Foreign FunctionsUser-defined Foreign Functions Usually coded with a geneUsually coded with a gene

ral-purpose programming lral-purpose programming language (e.g., Java)anguage (e.g., Java)

Receive in-memory represReceive in-memory representation of entation of target XML doctarget XML document fragmentsument fragments (e.g., DO (e.g., DOM)M)

Middleware should evaluatMiddleware should evaluate foreign functions: e foreign functions: since csince conventional RDBMSs do nonventional RDBMSs do not support such facilitiesot support such facilities

Middleware

<location><location> <coord><x>100</x><coord><x>100</x> <y>400</y></coord><y>400</y></coord> <coord><x>100</x><coord><x>100</x> <y>200</y></coord><y>200</y></coord> ......</location></location>

ForeignForeignFunctionFunction

XML FragmentXML Fragment


BackgroundBackground Overview of Our ApproachOverview of Our Approach The XPERANTO ApproachThe XPERANTO Approach Extension to the XPERANTO ApproachExtension to the XPERANTO Approach Query Processing ArchitectureQuery Processing Architecture Experimental EvaluationExperimental Evaluation Conclusions and Future WorkConclusions and Future Work

Our ApproachOur Approach

Processing Processing XML view queries including XML view queries including foreign functionsforeign functions By the cooperation of a conventional RDBMS By the cooperation of a conventional RDBMS

and a middleware systemand a middleware system Extension of the XPERANTO frameworkExtension of the XPERANTO framework

Proposal of Proposal of two query processing methodstwo query processing methods Two-step processing methodTwo-step processing method One-step processing methodOne-step processing method

Middleware

Two-Step Processing MethodTwo-Step Processing Method

QueryPlanning

Query Execution

Foreign FunctionForeign FunctionEvaluationEvaluation

Result XMLResult XML GenerationGeneration

RDBMS

SQLQuery

SQL QueryResult

SQLQuery

UserQuery

QueryResult

1st query1st query - to evaluate- to evaluate foreign functionsforeign functions2nd query2nd query - to generate the result- to generate the result

SQL QueryResult

Middleware

One-Step Processing MethodOne-Step Processing Method

QueryPlanning

Query Execution

Foreign FunctionForeign FunctionEvaluationEvaluation

Result XMLResult XML GenerationGeneration

RDBMS

SQLQuery

SQL QueryResult

UserQuery

QueryResult

combined querycombined query - to evaluate- to evaluate foreign functionsforeign functions - to generate the result- to generate the result

XML Views in XPERANTOXML Views in XPERANTO

A A default XML viewdefault XML view is a is automatically created froutomatically created from the underlying relatiom the underlying relational tablesnal tables

An An XML viewXML view is defined is defined over the default XML viover the default XML view using XQueryew using XQuery

Relational Tables

Default XML ViewDefault XML View<db>...</db><db>...</db>

View DefinitionXQuery


virtual virtual XML ViewXML View

automatic derivationautomatic derivation

virtualvirtualXML viewXML view

Default XML ViewDefault XML View AA default XML view default XML view is a is a

utomatically created froutomatically created from the underlying relatiom the underlying relational tablesnal tables

Each Each <row> element<row> element c corresponds to a relatioorresponds to a relational tuplenal tuple

<db><db> <city><city> <row><row> <cid>C0100</cid><cid>C0100</cid> <cname>A</cname><cname>A</cname> <population>16</population><population>16</population> </row></row> ...... </city></city> <location><location> <row><row> <cid>C0100</cid><cid>C0100</cid> <x>100</x><x>100</x> <y>400</y><y>400</y> </row></row> ...... </location></location> ......Fig. 6Fig. 6

corresponds tocorresponds toa city tuplea city tuple

XML View DefinitionXML View Definitioncreate view cities as {create view cities as { <cities><cities> for $city in view("default")/city/rowfor $city in view("default")/city/row returnreturn <city id=$city/cid><city id=$city/cid> <cname>$city/cname</cname><cname>$city/cname</cname> <population>$city/population</population><population>$city/population</population> <locations><locations> for $location in view("default")location/rowfor $location in view("default")location/row where $city/cid = $location/$cidwhere $city/cid = $location/$cid return <coord>return <coord> <x>$location/x</x><y>$location/y</y><x>$location/x</x><y>$location/y</y> </coord></coord> </locations></locations> ...... </city></city> </cities></cities>}}Fig. 7Fig. 7

An An XML viewXML view i is defined over s defined over the default XMthe default XML view using XL view using XQueryQuery

This view definThis view definition creates thition creates the XML view she XML view shown in Fig. 1own in Fig. 1

Query Processing in XPERANTOQuery Processing in XPERANTO


virtual virtual XML ViewXML View

Default XML ViewDefault XML View<db>...</db><db>...</db>

UserQuery

User QueryXQGM Graph

View DefinitionXQGM Graph

ViewViewCompositionComposition

ComposedXQGM Graph

XQGMGraph

TransformationTransformation- Computation PushdownComputation Pushdown- Tagger Pull-UpTagger Pull-Up

SQLSQLQueryQuery

XML TagXML TagOperatorsOperators

GenerationGeneration

View Definition XQGM GraphView Definition XQGM Graph Each node Each node

corresponds to an corresponds to an extended relational extended relational operator (shown in operator (shown in Table 1Table 1))

Contains a Contains a correlated correlated joinjoin operator operator

Node 11 is an Node 11 is an abbreviated abbreviated representation of node representation of node 1212 Node 12 contains tag Node 12 contains tag

operators shown in operators shown in Table 2Table 2

Fig. 8Fig. 8

User Query XQGM GraphUser Query XQGM Graph

Fig. 9Fig. 9

Some Problems and Our ExtensionSome Problems and Our Extension

The original XPERANTO approach [7] has some prThe original XPERANTO approach [7] has some problemsoblems It shows only simple transformation examplesIt shows only simple transformation examples It only considersIt only considers

Computation pushdown processing for top-level relationsComputation pushdown processing for top-level relations Simple XQuery query: selection query with one where clauseSimple XQuery query: selection query with one where clause Simple where clause with one conditionSimple where clause with one condition Simple return clauseSimple return clause

It does not consider It does not consider foreign functionsforeign functions Our extension to the XPERANTO approachOur extension to the XPERANTO approach

Devised user query graph generation (Fig. 9)Devised user query graph generation (Fig. 9) Incorporation of the intersection operator for multiple conIncorporation of the intersection operator for multiple con

ditions in a where clauseditions in a where clause Query translation that considers our extensionQuery translation that considers our extension

Query TranslationQuery Translation

Our extension to the XPERANTO approachOur extension to the XPERANTO approach Inclusion of foreign functions in where clauseInclusion of foreign functions in where clause Handling output specifications in return clauseHandling output specifications in return clause Treatment of multiple conditions in where clauseTreatment of multiple conditions in where clause Treatment of computation pushdown to subrelationsTreatment of computation pushdown to subrelations

Query translation consists of the following stepsQuery translation consists of the following steps DecorrelationDecorrelation View compositionView composition Computation pushdownComputation pushdown Tagger pull-upTagger pull-up

DecorrelationDecorrelation of View Definition of View Definition XQGM GraphXQGM Graph

A correlated join operA correlated join operator has high executioator has high execution costn cost

Decorrelation step eliDecorrelation step eliminates correlated joiminates correlated join operatorsn operators

View definition XQGM View definition XQGM graph (Fig. 8) is translgraph (Fig. 8) is translated as Fig. 10ated as Fig. 10

Fig. 10Fig. 10

Decorrelation of User Query Decorrelation of User Query XQGM GraphXQGM Graph

User query User query XQGM graph XQGM graph (Fig. 9) is also (Fig. 9) is also translated as Fig. translated as Fig. 1111

Fig. 11Fig. 11

View CompositionView Composition Compose a view Compose a view

definition XQGM definition XQGM graph and a user graph and a user query XQGM graph, query XQGM graph, then apply function then apply function composition rules in composition rules in Table 3Table 3

This step is almost This step is almost same as the original same as the original XPERANTO approachXPERANTO approach

Composition of Fig. 10 Composition of Fig. 10 and 11 yields Fig. 12and 11 yields Fig. 12

Fig. 12Fig. 12

Computation PushdownComputation Pushdown

Pushdown XQGM operators towards the leaves of Pushdown XQGM operators towards the leaves of the graph as much as possiblethe graph as much as possible For the efficient evaluation using the query processing For the efficient evaluation using the query processing

power of RDBMSpower of RDBMS However, we However, we cannot pushdown foreign function cannot pushdown foreign function

evaluationevaluation Foreign function evaluation is performed in the Foreign function evaluation is performed in the

middlewaremiddleware Evaluation in the middleware requires XML fragmentsEvaluation in the middleware requires XML fragments

Therefore, pushdown computation Therefore, pushdown computation except forexcept for foreign function evaluationsforeign function evaluations

Tagger Pull-UpTagger Pull-Up

Replace XML functions Replace XML functions with tag operatorswith tag operators

Pull-up tag operators Pull-up tag operators upward as much as upward as much as possiblepossible

Two SQL queries are Two SQL queries are generatedgenerated SQL-1SQL-1 from from wherewhere

clauseclause SQL-2SQL-2 from from return return

clauseclause

Fig. 13Fig. 13

Two-Step Processing Method (1)Two-Step Processing Method (1)

Middleware

QueryPlanning

Query Execution

Foreign FunctionForeign FunctionEvaluatorEvaluator

TaggerTagger

RDBMS

Result XML

SQL QuerySQL Query ControlControl

Tagger-1, 2Tagger-1, 2

SQL-1, 2SQL-1, 2

Tuple-1, 2Tuple-1, 2

FragmentFragment

KeysKeys

User Query

SQL-1SQL-1, , Tagger-1 Tagger-1 is used tois used toretrieve tuples to evaluateretrieve tuples to evaluateforeign functions (foreign functions (Tuple-1Tuple-1))

The qualified key value set The qualified key value set KeysKeysis combined with is combined with SQL-2SQL-2 to select to select the result tuples the result tuples

Two-Step Processing Method (2)Two-Step Processing Method (2)

Two approaches for the generation of SQL-2Two approaches for the generation of SQL-2 Two-Step Processing Method (where)Two-Step Processing Method (where)

The qualified key value set Keys (obtained by foreign fThe qualified key value set Keys (obtained by foreign function evaluation) is unction evaluation) is embedded into the whereembedded into the where clausclause of SQL-2 (e.g., e of SQL-2 (e.g., "where fid in Keys""where fid in Keys"))

Two-Step Processing Method (tmp)Two-Step Processing Method (tmp) First, First, one-column temporary tableone-column temporary table is created from the is created from the

key values in Keyskey values in Keys Then a Then a join operationjoin operation with the temporary table is incor with the temporary table is incor

porated in SQL-2porated in SQL-2

One-Step Processing MethodOne-Step Processing Method

Middleware

QueryPlanning

Query Execution

Foreign FunctionForeign FunctionEvaluatorEvaluator

TaggerTagger

RDBMS

Result XML

SQL QuerySQL Query ControlControl

Tagger-1, 2Tagger-1, 2

SQL-1, 2SQL-1, 2

TupleTuple

FragmentFragment

KeysKeys

UserQuery

SQL-1SQL-1 and and SQL-2 SQL-2 areareintegrated in one SQLintegrated in one SQLquery (query (SQL-3SQL-3))

The middleware selects tuples ofThe middleware selects tuples ofthe final result using the qualifiedthe final result using the qualifiedkey set key set KeysKeys

Outline of Experiments (1)Outline of Experiments (1) PostgreSQL on Linux PCPostgreSQL on Linux PC Relational TablesRelational Tables

No. of No. of citycity tuples: N = 1000 tuples: N = 1000 No. of No. of locationlocation tuples: 10N and 100N tuples: 10N and 100N No. of No. of facilityfacility tuples: 10N and 100N tuples: 10N and 100N

Four Types of Four Types of QueriesQueries Q1Q1: For each city whose area is larger than X, show its nam: For each city whose area is larger than X, show its nam

e and facilitiese and facilities Q2Q2: For each city whose area is larger than X, show its nam: For each city whose area is larger than X, show its nam

e, location information, and facilitiese, location information, and facilities Q3Q3: For each city whose area is larger than X and whose po: For each city whose area is larger than X and whose po

pulation is larger than Y, show its name and facilities (Q1 + apulation is larger than Y, show its name and facilities (Q1 + additional selection condition)dditional selection condition)

Q4Q4: Q3 + additional selection condition: Q3 + additional selection condition

Outline of Experiments (2)Outline of Experiments (2)

Selectivity factorsSelectivity factors Area condition: SArea condition: Saa = 0.1, 0.3, 0.5, 0.7, and 0.9 = 0.1, 0.3, 0.5, 0.7, and 0.9 Population condition: SPopulation condition: Spp = 0.1 and 0.3 = 0.1 and 0.3

Processing costsProcessing costs Foreign function evaluation and XML generation Foreign function evaluation and XML generation

are relatively small and almost equally included in are relatively small and almost equally included in both methods both methods

Cost of two-step processing methodCost of two-step processing method: processing : processing cost of SQL-1 and SQL-2cost of SQL-1 and SQL-2

Cost of one-step processing methodCost of one-step processing method: processing : processing cost of SQL-3 (SQL-1 + SQL-2)cost of SQL-3 (SQL-1 + SQL-2)

Q3 with SQ3 with Sp p = 0.3= 0.3

(no. of facility tuples = 10N) (no. of facility tuples = 10N) Q3: For each city Q3: For each city

whose area is largwhose area is larger than X and whoer than X and whose population is larse population is larger than Y, show itger than Y, show its name and facilitis name and facilitieses

No. of location tuplNo. of location tuples = 10N or 100Nes = 10N or 100N

Three methods haThree methods have similar costsve similar costs

Fig. 20Fig. 20

Q3 with SQ3 with Sp p = 0.3= 0.3

(no. of facility tuples = (no. of facility tuples = 100N100N)) Two-step methods Two-step methods

are better if selectiare better if selectivity of foreign functvity of foreign function is lowion is low filtering is well-perffiltering is well-perf

ormedormed Two-step method Two-step method

(where) is worse t(where) is worse than (tmp)han (tmp) embedding of key embedding of key

values is not efficievalues is not efficientnt

Fig. 21Fig. 21

Q3 with SQ3 with Sp p = = 0.10.1

(no. of facility tuples = 10N) (no. of facility tuples = 10N) Selectivity of the pSelectivity of the p

opulation attribute opulation attribute is small pushdo⇒is small pushdo⇒wn to RDBMS is cwn to RDBMS is can reduce the no. an reduce the no. of tuples of tuples

One-step method iOne-step method is betters better two-step methods two-step methods

have overheadshave overheads

Fig. 22Fig. 22

Q4Q4 with S with Sp p = 0.3= 0.3

(no. of facility tuples = 10N) (no. of facility tuples = 10N) Q4: For each city whoQ4: For each city who

se area is larger than se area is larger than X and whose populatiX and whose population is larger than Y, shon is larger than Y, show its name, ow its name, location ilocation informationnformation, and faciliti, and facilitieses

If no. of location tupleIf no. of location tuples is large (100N) and is is large (100N) and if the selectivity of foref the selectivity of foreign function is small, tign function is small, two-step method (tmp) wo-step method (tmp) is betteris better Fig. 23Fig. 23

Summary of ExperimentsSummary of Experiments Two-step processing method (where) is Two-step processing method (where) is worseworse than t than t

wo-step processing method (tmp) in most situationswo-step processing method (tmp) in most situations key value embedding is not a good ideakey value embedding is not a good idea

The cost of one-step processing The cost of one-step processing does not dependdoes not depend on on the selectivity of foreign functionthe selectivity of foreign function

If a query only contains a foreign function condition If a query only contains a foreign function condition (Q1 and Q2), two-step processing method (tmp) is g(Q1 and Q2), two-step processing method (tmp) is generally efficient when the selectivity of foreign functienerally efficient when the selectivity of foreign function is smallon is small

If a query contains additional conditions (Q3 and QIf a query contains additional conditions (Q3 and Q4), the efficiency is depend on the selectivity factors4), the efficiency is depend on the selectivity factors

If the processing cost of RDBMS is small, one-step pIf the processing cost of RDBMS is small, one-step processing method is efficientrocessing method is efficient

Guideline of UsageGuideline of Usage

Do not use two-step method (where)Do not use two-step method (where) If the processing cost in the RDBMS is quite If the processing cost in the RDBMS is quite

small, use one-step methodsmall, use one-step method If the query only contains foreign functions, uIf the query only contains foreign functions, u

se two-step method (tmp)se two-step method (tmp) If the query contains additional filtering condiIf the query contains additional filtering condi

tionstions we have to select an appropriate one from one-swe have to select an appropriate one from one-s

tep method and two-step method (tmp)tep method and two-step method (tmp) the selection is depend on the selectivity factors the selection is depend on the selectivity factors

ConclusionsConclusions Processing methods for XML view over relatiProcessing methods for XML view over relati

onal databases especially when queries inclonal databases especially when queries include foreign function callsude foreign function calls

Cooperation approach of middleware and RCooperation approach of middleware and RDBMSDBMS

Extension of XPERANTO frameworkExtension of XPERANTO framework Proposal of two query processing methodsProposal of two query processing methods

Two-step processing method (where/tmp)Two-step processing method (where/tmp) One-step processing methodOne-step processing method

Performance evaluation based on experimenPerformance evaluation based on experimentsts

Future WorkFuture Work

Broadening the supportable XML queriesBroadening the supportable XML queries Query optimizationQuery optimization

reduction redundancyreduction redundancy Development of other query processing Development of other query processing

approachesapproaches bitmap indexesbitmap indexes with clause in SQL:1999with clause in SQL:1999

Selection of an appropriate query processing Selection of an appropriate query processing methodmethod development of selection heuristicsdevelopment of selection heuristics

Thank youThank you

Q1Q1 (no. of facility tuples = 100N) (no. of facility tuples = 100N)

Q1: No filtering Q1: No filtering condition on condition on population population attributeattribute

Fig. 26Fig. 26

processing xml view queries including user-defined foreign functions on relational databases

Documents