The Design and Architecture of theτ -Synopses System

Yossi MatiasSchool of Computer Science

Tel Aviv Universitymatias@cs.tau.ac.il

Leon PortmanSchool of Computer Science

Tel Aviv Universityleonpo@cs.tau.ac.il

Natasha DrukhSchool of Computer Science

Tel Aviv Universitykreimern@cs.tau.ac.il

Abstract

Data synopses are concise representations of data sets,that enable effective processing of approximate queries tothe data sets. Approximate query processing provides im-portant alternatives when exact query answers are not re-quired. τ -Synopses is a system designed to provide a run-time environment for remote execution of various synopsesfor both relational as well as XML databases. It enableseasy registration of new synopses from remote platforms,after which the system can manage these synopses, includ-ing triggering their construction, rebuild and update, andinvoking them for approximate query processing. The sys-tem captures and analyzes query workloads, enabling itsregistered synopses to significantly boost their effectiveness(efficiency, accuracy, confidence), by exploiting workloadinformation for synopses construction and update. The sys-tem can also serve as a research platform for experimentalevaluation and comparison of different synopses.

1 Introduction

In large data recording and warehousing environments,it is often advantageous to provide fast, approximate an-swers to queries, whenever possible. The goal is to providea quick response in orders of magnitude less time than thetime to compute an exact answer, by avoiding or minimiz-ing the number of accesses to the base data.

Techniques for fast approximate answers can also beused in a more traditional role within a query optimizer toestimate plan costs, again with very fast response time.

Approximate query processing is supported by synopsesthat are compact representation of the original data, such ashistograms, splines, sampling, wavelets or other methods.

Increased interest in approximate query processing re-sulted with proliferation of new synopses addressing newproblems as well as proposed alternatives to previouslysuggested synopses. In particular, synopses are becom-ing more advanced, supporting updates to data, aware-

ness to workload, and adaptive to changes in workload(e.g., [2, 7, 8, 9, 12, 13, 20]).

Several systems and projects address approximate queryprocessing and data synopses. In the AQUA Project [14,4, 3], synopses are precomputed and stored in a DBMS. Itprovides approximate answers by rewriting the queries torun on these synopses, and enables keeping synopses up-to-date as the database changes. Various aspects of approxi-mate query processing were studied by Microsoft ResearchDB group (e.g., [7, 9]). The question of how to reconcilevarious synopses for large information sources with manytables was studied in [16, 15].

While the above research works deal mostly with rela-tional database systems, we see increasing interest towardsXML synopses. The extensible mark-up language (XML)is becoming ubiquitous as a data exchange and storage for-mat. Almost all commercial RDBMSs include support forXML data; native XML databases such as Xyleme [26] arespecially designed to store and query XML data on the web.Efficient query processing over XML data requires accu-rate estimation of the selectivities of the path expressionscontained in the query. Thus, maintaining concise synopsesstructures is crucial for the XML databases as well as for thetraditional relational databases. This problem has recentlyattracted the attention of the database research community,and several techniques [17, 21, 22, 11] have been proposedtargeting different aspects of the problem.

Operational systems require the management and con-solidation of many synopses. These include various syn-opses addressing different types of queries, each requiringits own type of synopsis; furthermore, even for the sametype of query, many different instances of the same synop-sis should be used to represent different data sets (e.g., dif-ferent relations, or different columns of the same relation).Finally, in some cases it would be beneficial to hold morethan one synopsis for the same type of queries, to benefitfrom the different properties of the various synopses (e.g.,having an answer based on all supporting synopses, lettingeach synopsis support a particular subset of the queries etc).

For research purposes, it would also be beneficial to sup-

1

port many synopses. Indeed, for each query there may bedifferent possible types of synopses that could be useful.Furthermore, for each particular synopsis, it would be use-ful to compare different implementations for evaluation andbenchmarking purposes. Therefore, it is advantageous tohave a system that can accommodatemultiple synopses, andhave an easy way to integrate new synopses and managethem.

The multiple synopses in use in either operational or re-search system could be placed in remote locations for var-ious reasons: they may be implemented on different typesof platforms, they may be summarizing remote data whosetransfer is undesirable or impossible due to performance orsecurity constraints, and it would be beneficial to share theload of operating a large number of synopses using differentsystems for load balancing and redundancy reasons. There-fore, it would be beneficial to have a system supportremoteexecutionof registered synopses.

Synopses are more effective when they are adaptive forpredicted workload, changes in workload, changes in data,and changes in performance requirements (query time, ac-curacy, confidence, and available memory). This motivatesthe use of a system ofmanaged synopses. This is a sin-gle system that registers all synopses, triggers their con-struction and maintenance (in response to new data, datachanges, predicted changed workload, or by demand). Hav-ing a managed synopses system enables providing informa-tion required by all relevant synopses from a single reposi-tory (e.g., data, workload, changes in data and workload).

1.1 τ -Synopses framework overview

Motivated by the above, theτ -Synopses system was de-signed to provide a run-time environment for remote execu-tion of various synopses. It enables easy registration of newsynopses from remoteSOAP-enabled platforms, after whichthe system can manage these synopses, including triggeringtheir construction, rebuild and update, and invoking themfor approximate query processing. The system captures andanalyzes query workloads, enabling its registered synopsesto significantly boost their effectiveness (efficiency, accu-racy, confidence), by exploiting workload information forsynopses construction and update. The system can serve asa research platform for experimental evaluation and com-parison of different synopses.

The τ -Synopses framework software demo was previ-ously presented at the EDBT’04 and ICDE’04 [27]. Sincethen the framework was extended to support XML datasources and manage XML synopses for approximate selec-tivity estimations. This paper, for the first time, reveals thedetailed description, architecture and design details of theτ -Synopses framework.

The τ -Synopses is a stand alone system, and can work

with data sources such as existing relational, xml or otherdatabase systems. It supports two types of users: synopsesproviders who register their synopses within the system, andend-users who send queries to the system. The system ad-ministrator defines available data sources and provides gen-eral administration of the system.

When a new synopsis is registered, the relevant data setand the supported queries are defined. A query submitted tothe system is executed using the appropriate synopsis, basedon the registration and other information. The result is re-turned to the user or optionally processed by other modulesin the system. The system transforms updated data from itsoriginal datasource to be consistent with the format knownto the synopses, so that synopses are not required to supportany data transformation functionality or database connec-tivity logic. Any relational/xml database or even real-timedata providers can be datasources in the system.

Workload information is recorded by the system and be-comes available to the registered workload-sensitive syn-opses.

Theτ -Synopses system has the following key features:

• various data sources: The system supports both rela-tional and XML data sources.

• multiple synopses: The system can accommodate var-ious types of synopses. New synopses can be addedwith their defined functionalities.

• pluggable integration: For integration purposes, a syn-opsis has to implement a simple interface, regardless ofits internal implementation. By utilizing a light-weighthost provided by the system, the synopsis can be exe-cuted on anySOAP-enabled platform.

• remote execution: Synopses can be transparently exe-cuted on remote machines, overTCP/IP or HTTP pro-tocols, within local area networks or over the internet.

• managed synopses: The system allocates resources tosynopses, triggers their construction and maintenance,selects appropriate synopses for execution, and pro-vides all required data to the various synopses.

• workload support: Workload is captured, maintainedand analyzed in a centralized location, and made avail-able to the various synopses for construction and main-tenance.

• research platform: The system provides a single, con-sistent source of data, training and test workload forexperimental comparison and benchmarking, as wellas performance measurements. It can therefore serveas an effective research platform for comparing differ-ent synopses without re-implementing them.

2

The rest of the paper is organized as follows. Section2 supplies the necessary background for query optimiza-tion and approximate queries in both relational and XMLdatabases. Sections 3 reveals in details the functionality oftheτ -Synopses framework, while sections 4-5 focus on thesoftware engineering aspects. We present our experimentalevaluation in Section 6 and draw conclusions in Section 7.

2 Background

This section supplies the necessary background for queryoptimization and approximate queries in both relational andXML databases.

2.1 Relational Databases

Approximate queries. So, what is fast approximate an-swers? In relational databases it used primarily for aggre-gate queries, calculation of which requires access to largepart of the original data. The primary goal is to quicklyreport for example leading digits of answers in second in-stead of hours. For example, a calculation of the averagesalary over database of all people in US will take about 10min, but approximate answers can be provided in 10 sec.The approximate answers can be achieved by answering thequery based on some synopses of data that are orders ofmagnitude smaller than the original data.

Synopses. Approximate query processing is supportedby synopses that are compact representation of the origi-nal data. To handle many base tables and many types ofqueries, a large number of the synopses may be needed.Moreover, for fast response times that avoid disk access al-together, synopses that are frequently used for query pro-cessing should remain in the main memory. Thus we evalu-ate effectiveness of a synopsis as a function of its size. Theeffectiveness of a synopsis can be measured by theaccu-racyof the answers it provides, andresponse time. Over thepast few years there has been extensive research in data syn-opses, including precomputed samples, various histograms,and wavelet synopses. Some of the techniques boost ac-curacy of query answers by adapting synopsis structure toavailable workload information.

Precomputed samples are based on the use of randomsamples as synopses for large data sets (see [13, 2, 9] andreferences therein). Histograms are commonly used syn-opses used to capture the distribution of the data stored ina database relation and are used to guide selectivity esti-mation as well as approximate query processing. Manydifferent histograms have been proposed in the literatureand some have been deployed in commercial RDBMSs.Equi-depth histograms (e.g., [2]) are popular histogramsthat are easy to implement. New histograms types canbe derived by combining effective aspects of different

histogram methods, including compressed histograms, v-optimal histograms, and maxdiff histograms. Recently,wavelet-based histograms, built from a wavelet decomposi-tion, were introduced as a mean for improved histogram ac-curacy [19, 23, 8]. Wavelets are a mathematical tool for hi-erarchical decomposition of functions, whose adaptive na-ture make them good candidate for a “lossy” data represen-tation. The use of wavelet-based synopses in database hasrecently drawn increasing attention. One of the latest de-velopments was the introduction of probabilistic waveletssynopses [12], a wavelet-based data reduction technique at-tempting to provide increased accuracy for individual ap-proximate answers.

Aggregate queries.Queries in relational databases aretypically formulated using the SQL query language. Thebelow aggregate range query is supported by most syn-opses:SELECT Sum(Data) FROM RelationWHERE filter > l AND filter < h

The above query calculates the sum of data values in thedata column in the specified range. There are number of dif-ferent aggregation functions: such as count, average, maxi-mum and minimum.

2.2 XML Databases

XML - extensible mark-up language, allows encapsulat-ing semi-structured data in tree structured documents thatmight be stored either as a regular ASCII file or inside anative XML database. XML Schema provides a structurevalidating mechanism for XML documents.

XML [6] has rapidly evolved from a mark-up languageto a de-facto standard for data exchange and integrationover the web. A testament to this is the increasing volumeof published XML data, together with the concurrent devel-opment of XML query processors that will allow users totap into the vast amount of XML data available on the In-ternet. The successful deployment of such query processorsdepends crucially on the existence of high-level declarativequery languages. There exist numerous proposals thatcover a wide range of paradigms, but a common charac-teristic among all XML-language proposals is the use ofpath expressionsas the basic method to access and retrievespecific elements from the XML database. A path expres-sion essentially defines a complex navigational path, whichcan be predicated on the existence of sibling paths or onconstraints on the values of visited elements. As a concreteexample, in a bibliography database, the path expression//author[book]/paper/sigmod/title (whichadheres to the syntax of the standard XPath language [10])selects the set of alltitle data elements discovered bythe label path//author/paper/sigmod/title , but

3

only for author elements that haveat least onebook child(a condition specified by theauthor[book] branch).Selectivity of a path expression is the number of dataelements retrieved by it (and not the retrieved data itself).

Similarly to relational optimization, optimizing XMLqueries with complex path expressions depends crucially onthe existence of concise summaries that can provide effec-tive compile-time estimates for the selectivity of these ex-pressions over the underlying (large) graph-structured XMLdatabase. This problem has recently attracted the atten-tion of the database research community, and several tech-niques [17, 21, 22, 11] have been proposed targeting dif-ferent aspects of the problem. Among the techniques thereare graph-structured solutions, markov based synopses andschema-based synopses. The synopses differ on the sub-set of supported XPath expressions, self-tuning capabilities,etc.

3 τ -Synopses Specification

The main operational processes supported by theτ -Synopses are: constructing and rebuilding of number ofpluggable synopses, interception and analysis of queryworkload, interception and analysis of data updates, ap-proximate query processing.

• Interception of Workload: User workload can betracked using various methods: analysis of databaseslog files, logging of all queries that executed directlyby τ -Synopses framework, execution database tracingutilities (like SQL Profiler in SQL Server). Collect-ing statistics of user workload will allow effective con-struction process of synopses. In addition, analyzingof workload change trends for triggering rebuilding ofsynopses when needed.

• Data updates: Updates issued in the underlying rela-tional data can be recorded using analysis of databaseslog files, execution of trace utilities or based ondatabase triggers. This updates will be analyzed andforwarded to synopses rebuilding processes.

• Approximate Query Processing: One of the aims ofτ -Synopses framework is to support general SQLqueries. The system then will be responsible forrewriting of incoming queries by using available re-lations in the calculated synopses. This will producebuilding and execution optimal query execution plans.

• Remote Integration: The system allows an integrationof remote synopses into the run-time environment.

The user interface provides an administrator user with acapability to manage data sources, synopses specifications,

Figure 1. Administration Module

updates and pre-defined workloads. The framework pro-vides also a user management mechanism. Figure 1 depictsthe main adminstration module.

• data sources: Relational data source is specified bythe database connection information (host, user, pass-word and database instance name). XML data sourceis specified by an absolute path to a data file and op-tionally a path for a validating schema file. As an XMLdata source represents single XML document, there isno need for any further specifications. With relationaldatabases, a user must also define a data relation - thesubset data of the database. It is specified by determin-ing the target table name plus filter and data columnnames.

• synopses: A synopsis is registered by providing thelocation of the execution host of the synopsis (servername plus the listening port) and the data relation forwhich the synopsis will be built. Invoking the con-struction of the synopsis requires setting the size limitand optionally the training workload. Both data rela-tion and the workload are chosen among those regis-tered in the system.

• updates: An update for particular data relation is de-termined by another instance of same relation, repre-senting the updated data.

• workloads: The queries workload is produced by thesystem based on user specifications. The users set thenumber of queries in the workload, the data relationto be processed and the workload construction param-eters. For workloads on relational data the parame-ters are the zipfian skew, low and high values, focusand range values. For xml workload the parameters in-clude the minimum and maximum lengths of the pathexpression and of the branching predicates, as well as

4

Figure 2. Query Mode

the maximum number of branching predicates and theprobability of their appearance.

The end-user is supplied with the capability to test andcompare different synopses that are registered in the sys-tem. The Query execution mode provides the user with theability to evaluate single synopsis at a time, figure 2 depictsthe Query mode UI adopted for querying an XML synopsis.The Benchmark mode enables multiple synopses evaluationover pre-defined workloads and their comparison using vi-sual display. The user interface for the benchmark mode isdepicted in figure 3. Full size screen shots are available athttp://www.cs.tau.ac.il/̃kreimern/TauSynopses.

Query Mode: The user selects the synopsis to beevaluated. The query for the relational databaseis currently limited to the following structure:SELECT Sum(Data) FROM RelationWHERE filter > l AND filter < h .The user supplies the range query parameters: low and highvalues; while the data and the filter column are those of thedata relation upon which the synopsis was built. Extendingthe support for native SQL queries is the term for futurework.

For XML synopsis, the query is an XPath expression.The system validates the user input expression for the XPathsyntax validity and the tag labels are validated against ex-isting labels of the underlying XML document.

After the query is executed (by the remote registered syn-opsis), the result is displayed. The system displays also thereal result of evaluating the query over the data source itself;this functionality depicts the accuracy of the tested synopsisand is relevant for research issues.

Query mode allows also evaluating multiple queries at atime by specifying a workload to be evaluated. The synop-sis results are depicted together with the real results com-puted by the system.

Benchmark Mode: The user chooses the synopses toevaluate and the workload to be used for the evaluation. For

Figure 3. Benchmark

the performance measurements, the minimum, maximumand step size for the synopses construction are set by theuser. The system invokes construction of the different syn-opses, then all the synopses evaluated over the chosen work-load. The system calculates the accuracy of the differentsynopses using several error metrics. MSE (mean squareerror), MRE (mean relative error) and MAXRE (maximumrelative error) performance measurements are depicted inthe results graph.

4 Architecture

4.1 Design Goals

In order to provide an effective operational and researchplatform τ -Synopses must commit to the following designgoals.

• research platform- The system should provide a sin-gle, consistent source of data, training and test work-load for experimental comparison and benchmarking,as well as performance measurements.

• multiple synopses- A system should be able to accom-modate multiple synopses

• easy integration- To allow external user to easily inte-grate his synopsis, the system should provide an avail-able, simple synopsis wrapper.

• pluggable integration- For integration purposes, asynopsis has to implement a simple interface, regard-less of its internal implementation. By utilizing a light-weight host provided by the system, the synopsis canbe executed on anySOAP-enabled platform.

• remote execution- Synopses can be transparently exe-cuted on remote machines, overTCP/IP or HTTP pro-tocols, within local area networks or over the internet.

5

Updates Logger

Synopses Manager

Synopses

Wavelet Sampling Histogram

Query Execution Engine

Workload Manager

Relation Database

Workload Original Data Updates log

Figure 4. Synopses Framework Architecture

• distributed client-server environment- The client ap-plication should reside locally on the end-station whilethe framework core should be centralized. In addition,the client and server deployment should be indepen-dent.

• flexibility and scalability - The framework designshould allow easy extending of the framework for non-relational data sources.

• privacy preservation- The system should enable itsusers to protect their resources (synopses, workloads).

• low bandwidth- Low bandwidth support is importantfor efficient client-server communication and a com-munication with the remote synopses.

Case Study: Theτ -Synopses framework actually hoststwo independent applications - relational synopses frame-work and xml synopses framework. These applications areindependent at first sight as they treat databases with to-tally different storage schema, different query languagesand they even target a slightly different problem of approx-imate processing. As it was stated earlier, at the beginning,the framework supported synopses for relational databasesonly. However, the modularity of the framework design andimplementation allowed easy adaptation of the already de-veloped relational synopsis framework to an xml synopsesframework.

4.2 High Level Design

The core of theτ -Synopses system architecture featuresthe following components, and depicted in Figure 4: QueryExecution Engine, Synopses Manager, Updates Logger, andWorkload Manager. In addition, it includes a query applica-tion which is used by end-users, an administration applica-tion used by the administrator and by synopses-providers,and a pool of registered synopses.

The Synopses Manager is used for registration and main-tenance of the synopses. A new synopsis is added to the

Framework

External Synopsis Host

SynopsisProxy

Figure 5. Synopses Integration

system by registering its parameters (including list of sup-ported queries and data sets) in the Synopses Manager Cat-alog.

The Query Execution Engine provides interface for re-ceiving query request from end-users and invoking the ap-propriate synopsis (or synopses), as determined by the Syn-opses Manager in order to process such query.

The Updates Logger provides all data updates to the reg-istered synopses by intercepting data updates information inthe data sources.

The Workload Manager captures, maintains and ana-lyzes workload information for building, maintaining andtesting synopses.

Figure 5 depicts integration process of a remote synopsiswithin the framework. The system provides a light-weighthost process, inside which the custom synopsis will be run-ning. The host is responsible for all communication withthe system and is transparent to the synopsis. This designenables unconstraint deployment. A remote synopsis canbe integrated into the system by deploying or adapting suchhost into the remote system, and connecting the synopsismodule locally into the host.

Figure 6 illustrates an overall view of the system in adistributed environment, consisting of multiple remote syn-opses, each representing its local data source.

4.3 Physical Architecture

To answer the design goals stated above, the system wasdesigned using the N-tier concept; Client -(remoting)- Fa-cade - Business Logic - Data Access Layer - Database. SeeFigure 7.

The client layer implements the user interface of the sys-tem. All user actions are treated by the client, however thenecessary processing is forwarded to the system server viaremoting. Facade interface of the server side provides thesingle point of access for the client application. The pro-cessing is further forwarded to an appropriate class insidethe business logic layer. The data access layer is responsi-ble for storing and retrieving information from the database.

The system stores all the internal data (administration in-formation, synopses specifications, etc) in the SQL Serverdatabase. The data access layer enables the application to be

6

Figure 6. Distributed Environment

Figure 7. N-tier Architecture

easily converted to use another commercial database. Sub-stituting the database provider remains transparent for therest layers of the system.

4.4 Theτ -Synopses Core Design

In this section we stand on the design ofτ -Synopsesserver side, i.e. the system core. The core implements allthe components revealed in the 4.2 section. Also, as statedin the 4.3 section, the server side is divided into severallayers (Facade, Business logic and Data Access). Below wewill reveal the major design principals.

The Facade layer is based on the the factory design pat-tern. The facade factory provides access to the followinginternal sub-systems:

• DataSourcesSystem- manages the data sources spec-ification ( database connection details for a relationaldata and data file location for an XML data)

• RelationsSystem- manages concrete row data specifi-cations for relational data sources. A relation is spec-ified for a pre-defined data source by determining thetarget table name, filter and data column names. Thismodule is also responsible for evaluating the queriesover the original data.

• SynopsesSystem- manages synopses specification aswell as synopsis construction invocation.

• UpdatesSystem- manages update specifications

• UsersSystem- manages users in the system, thus en-abling privacy and user privileges. ”

• WorkloadsSystem- manages workload specification,generation and storage.

• QuerySystem- responsible for invoking suitable syn-opsis for approximate estimation of a query.

• BenchmarkSystem- responsible for invoking the con-struction of various synopses and evaluating their ac-curacy based on the specified query workloads.

Each sub-system is divided into the above layers: facade -business logic - data access layer.

7

struct RelationData{

int size;

int *data;};

struct QueryData

{int low;

int high;

};struct WorkloadData

{int size;QueryData *queries;

};

// build synopsisSYNOPSIS_API int Build(int synopisSize,

RelationData relationData,

WorkloadData workloadData);

// update synopsisSYNOPSIS_API int Update(RelationData prevData,

RelationData newData,

WorkloadData workloadData);// Query synopsisSYNOPSIS_API double Query(QueryData

queryData);

Figure 8. Basic required interfaces

In order to integrate a new synopsis, it is sufficient tohave it implemented on aSOAP-enabled platform. Fig-ure 8 shows the interfaces required for the basic functional-ity. The synopsis should implement the Build, Update andQuery interface methods. After incorporating these inter-faces into a synopsis, it can already be used in the system.

4.5 Database Design

Figure 9 depicts the entities diagram (ERD) of theτ -Synopses. We maintain a separate table for each objectmanaged by the system: data source, data relation, work-load, synopsis, update and user. The dependency relationsbetween the objects are captured by foreign key relationsbetween the corresponding tables. For example we can seethat update, workload and synopsis objects reference the re-lation object.

While designing the implementation of XML synopsesframework, its requirements were mapped to existing ob-jects of the relational synopses framework. The databaseschema was extended for supporting XML synopses byadding additional columns to existing tables, thus reflectingthe adaptation of the objects to XML support.

5 System Implementation

Below are the major implementation characteristics ofthe τ -Synopses. The system modules were implementedin the .NET framework, with remote modules communicat-ing through the.NET Remoting. Both Client and Frame-work applications were developed using the C# program-ming language. The Client is a windows-forms applica-tion while the framework is a windows console application.SQL Server was used as a database provider.

The.NET Framework supplies a rich variety of the ready-to-use components which makes the development much

Figure 9. ERD

faster, the code much clearer and easier to maintain. Themost significant components of the.NET Framework thatwere utilized in our system are ADO.NET, Remoting, Seri-alization, XML support and Data Binding. We will brieflyreview these components and how they were used.

• .NET Remoting: Remoting enables client applicationsto use objects in other processes on the same computeror on any other computer available on its network. Inthe τ -Synopses the client application is connecting tothe server side using the .NET Remoting via a TCP/IPprotocol. Synopses are transparently executed on re-mote machines, overTCP/IP or HTTP protocols, withinlocal area networks or over the internet. The frame-work host and the synopsis host are console applica-tions. Both of them are registered on a user specifiedport in order to be invocable by the remoting.

• ADO.NET: ADO.NET is a set of classes that exposedata access services to the .NET programmer. Inτ -Synopses the SQL database server is accessed usingthe ADO.NET , each data object represented by atyped dataset.

• Data Binding: All user input is managed by data bind-ing of the input fields to the the corresponding columnin a typed dataset.

• XML support: The .NET Framework provide a com-prehensive and integrated set of classes, allowing youto work with XML documents and data. Inτ -SynopsesXML classes were used while processing the XMLdata sources.

• Serialization: Serialization is the process of convertingthe state of an object into a form that can be persisted

8

or transported. Binary serialization provided by the.NET framework Binary Formatter class was used forworkload storage implementation.

Network disconnected operational mode.The data ac-cess layer enables the application to be easily converted touse another commercial database. Substituting the databaseprovider remains transparent for the rest layers of the sys-tem. The storage of the system internal data can be evenimplemented inside XML data files. This feature was in-deed implemented in our synopsis framework in order tomake it more portable. Just by substituting the data ac-cess library we can create a light weight version of our sys-tem suitable for any disconnected laptop running MicrosoftWindows operational system. However in a disconnectedmode the system is limited to the XML data sources un-less there are any relational databases locally installed. Theversion without database storage was implemented by se-rializing the typed datasets to XML documents and storingthem in files.

6 Experience with the System

The τ -Synopses framework was successfully tested bydeploying remote synopses for both relational and xml datasources.

Several synopses for approximate queries over relationaldatabases were tested and evaluated using the frameworkrun-time environment. The platform allowed comparisonof different techniques - histogram based, sampling andwavelet based solutions were evaluated and compared.

The list of the currently integrated synopses :His-tograms - Equi-depth, Dynamic, Self-tuning;Samples -Random, ICICLES, Congressional;Wavelet Synopses -Ba-sic, Adaptive, Probabilistic, Weighted.

These synopses were developed by about 40 undergrad-uate and graduate students as part of their academic duties.They were implemented using different programming lan-guages, running on different platforms, and all operated bythe τ -Synopses system, through the .Net framework. Thesynopses were connected to the framework using API fromFigure 8. All the relational data sources used during theevaluation were stored in an SQL Server database.

The system allows to easily compare be-tween various combinations of synopses fora variety of data sets, as can be seen at”http://www.cs.tau.ac.il/̃kreimern/SynopsesTaxonomy”.These screen shots taken from [18].

The XML support of the system was tested withtwo already implemented synopses techniques. The XS-ketches [21] and the fXSketches [11] structured synopsesfor XML databases were integrated into the framework andsuccessfully evaluated. Figure 10 depicts the peformance

Figure 10. Benchmark

comparison for XSketch synopses constructed over differ-ent data sets.

τ -Synopses framework was proved to be an efficient re-search platform.

7 Conclusions

Theτ -Synopses framework provide a run-time environ-ment for remote execution and management of pluggablesynopses. It was proved to be an efficient research platformfor benchmarking and comparison of approximate queriestechniques for both relational and XML data sources.

We now encourage other research groups connect theirsynopses to theτ -Synopses system. This would allow ac-cess to a wide variety of different datasources and work-loads, and a fair comparison with other synopses with littleeffort.

References

[1] A. Aboulnaga, A. R. Alameldeen, and J. F. Naughton.“Estimating the Selectivity of XML Path Expressionsfor Internet Scale Applications”. InProceedings of the27th Intl. Conf. on Very Large Data Bases, 2001.

[2] A. Aboulnaga and S. Chaudhuri. Self-tuning his-tograms: building histograms without looking at data.In Proceedings of the 1999 ACM SIGMOD, pages181–192, 1999.

[3] S. Acharya, P. Gibbons, and V. Poosala. Aqua: A fastdecision support system using approximate query an-swers. InProceedings of the 1999 VLDB, 1999.

[4] S. Acharya, P. Gibbons, V. Poosala, and S. Ra-maswamy. The aqua approximate query answeringsystem. InProceedings of the 1999 ACM SIGMOD,1999.

9

[5] S. Acharya, P. B. Gibbons, V. Poosala, and S. Ra-maswamy. Join synopses for approximate query an-swering. InProceedings of the 2001 ACM SIGMOD,pages 275–286, 1999.

[6] T. Bray, J. Paoli, C. M. Sperberg-McQueen, andE. Maler. “Extensible Markup Language (XML) 1.0(Second Edition)”. W3C Recommendation (availablefrom http://www.w3.org/TR/REC-xml/ ),Oct. 2000.

[7] N. Bruno, S. Chaudhuri, and L. Gravano. STHoles:a multidimensional workload-aware histogram.SIG-MOD Record (ACM Special Interest Group on Man-agement of Data), 30(2):211–222, 2001.

[8] K. Chakrabarti, M. Garofalakis, R. Rastogi, andK. Shim. Approximate query processing usingwavelets. VLDB Journal: Very Large Data Bases,10(2–3):199–223, 2001.

[9] S. Chaudhuri, G. Das, and V. Narasayya. A robust,optimization-based approach for approximate answer-ing of aggregate queries. InProceedings of the 2001ACM SIGMOD, 2001.

[10] J. Clark and S. DeRose. “XML Path Language(XPath), Version 1.0”. W3C Recommendation (avail-able from http://www.w3.org/TR/xpath/ ),Nov. 1999.

[11] N. Drukh, N. Polyzotis, M. Garofalakis, andY.Matias. ”Fractional XSKETCH Synopses for XMLDatabases”. In”Proceedings of XSym’2004”, 2004.

[12] M. Garofalakis and P. B. Gibbons. Wavelet synopseswith error guarantees. InProceedings of the 2002ACM SIGMOD, pages 476–487, 2002.

[13] P. B. Gibbons and Y. Matias. New sampling-basedsummary statistics for improving approximate queryanswers. InProceedings of the 1998 ACM SIGMOD,pages 331–342, 1998.

[14] P. B. Gibbons, Y. Matias, and V. Poosala. AQUAproject white paper. Technical report, Bell Labs, Mur-ray Hill, New Jersey, U.S.A., 1997.

[15] A. C. Konig and G. Weikum. Automatic tuning of datasynopses.Information Systems, 28(1-2). Special Issueof papers from EDBT 2002.

[16] A. C. Konig and G. Weikum. A framework for thephysical design problem for data synopses. March2002.

[17] L. Lim, M. Wang, S. Padmanabhan, J. Vitter, andR. Parr. XPathLearner: An On-Line Self-TuningMarkov Histogram for XML Path Selectivity Estima-tion. In Proceedings of the 28th Intl. Conf. on VeryLarge Data Bases, 2002.

[18] Y. Matia, Y. Matias, and L. Portman. Synopses - tax-onomy and experimental evaluation. Technical Re-port, in preparation, 2004.

[19] Y. Matias, J. S. Vitter, and M. Wang. Wavelet-basedhistograms for selectivity estimation. InProceedingsof the 1998 ACM SIGMOD, pages 448–459, 1998.

[20] Y. Matias, J. S. Vitter, and M. Wang. Dynamic main-tenance of wavelet-based histograms. InThe VLDBJournal, pages 101–110, 2000.

[21] N. Polyzotis and M. Garofalakis. ”Statistical Synopsesfor Graph Structured XML Databases”. InProceed-ings of the 2002 ACM SIGMOD Intl. Conf. on Man-agement of Data, June 2002.

[22] N. Polyzotis and M. Garofalakis. ”Structure and ValueSynopses for XML Data Graphs”. InProceedings ofthe 28th Intl. Conf. on Very Large Data Bases, 2002.

[23] J. S. Vitter and M. Wang. Approximate computationof multidimensional aggregates of sparse data usingwavelets. InProceedings of the 1999 ACM SIGMOD,pages 193–204, 1999.

[24] W. Wang, H. Jiang, H. Lu, and J. X. Yu. Contain-ment join size estimation: Models and methods. InProceedings of the 2003 ACM SIGMOD Intl. Conf. onManagement of Data, 2003.

[25] Y. Wu, J. M. Patel, and H. Jagadish. ”EstimatingAnswer Sizes for XML Queries”. InProceedings ofthe 8th Intl. Conf. on Extending Database Technology(EDBT’02), 2002.

[26] Xyleme home page.

[27] Y.Matias and L.Portman. ”τ -synopses: a system forrun-time management of remote synopses”. In”Soft-ware Demo, ICDE’04, EDBT’04”, 2004.

10