massive scale-out of expensive continuous queries erik zeitler and tore risch uppsala database...

Massive Scale-out of

Expensive Continuous Queries

Erik Zeitler and Tore RischUppsala Database Laboratory

Uppsala University

31 Aug 2011 Erik Zeitler and Tore Risch 2

Outline

1. Introduction2. Stream splitting strategies for scale-

out3. Evaluating stream splitting strategies4. Cost model and heuristic5. Energy efficiency6. Related work7. Conclusions and future work


Input data streams

Query result data stream

DSMS

Queries

useror

programmer

meta-data

Query processing software

Stream data access software

storeddata

CQ: Continuous Queries (filters and transformations)

11001011

01001011

DSMS Data Stream Management System

SCSQ

Super Computer Stream Query processor

a Data Stream Management System


CQ

CQ

CQ

split merge

Research Questions

How to ensure scalable CQ execution• with growing input stream rate?• with high CQ execution cost?

CQs are scaled out by splitting the input stream.• applications require customizable input stream splitting,

called splitstream• both tuple route and broadcast allowed

splitstream CQ

By scale-out.


merge

Research Questions

How to ensure scalable CQ execution• with growing input stream rate?• with high CQ execution cost?

CQs are scaled out by splitting the input stream.• applications require customizable input stream splitting,

called splitstream• both tuple route and broadcast allowed

How to split massive streams over massively parallel CQs?• By parallelization of splitstream

By scale-out.

CQ

CQ

CQ

CQ

CQ

CQ

CQ

splitsplitstreamsplitstreamsplitstreamsplitstream


Outline

1. Introduction2. Stream splitting strategies for

scale-out3. Scale-up of stream splitting

strategies4. Cost model and heuristic5. Energy efficiency6. Related work7. Conclusions and future work


splitstream(stream s, integer q, function rfn, function bfn) vector of stream sv

User defines rfn and bfn rfn(object tpl, integer q) integer

rfnLRB(event e, integer q) integer as select expressway(e) where eventtype(e) = 0;

bfn(object tpl) booleanbfnLRB(event e) boolean as select eventtype(e) = 2;

rfn and bfn for streams are analogous to fragmentation and replication conditions in distributed DBMS

Unlike DDBMS, execution of rfn and bfn is parallelized

Defining stream splitting

splitstream

q

s sv


Naïve (flat) splitstream implementation: fsplit

fsplit

CQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ

Expensive stream splitting

computations

Bottleneck!

fsplit(stream s, integer q, function rfn, function bfn) vector of stream sv


Tree shaped splitstream implementation: maxtree

fsplit

fsplit

fsplitCQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ• Bottleneck is alleviated [Zeitler and Risch, DASFAA 2010]• but still problematic

maxtree(stream s, integer q, function rfn, function bfn) vector of stream sv


Scaled-out splitstream: parasplit

PR fsplit

fsplit

fsplitCQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ

Window router

distributes entire

windowsWindow splitter

Stream merge

parasplit(stream s, integer q, function rfn, function bfn) vector of stream sv


Parasplit:route – //fsplit – //(merge – CQ)

PR fsplit

fsplit

fsplitCQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ

Window router

distributes entire

windowsWindow splitter

Stream merge



Tree shaped window routing: parasplit*

PR

CQ2

CQ1

CQ3

CQ0

CQ5

CQ4

CQ7

CQ6

CQ2

CQ1

CQ3

CQ0

CQ5

CQ4

CQ7

CQ6

fsplit

fsplit

fsplit

fsplit

fsplit

fsplit

PR

PR

PR

fsplit

fsplit

fsplit

CQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ


Outline

1. Introduction2. Stream splitting strategies3. Scale-up of stream splitting



Experimental set-up

Hardware

Linux cluster Up to 70 nodes Each node has 2x

quad-core Intel® Xeon® [email protected], 6 MB L2$.

TCP/IP over GbE

Performance number L : Number of xways the DSMS can handle

www.cs.brandeis.edu/~linearroad


LRB result

name orgyea

rL

cores

comment

Aurora Brandeis, Brown,

MIT

2004

2.5 1

Commercial sys A

2004

0.5 1

SPC IBM200

62.5 170 3GHz Xeon

Xquery ETHZ200

71.5 1

DataCell CWI200

91 4 1.4s avg RT

stream schema ETHZ201

05 4

SCSQ maxtree UU201

064 48

D disabled (later verified in mySQL)

SCSQ parasplit UU201

1512

560 D disabled

Performance number L : Number of xways the DSMS can handle


Splitstream stream rate

0,00

200,00

400,00

600,00

800,00

1 000,00

0 100 200 300 400 500

q

Max

str

eam

rat

e [M

bps]

parasplit*parasplitmaxtreefsplit

CQ parallelism, q

1 Gbpswire

speed


Window router stream rate

PR fsplit

fsplit

fsplitCQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ

W – physical window size

p – number of parallel fsplit

pW


Impact of window size W in window router

0,00

200,00

400,00

600,00

800,00

1000,00

0 5 10 15W [kB]

Ma

x s

tre

am

ra

te [

Mb

ps

]

p=4 p=64

network bound for large enough windows


Impact of window size W in window router when scaling p

0,00

200,00

400,00

600,00

800,00

1000,00

0 5 10 15

W [kB]

Max

str

eam

rat

e [M

bp

s]

p=4 p=64

p=128 p=256

p=512


0,00100,00

200,00300,00

400,00500,00

600,00700,00

800,00900,00

1 000,00

0 100 200 300 400 500p

Ma

x st

rea

m r

ate

[Mb

ps]

.

window router tree (parasplit*)

single window router (parasplit)

Parasplit*Tree shaped window router

W = 16 kB fsplit parallelism, p


Outline




Eliminate p

PR fsplit

fsplit

fsplitCQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ

p

q


Given• Input stream rate ΦD

• Parallelism of continuous query q Automatically determine

• fsplit parallelism p


splitconsume ...R1

Rq

...emit(R1)

emit(Rq)

Cost model for fsplit

cr – read cost per tpl (read + de-marshal)cs – split cost per tpl (execute rfn and bfn)ce – emit cost per tpl (marshal + print)o – omit %r – routing %b – broadcast % q – number of output streams

)()( bqrcebqrocscrC fsplit

according to rfn and bfn


Cost model for merge in CQ

cr – read cost per tpl (read + de-marshal)cp – poll cost per tplcm – merge cost per tplO – cost of executing the CQ and emit its

result

OcmcppcrCCQ

merge compute split

S1 consume(S1)

Sp consume(Sp)

... ... ...R1

Rw

...emit(R1)

emit(Rw)


Cost model for parasplit

p can be eliminated using cost model, but

requires extensive profiling everywhere

OcmcppcrCCQ bqrcebqrocscrC Wfsplit

WWWPR cecscrC

PR fsplit

fsplit

fsplitCQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ

p

q


PR fsplit

fsplit

fsplitCQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ

p

q

Heuristic for estimating p

Assume• 1% broadcast tuples (configurable)• 0% omitted tuples (configurable)

Measure Φfsplit(1)

on rfn and bfn, q = 1: cs +ce = 1/Φfsplit(1)

Estimate p by

qcecsC fsplit 01.099.0ˆ

bqrcebqrocscrC Wfsplit

qpfsplit

D 01.099.0ˆ)1(

fsplit


0,00100,00200,00300,00400,00500,00600,00700,00800,00900,00

1 000,00

0 100 200 300 400 500q

Max

str

eam

rat

e [M

bps]

parasplitcost modelToo high p (p=q)Too low p (p=1)

p according to heuristics vs.p using exact cost model

CQ parallelism, q


Outline




Estimating energy efficiency, η

How much extra energy does parasplit consume in comparison to fsplit?

Conservatively assume energy consumption proportional to CPU usage:

Useful work • p ∙ Cfsplit

Overhead• CPR

• q ∙ CCQ(O=0)

)0(

OCQfsplitPR

fsplit

CqCpC

Cp

PR fsplit

fsplit

fsplitCQ

CQ

CQ

CQ

CQ

CQ

CQ

CQ


Measuring energy efficiency

0%10%20%30%40%50%60%70%80%90%

100%

0 100 200 300 400 500q

Eff

icie

ncy

parasplit*parasplitcost modelToo high p (p=q)Too low p (p=1)

CQ parallelism, q


Related work Nobody else has investigated strategies for scalable

customizable stream splitting IBM SPADE/System S [Andrade et al 2009]

• Splitstream operator with broadcast capabilities• Streaming throughput degrades when scaling q

Event based systems [Brenna et al 2009]• Custom stream splitting shown to be a bottleneck

Gigascope [Johnson et al 2008]• Assumes specialized stream splitting hardware• No customizable stream splitting

GSDM [Ivanova, Risch 2005]• Parallel execution of expensive UDFs• More limited parallelization

Streaming MapReduce [Condie et al 2010]• Does not handle scalable stream splitting

[Balkesen, Tatbul 2011]• Distributing entire windows over CQs• q ≤ 4


Conclusions and future work

Naïve stream splitting is prohibitive for scale-out of CQs

Parasplit• eliminates the bottleneck of stream splitting, providing

network bound stream rates Parasplit*

• provides network bound stream rates for highly scaled-out stream splitting

Push selection predicates from CQ to rfn of splitstream

Improve energy efficiency High Availability SCSQ home page

• http://www.it.uu.se/research/group/udbl/SCSQ.html

massive scale-out of expensive continuous queries erik zeitler and tore risch uppsala database...

Documents