distributed dbmspage 5. 1 © 1998 m. tamer Özsu & patrick valduriez outline introduction...
TRANSCRIPT
Distributed DBMS Page 5. 1© 1998 M. Tamer Özsu & Patrick Valduriez
Outline Introduction Background Distributed DBMS Architecture Distributed Database Design
Fragmentation Data Location
Semantic Data Control Distributed Query Processing Distributed Transaction Management Parallel Database Systems Distributed Object DBMS Database Interoperability Current Issues
Distributed DBMS Page 5. 2© 1998 M. Tamer Özsu & Patrick Valduriez
Design Problem
In the general setting :
Making decisions about the placement of data and programs across the sites of a computer network as well as possibly designing the network itself.
In Distributed DBMS, the placement of applications entails
placement of the distributed DBMS software; and
placement of the applications that run on the database
Distributed DBMS Page 5. 3© 1998 M. Tamer Özsu & Patrick Valduriez
Dimensions of the Problem
Level of sharing
Level of knowledge
Access pattern behavior
partialinformation
dynamic
static
data
data +program
completeinformation
Distributed DBMS Page 5. 4© 1998 M. Tamer Özsu & Patrick Valduriez
Distribution Design
Top-down
mostly in designing systems from scratch
mostly in homogeneous systems
Bottom-up
when the databases already exist at a number of sites
Distributed DBMS Page 5. 5© 1998 M. Tamer Özsu & Patrick Valduriez
Top-Down Design
User Input
View Integration
User Input
RequirementsAnalysis
Objectives
ConceptualDesign
View Design
AccessInformation ES’sGCS
DistributionDesign
PhysicalDesign
LCS’s
LIS’s
Distributed DBMS Page 5. 6© 1998 M. Tamer Özsu & Patrick Valduriez
Distribution Design Issues
Why fragment at all?
How to fragment?
How much to fragment?
How to test correctness?
How to allocate?
Information requirements?
Distributed DBMS Page 5. 7© 1998 M. Tamer Özsu & Patrick Valduriez
Fragmentation
Can't we just distribute relations? What is a reasonable unit of distribution?
relation views are subsets of relations locality extra communication
fragments of relations (sub-relations) concurrent execution of a number of transactions
that access different portions of a relation views that cannot be defined on a single fragment
will require extra processing semantic data control (especially integrity
enforcement) more difficult
Distributed DBMS Page 5. 8© 1998 M. Tamer Özsu & Patrick Valduriez
PROJ1 : projects with budgets less than $200,000
PROJ2 : projects with budgets greater than or equal to $200,000
PROJ1
PNO PNAME BUDGET LOC
P3 CAD/CAM 250000 New York
P4 Maintenance 310000 Paris
P5 CAD/CAM 500000 Boston
PNO PNAME LOC
P1 Instrumentation 150000 Montreal
P2 Database Develop. 135000 New York
BUDGET
PROJ2
Fragmentation Alternatives – Horizontal
New YorkNew York
PROJ
PNO PNAME BUDGET LOC
P1 Instrumentation 150000 Montreal
P3 CAD/CAM 250000P2 Database Develop. 135000
P4 Maintenance 310000 ParisP5 CAD/CAM 500000 Boston
New YorkNew York
Distributed DBMS Page 5. 9© 1998 M. Tamer Özsu & Patrick Valduriez
Fragmentation Alternatives – Vertical
PROJ1: information about project budgets
PROJ2: information about project names and locations
PNO BUDGET
P1 150000
P3 250000P2 135000
P4 310000P5 500000
PNO PNAME LOC
P1 Instrumentation Montreal
P3 CAD/CAM New YorkP2 Database Develop. New York
P4 Maintenance ParisP5 CAD/CAM Boston
PROJ1 PROJ2
New YorkNew York
PROJ
PNO PNAME BUDGET LOC
P1 Instrumentation 150000 Montreal
P3 CAD/CAM 250000P2 Database Develop. 135000
P4 Maintenance 310000 ParisP5 CAD/CAM 500000 Boston
New YorkNew York
Distributed DBMS Page 5. 10© 1998 M. Tamer Özsu & Patrick Valduriez
Degree of Fragmentation
Finding the suitable level of partitioning within this range
tuplesor
attributes
relations
finite number of alternatives
Distributed DBMS Page 5. 11© 1998 M. Tamer Özsu & Patrick Valduriez
Completeness Decomposition of relation R into fragments R1, R2, ..., Rn is
complete if and only if each data item in R can also be found in some Ri
Reconstruction If relation R is decomposed into fragments R1, R2, ..., Rn,
then there should exist some relational operator such that
R = 1≤i≤nRi
Disjointness If relation R is decomposed into fragments R1, R2, ..., Rn,
and data item di is in Rj, then di should not be in any other fragment Rk (k ≠ j ).
Correctness of Fragmentation
Distributed DBMS Page 5. 12© 1998 M. Tamer Özsu & Patrick Valduriez
Allocation Alternatives
Non-replicated partitioned : each fragment resides at only one
site Replicated
fully replicated : each fragment at each site partially replicated : each fragment at some of
the sites Rule of thumb:
If replication is advantageous,
otherwise replication may cause problems
read - only queriesupdate quries
1
Distributed DBMS Page 5. 13© 1998 M. Tamer Özsu & Patrick Valduriez
Comparison of Replication Alternatives
Full-replication Partial-replication Partitioning
QUERY PROCESSING
Easy Same Difficulty
Same DifficultyDIRECTORYMANAGEMENT
Easy orNon-existant
CONCURRENCYCONTROL
EasyDifficultModerate
RELIABILITY Very high High Low
REALITYPossible
applicationRealistic
Possibleapplication
Distributed DBMS Page 5. 14© 1998 M. Tamer Özsu & Patrick Valduriez
Four categories:
Database information
Application information
Communication network information
Computer system information
Information Requirements
Distributed DBMS Page 5. 15© 1998 M. Tamer Özsu & Patrick Valduriez
Horizontal Fragmentation (HF)
Primary Horizontal Fragmentation (PHF)
Derived Horizontal Fragmentation (DHF)
Vertical Fragmentation (VF)
Hybrid Fragmentation (HF)
Fragmentation
Distributed DBMS Page 5. 16© 1998 M. Tamer Özsu & Patrick Valduriez
Database Information relationship
cardinality of each relation: card(R)
PHF – Information Requirements
TITLE, SAL
SKILL
ENO, ENAME, TITLE PNO, PNAME, BUDGET, LOC
ENO, PNO, RESP, DUR
EMP PROJ
ASG
L 1
L 2 L 3
Distributed DBMS Page 5. 17© 1998 M. Tamer Özsu & Patrick Valduriez
Application Information simple predicates : Given R[A1, A2, …, An], a simple
predicate pj is
pj : Ai Value
where {=,<,≤,>,≥,≠}, ValueDi and Di is the domain of Ai.
For relation R we define Pr = {p1, p2, …,pm}
Example :PNAME = "Maintenance"BUDGET ≤ 200000
minterm predicates : Given R and Pr={p1, p2, …,pm}
define M={m1,m2,…,mr} as
M={ mi|mi = pjPrpj* }, 1≤j≤m, 1≤i≤z
where pj* = pj or pj* = ¬(pj).
PHF - Information Requirements
Distributed DBMS Page 5. 18© 1998 M. Tamer Özsu & Patrick Valduriez
Example
m1: PNAME="Maintenance" BUDGET≤200000
m2: NOT(PNAME="Maintenance") BUDGET≤200000
m3: PNAME= "Maintenance" NOT(BUDGET≤200000)
m4: NOT(PNAME="Maintenance") NOT(BUDGET≤200000)
PHF – Information Requirements
Distributed DBMS Page 5. 19© 1998 M. Tamer Özsu & Patrick Valduriez
Application Information
minterm selectivities: sel(mi)
The number of tuples of the relation that would be accessed by a user query which is specified according to a given minterm predicate mi.
access frequencies: acc(qi)
The frequency with which a user application qi accesses data.
Access frequency for a minterm predicate can also be defined.
PHF – Information Requirements
Distributed DBMS Page 5. 20© 1998 M. Tamer Özsu & Patrick Valduriez
Definition :Rj = Fj
(R ), 1 ≤ j ≤ w
where Fj is a selection formula, which is (preferably) a minterm predicate.
Therefore,A horizontal fragment Ri of relation R consists of all the tuples of R which satisfy a minterm predicate mi.
Given a set of minterm predicates M, there are as many horizontal fragments of relation R as there are minterm predicates. Set of horizontal fragments also referred to as minterm fragments.
Primary Horizontal Fragmentation
Distributed DBMS Page 5. 21© 1998 M. Tamer Özsu & Patrick Valduriez
Given: A relation R, the set of simple predicates Pr
Output: The set of fragments of R = {R1, R2,…,Rw} which obey the fragmentation rules.
Preliminaries :
Pr should be complete
Pr should be minimal
PHF – Algorithm
Distributed DBMS Page 5. 22© 1998 M. Tamer Özsu & Patrick Valduriez
A set of simple predicates Pr is said to be complete if and only if the accesses to the tuples of the minterm fragments defined on Pr requires that two tuples of the same minterm fragment have the same probability of being accessed by any application.
Example :Assume PROJ[PNO,PNAME,BUDGET,LOC] has two
applications defined on it.Find the budgets of projects at each location.
(1)Find projects with budgets less than $200000.
(2)
Completeness of Simple Predicates
Distributed DBMS Page 5. 23© 1998 M. Tamer Özsu & Patrick Valduriez
According to (1),
Pr={LOC=“Montreal”,LOC=“New York”,LOC=“Paris”}
which is not complete with respect to (2).
Modify
Pr ={LOC=“Montreal”,LOC=“New York”,LOC=“Paris”, BUDGET≤200000,BUDGET>200000}
which is complete.
Completeness of Simple Predicates
Distributed DBMS Page 5. 24© 1998 M. Tamer Özsu & Patrick Valduriez
If a predicate influences how fragmentation is performed, (i.e., causes a fragment f to be further fragmented into, say, fi and fj) then there should be at least one application that accesses fi and fj differently.
In other words, the simple predicate should be relevant in determining a fragmentation.
If all the predicates of a set Pr are relevant, then Pr is minimal.
acc(mi)–––––card(fi)
acc(mj)–––––card(fj)
≠
Minimality of Simple Predicates
Distributed DBMS Page 5. 25© 1998 M. Tamer Özsu & Patrick Valduriez
Example :
Pr ={LOC=“Montreal”,LOC=“New York”, LOC=“Paris”,
BUDGET≤200000,BUDGET>200000}
is minimal (in addition to being complete). However, if we add
PNAME = “Instrumentation”
then Pr is not minimal.
Minimality of Simple Predicates
Distributed DBMS Page 5. 26© 1998 M. Tamer Özsu & Patrick Valduriez
Given: a relation R and a set of simple predicates Pr
Output:a complete and minimal set of simple predicates Pr' for Pr
Rule 1: a relation or fragment is partitioned into at least two parts which are accessed differently by at least one application.
COM_MIN Algorithm
Distributed DBMS Page 5. 27© 1998 M. Tamer Özsu & Patrick Valduriez
Initialization : find a pi Pr such that pi partitions R according
to Rule 1 set Pr' = pi ; Pr Pr – pi ; F fi
Iteratively add predicates to Pr' until it is complete
find a pj Pr such that pj partitions some fk defined according to minterm predicate over Pr' according to Rule 1
set Pr' = Pr' pi ; Pr Pr – pi; F F fi if pk Pr' which is nonrelevant then
Pr' Pr' – pk
F F – fk
COM_MIN Algorithm
Distributed DBMS Page 5. 28© 1998 M. Tamer Özsu & Patrick Valduriez
Makes use of COM_MIN to perform fragmentation.
Input: a relation R and a set of simple predicates Pr
Output: a set of minterm predicates M according to which relation R is to be fragmented
Pr' COM_MIN (R,Pr)
determine the set M of minterm predicates
determine the set I of implications among pi Pr
eliminate the contradictory minterms from M
PHORIZONTAL Algorithm
Distributed DBMS Page 5. 29© 1998 M. Tamer Özsu & Patrick Valduriez
Two candidate relations : PAY and PROJ. Fragmentation of relation PAY
Application: Check the salary info and determine raise. Employee records kept at two sites application run
at two sites Simple predicates
p1 : SAL ≤ 30000
p2 : SAL > 30000
Pr = {p1,p2} which is complete and minimal Pr'=Pr Minterm predicates
m1 : (SAL ≤ 30000)
m2 : NOT(SAL ≤ 30000) (SAL > 30000)
PHF – Example
Distributed DBMS Page 5. 30© 1998 M. Tamer Özsu & Patrick Valduriez
PHF – Example
TITLE
Mech. Eng.
Programmer
SAL
27000
24000
PAY1 PAY2
TITLE
Elect. Eng.
Syst. Anal.
SAL
40000
34000
Distributed DBMS Page 5. 31© 1998 M. Tamer Özsu & Patrick Valduriez
Fragmentation of relation PROJ Applications:
Find the name and budget of projects given their no. Issued at three sites
Access project information according to budget
one site accesses ≤200000 other accesses >200000
Simple predicates For application (1)
p1 : LOC = “Montreal”
p2 : LOC = “New York”
p3 : LOC = “Paris” For application (2)
p4 : BUDGET ≤ 200000
p5 : BUDGET > 200000 Pr = Pr' = {p1,p2,p3,p4,p5}
PHF – Example
Distributed DBMS Page 5. 32© 1998 M. Tamer Özsu & Patrick Valduriez
Fragmentation of relation PROJ continued Minterm fragments left after elimination
m1 : (LOC = “Montreal”) (BUDGET ≤ 200000)
m2 : (LOC = “Montreal”) (BUDGET > 200000)
m3 : (LOC = “New York”) (BUDGET ≤ 200000)
m4 : (LOC = “New York”) (BUDGET > 200000)
m5 : (LOC = “Paris”) (BUDGET ≤ 200000)
m6 : (LOC = “Paris”) (BUDGET > 200000)
PHF – Example
Distributed DBMS Page 5. 33© 1998 M. Tamer Özsu & Patrick Valduriez
PHF – Example
PROJ1
PNO PNAME BUDGET LOC PNO PNAME BUDGET LOC
P1 Instrumentation 150000 Montreal P2DatabaseDevelop.
135000 New York
PROJ2
PROJ4 PROJ6
PNO PNAME BUDGET LOC
P3 CAD/CAM 250000 New York
PNO PNAME BUDGET LOC
MaintenanceP4 310000 Paris
Distributed DBMS Page 5. 34© 1998 M. Tamer Özsu & Patrick Valduriez
Completeness
Since Pr' is complete and minimal, the selection predicates are complete
Reconstruction
If relation R is fragmented into FR = {R1,R2,…,Rr}
R = Ri FR Ri
Disjointness
Minterm predicates that form the basis of fragmentation should be mutually exclusive.
PHF – Correctness
Distributed DBMS Page 5. 35© 1998 M. Tamer Özsu & Patrick Valduriez
Defined on a member relation of a link according to a selection operation specified on its owner.
Each link is an equijoin. Equijoin can be implemented by means of
semijoins.
Derived Horizontal Fragmentation
TITLE, SAL
SKILL
ENO, ENAME, TITLE PNO, PNAME, BUDGET, LOC
ENO, PNO, RESP, DUR
EMP PROJ
ASG
L1
L2 L3
Distributed DBMS Page 5. 36© 1998 M. Tamer Özsu & Patrick Valduriez
Given a link L where owner(L)=S and member(L)=R, the derived horizontal fragments of R are defined as
Ri = R F Si, 1≤i≤w
where w is the maximum number of fragments that will be defined on R and
Si = Fi(S)
where Fi is the formula according to which the primary horizontal fragment Si is defined.
DHF – Definition
Distributed DBMS Page 5. 37© 1998 M. Tamer Özsu & Patrick Valduriez
Given link L1 where owner(L1)=SKILL and member(L1)=EMP
EMP1 = EMP SKILL1
EMP2 = EMP SKILL2
whereSKILL1 = SAL≤30000(SKILL)
SKILL2 = SAL>30000(SKILL)
DHF – Example
ENO ENAME TITLE
E3 A. Lee Mech. Eng.
E4 J. Miller Programmer
E7 R. Davis Mech. Eng.
EMP1
ENO ENAME TITLE
E1 J. Doe Elect. Eng.E2 M. Smith Syst. Anal.E5 B. Casey Syst. Anal.
EMP2
E6 L. Chu Elect. Eng.E8 J. Jones Syst. Anal.
Distributed DBMS Page 5. 38© 1998 M. Tamer Özsu & Patrick Valduriez
Completeness Referential integrity Let R be the member relation of a link whose
owner is relation S which is fragmented as FS = {S1, S2, ..., Sn}. Furthermore, let A be the join attribute between R and S. Then, for each tuple t of R, there should be a tuple t' of S such that
t[A]=t'[A] Reconstruction
Same as primary horizontal fragmentation. Disjointness
Simple join graphs between the owner and the member fragments.
DHF – Correctness
Distributed DBMS Page 5. 39© 1998 M. Tamer Özsu & Patrick Valduriez
Has been studied within the centralized context
design methodology physical clustering
More difficult than horizontal, because more alternatives exist.
Two approaches : grouping
attributes to fragments splitting
relation to fragments
Vertical Fragmentation
Distributed DBMS Page 5. 40© 1998 M. Tamer Özsu & Patrick Valduriez
Overlapping fragments
grouping
Non-overlapping fragments
splitting
We do not consider the replicated key attributes to be overlapping.
Advantage:
Easier to enforce functional dependencies
(for integrity checking etc.)
Vertical Fragmentation
Distributed DBMS Page 5. 41© 1998 M. Tamer Özsu & Patrick Valduriez
Application Information Attribute affinities
a measure that indicates how closely related the attributes are
This is obtained from more primitive usage data Attribute usage values
Given a set of queries Q = {q1, q2,…, qq} that will run on the relation R[A1, A2,…, An],
use(qi,•) can be defined accordingly
VF – Information Requirements
use(qi,Aj) =1 if attribute Aj is referenced by query qi
0 otherwise
Distributed DBMS Page 5. 42© 1998 M. Tamer Özsu & Patrick Valduriez
Consider the following 4 queries for relation PROJ
q1: SELECT BUDGET q2:SELECTPNAME,BUDGETFROM PROJ FROM PROJWHERE PNO=Value
q3: SELECT PNAME q4:SELECTSUM(BUDGET)FROM PROJ FROM PROJWHERE LOC=Value WHERE LOC=Value
Let A1= PNO, A2= PNAME, A3= BUDGET, A4= LOC
VF – Definition of use(qi,Aj)
q1
q2
q3
q4
A1
1 0 1 0
0 01 1
0 01 1
0 0 1 1
A2 A3 A4
Distributed DBMS Page 5. 43© 1998 M. Tamer Özsu & Patrick Valduriez
The attribute affinity measure between two attributes Ai and Aj of a relation R[A1, A2, …, An] with respect to the set of applications Q = (q1, q2, …, qq) is defined as follows :
VF – Affinity Measure aff(Ai,Aj)
aff (Ai, Aj) (query access)all queries that access Ai and Aj
query access access frequency of a query
access
executionall sites
Distributed DBMS Page 5. 44© 1998 M. Tamer Özsu & Patrick Valduriez
Assume each query in the previous example accesses the attributes once during each execution.
Also assume the access frequencies
Then aff(A1, A3) = 15*1 + 20*1+10*1
= 45
and the attribute affinity matrix AA is
VF – Calculation of aff(Ai, Aj)
q1
q2
q3
q4
S1 S2 S3
15 20 10
5 0 0
25 2525
3 0 0
A A A A1 2 3 4
AAA
A
1
2
3
4
45 0 45 0
0 80 5 75
45 5 53 3
0 75 3 78
Distributed DBMS Page 5. 45© 1998 M. Tamer Özsu & Patrick Valduriez
Take the attribute affinity matrix AA and reorganize the attribute orders to form clusters where the attributes in each cluster demonstrate high affinity to one another.
Bond Energy Algorithm (BEA) has been used for clustering of entities. BEA finds an ordering of entities (in our case attributes) such that the global affinity measure
is maximized.
VF – Clustering Algorithm
AM (affinity of Ai and Aj with their neighbors) j
i
Distributed DBMS Page 5. 46© 1998 M. Tamer Özsu & Patrick Valduriez
Input: The AA matrix
Output: The clustered affinity matrix CA which is a perturbation of AA
Initialization: Place and fix one of the columns of AA in CA.
Iteration: Place the remaining n-i columns in the remaining i+1 positions in the CA matrix. For each column, choose the placement that makes the most contribution to the global affinity measure.
Row order: Order the rows according to the column ordering.
Bond Energy Algorithm
Distributed DBMS Page 5. 47© 1998 M. Tamer Özsu & Patrick Valduriez
“Best” placement? Define contribution of a placement:
cont(Ai, Ak, Aj) = 2bond(Ai, Ak)+2bond(Ak, Al) –2bond(Ai, Aj)
where
Bond Energy Algorithm
bond(Ax,Ay) = aff(Az,Ax)aff(Az,Ay)z 1
n
Distributed DBMS Page 5. 48© 1998 M. Tamer Özsu & Patrick Valduriez
Consider the following AA matrix and the corresponding CA matrix where A1 and A2 have been placed. Place A3:
Ordering (0-3-1) :cont(A0,A3,A1) = 2bond(A0 , A3)+2bond(A3 , A1)–2bond(A0 ,
A1)= 2* 0 + 2* 4410 – 2*0 = 8820
Ordering (1-3-2) :cont(A1,A3,A2) = 2bond(A1 , A3)+2bond(A3 , A2)–2bond(A1,A2)
= 2* 4410 + 2* 890 – 2*225 = 10150Ordering (2-3-4) :
cont (A2,A3,A4) = 1780
A A A A1 2 3 4
AAA
A
1
2
3
4
45 0 5 0
0 80 5 75
45 5 53 3
0 75 3 78
AA=
A A1 2
45 00 80
45 5
0 75
CA=
BEA – Example
Distributed DBMS Page 5. 49© 1998 M. Tamer Özsu & Patrick Valduriez
Therefore, the CA matrix has to
form
BEA – Example
A1 A2A3
45
0
45
0
45
5
53
3
0
80
5
75
Distributed DBMS Page 5. 50© 1998 M. Tamer Özsu & Patrick Valduriez
When A4 is placed, the final form of the
CA matrix (after row organization) is
BEA – Example
A AA A1 23 4
A
A
A
A
1
2
3
4
45
45
0
0
45
53 5
3
0
5
80
75
0
3
75
78
Distributed DBMS Page 5. 51© 1998 M. Tamer Özsu & Patrick Valduriez
How can you divide a set of clustered attributes {A1, A2, …, An} into two (or more) sets {A1, A2, …, Ai} and {Ai, …, An} such that there are no (or minimal) applications that access both (or more than one) of the sets.
VF – Algorithm
A1A2
Ai
Ai+1
Am
…A1 A2 A3 Ai Ai+1 Am
BA
. . .
. . . . . . TA
Distributed DBMS Page 5. 52© 1998 M. Tamer Özsu & Patrick Valduriez
DefineTQ = set of applications that access only TABQ = set of applications that access only BAOQ = set of applications that access both TA and BA
andCTQ = total number of accesses to attributes by
applications that access only TACBQ = total number of accesses to attributes by
applications that access only BACOQ = total number of accesses to attributes by
applications that access both TA and BA
Then find the point along the diagonal that maximizes
VF – ALgorithm
CTQCBQCOQ2
Distributed DBMS Page 5. 53© 1998 M. Tamer Özsu & Patrick Valduriez
Two problems :
Cluster forming in the middle of the CA matrix Shift a row up and a column left and apply the
algorithm to find the “best” partitioning point
Do this for all possible shifts
Cost O(m2)
More than two clusters m-way partitioning
try 1, 2, …, m–1 split points along diagonal and try to find the best point for each of these
Cost O(2m)
VF – Algorithm
Distributed DBMS Page 5. 54© 1998 M. Tamer Özsu & Patrick Valduriez
A relation R, defined over attribute set A and key K, generates the vertical partitioning FR = {R1, R2, …, Rr}.
Completeness The following should be true for A:
A = ARi
Reconstruction Reconstruction can be achieved by
R = KRi Ri FR
Disjointness TID's are not considered to be overlapping since they are
maintained by the system Duplicated keys are not considered to be overlapping
VF – Correctness
Distributed DBMS Page 5. 55© 1998 M. Tamer Özsu & Patrick Valduriez
Hybrid Fragmentation
R
HFHF
R1
VF VFVFVFVF
R11 R12 R21 R22 R23
R2
Distributed DBMS Page 5. 56© 1998 M. Tamer Özsu & Patrick Valduriez
Fragment Allocation
Problem StatementGiven
F = {F1, F2, …, Fn} fragments
S ={S1, S2, …, Sm} network sites
Q = {q1, q2,…, qq} applications
Find the "optimal" distribution of F to S. Optimality
Minimal cost Communication + storage + processing (read & update) Cost in terms of time (usually)
PerformanceResponse time and/or throughput
Constraints Per site constraints (storage & processing)
Distributed DBMS Page 5. 57© 1998 M. Tamer Özsu & Patrick Valduriez
Information Requirements
Database information selectivity of fragments size of a fragment
Application information access types and numbers access localities
Communication network information unit cost of storing data at a site unit cost of processing at a site
Computer system information bandwidth latency communication overhead
Distributed DBMS Page 5. 58© 1998 M. Tamer Özsu & Patrick Valduriez
File Allocation (FAP) vs Database Allocation (DAP):
Fragments are not individual files
relationships have to be maintained
Access to databases is more complicated
remote file access model not applicable
relationship between allocation and query processing
Cost of integrity enforcement should be considered
Cost of concurrency control should be considered
Allocation
Distributed DBMS Page 5. 59© 1998 M. Tamer Özsu & Patrick Valduriez
Allocation – Information Requirements
Database Information selectivity of fragments size of a fragment
Application Information number of read accesses of a query to a fragment number of update accesses of query to a fragment A matrix indicating which queries updates which fragments A similar matrix for retrievals originating site of each query
Site Information unit cost of storing data at a site unit cost of processing at a site
Network Information communication cost/frame between two sites frame size
Distributed DBMS Page 5. 60© 1998 M. Tamer Özsu & Patrick Valduriez
General Form
min(Total Cost)
subject to
response time constraint
storage constraint
processing constraint
Decision Variable
Allocation Model
xij 1 if fragment Fi is stored at site Sj 0 otherwise
Distributed DBMS Page 5. 61© 1998 M. Tamer Özsu & Patrick Valduriez
Total Cost
Storage Cost (of fragment Fj at Sk)
Query Processing Cost (for one query)
processing component + transmission component
Allocation Model
(unit storage cost at Sk) (size of Fj) xjk
query processing cost all queries
cost of storing a fragment at a siteall fragmentsall sites
Distributed DBMS Page 5. 62© 1998 M. Tamer Özsu & Patrick Valduriez
Query Processing CostProcessing component
access cost + integrity enforcement cost + concurrency control cost
Access cost
Integrity enforcement and concurrency control costs Can be similarly calculated
Allocation Model
(no. of update accesses+ no. of read accesses) all fragmentsall sites
xijlocal processing cost at a site
Distributed DBMS Page 5. 63© 1998 M. Tamer Özsu & Patrick Valduriez
Query Processing CostTransmission component
cost of processing updates + cost of processing retrievals
Cost of updates
Retrieval Cost
Allocation Model
update message cost all fragmentsall sites
acknowledgment cost all fragmentsall sites
(minall sitesall fragments cost of retrieval command
cost of sending back the result)
Distributed DBMS Page 5. 64© 1998 M. Tamer Özsu & Patrick Valduriez
Constraints Response Time
execution time of query ≤ max. allowable response time for that query
Storage Constraint (for a site)
Processing constraint (for a site)
Allocation Model
storage requirement of a fragment at that site all fragments
storage capacity at that site
processing load of a query at that site all queries
processing capacity of that site
Distributed DBMS Page 5. 65© 1998 M. Tamer Özsu & Patrick Valduriez
Solution Methods
FAP is NP-complete
DAP also NP-complete
Heuristics based on
single commodity warehouse location (for FAP)
knapsack problem
branch and bound techniques
network flow
Allocation Model
Distributed DBMS Page 5. 66© 1998 M. Tamer Özsu & Patrick Valduriez
Attempts to reduce the solution space
assume all candidate partitionings known; select the “best” partitioning
ignore replication at first
sliding window on fragments
Allocation Model