model management 2.0: manipulating richer mappings€¦ · model mgmt for cse544, phil bernstein...
TRANSCRIPT
![Page 1: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/1.jpg)
Model Mgmt for CSE544, Phil Bernstein
Phil Bernstein Microsoft Research
Nov. 26, 2007
1
Most slides come from SIGMOD 07 Keynote & “Bridging Apps & DB”, both with Sergey Melnik
![Page 2: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/2.jpg)
Model Mgmt for CDE 544, Phil Bernstein
Make it easier to write programs that access databases
Traditionally, for large IT departments
Much progress, but it’s still ~40% of the work
Core problem is developing and using complex mappings between schemas
2
![Page 3: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/3.jpg)
Model Mgmt for CSE 544, Phil Bernstein
Data translation
XML message mapping
Data warehouse loading
Query mediators
Forms managers
Report writers
Query designers
Object-relational wrappers
Portal generation from DB
OLAP databases
Application integration
Composing web services
3
![Page 4: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/4.jpg)
Model Mgmt for CSE544, Phil Bernstein
Most packaged business apps need to access an OO view of relational data
Requires an OR wrapper
App developer specifies a high-level mapping
Objects
Relationaldatabase
Business logic
4
A tool translates the mapping into executable code
![Page 5: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/5.jpg)
Model Mgmt for CSE544, Phil Bernstein
Person = HR ID,Name(Client)
Employee = HR ⋈ Empl
Customers = Client
Target: EER Source: SQLHR
Id Name
Client
Id Name Score ...
Empl
Id Dept
Customer
Person
IdName
Employee
Dept CreditScoreBillingAddr
5
Specified by app developer
![Page 6: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/6.jpg)
Model Mgmt for CSE 544, Phil Bernstein
SELECT VALUE
CASEWHEN (T5._from2 AND NOT(T5._from1)) THEN Person(T5.Person_Id, T5.Person_Name) WHEN (T5._from1 AND T5._from2)
THEN Employee(T5.Person_Id, T5.Person_Name, T5.Employee_Dept)ELSE Customer(T5.Person_Id, T5.Person_Name, T5.Customer_CreditScore,
T5.Customer_BillingAddr)END
FROM ( (SELECT T1.Person_Id, T1.Person_Name, T2.Employee_Dept,
CAST(NULL AS SqlServer.int) AS Customer_CreditScore,
CAST(NULL AS SqlServer.nvarchar) AS Customer_BillingAddr, False AS _from0, (T2._from1 AND T2._from1 IS NOT NULL) AS _from1, T1._from2
FROM ( SELECT T.Id AS Person_Id, T.Name AS Person_Name, True AS _from2FROM HR AS T) AS T1
LEFT OUTER JOIN (SELECT T.Id AS Person_Id, T.Dept AS Employee_Dept, True AS _from1FROM dbo.Empl AS T) AS T2
ON T1.Person_Id = T2.Person_Id )
UNION ALL (SELECT T.Id AS Person_Id, T.Name AS Person_Name,
CAST(NULL AS SqlServer.nvarchar) AS Employee_Dept, T.Score AS Customer_CreditScore, T.Addr AS Customer_BillingAddr, True AS _from0, False AS _from1, False AS _from2
FROM Client AS T)) AS T5
[ Melnik, Adya, Bernstein, SIGMOD 07]
6
![Page 7: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/7.jpg)
Model Mgmt for CSE544, Phil Bernstein
The main benefitIt’s easier to design mappings than to write code
The main problemsHelp the user develop mappings
Translate mappings into code
7
![Page 8: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/8.jpg)
Model Mgmt for CSE544, Phil Bernstein
Heterogeneity
Impedance mismatch
Insufficient abstraction
Potpourri of tools
And it’s getting harder due to more
[ Haas, ICDT 07]
8
Programming languages
Types of toolsData models
Schema sources
![Page 9: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/9.jpg)
Model Mgmt for CSE544, Phil Bernstein
Process
Prec
ision
Info Integration Workshophttp://db.cis.upenn.edu/iiworkshop/
9
![Page 10: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/10.jpg)
Model Mgmt for CSE544, Phil Bernstein
Motivation
Model Management
Operators & Scenarios
10
![Page 11: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/11.jpg)
Model Mgmt for CSE544, Phil Bernstein
Metadata Repository
ModelManagement
Engine
Wrapper
Generator
Query
MediatorETL
ToolsManipulate models & mappings as bulk objects
Customer
Order
Scheduled
Delivery
Product
Salespersonmapping
Operations• Match• Compose• Merge
• Diff/Extract• ModelGen• Inverse
Meta-model independent• relational, ER, OO, XML, RDF, OWL, SML, …
[ Bernstein, Halevy, Pottinger
SIGMOD Record 00]
11
![Page 12: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/12.jpg)
Model Mgmt for CSE544, Phil Bernstein
Choose a schema definition language
Choose a mapping language
12
![Page 13: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/13.jpg)
Model Mgmt for CSE544, Phil Bernstein
M1 M2mapM2 , map =
ModelGen(M1 , metamodel2)
M1 M2mapmap = Match(M1 , M2)
13
M1 M2mapFcn = TransGen(map)
Fcn
13
![Page 14: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/14.jpg)
Model Mgmt for CSE544, Phil Bernstein14
M1 M2map12
M3
M3 , map13 , map23 =
Merge (M1 , M2 , map12)
Plus a few more …
M1 M2mapM2 , map2 =
Diff(M1 , map1 )map1
Compose: map13 = map12 map23
map13
M1map12 M2 M3
map23
14
![Page 15: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/15.jpg)
Model Mgmt for CSE544, Phil Bernstein
More research focus on primary operations
More powerful operations
Hence better tools
More leverage from tool investments
More uniform behavior across tools
15
![Page 16: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/16.jpg)
Model Mgmt for CSE544, Phil Bernstein
Good NewsLots of progress on operations
Some practical applications
A lot has been learned
Bad News
Still waiting for the first reasonably-complete practical implementation
Good news
A lot of research left to do16
![Page 17: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/17.jpg)
Model Mgmt for CSE544, Phil Bernstein
I(S1) are the instances of schema S1
Each d in I(S1) is a database
(e.g., a set of relations)
I(S2) are the instances of schema S2
map12 I(S1) I(S2)
Usually, we represent a mapping by an
expression
V= R ⋈ S
R ⋈ S = T ⋈ U17
![Page 18: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/18.jpg)
Model Mgmt for CSE544, Phil Bernstein
Element correspondencesFirst step in aligning schemas
For lineage & impact analysis
Weak or no formal semantics
Mapping constraints relate instances of schemas
E.g., equality of relational expressions
Transformation is an executable mapping constraintConstructs target instances from source instances
E.g., SQL query, XSLT, C# program
SELECT Id, Name, Dept = SELECT Id, Name, DeptFROM Employee FROM HR JOIN Empl ON Id
[Catarci, Lenzerini. J. CoopIS 93]
[Miller, Haas, Hernandez. VLDB 00][Casanova, Vidal. PODS 83]
[Biskup, Convent. SIGMOD 86]
18
![Page 19: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/19.jpg)
Model Mgmt for CSE544, Phil Bernstein
What we want: first-order logic with
negation
aggregation
set and bag semantics
regular expressions
nested collections and lists
rich type constructors (e.g., to construct XML fragments),
user-defined functions
deduplication and other heuristic functions
What can we handle? … Much less.19
![Page 20: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/20.jpg)
Model Mgmt for CSE544, Phil Bernstein
Motivation
Model Management
Operators & Scenarios
20
![Page 21: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/21.jpg)
Model Mgmt for CSE 544, Phil Bernstein
2. Evolve mappings
Compose
Diff
Merge
Inverse
1. Create mappings
ModelGen
Match
ConstraintGen
TransGen
21
![Page 22: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/22.jpg)
Model Mgmt for CSE544, Phil Bernstein
Inputsource modeltarget metamodel
OO schema
Dept BldgIn
m:nO
O.Dept(d) S.Dept(d.key)
O.Bldg(b) S.Bldg(b.key)
O.In(d,b) S.In(d.key, b.key)
map
There are several credible prototypesDon't know of products, yet
[Atzeni, Torlone. EDBT 96][Bernstein, Melnik, Mork. VLDB 05] [Atzeni, Cappellari, Bernstein. EDBT 06]
22
Outputtarget modelconstraints
SQL schema
Dept BldgIn
FK
Sfrom to
FK
![Page 23: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/23.jpg)
Model Mgmt for CSE544, Phil Bernstein
Dept BldgInBldg
m:n
OO Model
O
[Atzeni & Torlone, EDBT '96]
Super Metamodel
Dept BldgInBldg
m:n
O
1
Super Metamodel
Dept BldgInBldg
ref
Sfrom to
ref
mapOS
2
SQL Schema
mapOS
Dept BldgInBldg
FK
Sfrom to
FK
3
![Page 24: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/24.jpg)
Model Mgmt for CSE544, Phil Bernstein
Data is transferred to super-metamodel DB
24
Input OOschema
Nativemeta-model
[Papotti, Torlone]
[Atzeni, Cappellari]
S1
Eliminate m:nassociations
Eliminateinheritance
Sn. . .Super
meta-model S0
Data is transferred to output schema’s database
Data is transformed within super-metamodel DB
Output SQLschema
Native meta-model
![Page 25: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/25.jpg)
Model Mgmt for CSE544, Phil Bernstein
Leverages Compose operator
Each mapi roundtrips data25
Input OOschema
S0
Supermeta-model S1
Eliminate m:nassociations
Eliminateinheritance
Sn
Output SQLschema
Nativemeta-model
map1 mapn
. . .
map1 ∘ map2 ∘ … ∘ mapn
mapInOut
TransGen
[Bernstein, Melnik, Mork VLDB’05, ER’07]
![Page 26: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/26.jpg)
Model Mgmt for CSE544, Phil Bernstein
[Miller, Haas, &
Hernandez, VLDB 00]
CorrespondencesModels
Customer
Order
Scheduled
Delivery
Product
Salesperson
Customer
Order
Scheduled
Delivery
Product
Salesperson
Foo(a,b,c)
Bar(d,e,f)
Foo(a,b,c)
Bar(d,e,f)
Match
Select ord#, prod#, cust# From Shipped
Select ord#, prod#, cust# From Order Join Item
on ord#
Constraints Transformations
TransGen
Select ord#, prod#, cust# From Shipped=
Select ord#, prod#, cust# From Order Join Item
on ord#
Constraints
ConstraintGen
Correspondences
26
![Page 27: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/27.jpg)
Model Mgmt for CSE544, Phil Bernstein
Produce candidate correspondences
Exploit lexical analysis of element names, schema structure, data types, thesauri, value distributions, ontologies, instances, and previous matches
Past Goal - improved precision & recallBig productivity gains are unlikely
Better goalsReturn top-k, not best overall match
Avoid the tedium. Manage work.
HCI – handle large schemas.
User studies – what would improve productivity?
S1
S2
Match
27
![Page 28: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/28.jpg)
Model Mgmt for CSE 544, Phil Bernstein
AnHai Doan
Alon Halevy
Pedro Domingos
Phil Bernstein
Erhard Rahm
Sergey Melnik
Jayant Madhavan
Jeffrey Naughton
Jaewoo Kang
Tova Milo
Pavel Shvaiko
Fausto Giunchiglia
Sonia Bergamaschi
Silvana Castano
Bin He
Kevin Chang
Namyoun Choi
Il-Yeol Song
Hyoil Han
Domenico Ursino
Luigi Palopoli
Dominico Sacca
Georgio Terracina
David Embley
David Jackman
Li Xu
Yihong Ding
Jacob Berlin
Amihai Motro
Hong Hai Do
Fabien Duchateau
Zohra Bellahsene
Ela Hunt
Toralf Kirsten
Andreas Thor
Alexander Bilke
Avigdor Gal
Michalis Petropoulos
Christoph Quix
Chris Clifton
Arnie Rosenthal
Wen-Syan Li
Hector Garcia-Molina
Sagit Zohar
Gio Wiederhold
Anna Zhdanova
Jerome Euzenat
Prasenjit Mitra
Natasha Noy
Anuj Jaiswal
Mikalai Yatskevich
Nuno Silva
Joao Rocha
David Aumueller
Sabine Massmann
Felix Naumann
28
![Page 29: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/29.jpg)
Model Mgmt for CSE544, Phil Bernstein
[Miller, Haas, &
Hernandez, VLDB 00]
CorrespondencesModels
Customer
Order
Scheduled
Delivery
Product
Salesperson
Customer
Order
Scheduled
Delivery
Product
Salesperson
Foo(a,b,c)
Bar(d,e,f)
Foo(a,b,c)
Bar(d,e,f)
Match
Select ord#, prod#, cust# From Shipped
Select ord#, prod#, cust# From Order Join Item
on ord#
Constraints Transformations
TransGen
Select ord#, prod#, cust# From Shipped=
Select ord#, prod#, cust# From Order Join Item
on ord#
Constraints
ConstraintGen
Correspondences
29
![Page 30: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/30.jpg)
Model Mgmt for CSE544, Phil Bernstein
For a given target element
Find all source elements linked by correspondences
Find all ways that source elements are related
Choose one of them and generate the transformation
EID
NameTelAID
AID
CityZip
SID
NameBirthDateCity
Empl
Staff
Addr
SID,Name,City(Staff) = EID,Name,City(Empl ⋈ Addr)
Source
Target
[Popa, Velegrakis, Miller, Hernandez, Fagin. VLDB 02][Velegrakis. PhD thesis 2005]
30
![Page 31: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/31.jpg)
Model Mgmt for CSE544, Phil Bernstein
Directly interpret correspondences as mapping constraints
If it’s a tree schema and keys correspond
1. EID(Empl) = SID(Staff)
2. EID,Name(Empl) = SID,Name(Staff)
3. EID,City(Empl ⋈ Addr) = SID,City(Staff)
EID
NameTelAID
AID
CityZip
SID
NameBirthDateCity
1
2
3
Empl
Staff
Addr
[Melnik, Bernstein, Halevy, & Rahm, SIGMOD 05]
31
![Page 32: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/32.jpg)
Model Mgmt for CSE544, Phil Bernstein
CorrespondencesModels
Customer
Order
Scheduled
Delivery
Product
Salesperson
Customer
Order
Scheduled
Delivery
Product
Salesperson
Foo(a,b,c)
Bar(d,e,f)
Foo(a,b,c)
Bar(d,e,f)
Match
Select ord#, prod#, cust# From Shipped
Select ord#, prod#, cust# From Order Join Item
on ord#
Constraints Transformations
TransGen
Select ord#, prod#, cust# From Shipped=
Select ord#, prod#, cust# From Order Join Item
on ord#
Constraints
ConstraintGen
Correspondences
32
![Page 33: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/33.jpg)
Model Mgmt for CSE544, Phil Bernstein
SELECT p.Id, p.NameFROM Persons AS p WHERE p IS OF (ONLY Person)
OR p IS OF (ONLY Employee)
SELECT Id, Name FROM dbo.HR=
SELECT e.Id, e.DeptFROM Persons AS e WHERE e IS OF Employee
SELECT Id, Dept FROM dbo.Empl=
SELECT c.Id, c.Name, c.CreditScore, c.BillingAddr
FROM Persons AS c WHERE c IS OF Customer
SELECT Id, Name,Score, Addr
FROM dbo.Client=
SELECT VALUE
CASEWHEN (T5._from2 AND NOT(T5._from1)) THEN Person(T5.Person_Id,
T5.Person_Name) WHEN (T5._from1 AND T5._from2)
THEN Employee(T5.Person_Id, T5.Person_Name, T5.Employee_Dept)ELSE Customer(T5.Person_Id, T5.Person_Name, T5.Customer_CreditScore,
T5.Customer_BillingAddr)END
FROM ( (SELECT T1.Person_Id, T1.Person_Name, T2.Employee_Dept,
CAST(NULL AS SqlServer.int) AS Customer_CreditScore, CAST(NULL AS SqlServer.nvarchar) AS Customer_BillingAddr, False AS_from0,
(T2._from1 AND T2._from1 IS NOT NULL) AS _from1, T1._from2FROM ( SELECT T.Id AS Person_Id, T.Name AS Person_Name, True AS
_from2FROM HR AS T) AS T1
LEFT OUTER JOIN (SELECT T.Id AS Person_Id, T.Dept AS Employee_Dept, True AS_from1FROM dbo.Empl AS T) AS T2
ON T1.Person_Id = T2.Person_Id )
UNION ALL (SELECT T.Id AS Person_Id, T.Name AS Person_Name,
CAST(NULL AS SqlServer.nvarchar) AS Employee_Dept, T.Score AS Customer_CreditScore, T.Addr AS
Customer_BillingAddr, True AS _from0, False AS _from1, False AS _from2
FROM Client AS T)) AS T5
[ Melnik, Adya, Bernstein, SIGMOD 07]
33
![Page 34: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/34.jpg)
Model Mgmt for CSE544, Phil Bernstein
Difficulty depends on
Whether the constraints are functions
The transformation language (e.g., SQL, XSLT)
Expressiveness of constraints
Optimization required
34
![Page 35: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/35.jpg)
Model Mgmt for CSE544, Phil Bernstein
Mapping: {QC1=QS1, …, QCn=QSn}
E.g.,
f
SELECT p.Id, p.NameFROM Persons p
SELECT Id, NameFROM ClientInfo=f: g:
Vf g
Client Store
g: V1=QS1
V2=QS2
Vn=QSn
…
f: V1=QC1
V2=QC2
Vn=QCn
…
f‘ g
query view
[Melnik, Adya, Bernstein
SIGMOD 07]
![Page 36: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/36.jpg)
Model Mgmt for CSE544, Phil Bernstein
I(S1) are the instances of schema S1
map12 I(S1) I(S2) map13 I(S2) I(S3)
map13 = {<d1 I(S1), d3 I(S3)> |
d2 I(S2) (<d1 , d2> map12 )
⋀ (<d2 , d3> map23 ) }
36
S1 S2 S3map12 map23
map12 • map23
Well known examples
View unfolding
Answering queries using views
S1 S2 S3v q
![Page 37: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/37.jpg)
Model Mgmt for CSE544, Phil Bernstein
Goal: Rewrite q to access S1 only
There are many solutions. A.Y. Halevy: Answering Queries Using Views: A Survey. VLDB J. 10(4): 270-294 (2001).
37
S1 S2 S3v q
DBView Queryresult
![Page 38: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/38.jpg)
Model Mgmt for CSE544, Phil Bernstein
38
Vf
f = Q f
Client
Q
Identity
Client
f = Q f
Use query rewriting to compute f’ from f
This is mapping composition.
Store
Vf g
fClient Store
f‘ g
![Page 39: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/39.jpg)
Model Mgmt for CSE 544, Phil Bernstein
2. Evolve mappings
Compose
Diff
Merge
Inverse
1. Create mappings
Match
ConstraintGen
TransGen
ModelGen
39
![Page 40: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/40.jpg)
Model Mgmt for CSE544, Phil Bernstein
schema
view
MapSV
database
evolvedschema
Schema evolves
What about database & view?
[ Rahm, Bernstein. SIGMOD Rec. Dec 06]
40
![Page 41: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/41.jpg)
Model Mgmt for CSE544, Phil Bernstein
schema
view
MapSV
database
evolvedschema
migrateddatabase
2. Generate a transformation
TransGen(MapES)
41
MapES
1. Create mapping: schema evolved schema
![Page 42: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/42.jpg)
Model Mgmt for CSE544, Phil Bernstein
schema
view
MapSV
database
evolvedschema
migrateddatabase
MapES
Compose MapSV and MapES to connect viewto evolved schema 42
![Page 43: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/43.jpg)
Model Mgmt for CSE544, Phil Bernstein
Some natural 1st-order mapping languages are not closed under composition
Sometimes, it's undecidable whether the composition is expressible in the input language
Can settle for a partial solution over 1st-order mappings
Or you can use a 2nd-order mapping language that’s closed under composition
There's a composition algorithm to compute it
Some prototype implementations reportedPractical applications needed
[Fagin, Kolaitis, Popa, Tan. TODS 05][Nash, Bernstein, Melnik. TODS 07][Yu, Popa. VLDB 05][Bernstein, Green, Melnik, Nash. VLDB 06]
43
S1 S2 S3map12 map23
map12 • map23
![Page 44: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/44.jpg)
Model Mgmt for CSE544, Phil Bernstein
S
V
p=MapSV
SEq=MapES
ES = Diff(SE , MapES)
V = Merge(V , S , p•q•r)E
VV
44
![Page 45: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/45.jpg)
Model Mgmt for CSE544, Phil Bernstein
There’s a big market looking for solutions
There’s progress on many operatorsBut it's incomplete
For mappings with limited expressiveness
Schema evolution is a particularly important scenario where the operators could have a major impact.
45
![Page 46: Model Management 2.0: Manipulating Richer Mappings€¦ · Model Mgmt for CSE544, Phil Bernstein Phil Bernstein Microsoft Research Nov. 26, 2007 1 Most slides come from SIGMOD 07](https://reader036.vdocuments.net/reader036/viewer/2022071215/6044fec846a44437d77df60e/html5/thumbnails/46.jpg)
Model Mgmt for CSE544, Phil Bernstein
© 2007 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it
should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
46