track your data across the fourth dimension

32
Text Tracking your Data Across the Fourth Dimension Wellington Waterloo Web Makers Meetup

Upload: jeremy-cook

Post on 26-May-2015

132 views

Category:

Technology


1 download

DESCRIPTION

Slides of talk given on 10.09.2014 introducing temporal databases.

TRANSCRIPT

Page 1: Track your data across the fourth dimension

Text

Tracking your Data Across the Fourth DimensionWellington Waterloo Web Makers Meetup

Page 2: Track your data across the fourth dimension

–Wikipedia

“In physics, spacetime is any mathematical model that combines space and time into a single

interwoven continuum.”

Page 3: Track your data across the fourth dimension

Databases are Good at ‘Now’

CRUD

Create data

Read data

Update data

Delete data

Page 4: Track your data across the fourth dimension

Bread and Butter Queries

How many people work in department X?

What’s the total amount paid in commissions by department?

What was the top selling product in the last year?

Page 5: Track your data across the fourth dimension

The Fourth Dimension…Compare reporting relationships one month ago, today and in one months time

Show me all changes made to this data over time

As of one month ago, what did I believe to be the truth about this data yesterday and what did I actually believe was the truth yesterday?

Page 6: Track your data across the fourth dimension

–Wikipedia

“A temporal database is a database with built-in support for handling data involving time…”

Page 7: Track your data across the fourth dimension

A Little History…

Need for temporal support identified as early as 1992

Initial attempts to get support for this into the SQL standard were rejected

Finally made it into the SQL:2011 standard

Page 8: Track your data across the fourth dimension

Some Sample Data

Owner Property

Jeremy ‘Some place’

Page 9: Track your data across the fourth dimension

What forms of temporal data are usually stored?

Page 10: Track your data across the fourth dimension

Temporal Aspects

Decision Time

Valid Time

Transaction Time

A table that implements one of these is temporal, two is bi-temporal, more is multi-temporal

Page 11: Track your data across the fourth dimension

Decision Time

Records when a decision was taken

Stored as a timestamp

You may be doing this already…

Page 12: Track your data across the fourth dimension

Decision Time Example

Owner Property Decision Time

Jeremy ‘Some place’ 2012-09-28

Adam ‘Some place’ 2014-02-21

Page 13: Track your data across the fourth dimension

Valid Time

The time during which a fact is true with respect to the real world

Modelled as a range between two timestamps

Closed at lower bound, can be open at upper bound

Page 14: Track your data across the fourth dimension

Valid Time Example

Owner Property StartVT EndVT

Jeremy ‘Some place’ 2012-09-28 2014-02-20

Adam ‘Some place’ 2014-02-21 ∞

Page 15: Track your data across the fourth dimension

Valid Time Example

Owner Property StartVT EndVT

Jeremy ‘Some place’ 2012-09-28 2014-03-20

Adam ‘Some place’ 2014-03-21 ∞

Page 16: Track your data across the fourth dimension

Transaction Time

The time period during which a fact stored in the database is considered to be true

Modelled as a range between two timestamps

Closed at lower bound, can be open at upper bound

Page 17: Track your data across the fourth dimension

Transaction Time Example

Owner Property StartVT EndVT StartTT EndTT

Jeremy ‘Some place’

2012-09-28

2014-02-20

2012-09-28

2014-09-09

Adam ‘Some place’

2014-02-21 ∞ 2014-02-2

12014-09-0

9

Jeremy ‘Some place’

2012-09-28

2014-03-20

2014-09-09 ∞

Adam ‘Some place’

2014-03-21 ∞ 2014-09-0

9 ∞

Page 18: Track your data across the fourth dimension

Valid Time and Transaction Time are Orthogonal

Valid

Tim

e

Transaction Time

Page 19: Track your data across the fourth dimension

Enough theory! How do I add this stuff to my db schema?

Page 20: Track your data across the fourth dimension

SQL:2011 Temporal Additions

!

PERIOD type for date ranges in table definitions

Can query for data in a period using new predicates CONTAINS, OVERLAPS, EQUALS, PRECEDES and IMMEDIATELY SUCCEEDS

Can also update or delete data matching a portion of a period using FOR PORTION OF {period_name} FROM {date} TO {date}

Page 21: Track your data across the fourth dimension

Implementing Valid Time

Done by adding columns for start and end with a period constraint on the columns

Period must also be included as part of the primary key

Page 22: Track your data across the fourth dimension

Table Definition using Valid Time

CREATE TABLE Emp( ENo INTEGER, StartVT DATE, EndVT DATE, EDept INTEGER, PERIOD FOR EValidTime (StartVT, EndVT), PRIMARY KEY (ENo, EValidTime WITHOUT OVERLAPS), )

Page 23: Track your data across the fourth dimension

Foreign Key Problems…

It’s possible that rows in child tables may reference rows in parent tables that are not currently valid and vice versa

Therefore foreign key constraints have to be updated where needed to include period data

Page 24: Track your data across the fourth dimension

Valid time foreign key definition

ALTER TABLE Emp ADD FOREIGN KEY (Edept, PERIOD EValidTime) REFERENCES Dept (DNo, PERIOD DValidTime)

Page 25: Track your data across the fourth dimension

Querying a Table using Valid Time

SELECT Ename, Edept FROM Emp WHERE ENo = 22217 AND EValidTime CONTAINS DATE ‘2011-01-02’; !SELECT Ename, Edept FROM Emp WHERE ENo = 22217 AND EValidTime OVERLAPS PERIOD (DATE ‘2010-01-01', DATE ‘2011-01-01');

Page 26: Track your data across the fourth dimension

Implementing Transaction Time

RDBMS automatically creates a snapshot when data is updated or deleted

Done by adding columns for start, end and a period for them

Must also add WITH SYSTEM VERSIONING to create table statement

Page 27: Track your data across the fourth dimension

Table Definition using Transaction Time

CREATE TABLE Emp ENo INTEGER, Sys_start TIMESTAMP(12) GENERATED ALWAYS AS ROW START, Sys_end TIMESTAMP(12) GENERATED ALWAYS AS ROW END, EName VARCHAR(30), PERIOD FOR SYSTEM_TIME (Sys_start, Sys_end) ) WITH SYSTEM VERSIONING

Page 28: Track your data across the fourth dimension

Querying a Table using Transaction Time

SELECT ENo,EName,Sys_Start,Sys_End FROM Emp FOR SYSTEM_TIME AS OF TIMESTAMP '2011-01-02 00:00:00’; !SELECT ENo,EName,Sys_Start,Sys_End FROM Emp FOR SYSTEM_TIME FROM TIMESTAMP '2011-01-02 00:00:00’TO TIMESTAMP '2011-12-31 00:00:00’;

Page 29: Track your data across the fourth dimension

Bi-Temporal table definitionCREATE TABLE Emp( ENo INTEGER, StartVT DATE, EndVT DATE, EDept INTEGER, PERIOD FOR EValidTime (StartVT, EndVT), Sys_start TIMESTAMP(12) GENERATED ALWAYS AS ROW START, Sys_end TIMESTAMP(12) GENERATED ALWAYS AS ROW END, EName VARCHAR(30), PERIOD FOR SYSTEM_TIME (Sys_start, Sys_end), PRIMARY KEY (ENo, EValidTime WITHOUT OVERLAPS), FOREIGN KEY (Edept, PERIOD EValidTime) REFERENCES Dept (DNo, PERIOD DValidTime) ) WITH SYSTEM VERSIONING

Page 30: Track your data across the fourth dimension

Who supports SQL:2011 Temporal?

IBM DB2. Called “Time Travel Queries”, although different syntax for FOR SYSTEM_TIME AS OF

Oracle 12c, in compliance with SQL:2011

Others?

Page 31: Track your data across the fourth dimension

Further Reading

‘Developing Time Oriented Applications in SQL’ by Richard Snodgrass

‘Temporal Features in SQL:2011’

‘Time and Relational Theory: Temporal Databases in the Relational Model and SQL'

Page 32: Track your data across the fourth dimension

Thanks for listening

Any questions?

Contact me:

@JCook21

[email protected]

I’d love some feedback!