Download - Data vault: What's Next
![Page 1: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/1.jpg)
Data Vault Modeling
WHAT’S NEXT???
© Dan Linstedt 2009-2012
This was a presentation I gave at an Array Conference In the Netherlands, in 2009.
![Page 2: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/2.jpg)
2
A bit about me…
• Author, Inventor, Speaker – and part time photographer…
• 25+ years in the IT industry• Worked in DoD, US Gov’t, Fortune 50,
and so on…
• Find out more about the Data Vault:o http://www.youtube.com/LearnDataVaulto http://LearnDataVault.com
• Full profile on http://www.LinkedIn.com/dlinstedt
04/10/2023LearnDataVault.com
![Page 3: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/3.jpg)
Who’s Using It?
04/10/2023LearnDataVault.com 3
![Page 4: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/4.jpg)
Featured Word from Our Customer
04/10/2023LearnDataVault.com 4
Jonathan Rice, BScING Real Estate/Development International/Operational Support Location HP A.05.071PO Box 90463, 2509 LL The Hague, The Netherlands
They are connecting MS SharePoint back-end to Data Vault Modeling EDW.
![Page 5: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/5.jpg)
What’s Changing the Game?
• Solid State Disk (SSD)• Hosted Cloud Computing• Column Based Databases• Unstructured Information• Ontologies / Taxonomies• Mining Engines• Broad Based Web Services
• Business Demand for Immediate Answers (pushing real-time)
• Compliance and Auditability (pushing volume)
04/10/2023LearnDataVault.com 5
• Data Visualization• Flash & Silverlight Front-Ends• Analytic functions in
Database Engines• Business Rules Engines
Melding with ETL and Web Services
• DW Appliances
![Page 6: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/6.jpg)
Current EDW + DV Architecture
04/10/2023LearnDataVault.com 6
Sales
Finance
Contracts
StagingDVEDW
StarSchemas
ErrorMarts
ReportCollections
Enterprise BI SolutionSOA
(real-time)
(batch)
(batch)
Business Rules Downstream!(the Lens Filter)
![Page 7: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/7.jpg)
First Change: Business Data Vault
04/10/2023LearnDataVault.com 7
Sales
Finance
Contracts
DVEDW
StarSchemas
ErrorMarts
ReportCollections
Enterprise BI SolutionSOA
(real-time)
Batch
Business Rules Downstream!(the Lens Filter)
BDVEDW
StagingStaging
![Page 8: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/8.jpg)
Next Change: Staging Removal
04/10/2023LearnDataVault.com 8
Sales
Finance
Contracts
DVEDW
StarSchemas
ErrorMarts
ReportCollections
Enterprise BI SolutionSOA
(real-time)
Business Rules Downstream!(the Lens Filter)
BDVEDW
Write Back
![Page 9: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/9.jpg)
Next Change: Virtual Marts
04/10/2023LearnDataVault.com 9
Virtual Marts
& Dynamic Cubes
Virtual Marts
& Dynamic Cubes
Sales
Finance
Contracts
DVEDW
Enterprise BI SolutionSOA
(real-time)
Business Rules Downstream!(the Lens Filter)
BDVEDW
Write Back
![Page 10: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/10.jpg)
04/10/2023LearnDataVault.com 10
Next Change: Unstructured Data
Unstructured Data Sets Ontologies/Taxonomies
• Email• Docs• Images• Movies• Sound
Unstructured Processing Engine
Raw Data Vault EDW
Joins through LINK Structures
On-DemandCubes
![Page 11: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/11.jpg)
A Look at Ontologies
04/10/2023LearnDataVault.com 11
Hierarchies of Data:Synonyms, Homonyms, Antonyms, Related Terms, Definitions, Categorizations, Organizations, Views of the data world
Ontologies HOLD THE KEY to understanding/conceptual relevanceOntologies can PIVOT raw data in to many different results
![Page 12: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/12.jpg)
04/10/2023LearnDataVault.com 12
Data Vault EDW• Stored• Analyzed / Scored
Plateau: Operational Data Warehouse
Virtual Marts
Real-TimeMiningEngine
Staging Area
Non-S.O.R.Historical Batch Data
SORReal-Time Data
Real-TimeCollector
Web Interface (usually)
OperationalSystems
OperationalAlerts
StrategicReports& OLAP
& W.SVCS
Operational Systems
UnstructuredSemi-Structured
Non-SORBatch Data
OperationalApplicationsMaster Data
OperationalMetadata
Management
Direct Edits
Direct Edits
• Flexible• Accountable• Compliant • Scalable
• Normalized• Dynamic• Granular• Historic
StrategicReports& OLAP
& W.SVCS
Dynamic Cubes
![Page 13: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/13.jpg)
Data Vault(EDW)
Operational Data Vault
04/10/2023LearnDataVault.com 13
Common Data Access
Layer
Operatio
nal
Applicatio
n
Read Current
Read/Lock
Insert Changes
Update/Unlock
![Page 14: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/14.jpg)
Why go Operational Data Vault?• Benefits Include
o Direct access to datao Removal of 90% of batch streams, replaced by Transactionso Better/faster alerting capabilitieso Direct control over changes to BI answer sets
• What are the risks?o The Data Vault EDW becomes an ODV, brings with it ALL
responsibilities of the Operational System
• What are the problems?o Row Lockingo Consistent Data Accesso Data Version Controlo Security of Data through access points
• Has it been done before?o Yes, Cendant Timeshare Resource Group did it in 2002!
04/10/2023LearnDataVault.com 14
![Page 15: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/15.jpg)
04/10/2023LearnDataVault.com 15
Plateau: Dynamic Data Warehouse
MetaMiningEngineH H H
H
H
H
L
L
L
L
DL
DL
Dynamically Constructed LinksDynamically Created HubsDynamically Altered SatellitesDynamically changed ETL/ELTDynamically Altered QueriesAutomatically Updated Cubes= Self Morphing (guided) Data Vault
![Page 16: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/16.jpg)
Impacts of Dynamic Data VaultsBusiness• New found knowledge• Evolving Data Warehouse models over time• Faster Reporting• Self-Maintaining Back-Ends
Technical• Guided Changes to Structures• Auto-Adapting Load Routines• Auto-Adapting Query Sets• Auto-Suggested Star Schemas
04/10/2023LearnDataVault.com 16
![Page 17: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/17.jpg)
Why go Dynamic?• Benefits include:
o Faster Turn Around Timeo Auto Configuration of “mundane tasks”o Discovery of new relationships (could result in increased revenue)o Self Healing Structures
• How Long before we See It?o 5 to 7 years out
• How do we get there?o Meta-Miningo Use of Ontologies inside the EDW
• What’s driving us there?o Business Users want faster turn around timeo Business Users want ingestion of Unstructured Data Setso Business Users need more control over their systems
04/10/2023LearnDataVault.com 17
![Page 18: Data vault: What's Next](https://reader034.vdocuments.net/reader034/viewer/2022051400/54bc2bd04a79591f528b45ba/html5/thumbnails/18.jpg)
18
Where To Learn More• The Technical Modeling Book: http://LearnDataVault.com
• The Discussion Forums: & eventshttp://LinkedIn.com – Data Vault Discussions
• Contact me:http://DanLinstedt.com - web [email protected] - email
• World wide User Group (Free)http://dvusergroup.com
04/10/2023LearnDataVault.com