oracle ’ s data warehouse & etl future
DESCRIPTION
Oracle ’ s Data Warehouse & ETL future. Communicatie. www.facebook.com/Vijfhart. #vijfhart. SSID: Vijfhart-IT Encryptie: Wpa2 (TKIP) Wachtwoord: 5hart-internet. Even voorstellen. Frank Kuijpers Business Intelligence consultant. Agenda. Oracle ETL tools OWB ODI Pauze - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/1.jpg)
5HART-IT 1
Oracle’s
Data Warehouse& ETL
future
![Page 2: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/2.jpg)
5HART-IT 2
Communicatie
www.facebook.com/Vijfhart
SSID: Vijfhart-ITEncryptie: Wpa2 (TKIP)Wachtwoord: 5hart-internet
#vijfhart
![Page 3: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/3.jpg)
5HART-IT 3
Even voorstellen...
Frank KuijpersBusiness Intelligence consultant
![Page 4: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/4.jpg)
5HART-IT 4
Agenda
Oracle ETL tools OWB ODI
Pauze
Overgang van OWB naar ODI Handmatig Converter
![Page 5: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/5.jpg)
5HART-IT 5
Oracle Warehouse Builder (OWB)
OWB 9.2 en 10.1=> Not supported by Oracle
OWB 10.2 => Premier Support gestopt in juli 2010.
OWB 11.1 => Premier Support stopt in augustus 2012.
OWB 11.2 => Vanaf september 2009, tot januari 2015
Future Geen geplande releases
![Page 6: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/6.jpg)
5HART-IT 6
Oracle Data Integrator (ODI)
Gekocht in 2006 van Sunopsis
ODI toevoegingen tov OWB Doel database Oracle en non-Oracle (DB2, Teradata, XML, SQL Server
etc) Productiviteits- en kwaliteitsverbetering tav realisatie Realtime datawarehousing mogelijk (CDC) Event and/or trigger-based integratie Service Oriented Architecture (SOA) integratie
![Page 7: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/7.jpg)
5HART-IT 7
Waarom ODI?
Goede Perfomance• ELT ipv ETL• Code specifiek voor een RDBMS (Oracle, SQLServer, Teradata)
Eenvoudig in gebruik• Intuïtief
Real-time Datawarehousing• CDC (Change Data Capture)
Decleratief Ontwerp• Ontwerper specificeert business rules in data model• Tool genereert specifieke RDBMS code
![Page 8: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/8.jpg)
5HART-IT 8
Waarom ODI?
Knowledge Modules• Productiviteitverbetering door voorgedefinieerde herbruikbare
code templates• Best practices snel te gebruiken
=> Vele voorbeelden te vinden op www• Specifiek voor een RDBMS• Aanpasbaar voor specifieke klantwensen
![Page 9: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/9.jpg)
5HART-IT 9
ODI en OBI-EE
ODI strategic tool for OBI EE-related ETL
Supports heterogeneous target Platforms
Future direction • merge ODI en OBI EE repositories?
![Page 10: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/10.jpg)
5HART-IT 10
1
2
3
ODI Architectuur - overview
![Page 11: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/11.jpg)
5HART-IT 11
Repository
Design Navigator:
Reverse-engineerOntwikkelprojectsScenario’s maken
Operator Navigator:
Sessies monitoren
Topology Navigator:
Infrastructuur vastleggen
Security Navigator:
Users enPrivileges vastleggen
ODI Studio
ODI Studio en interfaces
![Page 12: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/12.jpg)
5HART-IT 12
MasterRepository
Versioning
ExecutionExecution
Projects
Models
Work Repository(Development)
Work Repository(Production)
ODI Repositories
![Page 13: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/13.jpg)
5HART-IT 13
6 typen Knowledge Modules
Knowledge Module Description
Reverse-engineering KM(RKM)
Metadata ophalen en in ODI work repository zetten
Check KM(CKM)
Checks consistency van data tegen ODI constraints
Loading KM(LKM)
Laden verschillende platformen naar staging area
Integration KM(IKM)
Integratie/transformeren van data naar een target
Journalizing KM(JKM)
Change Data Capture (CDC) framework objecten in bronnen
Service KM(SKM)
Maakt data manipulation web services
![Page 14: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/14.jpg)
5HART-IT 14
ODI Data Kwaliteit overview1) Static Control op source data2) Flow Control tijdens integratieproces3) Static Control op target4) Corrigeren errors en herverwerken
1
2
3source target
ERRORS
ERRORS
TargettableSource
Table(s)Integratie
4
![Page 15: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/15.jpg)
5HART-IT 15
Standalone Agent voorbeeld
Repository
Design Navigator:
Reverse-engineerOntwikkel projectsScenario’s maken
Operator Navigator:
Sessies monitoren
Standalone Agent
Regiseert uit te voeren taken
4) Return code
Java – Ieder platform
Informatie systeem
Iedere RDBMS
Java – Ieder platform
3) Execute Job
5) Schrijf Reports
6) Monitor SessionsView reports
1) Gebruik Designer of Operatorom Job op te starten
2) Lees Session
![Page 16: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/16.jpg)
5HART-IT 16
Pauze
![Page 17: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/17.jpg)
5HART-IT 17
Handmatige overgang OWB naar ODI
Topology definiëren
Knowledge modules zoeken en aan eigen wensen aanpassen
Interfaces bouwen (= OWB mapping)
Packages bouwen (= OWB process flow)
Testen
![Page 18: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/18.jpg)
5HART-IT 18
Fasering conversietool OWB => ODI
Handmatig Converter
Fase: #dagen converter werk:
Intake ong. 5 dagen
Conversie tussen 10 en 20 dagen
Test
Hertest
![Page 19: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/19.jpg)
5HART-IT 19
Intake conversie OWB => ODI
1
![Page 20: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/20.jpg)
5HART-IT 20
Conversietool OWB => ODI
![Page 21: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/21.jpg)
5HART-IT 21
Overview conversie OWB => ODI
Input conversie• OWB repository metadata (MDL-file)• Database structuur export (exlcusief data)
Output conversie• ODI topology (XML-file)• ODI project (XML-file)
Supported versies • OWB => v9.2 (of hoger)• ODI => v10g (of hoger)
Critical Success Factors:• Project management• Communication meetings projectteam (Klant en Convertor)• Expertise ODI en DB in projectteam• Applicatie kennis vanuit projectteam
![Page 22: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/22.jpg)
5HART-IT 22
Conversietool OWB => ODI
OWBobject
ODIobject
OWB Project ODI project
OWB “data model” ODI model
OWB Module ODI Folder
OWB mapping ODI package met 1 of meer interfaces temporary interfaces
OWB process flow ODI package
![Page 23: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/23.jpg)
5HART-IT 23
Conversie staging mapping OWB => ODI
OWB mapping
- truncate/insert
ODI output
![Page 24: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/24.jpg)
5HART-IT 24
Conversie ODS mapping OWB => ODI
OWB mapping
- Merge obv NK
- lookup
ODI output
![Page 25: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/25.jpg)
5HART-IT 25
Complex voorbeeld conversie OWB => ODI
OWB multitarget mapping
ODI output • 2 interfaces
ODI ondersteund geen mutitarget
• Package
roept 2 interfaces in goede volgorde aan
![Page 26: Oracle ’ s Data Warehouse & ETL future](https://reader036.vdocuments.net/reader036/viewer/2022062309/56815932550346895dc66a62/html5/thumbnails/26.jpg)
5HART-IT 26
ODI goed leren kennen => ODI cursus vijfhartOnderwerpen:
Data Warehousing overview
ODI Architectuur
ODI Topology
Source metadata
Projects
Knowledge Modules
Staging metadata
Executen en monitoring
Data Store metadata
Data Mart metadata
Packages
Interfaces geavanceerde delen
Data Kwaliteit
Error handling en robuustheid
Change Data Capture (CDC)
ODI versie management
ODI documentatie
http://www.vijfhart.nl/oracle/opleidingen/oracle-cursus-pagina/training-id/8423/oracle-data-integrator-11g.htm
Conversie uitvoeren OWB naar ODIMail : [email protected]
ODI project starten Mail : [email protected] Tel : 06-54681902
Frank Kuijperssenior Oracle BI Consultant
Samenvatting