oracle warehouse builder to oracle data integrator 12c migration utility
TRANSCRIPT
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle Warehouse Builder to Oracle Data Integrator 12c Migrations
Noel Sidebotham – Oracle Data IntegrationHolger Dresing – Oracle Data IntegrationAshish Kashalkar – Oracle ConsultingAugust, 2015
Oracle Confidential – Internal/Restricted/Highly Restricted
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Oracle Confidential – Internal/Restricted/Highly Restricted 2
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |3
Agenda
• ODI 12c - Introduction
• Migration Utility Overview
• Migration Factory Services
• Demonstration
• Q&A
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle Data IntegrationIntroduction
Oracle Confidential – Internal/Restricted/Highly Restricted 4
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle Data IntegratorBulk Data Processing and Data Transformation
Oracle Data Integrator
OLTP
Applications
Legacy
Unstructured
Certified for leading technologies to deliver fast time to value
High-performance, low cost of ownership E-LT architecture
Lightweight deployment
Flexible, easy to enrich functionality
High Performance E-LT
Declarative Design
CEP Data Services
Extensible Knowledge Modules
Capable of Integrating Structured & Unstructured Data
Copyright © 2014 Oracle and/or its affiliates. All rights reserved.
Big Data: Hive, HBase, Sqoop
Best for Oracle: Merge, Spatial, Multi-Table Insert, Optimizer Hints and more
Real-Time: CDC with GoldenGate
Applications: E-Business Suite, Siebel, PeopleSoft, JD Edwards Enterprise One, JD Edwards World, SAP ERP and SAP BW
Heterogeneous: Optimizations for all major RDBMS: IBM DB2, Microsoft SQL Server, Teradata, Netezza, etc.
Applications
Legacy
Databases
Big Data
Cloud
Extended ConnectivityLeverages Existing IT, Faster Implementation
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Reverse Engineer Metadata
Journalize (CDC)
Load from Source to Staging
Check Constraints
Integrate, Transform
Data Service
Knowledge ModulesSimpler Physical Design and Shorter Implementation Time
9
Oracle Sqoop Hive HBaseOracle Merge
SAP ERP
SAP BWOracle
DatapumpOracle DBLink
JMSExternal Tables
Teradata
Oracle Spatial
SiebeleBusines
SuiteIBM DB2 Netezza SCD
Key Architecture Benefits Faster development and simpler
maintenance using templates Easy to extend and add new best practices Enforces predictability and reduces cost of
ownership
Pluggable Knowledge Modules Architecture
Sample out-of-the-box Knowledge Modules
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Key Architecture Benefits: 100% Java, Open APIs, fast E-LT
DA
B
FileC
C$_0
C$_1
LKM
LKM
IKM
I$ E$ (Errors)
CKMIKMRKM
JKM
Check-LoadTransformExtract-Load
ODI Agent
Application & SOA Business Intelligence& Data Warehouse
ODI Agent may be deployed in any part of the architecture
How ODI Works: E-LT ArchitectureHigh Performance, Flexible, Lightweight Architecture
10
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Declarative Flow-Based User Interface
• Reusable Mappings
• Step By Step Debugger
Improved User Experience and Productivity
• Parallel Target Table Load
• In-Session Parallelism
• Low Overhead Session
Core Features for Better Performance
• Integration with Common Admin Model
• Integration with OPSS Enterprise RolesEnterprise Deployment
• Integration with Enterprise Manager Cloud ControlImproved Manageability
• Integration with Oracle Warehouse Builder
• Integration with Oracle GoldenGate
Deeper Oracle Stack Integration
New Features in Oracle Data Integrator 12c
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Quick Overview of ODI StudioUI Overview
Object Tree
Source and Target Area
Navigators
Accordions
Toolbar
Components
Expression
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
A mapping can contain multiple targets
Any component can map result to multiple downstream components
Split component allows to split input logically based on expressions
Multiple Targets
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
• Capture common logic (sources, targets, transformations)
• Reuse logic in mappings
Reusable Mappings
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
Provide control to the user on the Session Execution Flow
Pause, Resume, Run to next Step, etc.
Define breakpoints on Steps in Packages
Toggle breakpoints in Sessions
Query data through Agent (gives access to uncommitted data)
Step By Step Debugger
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle Warehouse Builder to Oracle Data IntegratorMigration ToolIntroduction
Oracle Confidential – Internal/Restricted/Highly Restricted 16
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |17
Remarks
Oracle Data Integrator is the strategic go-forward ETL platform from Oracle
Statement of Direction: http://www.oracle.com/us/products/middleware/data-integration/odi-statement-of-direction-1922235.pdf
Existing OWB customers are NOT being forced to migrate to ODI
Oracle offers “Lifetime Support” option for Sustaining Support on all OWB features
Oracle Data Integrator 12c supports a phased migration approach
Execute and monitor existing OWB 11gR2 jobs from ODI 12c
Use the migration utility to convert existing OWB 11gR2 objects into their ODI equivalents
Start new developments using ODI 12c
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |18
OWB Support Policies
Release GA Date
Premier
Support Ends
Extended
Support Ends
Sustaining
Support Ends
OWB 9.2 Jul 2003 Oct 2007 Not Available Oct 2010
OWB 10.1 Apr 2004 Jul 2007 Not Available Jul 2010
OWB 10.2 Jul 2005 Jul 2010 Jul 2013 Indefinite
OWB 11.1 Aug 2007 Aug 2012 Aug 2015 Indefinite
OWB 11.2 Sep 2009 Jan 2015 Jan 2018 Indefinite
For more Information about Support Levels:http://www.oracle.com/us/support/lifetime-support/index.html
Customer Downloadable OWB Support Policies:http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf
Lifetime Sustaining Support for OWB Customers
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Goal Provide a migration path to OWB users Integrate OWB and ODI runtime for
phased migration
How OWB repositories can now be managed in
Topology OWB Jobs can be executed and monitored
in ODI OWB to ODI migration utility
Execute and monitor existing OWB 11gR2 jobs from ODI 12c
Oracle Warehouse Builder Integration
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |20
Migration Utility Overview
Versions supported
OWB 11.2.0.4 or 11.2.0.3 to ODI 12c
Requires an ODI-EE license
Design-time metadata migration only
Data Objects
Mappings
Separate command line utility
Uses OWB SDK and ODI SDK
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |21
Migration Scope Details
Module and Location
Data Objects
Table (Columns, Keys, Indexes)
View (Columns, Keys)
Materialized View (Columns, Keys, Indexes)
External Table (Columns)
File (Records, Fields)
Sequence
Supported OWB Design Time Metadata
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |22
Mappings
Classic Mappings, Code Template Mappings, Pluggable Mappings
Most Mapping Operators
Aggregator, Constant, Deduplicator
Expression, External Table, Filter
Flat File, Joiner, Key Lookup
Mapping Input Parameter, Materialized View, Pivot
Pre/Post Mapping Process, Sequence, Set
Sorter, Splitter, Subquery
Table, Table Function, Transformation
Unpivot, View
Migration Scope Details
Supported OWB Design Time Metadata
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |23
Migration Scope Details
Process Flows
Dimensional modeling metadata
Oracle Discoverer metadata and derived OBIEE metadata
Data Quality components
Custom PL/SQL
OMB*Plus scripts
OWB Experts
Unsupported OWB Design Time Metadata (sample)
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |24
ODI Related Enhancements
New components developed for ODI 12c (12.1.3)
Pivot
Unpivot
Table Function
Subquery
Enhancements to existing ODI Objects
Variable
Sequence (support for CURRVAL)
Lookup
Oracle Knowledge Modules
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |25
Migration Utility Syntax and Options
Command:
>$OWBHOME/bin/unix/migration.sh <odi_master_repos_pwd> <odi_user_pwd> <owb_wksp_owner_pwd> <config_file>
Configuration File
Connection Information to OWB and ODI
OWB design metadata to migrate
Migration options
3 running modes
Fast Check, Dry Run, Run
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |26
How to Get the Migration Utility?
Apply the following patches
OWB: #17830453
ODI: #17053768
Both are available on My Oracle Support
Supported Platforms: Windows 64-bit and Linux 64-bit
Supported Versions: OWB 11.2.0.3+ and ODI 12.1.2.0.0+
Migration documentation included in ODI documentation: http://docs.oracle.com/middleware/1212/odi/ODIMG/index.html
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |27
Migration Screenshots
Customers Mapping in OWBCustomers Mapping in ODI
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |28
Migration Screenshots
Customer Sales Mapping in ODICustomer Sales Mapping in OWB
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |29
Customer Experiences
• Average 80% of the mappings can be migrated automatically: customer with OWB in-database ETL up to 95% .
• Performance of the migrated SQL statements is comparable to the OWB processes. You have to keep in mind, that OWB generated long complex SQL-Statements with inline queries. ODI generated shorter SQL Statements, sometimes it generates more than 1 SQL Statements.
• The development of ODI mappings is faster than development of OWB mappings, about 30 %, reason is the use of the knowledge modules.
• ODI12c allowed us to maintain investment in OWB to execute non-migrated legacy code, whilst allowing us to use the new fantastic platform support like Hadoop.
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |30
Customer Examples
• International service provider in the area of:
• Supply Chain & Manufacturing,
• Solutions are based on Oracle technology
• Oracle Business Intelligence Enterprise Edition
• Endeca
• Oracle Data Integration Suite
• E-Business Suite
• Benefits
• Development time needed for EDW mappings decreased
by half
BI Infrastructure:
Enterprise dataware-
houseEDW
(Data Vault)
Analyticscubes
Reports
Statistics
Dashboards
Data-mart
DM
Staging
(pull)STG
SelfServicequeries
Pre-staging(push)
PRE
Process administration ADM
Data Sources
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Oracle ConsultingMigration Factory ServicesOWB to ODI 12c
Oracle Confidential – Internal/Restricted/Highly Restricted 31
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Op
erate
/ Main
tain
Migration Factory Approach
Dis
cove
r
Programme Governance
Platform Delivery
Tran
siti
on
Mig
rate
An
alyz
eto
to
to
Stre
am G
ove
rnan
ce
Further Migration Types
OWB ODI
BO OBIEE
SAP BW Oracle BI
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
OWB –ODI Migration: List of steps
Step DescriptionVerify your system environment
Before running the migration utility, verify that your system meets
requirements.
Review supported and unsupported objects
Make sure you understand what will and will not be migrated.
Edit the migration utility configuration file
Edit the migration utility configuration file and make sure the settings are
correct for your environment.
The configuration file contains connection information and other details
required for migration.
Run the migration utility
Run the migration utility to migrate OWB objects to ODI using the settings in
the migration utility configuration file.
Review the migration utility log file
After migration is complete, review the migration utility log file.
The file contains details about objects that were migrated, and error messages
if errors occurred.
Review the migration utility exclusion report
After migration is complete, review the migration utility exclusion report.
The report provides a summary of objects that were migrated, and lists
whether migration succeeded or failed for each object.
Update manually and migrate objects in Exclusion report
Manually update the interfaces to add functionality of objects that are not
covered by Utility
Verify your migration In ODI Studio, connect to your ODI environment and perform post-migration
testing to verify your migration.
Verify your system environment
Review supported and unsupported
objects
Edit the migration utility configuration
file
Run the migration utility
Review the migration utility log file
Review the migration utility exclusion
report. Manually fix.
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
Questions & Answers
Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Internal/Restricted/Highly Restricted 35