future strategy, direction, and roadmap of oracle’s data
TRANSCRIPT
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1
Future Strategy, Direction, and Roadmap of Oracle’s Data Integration Platform
Wankeun Lee ( 010-4167-5865)
Data Integration Solutions Specialist
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 2
Fast Action Delivers Value The Sooner You Know, The Better
Business decisions approach real-time
Time available to capture data is decreasing
Analysis of increasing data volumes = delays
Operational excellence requires immediate action
Real-time capture and action is available already
Business event
Action Time
Bu
sin
es
s V
alu
e
Data captured
Analysis completed
Action taken
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 3
Yes, in production at scale,
17.7%
Yes, in limited use,
12.2%
Yes, in pilot stage,
7.1%
Preliminary planning,
7.5% Under consideration,
10.6%
No, 35.4%
Don’t know/unsure,
9.5%
Public and Private Cloud Adoption Is Increasing
Source: IOUG ResearchWire member studies on Cloud Computing, conducted in Aug-Sept 2010 and Aug-Sept 2011
2011 Private Cloud 2011 Public Cloud
21% Use Public Clouds in 2011
50% increase from 2010
Yes, 20.9%
Under consideration,
13.6% No, 52.9%
Don’t know/
unsure, 12.6%
37% Have Private Clouds in 2011
28% increase from 2010
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 4
Ultimate Goal of Data Integration
Pervasive,
Continuous
Access to
Trusted Data
High
Performance,
Low Cost of
Ownership
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 5
Modernization MDM SOA Big Data
Oracle Data Integration Complete Offering for Enterprise Data Integration
Complete and best-of-breed
approach for enterprise data
integration
Maximum performance with
lower TCO, ease of use and
reliability
Certified for leading
technologies to deliver fast
time to value
Oracle Data Integrator
Oracle GoldenGate
Oracle Enterprise Data Quality
Oracle Data Service Integrator
OLTP
Applications
Legacy
Unstructured Oracle Customers Report:
80% lower TCO
Five times higher performance
70% reduction in development costs
Synchronization Custom BI
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 6
The Focus of Oracle’s Data Integration Strategy Five New Requirements for Data Integration
On-Premise,
On Cloud
Data
Quality
Any Data,
Any Source
Real-Time
Analytics
Zero Downtime,
Maximum
Performance
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 7
Big Data Drives Big Benefits
Transactions YOUR Data:
Decisions based on
your data
BIG Data: Decisions based on all
data relevant to you Machine-Generated
Data
Social Data
Documents
Any Data,
Any Source
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 8
Real World ROI Drivers for Big Data
Industry New Data What’s Possible Why?
Retail
One size fits all marketing
Weblog, click
streams
Micro-segmentation,
recommendations
Increase net margin by
60%
Banking
Fraud detection; risk analysis
Weblogs, transaction
systems, fraud
reports
Semantic discovery;
pattern detection
Billions of Dollars lost
in bank fraud annually
Healthcare
Improve Quality and
Efficiency
Practitioner’s notes;
machine statistics
Best practices,
reduced hospitalization
Increase industry value
by $300 B per year
Utilities
Resilient and adaptable grid
Smart meter
reading, call center
data
Real-time and
predictive utilization
analysis
Energy use expected
to grow by 22 percent
by 2030
Location-Based Services
Based on home zip code
Personal location
data
Geo-advertising, traffic,
local search, more.
Increase revenue for
providers by $100B+
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 9
Any Data,
Any Source
Oracle Strategy for Integrating Big Data
Big Data
Transformation
Bulk Data,
E-LT
OLTP OLAP Unstructured Legacy
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 10
Any Source, Any Target
IBM DB2 UDB
IBM DB2 z Series
IBM DB2 i Series
Enscribe
SQL/MP
SQL/MX
MySQL
Sybase ASE
Informix
JMS
Teradata
Netezza
Sample list of Sources and Targets
Oracle Database
Oracle Exadata
Oracle Big Data Appliance
Enterprise Data Quality
Oracle E-Business Suite
JD Edwards Enterprise One
Oracle Enterprise Service Bus
Oracle Hyperion
Oracle OLAP
Oracle PeopleSoft
Oracle Siebel CRM
Oracle Communications BRM
Hadoop
Sybase IQ
TimesTen,
PostgreSQL
Greenplum,
HP Neoview
SAS
SalesForce
SAP ERP & BW
Generic SQL
Hypersonic SQL
Microsoft SQL
Microsoft Access
Oracle BI EE
Oracle BI Apps
Linux
Sun Solaris
Windows 2000,
2003, XP
HP NonStop
HP-UX
IBM AIX
zLinux
…
New in Oracle GoldenGate 11g Release 2
IBM i Series
Postgres
MySQL v 5.5
IBM AIX v7.1
Sun Solaris v11
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 11
Real-time
Changed Data Mainframe
DB2 zOS &
DB for i
Reporting
(or OLTP
Systems)
Oracle GoldenGate
11
Leverage Your Mainframe Data
Bring mainframe data to
transactional and
analytical systems
Reduce MIPS
consumption by running
read-only queries on a
lower-cost, open system
Avoid having to upgrade
the legacy system to
maintain performance
Reduce MIPS, Avoid Future Mainframe Upgrades
Transaction
Processing
Read-only
Activity
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 12
On-Premise,
On Cloud
Integrating Cloud Applications
OLTP OLAP Unstructured Legacy
Big Data
Transformation
Real-time
Replication
Bulk Data,
E-LT
Data Quality (Product & Customer Data)
Service-Oriented Architecture, BPM
Data Services,
Federation
Cloud Application Foundation
Public
Cloud
Private
Cloud
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 13
Cloud Enablement
Seamless, Zero Downtime Migration
Continuous Availability
Certified for Oracle Exadata
Oracle Data Integration for the Private Cloud
Data-Layer Integration
Data Services for SOA/BPM
Real-Time Application Data Synchro
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 14
Migrated OLTP database workloads onto Oracle RAC running on Red Hat Linux v5.4 without interrupting business operations using Oracle GoldenGate
480 databases in 5 data centers are moving onto this architecture
Maintenance and upgrades done with near-zero downtime
Zero Downtime Operations
Customer Example: Fedex Database Consolidation to Oracle RAC Using GoldenGate
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 15
Oracle Strategy for Continuous Availability
OLTP OLAP Legacy
Oracle GoldenGate
Real-Time Replication
Eliminates planned and unplanned downtime
Bidirectional replication without distance limitation for Active-Active implementation
Supports heterogeneous databases and operating systems
Zero Downtime,
Maximum
Performance
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 16
• Storage failure
• Data recovery
• Backups
• Instance failure
• Server failure
• RAC rolling
maintenance
• Performance
scale-out
• Consolidation
• Continuous
point-in-time
recovery
• Granular repair
of logical
corruptions
• Transaction
• Table
• Database
• Database failure
• System failure
• Site failure
• Zero data loss
• Automatic database
failover
• Lost-write protection
• Database rolling
upgrade
• Offload read-only
workload and
backups
• Some migrations
• Zero downtime
upgrades and
migrations
• Online platform and
application upgrades
• Bi-directional and
multi-master replication
• Distribute read-only &
read-write workload
• An alternative to
physical replication
for site protection
• Flexible planned maintenance
and heterogeneous
migrations
Five Steps to Maximize Availability
ASM,
RMAN,
OSB
Oracle RAC
Flashback
Active
Data Guard
Oracle GoldenGate
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 17
Used GoldenGate for a phased migration
with more than 12 terabytes of data from
heterogeneous legacy systems supporting
SaaS integrated marketing offering
Gained the ability to complete the migration
in phases, enabling e-Dialog to test the new
environment over time
Reduced downtime during the massive
migration effort
Zero Downtime Migration
Customer Example: E-Dialog Business Continuity During Exadata Migration
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 18
Ensured continuous availability for the hundreds of thousands of customer transactions processed each day,
24/7 uptime for its e-commerce, risk management, investment, and payment-hub operations, as well as its business intelligence platform
Achieved greater than 99% service availability
Continuous Availability
Oracle GoldenGate Bank of Valetta
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 19
Oracle
GoldenGate
11gR2
Integrated Capture for Oracle Database
Update on Oracle GoldenGate: 11g Release 2
Enhanced Conflict Detection & Resolution
Manageability & Monitoring
Automated detection and resolution of data conflicts for active-
active or multi-master deployments
New Oracle GoldenGate Monitor product and integration with
Oracle Enterprise Manager
Moves Capture capabilities closer to the Oracle Database
Enables 100% support for compressed objects in Oracle
Database and Exadata
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 20
The Focus of Oracle’s Data Integration Strategy Five New Requirements for Data Integration
On-Premise,
On Cloud
Data
Quality
Any Data,
Any Source
Zero Downtime,
Maximum
Performance
Real-Time
Analytics
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 21
Real-Time
Analytics
Oracle Strategy for Real-Time Analytics
OLTP OLAP Unstructured Legacy
Big Data
Transformation
Real-time
Replication
Bulk Data,
E-LT
Data Quality (Product & Customer Data)
Business Analytics
Data Services,
Federation
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 22
Optimized for Oracle Engineered Systems
Optimized bulk loads, Leverages Infiniband, Smart Storage
No data transfer to/from separate ETL server
Transformations use Exadata optimized SQL
Native Batch And Real-Time integration capability
Zero down-time migrations to Exadata
Bridge for your Enterprise data and Big Data Appliance
Tightly integrated with Exadata, Exalytics
Real-Time
Analytics
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 23
ODI and Exadata – Real World Example: A Disruptive Improvement in Performance
A complex branch of the customer’s
tax allocation process runs 5 hrs 11
mins during quarter close
Exadata and ODI (E-LT) combined
is able to execute the process
42X faster (7mins 20 secs) 0
50
100
150
200
250
300
350
ODI on ExadataConventional ETL
Jo
b e
xecu
tio
n t
ime in
min
ute
s 5hrs 11mins
7min 20sec
42X
IMPROVEMENT
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 24
Integrated network operations systems to
the data warehouse to offer a global
reporting solution
Achieved 167% ROI after 9 months by
implementing an E-LT architecture
20X Performance improvement using
Oracle Data Integrator on Exadata.
The combined solution helped decrease
data preparation time for data mining from
27–29 days to 2–3 days
High-Performance E-LT for Data Warehousing
Customer Example: Turkcell Data Warehousing on Oracle Exadata
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 25
Faster Time to Insight Data Discovery using Oracle Data Integration
Faceted Data Model Enrichment
Unified
Querying
Interactive
Exploration
App
Composition
Endeca Information Discovery
Endeca Server
Oracle Data Integration
Helps organizations quickly
explore ALL relevant data
Combines structured & unstructured
data from disparate systems
Automatically organizes information
for search, discovery & analysis Oracle Data Integration
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 26
Your Data Is Changing
Companies Individuals Products Data
Quality In one hour… In one hour… In one year…
• 240 businesses will change addresses
• 150 business telephone numbers will change or be disconnected
• 20 corporations will fail
• 12 new businesses will open their doors
• 5,769 individuals in the US will change jobs
• 2,748 individuals will change address
• 515 individuals will get married
• On average 20% duplicates in product data
• 90% product introductions fail
• Retailers lost 40 billion or 3.5% of total sales lost each year due to item info inefficiencies
Faster Than You May Realize
Master data changes at rate of 2% per month
Source: D&B, US Census Bureau, US Department of Health and Human Services, Administrative Office of the US Courts,
Bureau of Labor Statistics, Gartner, A.T Kearney, GMA Invoice Accuracy Study
Compounded 27% per year 104% in three years
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 27
Your Data Contains Errors and Inconsistencies
Variation or Error
Example Variation or
Error Example
Sequence errors • Mark Douglas or Douglas Mark Transcription
mistakes • Hannah, Hamah
Involuntary corrections (자동수정)
• Browne – Brown Missing or extra
tokens
• George W Smith, George Smith,
Smith
Concatenated names
• Mary Anne, Maryanne Foreign sourced
data
• Khader AL Ghamdi, Khadir A.
AlGamdey
Nicknames and aliases
• Chris – Christine, Christopher, Tina Unpredictable
use of initials • John Alan Smith, J A Smith
Noise • Full stops, dashes, slashes, titles,
apostrophes Transposed
characters • Johnson, Jhonson
Abbreviations(약어)
• Wlm/William, Mfg/Manufacturing Localization • Stanislav Milosovich – Stan Milo
Truncations • Credit Suisse First Bost Inaccurate dates • 12/10/1915, 21/10/1951, 10121951,
00001951
Prefix/suffix errors ( 관용어)
• MacDonald/McDonald/Donald Transliteration
differences • Gang, Kang, Kwang
Spelling &
typing errors • P0rter, Beht Phonetic errors • Graeme – Graham
28
Data Governance Life Cycle
Understand
• Understand current data ‘fitness for purpose’
• Identify critical issues & quick wins for impact analysis and ROI
Improve
• Improve ROI and performance of existing applications
• Bring data to a known, baseline quality – ready to roll-out new applications and initiatives
Protect
• Continuous, consistent enforcement of standards
• High quality data drives ROI
Govern
• Monitor ongoing effectiveness
• Track and resolve issues and improve overall effectiveness
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 29
Oracle Enterprise Data Quality for Trusted Data
Easy-to-understand executive dashboards
for audit, profiling, parsing, and
standardization process results
Graphical views show data quality trends
over time
Allows continuous monitoring and quick
discovery of issues before they impact
business
Know and Manage Your Data Continuously
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 30
Created golden record for 64,000+
customers in over 150 countries
Reduced data contamination by 20% to
nearly 100% clean data
Delivered a single customer view to
2,000 information workers in 60 countries
Refreshed critical data 4x faster
EDQ standardization enabled 20%
increase in data volumes each year
without increasing resources
Data Quality for Customer Management
Customer Example: British Standards Institution Data Cleansing and Single Customer View
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 31
Putting it All Together: Oracle Data Integration
Any Data,
Any Source
On Premise,
On Cloud
Zero Downtime,
Max. Performance
Real-Time
Analytics
Data Quality OLTP OLAP Unstructured Legacy
Big Data
Connectors
Oracle
GoldenGate
Oracle Data
Integrator
Oracle Enterprise Data Quality
Oracle SOA Suite, BPM Suite, Oracle Business Analytics
Data Service
Integrator
Oracle Cloud Application Foundation
Public
Cloud
Private
Cloud
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 32
Enterprise-Class Data Integration with Oracle Multiple Use-Cases for Heterogeneous Systems
Real-Time Enterprise Data Warehouse and
Zero Downtime Migration
Real-Time Data Synchronization across Globe and Operational
Reporting
OLTP integration in Real Time, Data
Warehousing and Disaster Recovery
Data Warehousing for operations from
restocking to marketing
Real-Time Data Warehousing and Hot Standby for Disaster
Recovery
Operational Reporting for E-Business Suite
and Data Services for SOA
Data integration from Heterogeneous Sources
for SOA and BAM
Reliable Bulk Data Movement and
Transformation for High Volume, Transactions
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 33
Join the Data Integration Community
Twitter twitter.com/ORCLGoldenGate
Facebook facebook.com/OracleDataIntegration
Oracle’s Data GoldenGate YouTube channel youtube.com/oraclegoldengate
Oracle.com/goto/dataintegration
LinkedIn “Oracle Data Integration”
Oracle’s Data Integration blog blogs.oracle.com/dataintegration