1 copyright © 2012, oracle and/or its affiliates. all ... · pdf fileoracle big data...
TRANSCRIPT
![Page 1: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/1.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 1
![Page 2: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/2.jpg)
Oracle Big Data Appliance Releases 2.5 and 3.0
Ralf Lange
Global ISV & OEM Sales
![Page 3: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/3.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 4
Agenda
Quick Overview on BDA and its Positioning
Product Details and Updates
– Security and Encryption
– New Hadoop Versions and Components
– New NoSQL DB Features
![Page 4: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/4.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 5
Big Data Technology Stack
Big Data Applications
Business Analytics
Big Data Management System
Data Warehouse Data Reservoir +
Discovery Biz Intelligence +
by Industry & LoB
![Page 5: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/5.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 6
Big Data Management System
Data Warehouse Data Reservoir +
The Best of Both Worlds
![Page 6: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/6.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 7
Big Data Management System
0
1
2
3
4
5 Tooling maturity
Stringent Non-Functionals
ACID transactional requirement
Security
Variety of data formats
Data sparsity
ETL simplicity
Cost effectively store low value data
Ingestion rate
Straight Through Processing (STP)
Hadoop
Relational
The Best of Both Worlds
![Page 7: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/7.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 8
Oracle Big Data Management System
Oracle Big Data Management System
Data Warehouse Data Reservoir +
Oracle NoSQL
Database
Cloudera
Hadoop
Oracle R
Distribution
Big Data Appliance
Oracle Advanced
Analytics
Oracle
Database
Oracle Spatial
& Graph
Exadata
Oracle Big Data Connectors
Oracle Data Integrator
![Page 8: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/8.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 9
Agenda
Quick Overview on BDA and its Positioning
Product Details and Updates
– Security and Encryption
– New Hadoop Versions and Components
– New NoSQL DB Features
![Page 9: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/9.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 10
Big Data Appliance Product Family
Starter Rack is a fully cabled and
configured for growth with 6 servers
In-Rack Expansion delivers 6 server
modular expansion block
Full Rack delivers optimal blend of
capacity and expansion options
Grow by adding rack – up to 18 racks
without additional switches
![Page 10: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/10.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 11
Big Data Appliance X4-2
Sun Oracle X4-2L Servers with per server:
• 2 * 8 Core Intel Xeon E5 Processors
• 64 GB Memory
• 48TB Disk space
Integrated Software (3.0):
• Oracle Linux
• Oracle Java VM
• Cloudera Distribution of Apache Hadoop (CDH) 5.0
• Cloudera Manager 5.0 and Options
• Apache Spark
• Oracle R Distribution
• Oracle NoSQL Database
All integrated software (except NoSQL DB CE) is supported as part of Premier Support for Systems and Premier Support for
Operating Systems
![Page 11: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/11.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 12
Two Branches
BDA 2.5 (CDH 4.6)
BDA 2.6 (CDH 4.7)
BDA 3.0 (CDH 5.0)
BDA 3.1 (CDH 5.1)
BDA 4.0 (CDH 5.2)
Upgrade Points
![Page 12: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/12.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 13
Agenda
Quick Overview on BDA and its Positioning
Product Details and Updates
– Security and Encryption
– New Hadoop Versions and Components
– New NoSQL DB Features
![Page 13: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/13.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 14
Enhanced Big Data Security
Authenticate users with secure Kerberos protocol
Authorize access to data with fine grained controls
Audit activity and access with Oracle Audit Vault and Database Firewall
Encrypt data as it flows thru the system
More Info
![Page 14: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/14.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 15
First Kerberos and Sentry enabled Hadoop Appliance
Founding member of Apache Sentry bringing fine fine-grained authorization to Hadoop
Bring Oracle’s security expertise and commitment to Apache Hadoop
Kerberos and Apache Sentry
![Page 15: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/15.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 16
Oracle Audit Vault and Database Firewall
Databases Relational Data
Hadoop Non-Relational Data
Operating Systems
Audit Vault
One Consolidated, secure repository for all audit data
Centralized platform for audit reporting, alerting and policy management
![Page 16: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/16.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 17
Encryption at Rest and on the Network
Only Cloudera Hadoop appliance with Pre-Configured File System and Network Encryption
Transparent to applications and at no extra cost
Encryption at
Rest
Network
Encryption
![Page 17: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/17.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 18
Agenda
Quick Overview on BDA and its Positioning
Product Details and Updates
– Security and Encryption
– New Hadoop Versions and Components
– New NoSQL DB Features
![Page 18: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/18.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 19
Hadoop Technology Trends
Standardization of HDFS as the Big Data file system
Resource management opens doors to new processing
– YARN resource manager is GA, becoming the de facto standard
– Multiple processing frameworks can dynamically share resources
Processing frameworks move beyond MapReduce
– Avoid checkpoints to disk, cache data in memory
– More operators than map and reduce
– Contenders: Apache Spark (Databricks), Apache Tez (Hortonwork)
![Page 19: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/19.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 20
YARN Yet Another Resource Negotiator
What is it?
Benefits
Limitations
An execution-engine independent resource
management framework for Hadoop
Allows management of more than MapReduce
Management of RAM, CPU, (eventually) I/O
More complicated container model
Very few production implementations
![Page 20: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/20.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 21
MapReduce 2 MapReduce on YARN
What is it?
Benefits
Limitations
The MapReduce parallel processing framework
redesigned for use on YARN
Better support for multiple workload types
More complicated container model
Every job is a separate YARN executable
![Page 21: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/21.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 22
MapReduce 2 Launch Flow
MapReduce
Client
Resource
Manager
1. Submit MR AM Request
DataNode
And
Node Manager
Application
Master
(JobTracker)
2. Return MR Application Master
DataNode
And
Node Manager
DataNode
And
Node Manager
DataNode
And
Node Manager
DataNode
And
Node Manager
TaskTracker TaskTracker TaskTracker TaskTracker TaskTracker
3. Spawn Application Containers
4. Submit MR Job
![Page 22: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/22.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 23
Spark
What is it?
Benefits
Limitations
Rich Parallel-Processing with In-Memory Execution
A storage-independent, parallel-processing
framework for Big Data
Enables real-time streaming workloads
Much faster than MapReduce
Much more expressive than MapReduce
Very newly supported
Reliance on less common language (Scala)
Yet another framework to learn
![Page 23: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/23.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 24
In-Memory MapReduce
Spark is not “just” MapReduce with Caching
– Adopts and extends the “map/reduce” paradigm
Spark provides
– Fast, rich parallel –processing framework
– Interactive shells (Scala and Python)
– Real-time Streaming capabilities (Java, Scala)
– Machine Learning and Graph libraries out-of-the-box
Is Spark anything new?
![Page 24: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/24.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 26
What is Where?
BDA 2.x
Supports CDH 4.6
Delivered all security and encryption features
BDA 3.0
Builds on the features in 2.5 and
Delivers the new Hadoop components
BDA 3.0 - Available April 22nd
![Page 25: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/25.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 27
Agenda
Quick Overview on BDA and its Positioning
Product Details and Updates
– Security and Encryption
– New Hadoop Versions and Components
– New NoSQL DB Features
![Page 26: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/26.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 28
Oracle NoSQL DB Release 3.0 Enterprise Ready
Ease of Adoption
Security
Business Continuity
![Page 27: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/27.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 29
![Page 28: 1 Copyright © 2012, Oracle and/or its affiliates. All ... · PDF fileOracle Big Data Appliance Releases 2.5 and 3.0 Ralf Lange Global ISV & OEM Sales](https://reader031.vdocuments.net/reader031/viewer/2022030414/5aa06fe77f8b9a62178e1bd4/html5/thumbnails/28.jpg)
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. 30