Oracle Big Data - Overview

Hariharaputhran VaithinathanDirector of MembershipAIOUG

Sai Janakiram PenumuruVice PresidentAIOUG

2

Introduction

Sai Janakiram Penumuruo Fourteen years of experience - Oracle DBA / Oracle Apps DBA / Cloud Technologieso Lead Cloud Architect, AT2C Expert - HPEo Co-Fonder, Vice President, Director of Finance - All India Oracle Users Group (AIOUG)o Oracle ACE Director & Member of TOSCA - Topology and Orchestration Specification for Cloud Applicationso Oracle VM SIG Leader www.oraclevmsig.orgo Blog: www.oadba.com; www.oracle12c.infoo Contacts – ps.janakiram@gmail.com ; twitter - @sai_penumuru

3

V.Hariharaputhrano Twelve years in Oracle Development / DBA / Big Data / Cloud Technologieso All India Oracle Users Group (AIOUG) Evangelist o Leading AIOUG Chennai Chaptero Blog: www.puthranv.com

4

Plan of action for Next

Overview of Oracle Big Data

To make you an Big Data Expert

Minutes45

Agenda• Data Growth and A New Style of IT

• Defining Big Data: Market Drivers and Trends

• Why Big Data Now?

• Overview of Oracle Big Data Appliance

• Traditional vs Modern Systems

• Hadoop Overview

• HDFS Storage

• Map Reduce Processing

• Oracle Hadoop Integration

6

Internet of People to Internet of Things

QUALITY & CONSISTENCY MAINTAIN & REPAIRSMART SHOPPING MONITOR POLLUTION LEVELS

WILDLIFE PROTECTION FARMING ENERGY

Devices TALK to each other as they become SMART & generate DATA

7

A Real World Example:- Big Data- Micro-transactionsSensor data collected from US commercial jet engines during 1 year

1,041,600,500 TB

20 TB20 terabytes of information per engine every hour

2.5Average duration for US flights in hours

2twin-engine Boeing 737

days in a year

36528,537# of commercial flights in the sky in the United States on any given day

8

1030

This will take us beyond our decimal system

Geopbyte

This will be our digital universe tomorrow…

Brontobyte 1027

1024This is our digital universe today

= 250 trillion of DVDs

Yottabyte 10

21

1.3 ZB of network traffic by 2016

Zettabyte

Information from the Internet of Things: We have gone beyond the decimal system

Today data scientist uses

Yottabytes to

describe how much government data the NSA or FBI have on people altogether.

In the near future,

Brontobyte will be the

measurement to describethe type of sensor data that will be generated from the IoT

1018

1 EB of data is created on the internet each day = 250 million DVDs worth of information. The proposed Square Kilometer

Array telescope will generated an EB of data per day

Exabyte

1012

Terabyte500TB of new data per day are ingested in Facebook databases

1015

PetabyteThe CERN Large Hadron Collider generates 1PB per second

109

Gigabyte106

Megabyte

9

695,000 status updates

98,000+ tweets

698,445 Google searches

1,820TB of data created

11million instant messages

168 million+ emails sent

217 new mobile web users

Growing Internet of Things (IoT)

Pervasive Connectivity

Explosion of Information

Smart Device Expansion

Every 60 seconds

2013

A new style of IT required for IoT solutions

By 2020

40 Trillion GB(2)

10 Million(3)

… for 8 Billion(4)

(1) IDC Directions 2013: Why the Datacenter of the Future Will Leverage a Converged Infrastructure, March 2013, Matt Eastwood ; (2) & (3) IDC Predictions 2012: Competing for 2020, Document 231720, December 2011, Frank Gens; (4) http://en.wikipedia.org

A new era of accelerated innovationForever changing how consumers and businesses interact, enabling new opportunities

30 Billion(1)

DATA

Mobile Apps

Devices

10

What is Big Data? : Regular structured data

Over a longer periodFaster analysis

11

What is Big Data? : Meaning from Unstructured Data

Social media

Images Video Audio

Email Documents

Defining Big Data: Market Drivers and Trends

13

Big Data

Variety

Velocity

Volume

Big data is derived from a variety of sources

14

An intelligent end-to-end approach delivers the right information to the right person at the right time

A day in the life of Big Data

ExecutiveDashboards

Enterprise Search

Customer Interaction

PredictiveAnalytics

WebEngagement

Transactional Operational Strategic

Volume

Variety

Velocity

CRM(Sales)

Web ERP(Procurement)

Supply Chain(Ops)

Machine Generated Data

HR

Social Media Video Audio Email Texts TransactionalData

Word, Excel Logs Clickstream Data

ImagesMGD

Why Big Data Now?

16

How to make Smarter Decisions & Better Predictions

GUT

INTUTIONDATAGUESSBIG

17

Better,Right Time Decisions

Right information. Right person. Right time.

Command of information drives increased business performance

Foresight –What will happen?

Hindsight –what happened?

Insight –

What is happening now?

In Flightinformation

Activeinformation

Inactiveinformation

Right Time Decisions

18

Big Data Use Cases

Today’s Challenge New Data What’s Possible

HealthcareExpensive office visits

Remote patient monitoringPreventive care, reduced

hospitalization

ManufacturingIn-person support

Product sensors Automated diagnosis, support

Location-Based ServicesBased on home zip code

Real time location dataGeo-advertising, traffic, local

search

Public SectorStandardized services

Citizen surveysTailored services,

cost reductions

RetailOne size fits all marketing

Social media Sentiment analysis segmentation

19

Innovative analytic use cases are cutting across structured, unstructured and semi structured data

Big Data opportunities across industries and use cases

Government Telecom Manufacturing Healthcare

• Sentiment analysis

• Social CRM / network analysis

• Churn mitigation

• Brand monitoring

• Cross and Up sell

• Loyalty & promotion analysis

• Web application optimization

• Marketing campaign optimization

• Brand management

• Social media analytics

• Pricing optimization

• Internal risk assessment

• Customer behavior analysis

• Revenue assurance

• Logistics optimization

• Clickstream analysis

• Influencer analysis

• IT infrastructure analysis

• Legal discovery

• Equipment monitoring

• Enterprise search

• Drug development

• Scientific research

• Evidence based medicine

• Healthcare outcomes analysis

• Supply chain optimization

• Defect tracking

• RFID Correlation

• Warranty management

• Broadcast monitoring

• Churn prevention

• Advertising optimization

• Law enforcement

• Counter terrorism

• Traffic flow optimization

Horizontal use cases

Sources: IDC: 2012 “Worldwide Big Data Technology and Services Forecast: 2011-2015, Gartner: 2012 “Big Data Drives Rapid Changes in Infrastructure and $232 Billion in IT Spending Through 2016

Finance

• Fraud detection

• Anti-money laundering

• Risk management

Energy

• Weather forecasting

• Natural resource exploration

Overview ofOracle Big Data Appliance

21

Oracle Big Data Appliance

Sources: http://www.oracle.com/technetwork/database/bigdata-appliance/overview/bigdataappliance-datasheet-1883358.pdf

22

Prepare your host system

Oracle Big Data Lite Virtual Machine

To get started:• Download and install Oracle VM VirtualBox and 7-zip

• Download each of the 7-zip files

• Run the 7-zip extractor on the BigDataLite421.7z.001 file only. This will create the BigDataLite-421.ova VirtualBox appliance file

• In VirtualBox, import BigDataLite-421.ova

• Start BigDataLite-4.2.1

• Log in as oracle/welcome1

Version 4.2.1

Technical Requirements:

• Dedicate 2 cores

• 5 GB memory

• 50GB disk space to the virtual machine

• Install will require ~53 GB disk space including temporary files

23

http://www.oracle.com/technetwork/database/bigdata-appliance/oracle-bigdatalite-2104726.html

Download Oracle Big Data Lite Virtual Machine

File Description

Deployment Guide •Start Here!Deployment Guide provides step-by-step instructions for download and deployment.

BigDataLite421.7z.001 (2147483648 bytes)BigDataLite421.7z.002 (2147483648 bytes)BigDataLite421.7z.003 (2147483648 bytes)BigDataLite421.7z.004 (2147483648 bytes)BigDataLite421.7z.005 (2147483648 bytes)BigDataLite421.7z.006 (2147483648 bytes)BigDataLite421.7z.007 (2147483648 bytes)BigDataLite421.7z.008 (2147483648bytes)BigDataLite421.7z.009 (2147483648 bytes)BigDataLite421.7z.010 (2147483648 bytes)BigDataLite421.7z.011 (2147483648 bytes)BigDataLite421.7z.012 (2147483648 bytes)BigDataLite421.7z.013 (147967130 bytes)

md5sum.txt (346 bytes)

To get started:•Download and install Oracle VM VirtualBox and 7-zip•Download each of the 7-zip files•Run the 7-zip extractor on the BigDataLite421.7z.001 file only. This will create the BigDataLite-4.21.ova VirtualBox appliance file•In VirtualBox, import BigDataLite-421.ova•Start BigDataLite-4.2.1•Log in as oracle/welcome1See the Deployment Guide for details.

Cloudera JDBC Drivers Download and install the Cloudera JDBC drivers to enable Oracle SQL Developer and Data Modeler to connect to Hive.

Oracle Enterprise Linux 6.6Oracle Database 12c Release 1 Enterprise Edition (12.1.0.2) - including Oracle Big Data SQL-enabled external tables, Oracle Multitenant, Oracle Advanced Analytics, Oracle OLAP, Oracle Partitioning, Oracle Spatial and Graph, and more.Cloudera Distribution including Apache Hadoop (CDH5.4.0)Cloudera Manager (5.4.0)Oracle Big Data Discovery 1.1Oracle Big Data Connectors 4.2

Oracle SQL Connector for HDFS 3.3.0Oracle Loader for Hadoop 3.4.0Oracle Data Integrator 12cOracle R Advanced Analytics for Hadoop 2.5.0Oracle XQuery for Hadoop 4.2.0

Oracle NoSQL Database Enterprise Edition 12cR1 (3.3.4)Oracle Big Data Spatial and Graph 1.0Oracle JDeveloper 12c (12.1.3)Oracle SQL Developer and Data Modeler 4.1Oracle Data Integrator 12cR1 (12.1.3.0.1)Oracle GoldenGate 12cOracle R Distribution 3.1.1Oracle Perfect Balance 2.4.0Oracle CopyToBDA 2.0

Version 4.2.1

Thank you