The AMIS Team

Oracle OpenWorld 2016, Nieuwegein, 13th October 2016

Oracle OpenWorld 2016 ReviewData - Database Development, BigData, BI

2

Data

• ACID is expensive• OLTP is a niche• Moving data takes long – bring processing to data• SQL is omnipresent – expose all data in an SQL friendly way

– Including NoSQL and data on Hadoop• Data from the past should be able to help us predict the future

– Bring on machine learning (aka AI aka predictive analytics)– Aided by the citizen data scientist in (Big) Data Discovery

• Fast Data (big data at high velocity) should be handled in real time– Enter: Streaming Analytics & Apache Kafka

• Oh and ehm …. a next major release of Oracle Database is available– 12cR2 – only on the cloud for now– Highlights: Sharding, More PDB (“virtual database”),

Approximate Query Processing, Leverage In-Memory even more, JSON document generation and faster JSON processing, Analytic Views

3

Learning

• How do we learn?– Try something (else) => get feedback => learn

• Eventually:– We get it (understanding) so we can predict the outcome

of a certain action in a new situation– Or we have experienced enough situations to predict

the outcome in most situations with high confidence• Through interpolation, extrapolation, etc.

– We remain clueless

4

Machine Learning

• Analyze Historical Data (input and result – training set) to discover Patterns & Models

• Iteratively apply Models to [additional] Input (test set) and compare model outcome with known actual result to improve the model

• Use Model to predict outcome for entirely new data

5

Data Discovery

A B C D E F G1104534 ZTR 0.1 anijs 2 36 T

631148 ESE 132 rivier 0 21 S

-3 WGN 71 appel 0 1 -

1262300 ZTR 56 zes 2 41 T

315529 HVN 1290 hamer 0 11 -

788914 ASM 676 zwaluw 0 26 T

157762 HVN 9482 wie 0 6 -

946681 DHG 42 rond 1 31 T

-31539 WGN 2423 bruin 0 0 -

47338 HVN 54 hamer 0 16 P

6

Scatter PlotAttribute F (Y-axis)vs Attribute A

-500000 0 500000 1000000 150000005

1015202530354045

Y-Values

Y-Values

7

Scatter PlotAttribute F (Y-axis)vs Attribute A

1960 1970 1980 1990 2000 2010 202005

1015202530354045

Age of Lucas Jellema vs Year

Y-Values

8

Data Discovery

Time City - - #Kids Age Level of Education

1104534 ZTR 0.1 anijs 2 36 T

631148 ESE 132 rivier 0 21 S

-3 WGN 71 appel 0 1 -

1262300 ZTR 56 zes 2 41 T

315529 HVN 1290 hamer 0 11 -

788914 ASM 676 zwaluw 0 26 T

157762 HVN 9482 wie 0 6 -

946681 DHG 42 rond 1 31 T

-31539 WGN 2423 bruin 0 0 -

47338 HVN 54 hamer 0 16 P

9

Machine Learning, Data Mining & Predictive Analytics

10

Recent developments

• More compute capacity, smarter algorithms and better analytical tooling– Evolving Machine Learning – Smart text and speech analysis (NLP, ESA)– Real time predictions become a reality– Streaming (event) Analytics– Visualization– Citizen Data Scientist– SQL against Big Data

• More data available & accessible (IoT, Social, Media, IT operations,business processes,…)

• Better/larger/cheaper/faster data storage capabilities

11

Many cloud services around Big Data & Analytics

12

Big Data Integration Reference Architecture

IngestPrepare

Transform, Merge, Enrich

Save

GovernGovern

Serve

Analyze & Act

Present, Leverage & ‘Action’

Extract

Explore

Purge

13

Data Integration Platform

14

Mapping Oracle portfolio to Reference Architecture

Big Data Discovery

Data Visualization

BI CS

IT Analytics

Security Analytics

Log Analytics

15

Tip: OEMM - Oracle Enterprise Metadata Management

16

Citizen Data Scientist

• Data Visualization CS

• Big Data Preparation CS• Big Data Discovery CS

• Machine Learning CS

17

Oracle Machine Learning Cloud Service

18

19

20

https://www.zeppelinhub.com/

21

Relational Data& friends

22

Traditional approach

• All enterprise data is in the Oracle [relational] Database– Except very unstructured documents - and sometimes even those

23

Center of the Data Universe is shifting

24

Variety in data – Data Tiering

• How long relevant (hot vs cold vs dead)?• How fine grained and how accurate?• What is it used for?

– By whom, where, in what way, using which tools• What format is it in/should it be in?• How to be processed?• What confidentiality & integrity is required?• How much of it?

25

Trends around data storage and data processing

• Take processing to data [to reduce data movement]– Exadata SmartScan in Storage Cells (SQL & R processing)– Hadoop MapReduce/Spark– Coherence Processors– Streaming Analytics– Microservices, stand alone data domains

• Distributed data partitions – for scalability and parallelization [and fault tolerance when also replicated]: – Shards (Oracle Database 12cR2) and Partitioned External Tables– TimesTen Velocity Scale – distributed In-Memory OLTP– Hadoop HDFS, Apache Kafka

• New paradigms regarding transactional data– CQRS (for example Oracle Database In Memory (read) / In Flash/On disk

(read/write), Write behind cache)– Event Sourcing, Transaction Log

26

Oracle Database

• How much of your data– Arrives through (business) transactions that require true ACID?– Is involved in current business operations?– Will ever be updated [again]?– Plays a direct role in integrity [of other records]?– Is actively accessed [on a regular basis] ?– Really has to be in the OLTP engine?

• How much of the data currently in your OLTP engine could be off-loaded– If that data remains accessible through SQL (even from within the OLTP engine,

without altering existing queries) with reasonable response times• What if such off-loading

– Improves performance of the OLTP engine for transactions– Shortens batch jobs [by engaging distributed, scale out processing options]– Opens up possibilities for advanced analytics– Potentially lowers the cost [licenses & specialized hardware] for the OLTP engine– Introduces some change and complexity

27

Oracle Big Data SQL

• Big Data SQL: A ‘franchised query engine,’ enables scalable, integrated access in situ to the entire Big Data Management System (BDMS)– Meta data, Query execution, Workload Management, Data Optimization– Primary role for Oracle Database – foundation for BDMS

See Statement of Direction: http://www.oracle.com/technetwork/database/bigdata-appliance/overview/sod-bdms-2015-04-final-2516729.pdf

28

Oracle Database Development

29

Oracle Forms

• Release 12.2.1.1 is available– 12.2.2 (or 12.2.1.2) is planned for late 2016– Support for Forms 12c: Premier Oct 2020, Extended 2023 – (and moving)

• Forms usually runs in browser – using the Java JRE plugin for Applets– Modern browsers have stopped or will stop supporting the Java plugin– Forms will either have to run on outdated browsers (IE, old versions of Firefox or

Chrome) or run outside the browser– The main changes around Forms are around running Forms outside the browser – as

standalone Java Web Start (jnlp) application– Also: Forms Helper – script for customizing post-install environment (simplified

WLST)• On Reports:

– Reports 12c exists – it is the terminal release– From now on, reporting should be done using BI Publisher– BI Publisher has become part of the Developer Suite and will be included in the

WebLogic Suite

30

Forms in the Cloud

31

APEX – 5.1

• Interactive Grid – A new rich, client-side region type that allows editing multiple rows of data in a dynamic, JSON-enabled grid, and supports multiple grids on a single page.

• Master / Detail / Detail – Provide a wizard interface to define declarative master/detail/detail regions.

• New Charting Engine – Include a new JavaScript (Oracle JET) based charting engine developed by Oracle which produces responsive and accessible HTML5 charts.– AnyChart is on the way out

• Ability to have multiple tabs open to the same APEX application and isolate session state 

• Improved Wizards - fewer steps and more attributes set by default.• Declarative RTL Support –declarative methods to control user interface direction-

support for both Left-to-Right and Right-To-Left languages.• Packaged Applications – Improved framework and enhancements to the

packaged applications.• Status: EA 2 is available (hosted) as of September 2016

– APEX 5.1 Production – early 2017?

32

New in SQL in 12cR2

• Listagg improvements• Error handling for CAST function & new Validate_Conversion function• Materialized Views

– Real Time Materialized Views (stale plus logs)– Statement Level Refresh

• AL32UTF8 As the Default Database Character Set• New capabilities for generating JSON documents directly from SQL

queries, improved JSON support in In Memory processing• Beyond 12cR2• Approximate Query Processing (using HyperLogLog)• Analytical Views• Band Join- better performance for non-equijoins• Temporary, cached in memory tables for duration of cursor• Partitioned External Tables

33

New in PL/SQL in 12cR2

• Deprecated procedures and functions• Accessible by at procedure or function level• JSON support: generation of JSON documents using PL/SQL API and

Oracle supplied Object Types (somewhat akin to XMLType)– JSON SQL functions available in PL/SQL expressions

• Supplied package dbms_plsql_code_coverage to identify code units not touched in specific [test] scenarios

• PL/Scope enhancements – more fine grained reporting

• Edition Based Redefinition does ‘garbage collection’ –editioned objects no longer in use are cleaned up

34

Other Database Development News

• JS Stored Procedures• SQL Developer GUI Debugger

– One session can have another start debugging– At breakpoint: execute SQL to inspect run context – including PL/SQL state

• SQLcl

• ORDS – Oracle REST Data Services

35

36

Summary of Oracle OpenWorld 2016

• (5 days filled to brim with 1800+ sessions, 12 keynotes, 150+ demo booths, hundreds of vendors and quite a few rumors & hallway tales)

• Infrastructure [as a Service]– Generation 2 Data Center– Network & IOPS (storage, NVMe, Flash)– Exadata SL

• Abdication of the single, central, enterprise Oracle RDBMS – and the limelight for data– PDBs – Sharding– Hadoop & Spark (& SQL & R)– Machine Learning

• Adoption of open source projects, industry trends & community darlings– Node.js, Docker, Microservices, Git(Hub), Python, Slack, …

37

Summary of Oracle OpenWorld 2016 (2)

• Cloud [First] strategy– Migrate & Extend i/o [bidirectional] Lift & Shift– Cloud@Customer– Status and future of On Premises software (and yet Engineered Systems)– Ops in Oracle Data Centers– Subscription Models, Suites (i/o a la carte)– How fast can Oracle move [without spreading itself too thin]?

• SaaS and [Unlimited] Applications– SaaS portfolio quite extensive– UX is important asset of the SaaS applications– Real cloud elements are improving (APIs, extensibility)– Traditional Apps are still evolving [as promised] – and seem to benefit from SaaS and

technological advances across the board• Oracle Public Cloud consistency, architecture and the Dogfood Doctrine

– Fabric and foundational components– Designated capabilities and mutual integration

38

Oracle OpenWorld 2016 Tag Cloud