Burak AkkusExpert Bigdata and Database Architect

First experience with ADWC in a missioncritical DW environment

Agenda

• About Me• Turkcell Overview• Turkcell DWH • Exadata Journey• Projects• ADWC experience• Q&A

About Me

• Computer Engineer• 12+ IT Experience• Database and Bigdata Expert• Sailor• Blogger

Turkcell Group Overview

Most admired company

Most valuable brand in Turkey

Listed on NYSE and BIST

Market Cap: $7.12 billion

2018 Revenue: $3.7 billion

50 M subscribers in 9 countries

Leading operator in 5 countries

3rd biggest in Europe

Turkey

Kazakhistan

Azerbaijan

Georgia

TRNC

Belarus

Ukraine

Moldovia

DWH Environment in Turkcell

• 405 Oracle Database• 120 Production ( 92 RAC )• 105 Physical Standby - Ankara• 200 Test&Dev DB

• 13 PB Total Database Size• 350 Physical Server• 14+ Exadata• Oracle Bigdata Applicance• Oracle Bigdata Sql• Over 6 PB Total Hadoop Size

Turkcell DWH DOMAIN

ReportingMSTR

ETLAB Initio

Oracle ODIGolden Gate

Hadoop

SourceDBs

Exadata

OtherDWH DBs

• 5 Billion CDR per Day• Appr. 5 TB raw data extracted from 20+ source databases

and Hadoop• 20 TB Data on file system processed• 10-12 TB loaded into databases, all into Exadata• Appr. 2 PB Customer data stored in multiple Databases

Amount of Data

Turkcell DWH Domain II

• ETL Environment✓ AB Initio – runs on 25 Node Linux Cluster✓ Data loaded daily between 19:00 - 08:00 (SLA) ✓ Sql*Loader used to load tables with 96 parallel

threads✓ Implemented GoldenGate✓ Reporting Environment✓ MSTR (Microstrategy) is used mostly.✓ 4000+ Users, 80 power Users✓ 5000 distinct reports with 50k run per month✓ Runs between 08:00 – 20:00 (SLA)

Application Footprint

Oracle Exadata Journey I

DWH

RDWH

NRDSDB

RepDB

KPIDB

NDMDB

Arc DWH

2012 2013 2014 2015 2016

ExadataX2-22 Rack50 TB

X3-22 Rack80 TB

ExadataV21 Rack50 TB

HP-UX Superdome40TB

12 TBHP servers

10 TB

X2-22 Rack55 TB

V21 Rack20 TB

X4-22 Rack110 TB

X2-22 Rack70 TBV

X3-22 RackSTBY

V21 Rack25 TB

X5-22 Rack150 TB

X4-22 Rack90 TB

X3-2+X5-23 RackSTBY

X2-22 Rack4 TB

X5-22 Rack180 TB

X4-22 Rack120 TB

X2-22 Rack12 TB

X3-2+X5-23 RackSTBY

X4-2 Dev&Test1 Rack130 TB

V VV

Oracle Exadata Journey II

DWH

RDWH

NRDSDB

RepDB

KPIDB

NDMDB

Arc DWH

2016

X5-22 Rack180 TB

X4-22 Rack180 TB

X4-2 Dev&Test1 Rack130 TB

20182017

MASSDB

X5-22 Rack280 TB

X4-22 Rack290 TB

X4-2 Dev&Test1 Rack190 TB

X2-22 Rack12 TB

X6-2 elastic2 Rack250TB

X3-2+X5-23 RackSTBY

X3-2+X5-23 RackSTBY

X7-22 Rack450 TB

X4-22 RackSTBY

X4-2 Dev&Test1 Rack220 TB

X6-2 elastic2 Rack250TB

X3-2+X5-23 RackSTBY

X5-22 Rack280 TB

V

V

X6-2 elasticStandby/Dev

X6-2 elasticStandby/Dev

DW Projects

Turkcell Exadata MAA

2x X7-2 2x X5-2ZFS

BACKUP

ZS5-4 DR SITE DR Site ADG

2x X6-2

2x X6-21x X5-22x X3-2 2x X4-2

X4-2 Test&Dev

ZDLRA

Bigdata SQL Performance

Bigdata SQL Tests

Active DG

BDSQL

BDSQL selecttime

BSQL CTAS time

HIVE CTAS time

HIVE select time *

SCENARIO I 20 22 191Subselect ERROR

SCENARIO II 16 16 85SubselectERROR

ArchitectureREPORTING

Teradata to Exadata Migration Project

Active DG

Tape backup

RPO < 5 minRTO < 10 hour

BDSQL

ZDLRA backup withrealtime redolog copy15 days retentionRPO > 1 Day

RTO > 5 Days

Data Offloaded with Bigdatasql

ILM with Bigdata SQL

Active DG

BDSQLBDSQL

MASSIVE DB RAW + SUMMARY SIZE (TB) EXADATA RACK INFO HADOOP

DATE COMPRESSED UNCOMPRESSED NODE DETAIL NODE DETAIL

2017 Q4 360 1620 20 Cell node+14 DB node+ 4 temp cell node

2018 Q4 540 2430 EXADATA 800 TB/COMPRESSION RATIO ->HADOOP

2019 Q4 702 3159 EXADATA 1600 TB/COMPRESSION RATIO -> HADOOP

2020 Q4 877,5 3948,75 EXADATA 2400 TB/COMPRESSION RATIO -> HADOOP

DATAINSIGHT DB RAW + SUMMARY SIZE (TB) EXADATA RACK INFO

DATE COMPRESSED UNCOMPRESSED NODE DETAIL

2017 Q4 4,29 21,45 1/8 RACK

2018 Q4 4,719 23,595 1/8 RACK

2019 Q4 5,1909 25,9545 1/8 RACK

2020 Q4 5,71 28,54995 EXADATA+ 5 TB/COMPRESSION RATIO -> HADOOP

DataLake Project

BIGDATA SQL

ADWC Experience

Life with On-premises

• Takes 15 days for virtual, one and a half month for physicalinstallation

• Tuning physical infrastructure may even take months or years

• Understanding app behavior before resource management

• On premises environment has constraints and security efforts

• Needs lots effort for Test/dev env. Management

• Scaling needs huge investment,should be planned 6 months before

• Patching should be done manually

• Manual backup configuration

• Creating high avalibility may need a project and huge effort

• DBA expertise needed

ADWC vs On-Premises

Life with ADWC

• No initial setup,only 1-2 minutes

• Less time on physical infrastructure,optimized for DW workload

• Resource management is simple

• Connectivity is simple and secure

• Easy data ingestion

• Limited effort for Test/Dev env Management

• Instant scaling,flexible

• Autonomous Backup and Patching

• Less time on ensuring avalibility

• No database expertise needed

ADWC vs On-Premises

AutonomousDW Cloud

Changes in life of DBA with ADWC

• Less effort on dealing with end-user problems

• Less effort on dealing with dev/test users

• More time research&development

• Less time on query tuning

• Less time on dealing with infrastructure

• More time on database design and data modelling

• More time on securing data

• More time on data analytics,machine learning

• More time for application level tuning

• Less time on generic tasks

ADWC Benefits

AutonomousDW Cloud

ADWC vs Exadata X4-2

• Multirack X4-2 Exadata production

• 15 ETLS and 20 REPORTS

• 800 GB of data extracted with data pump

• Oracle Cloud Object Storage for staging area

• Oracle CLI for fast data upload

• Autonomous Data Warehouse Cloud with 8 OCPU’s

• Fixed parallelism is used

• CTAS used to eliminate fetch time

• Without tuning

• Average elapsed calculated

ADWC Performance Tests

AutonomousDW Cloud

Round I -ETL tests

ADWC Performance Tests

AutonomousDW Cloud

ADWC TIME(Seconds)

EXADATA X4-2 TIME (Seconds)

ETL1 198 258

ETL2 9 6

ETL8 130 96

ETL11 340 233

ETL9 84 172

ETL5 211 381

ETL4 15 12

ETL13 2031 1372

ETL14 645 2041

ETL15 364 1433

ETL16 176 1167

ETL17 86 900

ETL18 490 4458

ETL19 43 900

TOTAL 4822 13429

Round II - REPORT tests

ADWC Performance Tests

AutonomousDW Cloud

ADWC TIME(Seconds)

EXADATA X4-2 TIME (Seconds)

REPORT1 182 545REPORT2 48 4

REPORT3 6 4REPORT4 50 6REPORT5 32 27REPORT6 33 935REPORT7 55 64

REPORT10 13 209REPORT11 3 371

REPORT9 20 16

REPORT12 32 233REPORT13 31 253

REPORT14 65 4REPORT15 25 118REPORT16 10 10

REPORT17 9 18REPORT18 7 3

REPORT19 223 1576REPORT20 27 25TOTAL 871 4421

ADWC use cases

ADWC

• Instant Datamarts

• Self service BI apps

• Test/Dev environments

• Poc environments

• Less frequently accessed data needsspeed with low cost

ADWC Info

My Personal Blog

Step by Step guide for ADWC

http://cloudba.blogspot.com/

Adwc Forum

https://cloudcustomerconnect.oracle.com/resources/32a53f8587/summary

Mos

Autonomous Data Warehouse Cloud Tools and Applications Certification Matrix (Doc ID 2369441.1)

Twitter

https://twitter.com/yasinbaskan

ADWC

Q&A

Thanks

Questions ?