con2715 baumann oow
Post on 06-Jul-2018
222 Views
Preview:
TRANSCRIPT
-
8/17/2019 Con2715 Baumann Oow
1/46
Top Five Ways Swiss
Improved Business Value
Database In-Mem
aolo !reth
Head of Data Manage
Thomas Bauman
Head of IT Performance M
-
8/17/2019 Con2715 Baumann Oow
2/46
In-Memory Ma"in# $eadlines at Swiss MobiliarNew business insights due to real-time analytics
ANALYTICS ON OLTP DATAE
PORTS AUF OLTP DATEN
-
8/17/2019 Con2715 Baumann Oow
3/46
IN 9 VON 10 FÄLLEN KEIN SQL QUERY TUNII
N 9 VON 10 FÄLLEN KEIN SQL QUERY TUNI
In-Memory Ma"in# $eadlines at Swiss MobiliarMassively reduced tuning efforts for !" #uery te$ts
UT OF 10 QUERIES DON‘T NEED SQL
REPORTS AUF OLTP DATEN
-
8/17/2019 Con2715 Baumann Oow
4/46
REDUCED ETL FLOW
In-Memory Ma"in# $eadlines at Swiss Mobiliar % &aradigm change' ()ring %nalytics to the Data(
Verz!"# $%& ETL
-
8/17/2019 Con2715 Baumann Oow
5/46
%#enda
wiss Mobiliar in a Nutshell
%nalytics on ,"TP data where we are today %nalytics on ,"TP data Po. with Database In-M
Po. Database In-Memory /esults
0urther $&eriences and Ne$t te&s with Databas
In-Memory at wiss Mobiliar ummary
-
8/17/2019 Con2715 Baumann Oow
6/46
legal form of a coo&erative ass
com&any4
wit5erland6s number one insuhousehold contents7 business
life insurance4
close to customers throughout
than8s to around 9: general ag
locations4
over 14< million insured &erson
over *7*:: em&loyees and 3=+
Swiss Mobiliarwit5erland>s most &ersonal insurer
&'( continuously )**'-)*&+
-
8/17/2019 Con2715 Baumann Oow
7/46
Overall ,on-i.e Insurance Mar"et /rowth in Swit0erland.lose to =?3 of Mar8et @rowth to wiss Mobiliar in =:1*
@rowth Mobiliar Mar8et @rowth
in Mio .H04 ource' chwei5erische
-
8/17/2019 Con2715 Baumann Oow
8/46
ystems' i,7 "inu$7 Bindows and 5?,
+::: Noteboo8s7 19:: iPhones7 3:: iPads
D)M' ,racle7 D)=7 IM7 M !" erver7 Neo*C
tandard-B 2%rc@I7 %P7 ),7 iebel7 .,/ "ife etc4
Many home-grown a&&lications
Swiss Mobiliar1s IT Misson Statement2 De&loy and /uninnovative technologies for efficient business &rocesses
S
-
8/17/2019 Con2715 Baumann Oow
9/46
The Spea"er Thomas )aumann
f Technology7 Eurich
istics
ata
Management ystems and Performance since 1=
al conferences
obiliar
dedicated to &erformance since 1;3
Fdedicated to &erformance
also &roduces this search
-
8/17/2019 Con2715 Baumann Oow
10/46
%#enda
wiss Mobiliar in a Nutshell
%nalytics on ,"TP data where we are today %nalytics on ,"TP data Po. with Database In-M
Po. Database In-Memory /esults
0urther $&eriences and Ne$t te&s with Databas
In-Memory at wiss Mobiliar ummary
-
8/17/2019 Con2715 Baumann Oow
11/46
%nalytics on OT Data at Swiss Mobiliar %rchitecture =:1*
,"TP
Decision u&&ort Business
Intelli#ence
Data Mart
Data
Warehouse
3ross
In.ormation Systems
3ore
In.ormation
Systems
%ccessIn.ormation
Systems
%nalytical
2,"%P
,&erational
2,"TP
,"TP %nalytics
co&e
% l ti OT d t t S i M bili
-
8/17/2019 Con2715 Baumann Oow
12/46
%nalytics on OT data at Swiss Mobiliar %rchitecture =:=:
,"TP
Decision u&&ort Business
Intelli#ence
Data Mart
Data
Warehouse
3ross
In.ormation Systems
3ore
In.ormation
Systems
%ccessIn.ormation
Systems
%nalytical
2,"%P
,&erational
2,"TP
,"TP %nalytics
co&e
%rchitecture )*&4
-
8/17/2019 Con2715 Baumann Oow
13/46
Desi#n atterns
.olumn oriented data storage
Data re&lication close to real time based onlog records7 not transactionally consistent
!uery /e-/outing decided by o&timi5er7
trans&arent for a&&lication
No need for inde$es
Aery high com&ression rate
%rchitecture )*&4ID%% 2I)M D)= %nalytics %ccelerator Aalue Delivery
D)= 5,
2(Mainframe(
I)M
Nete55a
!" !uery
2to D)=
/esult et
%utomatic #uery re-routing o
intensive #ueries1 to data co
a&&liance
1 0or a&&lications which don>t re#uire transactionall
acce&t data delayed by a few minutes
imilar design &atterns for ,racle Database In-Memory
%re there similar results as wellG
Ma5or 6esults
Increased ,"%P #uery
1:: times faster in ave
0aster inserts on D)= a
scalability
Due to elimination of m
hort timeframe betwe
and analysis
More 6eal Time %nalytics $eadlines at Swiss Mobiliar
-
8/17/2019 Con2715 Baumann Oow
14/46
( REDUCTION OF MAINFRAME CPU
More 6eal-Time %nalytics $eadlines at Swiss Mobiliar/educed .P consum&tion during &ea8 time
REDUKTION MAINFRAME CPU RESSOURCE
ID%% in more detail
2or what is different to ,racle Database In-Memory
Massive arallel rocessin# %rchitecture
3= nodes on = blades at wiss Mobiliar Ma$ .a&acity 1:=* nodes 29 rac8s
Data rocessed ocally 0P@% 2HB encoded logical arrays'
Decom&ression7 ProCection7 Predicate
%&&lication all done locally within 0P@%
oins &rocessed locally if &ossible7 dataredistribution among nodes otherwise
Data clustering within node to su&&ort s8i&s
for data scanning 25one ma&s
Tables completely replicated 2all
rows?columns
Tunin# !nobs'
Data distributio
Data clustering
)oth are inde&
&artitioning and
Step by Step ID%% 7sa#e
-
8/17/2019 Con2715 Baumann Oow
15/46
0irst )usiness %&&lications
%d-hoc re&orts from business end users
Im&roved end-of-month &rocessing
"og analysis based on D)= tables for access &attern analytics
Im&roved T" flow
treamline Mainframe for ,"TP
liminating inde$es used for analytics only
liminate M!T and other au$iliary structures for analytics
/educed demand for reorg More efficient inserts
Step-by-Step ID%% 7sa#e1= month from installation to broad usage
1= months
New )usiness 0unctions
Be e$&ect a similar timeline for ,racle Database In-Memory 2see later
,ew Business 6eports 21 of =
-
8/17/2019 Con2715 Baumann Oow
16/46
,ew Business 6eports 21 of =How wiss Mobiliar>s ./M system is usedG
$8
; ;4+ <
-
8/17/2019 Con2715 Baumann Oow
17/46
hich car ma"es do also have many other contracts at Swiss Mobiliar9
,ew Business 6eports 2= of = .ross elling
with partner_MFZ (PNR, Make) as(SELECT DISTINCT PAR.C!"#$ AS PNR, MFZ.D%&' as MAE
FRM D*"P+IE.+-ES-PA" PAR, D*"P+IE.+ALL' +ER, D*"P+IE.+PA'$ +ER_*/,
D*"P+IE.+*MFZP$ MFZ
0ERE PAR.C#'"% 1 +ER.C#'"% AND +ER.C#'"%1+ER_*/.C#'"% AND +ER_*/.C#'&%_$1MFZ.C#'&%
AND
+ER_*/.C#'%"1MFZ.C#'%" AND PAR.C!$# 1 2$! AND PAR.C% 1 AND 3EAR(PAR.C")
1
AND 3EAR(PAR.C#) 1 AND 3EAR(+ER_*/.C#)1 AND 3EAR(+ER_*/.C")1
AND
3EAR(MFZ.C#)1 AND 3EAR(MFZ.C")1 AND +ER.C#'&% 1 $ AND +ER.C% 1
AND
3EAR(+ER.C#) 1 AND 3EAR(+ER.C") 1 AND +ER.D#'& IN ("2,"#,"%,"!) ),
partner_a44 (PNR, N567er) as
(SELECT PAR.C!"#$ AS PNR, C8NT(9)
FRM D*"P+IE.+-ES-PA" PAR, D*"P+IE.+ALL' +ER
0ERE PAR.C#'"% 1 +ER.C#'"% AND PAR.C!$# 1 2$! AND PAR.C% 1 AND 3EAR(PAR.C") 1
AND
3EAR(PAR.C#) 1 AND +ER.C#'&% 1 $ AND +ER.C% 1 AND 3EAR(+ER.C#) 1
AND
3EAR(+ER.C") 1 -R8P *3 PAR.C!"#$)
se4e:t 6ake, a;_:=ntra:ts, :=5nt(9) as n567er_=>_:5st=6ers
>r=6 Partner_6>?, partner_a44where partner_6>?.pnr1partner_a44.pnr
-
8/17/2019 Con2715 Baumann Oow
18/46
6eal-Time %nalysis2 6is"snvalidated correlations
)oth statistics7 number of stor8s observed7 and number of new
children counted in same area and year7 are correlated with tim
both were decreasing7 inde&endent from each other4
No4 of new-born
children in samearea and year
No4 of stor8s
observed
-
8/17/2019 Con2715 Baumann Oow
19/46
%#enda
wiss Mobiliar in a Nutshell
%nalytics on ,"TP data where we are today
%nalytics on ,"TP data Po. with Database In-
Po. Database In-Memory /esults
0urther $&eriences and Ne$t te&s with Databas
In-Memory at wiss Mobiliar ummary
Oracle Database In Memory o3
-
8/17/2019 Con2715 Baumann Oow
20/46
Oracle Database In-Memory o3,bCectives
The o3 2)eta-test of ,racle 1=c Database In-Memory shall &rov
com&arable7 if not better7 res&onse times might be achieved for the
and the same #uery com&ared with D)=?Nete55a4 the number of #ueries with demand for manual #uery statement tu
reduce by at least :J4
analytics on OT data will be: after migration D)=,racle7
K very e..icient4
K trans&arent for users and a&&lications4
K without impact on OT &rocessing4
-
8/17/2019 Con2715 Baumann Oow
21/46
Oracle Database In-Memory
column-oriented
Memory
data
Memory
row-oriented
data
o far' Data organi5ed in rows: data loaded into memory at .irst usa#e
designed for
transactions
ew2 %dditional' Data organi5ed in columns: permanently stored in memor
desig
ana
automated7 near real-time
re&lication
Oracle Database In-Memory o3
-
8/17/2019 Con2715 Baumann Oow
22/46
O..er and 3ontract data
Migration D)=,racle of < tables with No4 of rows between : an
,ffer data 2++:L contracts in &rogress
.ontract data 2+4*M contracts
In other words' very 1:th contract is in &rogress4
This data is not &art of the traditional data warehouse4
%nalytics of this data needs to be close to real time
%ll tests were &erformed by a&&lying &' real user reports with real da
Oracle Database In-Memory o3.onfiguration
Oracle Database In-Memory o3
-
8/17/2019 Con2715 Baumann Oow
23/46
14 How e$&ensive is the migration of the D)= data into ,racleG
=4 How much tuning effort was re#uired after activating ,racle Database In-Mem
34 %re the res&onse times better than with D)=G Bhich &art of the im&rovement
Database In-MemoryG
*4 How much administration effort is necessary to de&loy and run Database In-M
+4 Bhich are the &rere#>s to a&&ly Database In-MemoryG
;4 Bhich are the strengths and wea8nesses of Database In-MemoryG
Oracle Database In-Memory o3The Po. should answer the following ; #uestions
-
8/17/2019 Con2715 Baumann Oow
24/46
The Spea"er
-
8/17/2019 Con2715 Baumann Oow
25/46
pPaolo Lreth
tics at @enoa niversity
19 ,racle 9i ,bCect /elational Database
cal "ead Mobi
ou&
8uestion &2 Database mi#ration .rom DB) to Oracle
-
8/17/2019 Con2715 Baumann Oow
26/46
8uestion &2 Database mi#ration .rom DB) to Oracle
K Data was successfully migrated from D)= to ,racle' 3 days effort4
K %dCustments from D)= to ,/%."− TIMT%MP Data Ty&e conversion =*':: =3'+'
2,racle does not recogni5e =*'::
− %dCustments on DD"-definitions for em&ty trings?Numbers?Timestam
CREATE TABLE TEO_TBENPRO
( C43087 CHAR(30) DEFAULT ON N
NOT NULL, C…. NUMBER(30) DEFAULT ON NU
NOT NULL,
…. )
KNo runtime errors4
8uestion )2 $ow much tunin# e..ort was re
-
8/17/2019 Con2715 Baumann Oow
27/46
# <
activatin# Database In-Memory9
K No tuning was re#uired4K ,ne single o&timi5ation'
− Be changed the &artitioning layout of a table to o&timi5e it for the re#u
2%mount' 1 hour
-
8/17/2019 Con2715 Baumann Oow
28/46
8uestion '2 $W-3omparison between ID%% and Oracle
-
8/17/2019 Con2715 Baumann Oow
29/46
8uestion '2 $W-3omparison between ID%% and Oracle
IBM ,ete00a ID%% vs; Oracle &)c Database In-Memory
O in the red co
Oracle &)c mit
K Hitachi nifiedPlatform
K 1; .ore
K 39* @)
K no attache P
K ,' /edHat
K ,racle 1=41
Oracle &)c mit
K Hitachi nifiedPlatform
K 1; .ore
K 39* @)
K no attache P
K ,' /edHat
K ,racle 1=41
O in the blue cornerO4
IBM ,ete00a ID%%2
K PureData ystem for %nalyticsN1::1-::=
K =* .ores
K
-
8/17/2019 Con2715 Baumann Oow
30/46
:41:
14::
1:4::
1::4::
ID%%
,racle IM
.olumn tore
8uestion '2 Oracle &)c Database In-Memory vs; IBM ,ete00a
K .om&arison of ela&sed time O
O Nete55a ID%% vs4 ,racle Database In-Memory
Total e
ID%%
,/%.
8uestion 42 %dminstration o. Oracle Database In-Memory
-
8/17/2019 Con2715 Baumann Oow
31/46
8uestion 42 %dminstration o. Oracle Database In Memory
K +: @) /%M was reserved for the in-memory column store
−. O*1 @) used 2Memcom&ress for !uery
−. Oon %N-these data occu&ied 1
-
8/17/2019 Con2715 Baumann Oow
32/46
8uestion +2 re conditions .or optimal usa#e
Which are the pre conditions .or an optimal usa#e o. Oracle Database In-M
a se of ,/%." built in functionalities to restrict the data set 2Partitioning
−
The o&timal &artitioning method can significantly im&rove the #uery &b 0ind the o&timal degree of &arallelism in 1=c
− In our tests we found that the o&timal degree of &arallelism was 9
2this value de&ends on hardware we tested on the .Ps were unab
data from memory
− More &arallel slaves &roduced only more overhead
− 0or different hardware this value could differ
c Plan tability
− ,f relevant im&ortance is the stability of e$ecution &lans4 Therefore th
statistics have to be u& to date4
8uestion A2 Stren#ths and Wea"nesses o. Database In-Memory
-
8/17/2019 Con2715 Baumann Oow
33/46
8uestion A2 Stren#ths and Wea"nesses o. Database In Memory
Stren#ths2
K It scales very good incrementing the degree of &arallelism4 %lso small deg
show big &erformance im&rovements4
K /ead and 0ilter of large data volumes
K asy administration
You can improve the performance of an application within minutes
Wea"nesses2
K orts of large data volumes is still a bottlenec8 2 1:: @) orts still use tem&orary tables&ace
K Pro&osal to O6%3'
Tem& egment in Memory for .olumn store G
0eedbac8 to ,racle has been &rovided4
%#enda
-
8/17/2019 Con2715 Baumann Oow
34/46
%#enda
wiss Mobiliar in a Nutshell
%nalytics on ,"TP data where we are today
%nalytics on ,"TP data Po. with Database In-
Po. Database In-Memory /esults
0urther $&eriences and Ne$t te&s with Databa
In-Memory at wiss Mobiliar ummary
6esults within the MobiliarFIS ro5ect
-
8/17/2019 Con2715 Baumann Oow
35/46
5
What were the straits within the MobiFis ro5ect2
Mobi0is eDBH is still in develo&ment
Be could wor8 only on a small set of #ueries and
re&orts
Solution2
Be did the tests only on !ueries on a fact table and the
corres&onding dimension tables
)eside the small #uery set on to& of the re&orts we got from develo&ment
synthetic #ueries trying to simulate future re&orts by creating'
ums7 averages and other grou& functions on the e$isting tables
@rou&ing of values
Be issued these #ueries on the following data volumes
K 1 0act-Table with14 Mrd4 /ows
K & to1: Dimension Tables between 1:: and =4+ Mio4 /ows
6esults on MobiliarFIS
-
8/17/2019 Con2715 Baumann Oow
36/46
6esults on MobiliarFIS
3omparison o. elapsed times
8uery lapsed Time
& &)c
,verview Tb "evel 2),-/e&ort * 193
Numer of contracts for one &roduct in :1?1* 3= =
)uilding bloc8s &er single contract =
-
8/17/2019 Con2715 Baumann Oow
37/46
6esume MobiliarFIS
Summary
K The /esults from the Po. on ,racle 1=c
QIn-Memory .olumn toreQ showed an im&rovement of one or two magnitu
K Ty&ical DBH #ueries li8e sums7 averages etc4 can be im&roved dramatical
K The assum&tions made at the start of the Po. could be confirmed4
,e(t Steps
K Mobiliar0is has been migrated on 1=414:4= and we are beginning to develo&
will benefit from ,racle Database In-Memory4
Test on an e(istin# %pplication - 6I3O
-
8/17/2019 Con2715 Baumann Oow
38/46
Be tested on the database of our ris8 controlling a&&lication 2/I.,
The /ico Database occu&ies about 1 Tb of Data
The /ico a&&lication is over =: Rears in &roduction
Be collected the most im&ortant #ueries from the user>s &ers&ective
These #ueries were the candidates for im&rovement4
Tests on 6I3O = Be.ore startin#
-
8/17/2019 Con2715 Baumann Oow
39/46
Be analy5ed the e$isting a&&lication and investigated &erformance o&timi5ation
before 1=c'
.om&ression
,&timi5ation of P.T0/ &arameter
.hange of the &artitioning schema of some tables
Be started using In-Memory after these changes
-
8/17/2019 Con2715 Baumann Oow
40/46
6esults 6I3O
-
8/17/2019 Con2715 Baumann Oow
41/46
6esults 6I3O
&eed & ,racle 1=c In-Memory vs4 ,racle 1=c
Parallel Degree !uery 1 !uery = !uery 3 !uery * !uery
1 1:949 =+431 *94
-
8/17/2019 Con2715 Baumann Oow
42/46
6esults 6I3O
)oth slides combined' &eed & ,racle 1=c In-Memory vs4 ,racle 11g
Parallel Degree !uery 1 !uery = !uery 3 !uery * !uery
1 39*4:* *;41; 3
-
8/17/2019 Con2715 Baumann Oow
43/46
/ico has been migrated on uly on 1=414:4=
% develo&ment environment has been built to test the u&grade and the
Memory
Be are using the !" Performance %naly5er of the /eal %&&lication T
to identify im&rovements and regression
Be are involved in the ,racle 1=c /eference Program and we are wor
with ,racle4
,racle In Memory will be activated after the Mobiliar oftware /elease
,ctober
%#enda
-
8/17/2019 Con2715 Baumann Oow
44/46
wiss Mobiliar in a Nutshell
%nalytics on ,"TP data where we are today
%nalytics on ,"TP data Po. with Database In-
Po. Database In-Memory /esults
0urther $&eriences and Ne$t te&s with Databa
In-Memory at wiss Mobiliar
ummary
Oracle Database In-Memory Bene.itsnd ser Perce&tion
-
8/17/2019 Con2715 Baumann Oow
45/46
nd ser Perce&tion
"i8e an additional7 fast7 access &ath
0astest way from an 0rancisco to .hicagoG
!" Tuning D) In-Memory
)eam me u&731 hours *h 3:min21h 3:min
Than" you@
-
8/17/2019 Con2715 Baumann Oow
46/46
thomas4b
&erf
&a
mailto:thomas.baumann@mobi.chmailto:performance@mobi.chmailto:paolo.kreth@mobi.chmailto:paolo.kreth@mobi.chmailto:performance@mobi.chmailto:thomas.baumann@mobi.ch
top related