© 2015 IBM Corporation

From Concept to Production: Nationwide Insurance IBM BigInsights Journey with Telematics # 2404 Krish Rajaram & Rajesh Nandagiri – 10/26/2015

Big Data and Analytics Helps Nationwide Customers Become Better Drivers

Agenda

Introduction Architecture Data Processing Data Access Business Benefits

2

0.67

8.30 8.30

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

0.00

100.00

200.00

300.00

400.00

500.00

600.00

700.00

1 Hr Batch 12 Hrs Loop 12 Hrs batch

Dat

a Vo

lum

e in

GB

Elap

se T

ime

in M

ins

Cycles

After Redesign

First Iteration

Introduction

About Nationwide

SmartRide Program

SmartRide Data

About Nationwide

4

16+ MILLION POLICIES 25

MILLION CONTRIBUTED TO NONPROFITS

AND COMMUNITIES

$ 1 #

INSURER OF FARMS AND RANCHES

7 LARGEST

HOMEOWNER AND AUTO INSURANCE

PROVIDER IN THE U.S.

th

GALLUP GREAT PLACE TO WORK AWARD WINNER 3 YEARS RUNNING

LARGEST PET INSURER IN THE U.S.

9 th

LARGEST COMMERCIAL

INSURER

$23.9 BILLION IN REVENUE FOR 2013

Nationwide has approximately 31,000 associates serving customers in nearly every state.

1 # PROVIDER OF

PUBLIC-SECTOR RETIREMENT

PLANS

FOUNDED IN 1926 BY MEMBERS OF THE

OHIO FARM BUREAU

28 th

COMPUTERWORLD GREAT PLACE TO

WORK IN IT

About SmartRide • SmartRide is Nationwide's version of Telematics, offered to

customers to help them improve their driving behavior and save on insurance premiums.

5

• Customers install a small device into their vehicle for 6 months which measures…

SmartRide Data Characteristics Multiple vendors Files of different layouts arriving at different frequencies:

Hourly Every 4 hrs

Four CSV files per vendor ~ 30 GB to ~ 60 GB of data per day Data challenges

Late arriving trips Partial trips Duplicate trips Orphan trips

6

Trip Data Characteristics • Missing Timestamp & Speed Spike • Acceleration Lag

7

vin_nb trip_nb position_ts Speed engine_rpmabc 123 2015-07-21 12:31:36.0 54 1600abc 123 2015-07-21 12:31:39.0 55 1800abc 123 2015-07-21 12:31:42.0 57 1500abc 123 2015-07-21 12:31:43.0 82 1600abc 123 2015-07-21 12:31:44.0 58 1500

vin_nb trip_nb position_ts Speed engine_rpmabc 123 2015-06-30 21:25:05.0 0 700abc 123 2015-06-30 21:25:06.0 0 700abc 123 2015-06-30 21:25:07.0 0 1000abc 123 2015-06-30 21:25:08.0 8 1800abc 123 2015-06-30 21:25:09.0 15 2000

Architecture

Logical Data Flow

IBM® BigInsights™ Configuration

Decision Catalog

Job Orchestration

Logical Data Flow

9

IBM® BigInsights™ for Apache™ Hadoop Configuration

• Version 2.1.2 6 Management Nodes and 16 Data Nodes Each with 128 GB RAM and 18 TB of storage Hadoop 2.2, BigSQL 1.0, Hive 0.12, Hbase 0.96

• Three environments Dev, Test, and Production All same configuration

• Limitations No workload management No environment for DR Used Test Cluster for Hbase failover

10

Decision Catalog

11

Job Orchestration

12

Data Processing

Design Considerations

Phases of Data Movement

Batch Performance Metrics

Design Considerations • One hour window for end

to end processing Handling data issues Summarization Multiple cycles per

day • Predictable run time for

backlog processing when jobs fail

• Reloading incorrect batch

• Restart failed batch

14

0.67

8.30 8.30

0.00

1.00

2.00

3.00

4.00

5.00

6.00

7.00

8.00

9.00

0.00

100.00

200.00

300.00

400.00

500.00

600.00

700.00

1 Hr Batch 12 Hrs Loop 12 Hrs batch

Elap

se T

ime

in M

ins

Cycles

TripScrub TripSecDetails TripSum

State Median Hbase load AuditEn

raw2can Input Size in GB

Acquire Phase

15

Raw trip files copied into HDFS using WebHDFS protocol

Folders created by

vendor, file, load event ID, and batch #

Used Sqoop to

transfer 4 TB of historical data from Data Warehouse

Hive external tables for each file

Partitioned by load event ID and batch #

Used both BigSQL 1.0 and HiveQL

Partitioned external tables helped in o Processing backlog data o Reprocessing incorrect batches

Standardize Phase

16

Select data from external tables based on load event ID

Each load event ID can include one or more batches

More than one load event ID can be processed in one cycle

Data moved to next stage only from work tables Helped in performance

Dynamic partitions helped in loading multiple batches Partitions get overwritten if already exists

Helped in reprocessing incorrect batch

Work tables contain data for CURRENT

processing cycle

Canonical tables partitioned by

source and batch #

Load using dynamic

partitioning

Data Scrubbing & Event Calculation

17

Trip summary

Trip point

Map side join Single read multi write

Orphan trips

Trip points (Work table)

Java M/R program for o Scrubbing o Events calculation Night time driving Hard brake Fast acceleration Miles driven

Events at seconds level (Work table)

Very good performance gain Using Java for complex scrubbing rules Single read multiple writes Only required data points processed No data persisted to corpse tables

Summarization Phase

18

Events at seconds level (Work table)

Gather all trips related to devices from current trip and aggregate at various levels

Union ALL UDF to store data points

for trip graph Replace new summary

info into final table

SRE summary (Work table)

SRE summary partitioned by

source

SRE summary in HBase

Parallelized the Union All operation Partitioning by Source enabled both Vendor

data to be processed at same time if overlap happens

PUT from Hive to Hbase, WAL disabled

Shorten column names

Changed to epoch time

Prefix salting key Generate rowkey Column family

mapping

Batch Performance Metrics

19

1 Hr Batch SLA

0

0.5

1

1.5

2

2.5

0 5

10 15 20 25 30 35 40

Dat

a Si

ze in

GB

Run

tim

e in

Min

s

Cycle Schedule Time

Avg Run Times for Hourly Cycles

0

2

4

6

8

10

12

0

10

20

30

40

50

60

0000 0400 0800 1200 1600 2000

Dat

a Si

ze in

GB

Run

tim

e in

Min

s

Cycle Schedule Time

Avg Run Times for 4 hr Cycles

Trip Second Details

Standarize

SRE Trip Summary Hive

SRE Trip Summary Hbase

Audits

Acquire

Size in GB

Data Access

SmartRide Web Page

Application Layer

Column Family and Row Key Design

Performance Metrics

SmartRide Web Page

21

SmartRide Web Page – Daily

22

Application Layer

23

Data Access Layer

HBASE API

Restful Service

Single Page Web App

BigSQL &

Hive

Aggregates

Daily

HDFS

HBase

HRegion Server

HRegion

HLog

Memstore

HFile

ODS – DB2 ODBC

Column Family & RowKey Design

24

RowKey – Pfx_pgmId_pdflg_timestamp

Column Family – Summary Data

Column Family – Trip-point Data

12_8798782_Tp_2015080912000000

SM:miles,1500001245,’15’, SM:hb,1500001245,’2’, SM:fa,1500001245,’5’, SM:nt,1500001245,’Y’

TP:Trip,1500001245,’{JSON BLOB}’ S

orte

d Le

xico

grap

hica

lly

• Column family (CF) helps in grouping the related columns depending on access pattern.

• Co-locating the keys related to one customer in one region to access data using filter from one region server.

Performance Metrics

Scenarios – 1x, 2x, 3x concurrent users, Zookeeper node going down, Datanode unavailable Tools used – Initial test using custom program, LoadRunner for final test, SiteScope for monitoring resource consumption

25

SLA for aggregates – 5 sec # of concurrent users - 1200

HBase Data Distribution – Using Hannibal

26

SmartRide Data Distribution

Business Benefits • Deeper Engagement with Members

Over 2 million website page views since the July launch. To put in perspective, our vendor-hosted website would receive 100,000 views in a 12 month period.

Over 60K users have accessed the new site and 90% of those are new users.

• Increase in bind ratios across all channels

• Improvement in loss ratios

• Enterprise first "big data" implementation at Nationwide

27

Future scope – Personal and Commercial Fleet

28

Insights Give Nationwide Competitive Advantage

Weather Data

GPS Data

Hourly Trip Data

from Device

Claims Data

Other Public

Records

© 2015 IBM Corporation

Thank You

We Value Your Feedback!

Don’t forget to submit your Insight session and speaker feedback! Your feedback is very important to us – we use it

to continually improve the conference.

Access your surveys at insight2015survey.com to quickly submit your surveys from your smartphone, laptop, or

conference kiosk.

30

31

Notices and Disclaimers Copyright © 2015 by International Business Machines Corporation (IBM). No part of this document may be reproduced or transmitted in any form without written permission from IBM.

U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM.

Information in these presentations (including information relating to products that have not yet been announced by IBM) has been reviewed for accuracy as of the date of initial publication and could include unintentional technical or typographical errors. IBM shall have no responsibility to update this information. THIS DOCUMENT IS DISTRIBUTED "AS IS" WITHOUT ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL IBM BE LIABLE FOR ANY DAMAGE ARISING FROM THE USE OF THIS INFORMATION, INCLUDING BUT NOT LIMITED TO, LOSS OF DATA, BUSINESS INTERRUPTION, LOSS OF PROFIT OR LOSS OF OPPORTUNITY. IBM products and services are warranted according to the terms and conditions of the agreements under which they are provided.

Any statements regarding IBM's future direction, intent or product plans are subject to change or withdrawal without notice.

Performance data contained herein was generally obtained in a controlled, isolated environments. Customer examples are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual performance, cost, savings or other results in other operating environments may vary.

References in this document to IBM products, programs, or services does not imply that IBM intends to make such products, programs or services available in all countries in which IBM operates or does business.

Workshops, sessions and associated materials may have been prepared by independent session speakers, and do not necessarily reflect the views of IBM. All materials and discussions are provided for informational purposes only, and are neither intended to, nor shall constitute legal or other guidance or advice to any individual participant or their specific situation.

It is the customer’s responsibility to insure its own compliance with legal requirements and to obtain advice of competent legal counsel as to the identification and interpretation of any relevant laws and regulatory requirements that may affect the customer’s business and any actions the customer may need to take to comply with such laws. IBM does not provide legal advice or represent or warrant that its services or products will ensure that the customer is in compliance with any law.

32

Notices and Disclaimers (con’t)

Information concerning non-IBM products was obtained from the suppliers of those products, their published announcements or other publicly available sources. IBM has not tested those products in connection with this publication and cannot confirm the accuracy of performance, compatibility or any other claims related to non-IBM products. Questions on the capabilities of non-IBM products should be addressed to the suppliers of those products. IBM does not warrant the quality of any third-party products, or the ability of any such third-party products to interoperate with IBM’s products. IBM EXPRESSLY DISCLAIMS ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.

The provision of the information contained herein is not intended to, and does not, grant any right or license under any IBM patents, copyrights, trademarks or other intellectual property right.

• IBM, the IBM logo, ibm.com, Aspera®, Bluemix, Blueworks Live, CICS, Clearcase, Cognos®, DOORS®, Emptoris®, Enterprise Document Management System™, FASP®, FileNet®, Global Business Services ®, Global Technology Services ®, IBM ExperienceOne™, IBM SmartCloud®, IBM Social Business®, Information on Demand, ILOG, Maximo®, MQIntegrator®, MQSeries®, Netcool®, OMEGAMON, OpenPower, PureAnalytics™, PureApplication®, pureCluster™, PureCoverage®, PureData®, PureExperience®, PureFlex®, pureQuery®, pureScale®, PureSystems®, QRadar®, Rational®, Rhapsody®, Smarter Commerce®, SoDA, SPSS, Sterling Commerce®, StoredIQ, Tealeaf®, Tivoli®, Trusteer®, Unica®, urban{code}®, Watson, WebSphere®, Worklight®, X-Force® and System z® Z/OS, are trademarks of International Business Machines Corporation, registered in many jurisdictions worldwide. Other product and service names might be trademarks of IBM or other companies. A current list of IBM trademarks is available on the Web at "Copyright and trademark information" at: www.ibm.com/legal/copytrade.shtml.

• IBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.

• Information regarding potential future products is intended to outline our general product direction

and it should not be relied on in making a purchasing decision. • The information mentioned regarding potential future products is not a commitment, promise, or

legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract.

• The development, release, and timing of any future features or functionality described for our

products remains at our sole discretion. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

Please Note:

2