Driving Business Performance in Telco with Hadoop

21 Use Cases with Lessons Learned at the Intersection of People, Processes and Technology

Juergen Urbanski

Board Member for Big Data & Analytics, BITKOM

Agenda

•  The Journey to a Data Driven Organzation

•  21 Telco Use Cases for Hadoop

•  Overall Lessons Learned

– 2 –

Business Value from Hadoop Flight Plan for a Journey in Four Phases

* Timeline varies by company size. Often smaller or focused online businesses achieve milestones at the shorter end of the range.

1 2 Evaluation –

Business Value

Awareness & Interest

Evaluation – Technical

Enterprise Deployment

Enterprise Production

Industry Leadership

Point Deployment

Point Production

3 4 Operational Value Strategic Value Data-Driven

Organization

Flight plan – typical elapsed time* from start of phase 1 in months:

2-6 9-15 18-36

Potential Value

– 42 –

1 2 3 4

What Would You Like to Accomplish? Levels of Success with Hadoop

Potential Value Operational Value Strategic Value Data-Driven Organization CXO •  Recognition of

potential •  Mandate to explore

•  Recognition of value realized •  Sponsorship to expand use

•  Recognition of material value realized •  Sponsorship to transform organization

•  Competitive advantage •  CDO part of Exec Team

Line of Business

•  Basic understanding of the value of Hadoop to the business

•  Value realized in 1 area ‒  Customer intimacy ‒  Operational excellence ‒  Risk, security,

compliance ‒  New business

•  Value realized and tracked in many areas ‒  Customer intimacy ‒  Operational excellence ‒  Risk, security, compliance ‒  New business

•  Data managed like capital •  Intelligence at the front

line •  JIT decision making •  Widespread value

creation

Analytics & Applications

•  Basic understanding how Hadoop fits into existing landscape

•  BI and EDW access to Hadoop

•  Some new analytic apps, often batch

•  Few use cases and processing engines

•  Many sources and time periods

•  Mostly departmental silos •  10-50 enterprise users

•  Hadoop consumable by any department, both technically and process-wise

•  New apps natively on Hadoop, often transactional or real-time

•  Many use cases and processing engines

•  Multiple lenses into common data pool •  Emerging data science team •  50-500 enterprise users

•  Data-driven culture •  High-performing data

science team •  Use cases build on each

other •  500-5000 enterprise

users

Data Mgt. & Security

•  Basic understanding how Hadoop fits

•  Benefitting from schema on read

•  Professionalizing data definitions and models

•  Collaboration and granular security controls governing use of shared data

•  Incentives and process to encourage consumption of shared data

Infra-structure

•  Basic fluency with core technical concepts of Hadoop

•  1 or more production environments

•  Multi-tenant shared service worldwide •  Data Lake •  Service Desk / CoE

•  Hadoop community participation and contribution

– 43 –

Early Stage Center of Excellence Mastering the Hadoop Journey 2

– 49–

Roles

Analytic Application Development (LOB)

Infrastructure Operations

Change & Program Management

Architecture Head of Big Data

Business Analysts (usually in LOB)

Roles

Later Stage Center of Excellence Mastering the Hadoop Journey

Legend Red = New at this stage

– 50 –

3

Analytic Application Development (LOB)

Infrastructure Administration

End User Service Desk

Change & Program Management

Advocacy & Demand Management

Data Science & Machine Learning

Operations Business Team

Architecture & Service Portfolio Management

SVP Big Data / Chief Data Officer

Group CIO or EVP

Business Analysts (usually in LOB)

A Data Lake Establishes Hadoop as a Shared Service Mastering the Hadoop Journey

Data Lake Characteristics

•  Timely insights for all authorized users / tenants

•  Many use cases, often building on each other

•  The right processing engine for the right job

•  All data, across all time periods

•  Multiple lenses on the same data

3

– 45 –

User

Use case

Processing engine

Data

Stakeholder Expectations of a Shared Service Mastering the Hadoop Journey

•  Multi-tenancy, workload fencing, resource isolation •  Data security, governance and workflows •  Data privacy, policy and regulatory compliance •  Consumption models (e.g., self-service, charge-back,

on-boarding) •  Data publication guidelines (availability, stability, quality) •  Operational processes, standards, service tiers with

SLAs

Stage 3 – Enterprise Data Lake Departmental Project Silos

•  Security largely via restricting physical access to a few friendly users

•  Best efforts service for batch use cases •  Data largely owned by each

department

3

– 46 –

User

Use case

Processing engine

Data

Hadoop Can Create Competitive Advantage Mastering the Hadoop Journey

2-3

Data Science & Machine Learning*

Wave C

Wave B

Wave A Use case

New Front-line Applications, often Real-Time

Integration with Front-line Applications, often Real Time

Data Mining – New Analytic Applications

Data Refinery – Integration with Analytic Applications

Active Archive and Data Offload

Creates Operational Value

Drives Competitive Advantage

* A practice area that is relevant to many Hadoop workloads.

– 44 –

Agenda

•  The Journey to a Data Driven Organzation

•  21 Telco Use Cases for Hadoop

•  Overall Lessons Learned

– 2 –

n  Network capacity planning n  Network upgrades n  Network maintenance n  Network performance management n  Network traffic shaping

21 Telco Use Cases for Hadoop

– 11 –

Use Case

Network Infrastructure

Function

n  Customer experience analytics n  Contact center productivity n  Field service productivity n  Data protection and compliance n  End-user device security

Service and Security

n  360-degree view of customer value n  Personalized marketing campaigns n  Upselling and cross-selling n  Next-product-to-buy (NPTB) n  Churn reduction

Sales and Marketing

n  New product development n  Actionable intelligence serving:

l  Advertisers l  Merchants/retailers l  Payment processors l  Federal governments l  Local governments

New and Adjacent Business

Network Care Sales

New Biz

Network Infrastructure – Network Capacity Planning

– 12 –

Business Problem n  The consumption of services and

resulting bandwidth in a particular neighborhood may be out of sync with a telco’s plans to build new towers or transmission lines in that same neighborhood.

n  This leads to a mismatch between expensive infrastructure investments and the actual revenue from those investments.

n  Examples: l  4G (LTE) l  FTTC (fiber to the curb)

l  FTTH (fiber to the home)

n  One European carrier used Hadoop to optimize the rollout of 4G coverage in time and space to match the likely pick-up in service revenue, based on detailed cell tower traffic data of the last few years.

n  With their prior, less informed approach, they would have had to spend 10% more capex for the same outcome.

Value Realized

Network Care Sales

New Biz

Hadoop in Network Infrastructure – Network Upgrades Improve the Customer Experience

– 13 –

•  Correlate network congestion and customer experience

•  11 different data sources

•  Millions of subscriber records, work orders, calls, IPDRs, Tivoli NPMs

•  Finding: Only a few nodes responsible for most of the negative customer experience

Network Node

TNMP CMTS

Performance

Network Sensors

IPDR Cable

Modem Usage

CompetitiveSpendData

HouseholdHousehold

Master Subscriber

Record

Marketing Demo-

graphics

Caller Experience

Work Orders

Mobile Devices

CustomerPremise

Equipment

OnlineTransactions

Social MediaInteractions

SOURCE DATA

Network Care Sales

New Biz

Service and Security – Customer Experience Analytics Based on Call Detail Records (CDRs)

– 14 –

Business Problem n  A typical mobile service provider

generates >1 billion CDRs per day, ingesting millions of CDRs per second.

n  System holds >100 billion records, half a petabyte added every month!

n  Due to the cost of existing solutions, the data expires after 60 days

n  CDRs need to be analyzed and archived for compliance, billing and congestion monitoring.

n  Example: forensics on dropped calls and poor sound quality.

n  High volume makes pattern recognition and root cause analysis difficult.

n  Often those need to happen in real-time, with a customer waiting for answers.

n  With Hadoop the carrier can to retain some data for up to three years

n  Hadoop provides both a cost advantage – Hadoop provides storage 20x cheaper than enterprise-grade storage – and better insights.

n  Better analysis to continuously improve call quality, customer satisfaction and servicing margins.

Value Realized

Network Care Sales

New Biz

Service and Security – Contact Center Productivity

– 15 –

Business Problem n  A US-based mobile provider struggled

with a combination of high costs but low customer satisfaction related to customer care.

n  An increasing share of support cases are related to mobile data usage and associated charges.

n  Traditionally, contact center agents did not have granular insights into a particular customer’s data usage, hence were unable to provide effective call resolution.

n  With Hadoop, one operator detected that 25% of callers were contacting the call center merely to have their late fees on the monthly bill waived.

n  The provider was able to off-load these cases to online self-service and interactive voice recognition.

n  Frees up the agents to focus on more valuable customer interactions.

n  The provider is now extending this solution to focus on issue resolution.

Value Realized

Network Care Sales

New Biz

Service and Security – Field Service Productivity

– 16 –

Business Problem n  A provider’s contact center agents had

insufficient ways of diagnosing what was wrong with customers, leading to many unnecessary truck rolls.

n  In particular, the agents were not able to triage network vs. home-based problems accurately enough.

n  Therefore, technicians were dispatched to the customer premises for problems that reside within the network.

n  The provider was able to avoid a large number of “false positive” truck rolls.

n  With each truck roll costing about $150 fully loaded, the provider was able to save several million dollars already in the first year.

Value Realized

Network Care Sales

New Biz

Sales and Marketing – 360 Degree View of Customer Value

– 17 –

Business Problem n  Telcos and cable companies interact

with customers across many channels and points in time.

n  Data about those interactions is stored in silos.

n  Difficult to correlate data about customer purchases, marketing campaign results, and online browsing behavior.

n  Problem is exacerbated by recent acquisitions and a proliferation in the volume and type of customer data.

n  Merging that data in a relational database structure is slow, expensive and technically difficult.

n  Enterprise-wide data lake of several petabytes

n  360-degree unified view of the customer (or household) life time value based on usages patterns across time, products and channels.

Value Realized

Network Care Sales

New Biz

Sales and Marketing – Personalized Marketing Campaigns

– 18 –

Business Problem n  Mobile phones not only follow their

owners everywhere, but also reveal a lot about their owners’ interests through browsing behavior and the applications present on the phone.

n  Telcos are looking for ways to mine that information.

n  Provider risked losing substantial revenue as prepaid customers were starting to switch to a competitor as a result of a particularly effective marketing campaign.

n  Pinpoint those individual customers most at risk of churning, and then built a highly targeted campaign to retain the remaining customers in that segment.

n  A churn alarm system was established and revenue leakage was minimized.

n  Telesales revenue increase by 50% by tracking competitors web-sites visited and counter offers to products searched

n  +20% conversion rate increase by optimizing and personalizing the path-to-transaction

n  $1.65 ARPU increase for 1 million customers boosts topline by $20 million per year.

Value Realized

Network Care Sales

New Biz

Sales and Marketing – Up-selling and Cross-selling

– 19 –

Business Problem n  The provider needed to find an

approach to upsell smart phones into a user base that was still largely on legacy feature phones.

n  The operator converted many hundred thousand feature phone users to smart phones with associated data plans.

Value Realized

Network Care Sales

New Biz

Sales and Marketing – Next Product to Buy (NPTB)

– 20 –

Business Problem n  As telco product portfolios grow more

complex, there are ever more opportunities to sell additional services to the same customer base.

n  Many sales reps however are overwhelmed with that complexity and struggle to translate the breadth of the product portfolio into incremental sales.

n  Confident NPTB recommendations, based on data from all its customers, empower sales associates and improve their interactions with customers pre-transaction.

Value Realized

Network Care Sales

New Biz

Sales and Marketing – Churn Reduction

– 21 –

Business Problem n  A North American provider faced the

following challenge: 50% of new customers churned off within 6 months of acquisition.

n  The average customer life time in this segment was 13 months, well short of the 18 months needed to break even.

n  The provider increased the “right” customer acquisitions by 27% and decreased subsequent churn in this segment by 50%.

n  Price related churn down by 40% n  Reducing cable subscriber churn (“cord

cutting”). Every 100,000 subscribers equates to customer lifetime value of $1 billion

Value Realized

Network Care Sales

New Biz

New and Adjacent Businesses – Actionable Intelligence Serving Advertisers

– 22 –

Business Problem n  Europe’s leading real estate

marketplace Scout24 – a subsidiary of Deutsche Telekom – features more than one million properties for rent or sale at any given time, and has facilitated more than 20 million property transactions over the last few years.

n  The company wanted to drive more market share to Scout24 by offering advertisers – typically real estate agents and brokers – an even better service.

n  A small team consisting of a product manager, a data scientist and a few developers was able to make a meaningful contribution to revenue growth.

Value Realized

Network Care Sales

New Biz

Big Data as a Product: ImmobilienScout (Deutsche Telekom)

– 23 –

Network Care Sales

New Biz

New and Adjacent Businesses – Actionable Intelligence Serving Payment Processors

– 24 –

Business Problem n  Credit card issuers experience

increasing fraud when their card members are travelling abroad.

n  95% of travelers opted into the SMS alerting service, resulting in a substantial decrease in fraud related to card use in foreign countries.

Value Realized

Network Care Sales

New Biz

New and Adjacent Businesses – Actionable Intelligence Serving Federal Governments

– 25 –

Business Problem n  The Eastward expansion of the

European Union has resulted in a longer and more porous border to non-EU member states.

n  This has made it more difficult to protect the EU against a stream of illegal goods and refugees, which often travel over land from the EU’s Eastern and South-Eastern neighbors.

n  Law enforcement agencies are able to target their scarce resources much more effectively, for instance choosing to intercept suspicious cars traveling in certain directions at speeds above 130km/h.

n  This radically increases their hit rate per mission.

Value Realized

Network Care Sales

New Biz

Agenda

•  The Journey to a Data Driven Organzation

•  21 Telco Use Cases for Hadoop

•  Overall Lessons Learned

– 2 –

Lessons Learned – Technology Disciplines

– 48 –

Security. XA Secure is a big step forward. But internal security sign-offs are complicated. New possibilities opened up by the data lake imply a steep learning curve.

Data quality. Early use case was volume of structured data. Gets harder now with unstructured data use cases.

Metadata management needs to be unified across BI and Hadoop.

Coordination between data publishers and consumers on our internal social network breaks down barriers.

Lessons Learned – Business Disciplines Sponsorship. Even our CEO is aware that Hadoop has a role to play in the transformation of our business.

Ownership of projects always sits with business people. Success is measured.

On-boarding 10 new use cases per year.

Governance is difficult in a federated organization. Hence the CEO pushes for success in a few areas, and the rest of the organization can opt in.

Funding and incentives. We pay the internal data producers, there is no free data. If you consume our data, you have to share all of your data, no cherry picking.

Application development not that different. Important to apply usual coding best practices here as well.

Cloud is intriguing for rapid prototyping, side-stepping procurement or where sources are in the cloud. However, different clouds are not fully interoperable, resulting in some lock-in.

Power plays. Data has power. A data lake brings lots of power. Beware of internal politics over who should own Hadoop.

– 47 –

Driving Business Performance in Telco with Hadoop

21 Use Cases with Lessons Learned at the Intersection of People, Processes and Technology

Questions? @juergenurbanski LinkedIn: juergenurbanski