digital summit dallas 2015 - research brings back the 'human' aspect to insights

Intended for Knowledge Sharing only

Research is to Human Learning, what Analytics is to Machine

LearningDigital Summit

Dec 2015


Disclaimer: Participation is purely on a personal basis and does not represent VISA,Inc. in any form or matter. The talk is based on learning from work across industries and firms. Care has been taken to ensure no proprietary or work related info of any firm is used in any material.

Director, Insights at Visa, Inc. Enable Decision Making at the Executives/ Product/Marketing level via actionable insights derived from Data.

RAMKUMAR RAVICHANDRAN

Data Analytics Engineer. Data Architecture & Reporting Solutions to enable Decision Making

NIRANJAN SIVARAMAN


Quick recap of what it is


What makes a Company Iconic?

WHAT MAKES A COMPANY ICONIC


5% Brand

50%

What they want before they know

it

15% What they need

5% Sales & Service

20% What they want

5% Product

Good

Great

ICONIC

IN HIS OWN WORDS…


https://www.youtube.com/watch?v=2U3w5Blv0Lg

https://www.youtube.com/watch?v=2U3w5Blv0Lg

WHY IS RESEARCH NOT AS SEXY AS DATA SCIENCE


Customers sometimes may not know what they want

You cannot satisfy everyone all the time

Low Response rates, since no incentive for the Customers

What they say vs. what they do

Don’t always understand the questions correctly

Not easy to scale

Legal/privacy issues

HOW CAN WE HELP?


Product Sales & Service Brand What they

wantWhat they want

before they know itWhat they

need

Analytics

Monitoring

A/B Tests

Research

Reports+

Analytics+

Research+

Testing+

Mining

Active listening & pattern analysis could reveal unexpected opportunities…




Really, how can say so?

RESEARCH VS. ANALYTICS


RESEARCH ANALYTICS

Cost/Speed of doing it

Ease of Analyzing (Structure)

Sample Size

Type of Insights Attitudinal Behavioral

Attribution Inferred Direct

Greatest strength?

Finding out a hypotheses!

Testing the hypothesis

Analytics is the yang to the Research’s yin…




What can we do about it?

THE BIG TRENDS


Cost & ease of storing and analyzing unstructured data going down

Need for “Causal” answers vs. “Correlated” insights

Increased investment & avenues for getting Customer feedback – Social; Transactional; Voice

Text Analytics is maturing and getting integrated into Analytics suite

More executive sponsorship and incorporation of metrics like NPS into Corporate goals

LISTENING AS PART OF ANALYTICS MATURITY CYCLE


Inform

Act

Listen

PredictOptimize

Maturity phases of Analytics Practice

Valu

e A

dditi

on

Envision

Mine

TREAT CUSTOMER RESEARCH INTERACTION LIKE ANOTHER TRANSACTION


1 Record: Store every Research interaction and make it a “Profile” field to be used in predictive modeling

Relevant: Customize Research by the type and engagement level of the customers2

UED: Gamify the Research (not just financial incentive), make it easy/contextual/timely/deviced3

Accountability: If a user feedback went into new product design, “thank and inform them”4

Quantify: Has the customer satisfaction improved over time? Is it different across product types? Did the Lifetime Value go up?5

…Analysis & optimization of Research funnel mandatory to improve data and insight collection progressively

STRATEGIC EXECUTION OF RESEARCH


• Objective: Exploratory, Target, Monitoring• Target Customers: Engaged/Inactive, etc. • Where and how will it be shown: Focus/Trigger/Deferred, etc. • Success metric & criteria• Minimum sample size needed & time to run• Expected Corporate KPI bucket

STEPS

Strategy

Measure

Analyze

Planning

• Metrics instrumentation & logic(Conversions/Satisfaction/Share of Voice/Brand Awareness/Sentiment & Open/Click/Complete rate)

• Dimensions: Engagement Bucket, Devices, Time to Survey, Type of Survey, Geo, Type of Customers, Profile

DESCRIPTION

• Analysis of Survey response, the insights readout & recommendations (Sizing of opportunity, consistency vs. statistical significance)

• Text Analytics on the open commentary section – Entity extraction, theme identification, categorization, pattern identification, time series, structural analysis

• Vetting, validation & storyling across various sources.

• Additional research – in house/labs focus groups• Feature/Product planning & prioritization• A/B Testing

RESEARCH METHODOLOGY MATRIX


Research Methods

• Attitudinal vs. Behavioral – What do Consumers say vs. What do they do• Qualitative vs. Quantitative – Direct data gathering (surveys) vs. Implicit data inferences (Logs)• Context for Product Use – Lab vs. close to real life

RESEARCH METHODOLOGY MATRIX


Method DescriptionFactors

Speed Cost Inference Dev Stage

Prototyping

Usability Studies

Focus Group

Surveys & Feedback

Pre-Post

A/B Testing

Create & Test prototypes

internally (external, if needed)

Standardized Lab experiments –

Panel/s of thought-leaders;

Employees; Influencers

In-depth interviews for Feedback

Email/Pop-ups Surveys

Roll-out the changes and then

test for impact

Different experiences to users and then measure delta

Quickest (Prototypes)

Quick (Panel,

Questions, Read)

Slow (+Detailed interviews)

Slower (+Response

rate)

Slower (Tied to Releases)

Slowest (+Sampling+

Profiling+ Statistical

Inferencing)

Inexpensive

(Feedback incentives)

Relatively expensive

(+Lab)

Expensive (+Time)

Expensive (Infra to

send, track & Read)Costly (+Tech

resources)

Very Costly (+Tech

+Analytics +Time)

Directional

+Consistency across

users

+additional context-

Why?

+scale+Rigorous (Statistical

Significance). *Risk of

bad experience.

*Risk of bad experience reduced.

Ideation Stage

Ideation Stage

Ideation Stage

Ideation/Dev/ Post Launch

Post Launch

Pre Launch (after Dev)




A note on Social

CONSIDER SOCIAL…


1-in-7 are Mobile-Social daily and spend more than half hour a day!

Every 1-in-6 page goes Viral!

Social is cheap and easy!

Personalized!

1-in-5 people in the world are Social!

…BUT WITH CARE


Metrics may obfuscate reality

Like/follow & forget

Not always actionable or relevant (don’t always know fan vs. customer)

When it’s good, it’s difficult – lingo, emoticon, dialect, sarcasm

Bad data-spam/gaming/bad behavior

…it’s mostly reinforcement, not always influential (friend vs. expert)




Putting it all together

MAY PLAY A ROLE ACROSS THE BOARD


Product

Marketing

Operations

Fraud

Strategy

1. Monitoring throughout PLC2. User Experience issues3. Personalization – FB Connect

1.Promotion effectiveness 2.Brand/Public Relations initiatives3.Cross & Up-sell/Campaign designs

1.Platform uptime2.Conversion3.Quicker sales

1.CRM Effectiveness2.Proactive solutions

1.Brand Awareness, Share of Voice2.Engagement3.CLV

1.Needs assessment & roadmap2.Competitive assessments

1.Fraud/gaming2.Information Security

1.Reduced incoming calls & response times2.Relational NPS

1.Fraud rates2.Complex pattern identifications3.Post incident response

1.Industry and consumer pulse2.Consumer relationship stickiness

Function Possible applications Possible metrics that it can help

NEEDS FOR IT TO BE SUCCESSFUL…


1 Alignment with Strategic Goals and Outcome Focused approach

2Thought through Research Design –Strategic goals, success criteria, KPIs, initiatives, budget, executive ownership and cross checking with other information sources.

3Customized by Customer Type (Influencer/Engaged/Inactive/Prospect), context, device & a Strong Value Prop for customers to respond

4 Record & profile users and analyze Research funnel and improve the response rates and quality of feedback and insights.

5

Establish “Text Analytics” practice that translates findings into recommendations with estimated impact sizes that helps prioritization. This also helps in connecting dots across the organization (Analytics, Research, Reports, A/B Testing, etc.)

…Executive Support & sponsorship is assumed as a default necessity

SUMMARY

23

• “Know” that Research & Analytics each complete one half of the picture

• “Must have” goals, success criteria & tie up with Corporate KPIs

• “Ensure” Value Prop for consumers to respond

• “Develop” ‘learn-listen-test-learn’ framework

• “Prepare” for ever more increasing personal-mobile-social world and the possibilities & challenges of the new era.





Appendix

THANK YOU!


Would love to hear from you on any of the following forums…

https://twitter.com/decisions_2_0

http://www.slideshare.net/RamkumarRavichandran

https://www.youtube.com/channel/UCODSVC0WQws607clv0k8mQA/videos

http://www.odbms.org/2015/01/ramkumar-ravichandran-visa/

https://www.linkedin.com/pub/ramkumar-ravichandran/10/545/67a

RAMKUMAR RAVICHANDRAN

25

https://www.linkedin.com/in/niranjansivaraman

NIRANJAN SIVARAMAN

https://twitter.com/decisions_2_0






https://www.linkedin.com/in/niranjansivaraman






digital summit dallas 2015 - research brings back the 'human' aspect to insights

Data & Analytics