bsc 3362 - big data and social analytics - iod conference (ibm)

#ibmiod#ibmiod

Revolutionizing How Business Understands Customers -- Big Data Meets Social Analytics

Session Number BSC-3362

Aya Soffer | Director, Information Management & Analytics Research | IBMMark Heid | Program Director, Social Analytics | IBM

#ibmiod

Please noteIBM’s statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion. Information regarding potential future products is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information mentioned regarding potential future products is not a commitment, promise, or legal obligation to deliver any material, code or functionality. Information about potential future products may not be incorporated into any contract. The development, release, and timing of any future features or functionality described for our products remains at our sole discretion.

Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user’s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here.

#ibmiod3

Agenda

12

3

Our Perspective on Big Data Analytics

A Look at Big Data Social Analytics• Multi-channel Marketing• Customer Care and Insight• End-to-End Demo

IBM Research: Driving the Revolution in Big Data Social Analytics

#ibmiod4

Variety

Volume Velocity

Veracity

of Tweets create daily

12 terabytes

trade eventsper second

5 million

Of video feeds from surveillance cameras

100’s

We’ve Moved into a New Era of Computing

“We have for the first time an economy based on a key resource [Information] that is not only renewable, but self-generating.

Running out of it is not a problem, but drowning in it is.”

– John Naisbitt

#ibmiod5

Challenges of Big Data – The New Mix of Information

Enterprise Data Machine Data Social Data

• Volume

• Structured

• Throughput

• Velocity

• Semi-structured

• Ingestion

• Variability

• Highly unstructured

• Veracity

#ibmiod6

Compute Intensive

Storage Intensive

• Fraud Detection• Smart Grids and Smarter Utilities

• Risk Management and Modeling• Asset Management and Optimization • Call Detail Records• Call Center Transcripts• Log Analytics

• 360°View of the Customer• Data Warehouse Evolution

Gain more complete answers to business decisions to make better decisions faster

Ask new questions about their business to uncover new value or realize cost-savings

Explore and experiment to find new opportunities and create new business models

Typical Client Use Cases with New Types of Analytics

#ibmiod7

IBM Big Data – Analytics and Platform

• Addresses 4Vs of information

• Harnesses the next wave of analytics that exploits value from a rich information mix

• Fosters a new era in analytical applications

Visualize and Experiment

Predict

Integrate and Govern

HadoopSystem

Stream Computing

DataWarehouse

Analyze Real-time

Search and Discover

IBM Big Data –Analytics and Platform

#ibmiod8

Pre-processing • Ingest and analyze unstructured data types

and convert to structured data

Combine structured and unstructured analysis • Augment data warehouse with additional external

sources, such as social media

Combine high velocity and historical analysis • Analyze and react to data in motion; adjust models

with deep historical analysis

Reuse structured data for exploratory analysis

• Experimentation and ad-hoc analysis with structured data

Visualize and Experiment

Predict

Integrate and Govern

HadoopSystem

Stream Computing

DataWarehouse

Analyze Real-time

Search and Discover

IBM Big Data - Analytics and Platform

Most Client Use Cases Combine Multiple Technologies

#ibmiod9

The intersection of social media and big data

#ibmiod10

Agenda

12

3




#ibmiod11

Source: Q4 2010, Unica’s Global Survey of Marketers

About half of marketers admit that their social

media marketing efforts are totally siloed

Measurement and ROI are elusive

Campaigns are poorly integrated

Only brand / mass marketing techniques are employed

Opportunity to engage individuals is ignored

11

Even though social media is pervasive, using it successfully in marketing campaigns today is hit or miss

#ibmiod12

By linking together social and customer data, we can help our clients market more effectively across multiple channels

Planning, coordinating and executing marketing campaigns to stimulate demand – it’s a process that includes social media

Create relevant

messages

Deliver targeted messages and offers

Optimize email, display and search ad programs

Insights from social media

and other data sources

Capture & analyze responses and

refine

12

#ibmiod13

Introducing: Multi-channel campaign management with integrated social analytics

13

An integrated approach which allows organizations to measure, adjust and, ultimately, use social media data to gain greater precision for their campaigns.

How can I leverage social analytics to optimize return on my campaigns?

How can I maximize the value of our social insights

for marketing?S oc ia l Media

Ana lys t

Marketing Mana ger

• Measure the social impact of campaigns through earned and owned media

• Gain greater campaign precision by applying predictive models to socially-derived segments

• Evolve and align marketing and social campaigns through a centralized workspace

#ibmiod14

Big Data Social Analytics inSocial Business & Smarter Commerce

#ibmiod15

“Benjamin's Grocery” - Winning with Social Analytics & Smarter CommerceHow does it work?

Customer Website Behavior• Clicks• Searches• ViewsPrevious Campaign Data• Contact history• Response/purchases• Test campaigns Modeling Scoring Campaigns

Rank best offers

Social Media• Tweets• Blogs• ForumsCommunities• Surveys• Advocate dialog• Discussions

Derive ideas, insights and actions from Social Media

Execute the campaign using Individual Data for consumers who opted-in

Multi-Channel Marketing

AnalyticsSentiment dashboard

Emerging Topics Affinities Conversations you asked

about and those you didn'tWhat is correlated with what?

Pulling consumers from where the conversation is on the web, match them to segments based on

their actions on Benjamin's website

1

2

3

Perceptual Map Spatial alignment of attributes

Predict who is likely to respond

#ibmiod16

“Benjamin's Grocery” - Winning with Social Analytics & Smarter CommerceWhat is the storyline?

Introducing Benjamins Grocery Stores Competition in the grocery business can be intense and Benjamins faces their fair share with Jurassic, a low-price chain with broad presence in the market.

The Market Event On January 20th, 2012, Jurassic announces the end of ad hoc campaigns and the beginning of “every-day low prices”. They drop prices by 12-15% for 3000 products.

Benjamins' Research Knowing that they can't profitably copy Jurassic's price strategy, Benjamins mobilizes a team of experts to search for a better response. They discover that customers have a core un-met need for “healthy, interesting meals at a fair price”.

Benjamins' Response The Benjamins team rapidly tests a creative plan to hire well-known chefs to sponsor new recipes that use Benjamins store brand products. Their communities-of-interest like it – particularly “Moms”, “Singles” and “Gourmets”. They kick-off a new 1:1 cross-channel campaign that lasts through the rest of Q1.

The Results Over the two-month campaign, Benjamins gains market share and grows profit by 8%.

#ibmiod17

“Benjamin's Grocery” - Winning with Social Analytics & Smarter CommerceWhat products are used?


Rank best offers


Derive ideas, insights and actions from Social

Media

Execute the campaign using Individual Data for consumers who

opted-in



Emerging Topics

Affinities Conversations you asked




1

2

3



How can Benjamin's quickly understand their differentiators and competitor vulnerabilities?

Where can all of the relevant information be brought together for productive decision-making?

What optimization can be applied to campaign parameters?

1

2

3

What can they use to do root cause analysis and uncover un-met needs among their target customers?

How can Benjamin's pivot from aggregate to individual data?

#ibmiod18

“Benjamin's Grocery” - Winning with Social Analytics & Smarter CommerceWhat products are used?


Rank best offers


Derive ideas, insights and actions from Social

Media

Execute the campaign using Individual Data for consumers who

opted-in



Emerging Topics

Affinities Conversations you asked




1

2

3



● Cognos Consumer Insight 1.1● SPSS Modeler 15.0● Cognos 10.1● Connections 4.0

● Coremetrics Web Analytics ● Cognos Consumer Insight 1.1

● Unica Campaign● SPSS Modeler 15.0● Cognos Consumer Insight

1

2

3

© 2011 IBM Corporation19

Speakers• Leslie Ament, Vice President, Research & Client Advisory, Hypatia Research Group, • Mark Heid, Program Director, Social Analytics, IBM

Join IBM & Hypatia Research Group for insightful November 6th WebcastSocial Analytics & Intelligence: Converting Contextual to Actionable Insights Creating social intelligence by mining social media networks is no longer the sole purview of elite decision scientists or statisticians. Social analytics is increasingly integrated into work-flows and processes driven every day by business users.

This webinar will review the recent findings from Hypatia Research Group’s benchmark study, Social Analytics & Intelligence: Converting Contextual to Actionable Insights, and demonstrate how business users and analysts collaborate to transform a multitude of online contextual sources into insight, predict optimal next best actions and outcomes and act upon this consumer insight for business gain.

November 6th, 1:00-2:00 pm EThttp://events.unisfair.com/rt/IBM~SocialAnalytics

Converting Contextual to Actionable InsightsNovember 6th, 1:00-2:00 pm EThttp://events.unisfair.com/rt/IBM~SocialAnalytics

#ibmiod20

Business Analytics and Big Data Platform Integration

Business Analytics

InfoSphere BigInsights

Hadoop (Map-reduce)

File system (GPFS, HDFS)

Hive HBase

Cognos Insight

Cognos BI

Export and Explore

Unstructured Analysis

Reporting / Analysis Dashboards

CCI

Load through UDFs

SPSS Predictive

Cognos RTM

Real-time Analytics

Predictive

BigSheets BigIndex

InfoSphere Streams

IBM Confidential: References to potential future products are subject to the Important Disclaimer provided earlier in the presentation

Data Warehouse

#ibmiod21

Agenda

12

3




#ibmiod22

Social Analytics in IBM Research - moving up the value stack to extract actionable insight

Filtering social media is challenging and critical

Summarization is critical in diffuse content streams)Relevance Filtering Topic Modeling

Information Summarization

Sentiment Lexical Pattern ExtractionNeeds to be multi-lingual and tuned to specific domains

Detecting intent to buy or intent to act or mood or brand attributes

Lexical Extraction

Influence Community DetectionInfluence is critical component for social media filtering and Enterprise expertise

Discover hidden pockets of expertise in an enterprise setting

Influence and Communities

Customer Modeling Situational ContextExtract customer demographic features that can be joined with legacy attributes

Context (eg location) is key differentiator in an increasing number of applications

User Modeling

#ibmiod23

Social Pulse – What are employees saying about their company’s brand

Social Pulse

• A Social Analytics Solution for marketing and communications professionals

• Focuses on internal versus external consumer perception of your brands and products

• Based on the idea of your workforce being brand ambassadors

• Experimenting within IBM• Externally

>25,000 employees on Twitter, >300,000 on LinkedIn, and > 198,000 on Facebook

• And Internally > 300,000 IBMers use IBM Connections Communities, Blogs, Wikis, Profiles, Forums etc.

#ibmiod24 24

The UsersWhat brand

related topics are IBMers talking

about this week?Is everyone on board with our new

Smarter Planet strategy?

Which business units get the

message, which ones are still struggling?

Are our management teams helping our brands to be presented in

the best light?

Social PulseSocial Pulse

25

View Topics and Sentiment of your Workforce by Country

26

By Business Unit & Common Topics Across Business Units

Search for brand specific topics

27

Not All Business Units are Positive

Let’s see if there aredifferences across countries

Within S&D

28

S&D Ireland Very Positive, Opening New Technology Center, Ireland Research (= new

Technology Center) is reserved.

#ibmiod29

Brandy – Associating brand perceptions with customer traits

Mining of customer traits• Demographics• Personality• Fundamental needs• Preferences• …

• Integrating mined information with existing customer data

• Associating brand perceptions with customer traits especially their “needs map”

[Ford, 2005][Ford, 2005]

Brandy

inventive/curious vs.

consistent/cautious

friendly/compassion

ate vs. cold/unkind

outgoing/energetic vs. solitary/reserved

sensitive/nervo

us vs.

secure/confident

effic

ient

/org

aniz

ed

vs. e

asy-

goin

g/ca

rele

ss

#ibmiod30

Example: Modeling and Deriving Personality

0% 20% 40% 60% 80%

Openness

Conscientiousness

Extraversion

Agreeableness

Neuroticism

[Tausczik&Pennebaker 2010, Yarkoni 2010] [Tausczik&Pennebaker 2010, Yarkoni 2010]

Map the use of words, frequency, & correlation with Big5 based on LIWC

“Agreeableness”wonderful (0.28), together (0.26) …porn (-0.25), cost (-0.23)

Brandy

#ibmiod31

All Brands

Retailer 1

Retailer 2

Retailer 3

Openness – Liberalism Conscientious - Cautiousness

Example comparing 3 Retailers Brandy

#ibmiod32

Campaign management: a Retail Example

Openness: 83% Idealist: 62%Interest: Dining50% close ties: openness 75%

Openness: 83% Idealist: 62%Interest: Dining50% close ties: openness 75%

Openness: 23% Realist: 87%

Interest: Travel35% close ties: interested in travel

Openness: 23% Realist: 87%


Help Retailer identify customer segments to launch “CoolBrand” collectionHelp Retailer identify customer segments to launch “CoolBrand” collection

… Want your luggage to stand out at the airport? Never need to dust it? Here comes “CoolBrand” collection…

Save 5% by sharing this with your 5 (travel-loving) friends such as…

… Want your luggage to stand out at the airport? Never need to dust it? Here comes “CoolBrand” collection…

Save 5% by sharing this with your 5 (travel-loving) friends such as…

… experience fine dining at home in Italian fashion style: “CoolBrand” dinnerware…

Save 5% by sharing this with your 5 (open-minded) friends such as …

… experience fine dining at home in Italian fashion style: “CoolBrand” dinnerware…

Save 5% by sharing this with your 5 (open-minded) friends such as …

Brandy

#ibmiod33

A Smarter Cities Example

Conscientiousness: 23% Realist: 92%Interest: Foodies50% close ties: Conscientiousness 25%

Conscientiousness: 23% Realist: 92%Interest: Foodies50% close ties: Conscientiousness 25%

Neuroticism: 53% Idealist: 71%


Neuroticism: 53% Idealist: 71%


Help DMV identify suitable segments for different campaignsHelp DMV identify suitable segments for different campaigns

… Your current insurance policy is up for renewal …

Share this with your 5 (travel-loving) friends such as… and ask them to follow us to receive reminders…

… Your current insurance policy is up for renewal …

Share this with your 5 (travel-loving) friends such as… and ask them to follow us to receive reminders…

… Holiday is around the corner …Here are holiday safe driving tips: http://dmv.ca.gov/...

share this with your close friends such as …

… Holiday is around the corner …Here are holiday safe driving tips: http://dmv.ca.gov/...

share this with your close friends such as …

Brandy

http://dmv.ca.gov/

http://dmv.ca.gov/

http://dmv.ca.gov/

http://dmv.ca.gov/

#ibmiod34

Emergencies, call for help

Limited coverage

Analytics and fusion

in near-real-time

• Event / fact summarizations

• KPIs

sensorssensors

Crowdsource(voice& text)

Socialmedia

COPS – Crowdsource Oriented Public Safety

Automatic detection of Public Safety incidents and KPIs, from crowdsourcing data, which is incomplete, inaccurate and noisy

Use innovative “fusion analytics” to reliably detect incidents and trends from uncertain data, textual, spoken and numerical

COPS

#ibmiod35

Sample Use Case (Managing Natural Disasters)

Event 1 – 10:10 river water surging (from accumulation of tweets)

Event 2 – 11:15 fast moving water (from accumulation of

mobile messages)Event 3 – 11:15 – flood, major

road blocked (from accumulation of tweets and mobile messages)

Event 4 – 12:30 – flood (from accumulation of tweets and

mobile messages)Event 5 – 12:30 – traffic

accident (from accumulation of mobile messages)

COPS

#ibmiod36

COPSSystem automatically aggregates and filters the data

36

Event 1 – 10:10 river water surging (from accumulation of tweets)

Event 2 – 11:15 fast moving water (from accumulation of

mobile messages)Event 3 – 11:15 – flood, major

road blocked (from accumulation of tweets and mobile messages)

Event 4 – 12:30 – flood (from accumulation of tweets and

mobile messages)Event 5 – 12:30 – traffic

accident (from accumulation of mobile messages)

Crowd-source events that are progressive – updated as more crowd-source data becomes available

Crowd-source events that display the inherent uncertainty (confidence) – from the event description to the location

Crowd-source events that reflect aggregated data – to avoid overloading by large volume of crowd-source data

and to reduce uncertainty by fusing multiple posts

#ibmiod37

Main Module - Event Profile Generation

37

Event Event representationrepresentation

Join/FuseJoin/Fuse

/Aggregate/Aggregate

Entity/ Entity/ Event Event ExtractionExtraction

Unstructured data sources

BigInsights /BigData Platform

(3) Automatic Model Generation from entity schema to Event model on BigInsights

Event Schema

(2) Extraction/Integration Flow from unstructured data (tweets and crowd data) to JSON objects

FiltersFilters

Fuse & Fuse & AggregateAggregate

Data Data ingestingest

Streams / BigData Platform

(1) Data Ingestion filter relevant information from millions of messages

Event Detection

(4) Event Detection Statistical detection & model-based detection

(5) Reporting/Alerting/Dashboarding

Events, event summaries, trends, KPIs, Predictions

Statistical Statistical patternspatterns

COPS

#ibmiod38

Microcosm - uncover the commercial potential of local microcosms

• Understand the marketing potential of particular locations beyond the individual level

• Understand the potential of viral marketing• Identify promising community types and target marketing to them• Lower marketing costs by targeting earned media

Microcosm

#ibmiod39

Social Analytics to extract communities and Locations

Extended communityof people that talk about some subject

Identifying participants locationbased on profiles and discussions

Microcosm

#ibmiod40

Geographical Analytics – How it works

• GPS Geotagging (<5% of tweets)• Even if explicit in profile – disambiguation might be needed:

• E.g., “Springfield” by itself can refer to 30 different cities in the USA.• Techniques used

• Rule-basedE.g., “I live in ..”, “lets meet at ..”

• Machine learning (supervised):Statistical methods- find the most characteristic terms of people that report they live in some location x. E.g., “The Strip”, “Bellagio fountains”, “Freemont St.”…-> Las Vegas

• Based on Social Network, • i.e. learn location of people

based on the locations of their friends

Location 1 Location 2 Location 3

Microcosm

#ibmiod41

Community Analytics - How it works:

How we build the communities:• Build social graph based on the data flow in the social media. For

example, in Twitter, using the @Reply tag.• Extend the connections with friends, followers, following, etc.• Then use clustering-based approach

What we gain from the communities analysis?• which features have commercial significance• which features can be acted upon

Microcosm

#ibmiod

Thank You!

Your Feedback is Important! • Access SmartSite to complete your session surveys

o Any web or mobile browser at iodsmartsite.com o Any SmartSite kiosk onsiteo Each completed session survey increases your chance to win

an Apple TV with daily drawing sponsored by Alliance Tech

#ibmiod43

bsc 3362 - big data and social analytics - iod conference (ibm)

Documents

structured data ibm

customer data

big data social analytics3

big data analytics2a

challenges of big data

integrated social analytics

unstructured data typesand

social media search