predictive analytics, ai and the promise of personalization

Copyright © 2016 Earley Information Science1

Predictive Analytics, AI and the

Promise of Personalization

May 25, 2016

Copyright © 2016 Earley Information Science

Seth Earley, EIS

Dino Eliopulos, EIS

Julie Penzotti, Amplero

Adam Pease, Articulate Software


Today’s Agenda

• Welcome & Housekeeping

• Dino Eliopulos, Managing Director, Earley Information Science

(@DEliopulos)

• Session duration & questions

• Session recording & materials

• Take the polls & the survey!

• The Panelist Point of View

• Seth Earley, CEO, Earley Information Science (@SethEarley)

• Julie Penzotti, VP, Customer Analytics, Amplero

• Adam Pease, CEO & Principal Consultant, Articulate Software

• Expert Panel Discussion

• Questions & Answers

• Join the conversation: #earleyroundtable

Copyright © 2016 Earley Information Science3 Copyright © 2016 Earley Information Science

Predictive Analytics, AI and the Promise of

Personalization


Dino Eliopulos - Biography

Dino Eliopulos

Managing DirectorEarley Information

Science

Deep specialization User experience and highly complex business

applications

Over 20 years of experience Machine Learning, Data Mining and other AI techniques

applied to deliver rich content-driven solutions

Financial Services, Retail / CPG, Telecommunications,

Travel and Entertainment, Healthcare, Pharmaceuticals,

Hi-Tech Manufacturing and Energy

Strategy, planning, forecasting, budgeting, measurement,

sales, talent acquisition / management and retention,

career stewardship, program management and service

delivery

IT professional services

Highly collaborative and results-oriented management

style delivers outstanding outcomes for clients, employers

and teams

Industry experience

Experienced leader and innovator in industry and high-end professional IT consulting

Outstanding outcomes


Predictive Analytics, AI and the Promise of Personalization

5

Personalization has been the big promise for the past 15 years. The problem is that this

vision is still a long way from reality.

Meaningful personalization requires

• meaningful knowledge and content assets

• the use of analytics to understand and model customers

• prediction to anticipate what they need and principles of AI

to fulfill the promise

This roundtable will review the state of the industry and discuss the problems and

challenges inherent in understanding and anticipating user needs and ways that

organizations can move the needle, improve engagement and move up the

personalization functionality maturity curve.


Personalization, predictive analytics and product data

6

Personalization, Predictive Analytics and Product Information three aspects of making

more appropriate recommendations for customers.

• Well curated product information in an ontology is at the core of ecommerce

offerings

• Predictive Analytics draws from sales and customer data sources in order to

provide product recommendations

• Personalization depends on rich structured product content and digital assets

• Machine Learning algorithms can improve the effectiveness of targeting and

improve effectiveness of merchandizers

• A combination of crowdsourcing, merchandizing expertise, curation and

automated techniques can be leveraged in an optimal solution


Personalization Components

Grouped Product

SetsProduct Data

Sales Data

Contextualized

filtering

Customer

Profile Data

Search History

Solution DomainsSolution

Landing Pages

Natural Language

Processing &

Inference

Domain

KnowledgeInformation

Extraction

PDF/

Unstructured

7


Seth Earley - Biography

Seth EarleyCEO and Founder

Earley Information Science

Over 20 years experience

Current work

Co-author

Editor

Member

Former Co-Chair

Founder

Former adjunct professor

Guest speaker

AIIM Master Trainer

Course Developer &

Master Instructor

Data science and technology, content and knowledge

management systems, background in sciences (chemistry)

Enterprise IA and Semantic Search

Information Organization and Access

US Strategic Command briefing on knowledge networks

Northeastern University

Boston Knowledge Management Forum

Long history of industry education and research in emerging fields

Academy of Motion Picture Arts and Sciences, Science

and Technology Council Metadata Project Committee

Editorial Journal of Applied Marketing Analytics

Data Analytics Department IEEE IT Professional Magazine

Practical Knowledge Management from IBM Press

Cognitive computing, knowledge and data

management systems, taxonomy, ontology and

metadata governance strategies


• Personalization is based on “electronic body language”

– Web site behaviors, click streams, downloads, consumed content

– Past purchases

– Social media

– Social graph

– Explicit preferences

– Derived attributes

– Hidden characteristics

Personalization signals


• The question is, what do we offer?

• Once we know something about a user, what do we do with that

knowledge?

• We are trying to give them something we think they want

– In the context of their task

– To meet a specific need

– Solve a particular problem

• Personalization is making a recommendation about a product, service,

solution, piece of content or next action based on what we know in

advance and what the customer is telling us at that moment

Personalization as Recommendation


• Offers – need offers that can be recombined

• Content – content to support the user’s task

• Products – what products will be presented to the user

• Rules (derived or developed) – how will assets and content be

assembled for the user

• Need to identify and understand customer segments, behaviors

and content to drive desired behaviors

Components of Personalization


Mine data sources for customer behaviors and product groupings

12

• Product Attributes derived from analytics

• Correlating POS data, PIM, Tech Support

• Structured textual data mining text

• Reuse rich and mature ontology

• Inference engine to deduce relationships

• Derived, curated and synthesized product data to support customer tasks and processes

• Integrated into user experience to generate custom suggested search results

How can we infer what products customers want to see when they enter a search term?

Can we improve conversions of products based on search engine marketing (paid search)?

Knowledge Content

• Portion of revenue from high value customers

• Time between purchases

Sales analysis

• High value customers

• One time buyers

• Lapsed customers (retargeting)

• Tasks, solutions, interests

Customer Profiles

• Keyword searches and subsequent behaviors (conversions vs abandonment)

Web Behaviors

• Hi value product bundles, product bundles

• Segment and product bundle relationships

Product Data

• Organizing principles and related content

Competitors/Suppliers

PRODUCT DATA ENHANCEMENT

DATA MINING

DATA SOURCESEXISTING ECOMMERCE PLATFORM

RESULTING DATA ASSETS

What products are purchased together?

What keywords lead to what behaviors?

How are customers described and grouped?

+


Analysis Approach

13

Use case Input Analytical

approach

Output Purpose or Benefit

Sales pattern

analysis

Order size, product

mix

Unsupervised to

identify clustering

Sales correlated with

customer types, segments

and product combinations

Repeat customers, one time buyers, lapsed customers (for personalized

retargeting offers), time between purchases – customer journey

(segment and product bundles), customer value, top value customers

generating most revenue, highest profit, portion of revenue from high

value customers and related clickstream behaviors

Real time behavior

(electronic body

language)

Search logs, paid

search, click

stream data, email

marketing results

Supervised learning

to identify keywords

leading to high

margin sales clusters

Keywords and messaging

clustered with concepts and

customer attributes leading

to conversions

Insights on keywords and concepts related to purchase funnel – what

people do once they are on the web site, where they abandon based on

intent inferred from email traffic, organic search, paid search and onsite

search and subsequent behaviors

Competitor

analysis

Product results

from keywords

Crawling, content

mining, graph

building

Target product classes for

optimization

Compare search results from failed searches with competitor results to

identify opportunities for improved experience

Interests

questionnaire

Customer

responses, sales

data

Combination of

supervised and

unsupervised

Topic map of possible

offering areas aligned with

customer interests

Correlate interests with RFM, seasonality, industry, product bundles,

high value customers, tasks, solutions

Failed conversions Web traffic logs,

web analytics

Tree search

methods (binary,

monte carlo, etc)

List of

terms/phrases/concepts for

optimization

Supervised learning to identify customers who are abandoning cart

versus not abandoned, what search terms are driving customers to

abandon, areas for remediation (content for search terms)


Discovery of product combinations

Identify competitive differentiators, strategic initiatives, priority categories.

5 – 10 target processes

Products grouped to support task, process or solution

MERCHANDIZERSTarget categories

Target processes

Intelligent Parser

USE CASES TARGET PROCESSES

PRODUCT COMBINATIONS

KNOWLEDGE AND EXPERTISE CONTENTCustomer Support Content

Maintenance manuals

Key Opinion Leaders

What products are used in combination?

Supports SEO, surfaces expertise and related content

RELATED CONTENT

14


Poll Question #1

What is the maturity level of your knowledge and use of data-driven

personalization?


• None

• Dabbling

• Successful proof-points

• Concentrated capability development

• Core strength

What is the maturity level of your knowledge and

use of data-driven personalization?


• Continued development of Amplero, a self-learning

personalization technology platform for B2C marketing

automation

• Focus on deep data understanding, developing key findings,

driving customer interpretation/understanding

• Previously a scientist and consultant in the pharmaceutical

industry, specializing in data mining and analytics for drug

discovery

• 25+ publications and several patents.

• Earned a Ph.D. in Bioengineering and M.S. in Physical Chemistry

from the University of Washington and received her B.S. in

Biomedical Engineering from Duke University.

Julie Penzotti - Bio

Julie PenzottiVP, Customer Analytics

Amplero

CONFIDENTIAL

Purchases,Usage,Contacts,Demographics,Social Connections& their demographics,Propensity ModelsOther…

Big Data and the Age of the Customer

Channel,Day,Time,Location,Other…

Execution

Offer,Offer Expiry period,Incentive Type,Incentive Amount,Message,Creative,Semantic Tags,Other…

Experience

Context

Customer

Today’s customers…• Expect you know them• Are fickle and jaded• Tell others what they think• Are always connected• Are empowered to act

CONFIDENTIAL

Unfortunately, rules-based approaches don’t scale for B2C

[ 19 ]

With20 – 30

targeting ruleswhich onewill work

best?

?

CONFIDENTIAL

Machine Learning to automate and optimize targeting at scale

[ 20 ]

Modelling & Enrichment

Marketing Asset

Library

Machine Learning

Experimenting

Enriched Data

Decision Tree

OffersCustomer

DataCustomers

Marketer

Decisioning

Tomorrow’s marketer…• Agile and responsive• Runs campaigns in a loop• Gathers and applies

insights constantly• Thinks empirically rather

than intuitively• Let’s the machine do the

heavy lifting

CONFIDENTIAL

Machine learning to discover personalized contexts that optimize performance

[ 21 ]

DISCOVERED BY AMPLERO

Revenue Lift: +4%Confidence: Low

CONFIGURED BY CAMPAIGN MANAGEROffer: Unlimited Upgrade

Eligibility: International Saver Plan Subscriber

Revenue Lift: -4%Confidence: Medium

Revenue Lift: +8%Confidence: Medium

Condition:+Voice Consumption Cluster 5

Condition:+Voice Consumption Cluster 4

Revenue:+14%High

Revenue: -1%High

Revenue:-5%High

Offer Price: $10 $15 $20

Revenue:+6%High

Revenue:-10%High

Smart Package Owner: No Yes

KPI

Targets

KPI

Controls

Revenue Lift: +10%

Confidence: High

CONFIDENTIAL

Multi-armed bandits to manage decisioning for marketing contexts:

– Hedge bets about which choice is best

– Increasing certainty as more response data is gathered from customers

– Exploration/exploitation trade-off permits agility and adaptation

– Generalized learning over customer and marketing attributes

– Automatically segments population according to responses to different experiences

[ 22 ]

Machine learning for adaptive personalization and maximum benefits

Mean Lift Estimates of PerformanceContext 1 Context 2 Context 3 Context 4

Probabilityof Selection

Bandit Policy

Customer Attributes + Experience + Execution

OptimizationModels


Poll Question #2

What is the appetite and interest of applying machine learning in

your organization?


• Organization still skeptical

• Open to, but not sure where to dive in

• Trying out techniques

• Clear identified ROI and priorities

What is the appetite and interest of applying machine

learning in your organization?


Adam Pease - Bio

Adam Pease

CEO & Principal

Consultant

Articulate Software

[email protected]

History

Undergrad CompSci, Doctorate Linguistics

Program Manager, Teknowledge (mostly DARPA contracts)

CEO, Articulate Software (commercial consulting in data

modeling)

Cognitive R&D Manager, IPsoft

Specialties

Suggested Upper Merged Ontology

Open source, higher-order logic, 15 yr history, mapped to

WordNet

http://www.ontologyportal.org

Sigma

Open source, Reasoning, ontology modeling, deep NLP

“Ontology: A Practical Guide” - 2011

http://www.ontologyportal.org/

Adam Pease – Articulate Software Earley Executive Roundtable


Don't forget about knowledge based methods

• What's the problem you're trying to solve?

• There's more than just matching to do

• Matching methods reaching asymptote on many tasks

• Semantics is often what's missing

• Semantics and KR matters

• What's most popular may not be the best technical

solution


Personalization as Dialogue

• Are we making the problem too hard?

• Billings and Reynard (1981) – 73% of air traffic incident

reports involved problem in communication

• People have problems answering questions and communicating too

• Dialog is how we address the problem with people


Knowledge Discovery

• Use Data Mining to discover trends and relationships

• Express them in computable semantics

• Can be explained

• Spurious correlations can be understood and corrected

• Consolidate gains – don’t learn things that are already known


Suggested Upper Merged Ontology

• Initial versions: 1000 terms, 4000 axioms, 750 rules

• Mapped by hand to all of WordNet 1.6

• then ported to 3.0 and continually updated

• Associated domain ontologies totalling 20,000 terms and 80,000 axioms

• Now linked with factbases including YAGO for millions of facts

• New ontologies of Hotels and Dining

• If-then rules, not just a taxonomy or semantic web structure

• Free

• SUMO is owned by IEEE but basically public domain

• Domain ontologies are released under GNU

• www.ontologyportal.org


Poll Question #3

Does your organization take a rules-based or statistical based

approach to personalization?


• Primarily rules-based

• Primarily statistical-based

• Both

• None

Does your organization take a rules-based or

statistical based approach to personalization?


Panel Discussion


Roundtable Discussion

Dino Eliopulos

Managing Director Earley Information

Science

Seth Earley

CEOEarley Information

Science

Adam Pease

CEO & Principal ConsultantArticulate Software

Julie Penzotti

VP, Customer AnalyticsAmplero


Suggested Resources

• Introductory video on ontology - http://www.youtube.com/watch?v=EFQRvyyv7Fs

• Pease, Adam. “Ontology: A Practical Guide” - http://www.ontologyportal.org/Book.html

• SUMO on line - http://54.183.42.206:8080/sigma/Browse.jsp?kb=SUMO

• Ontology Publications - http://www.adampease.org/professional/

• Adam Pease’s Podcasts & Blog - http://www.ontologyportal.org/

• Earley, Seth. "Cognitive Computing, Machine Learning and Personalization: New Marketing Constructs or

New Capabilities?" KMWorld, November/December 2015. http://www.kmworld.com/

• “Making it Personal: Strategies for Creating Meaningful Customer Interactions”

http://www.earley.com/blog/making-it-personal-strategies-creating-meaningful-customer-interactions

• “Contextualizing Customer Journeys” Earley Executive Roundtable, Nov 2015

http://info.earley.com/roundtable-contextualize-customer-journeys

• Penzotti, Julie, “Marketing in the Age of Machine Learning: How optimising personalization granularity

leads to better performance in a dynamic market”, Applied Marketing Analytics, Vol 2 (1):41-51

• Amplero research links: http://www.amplero.com/research/article/?s=why-offer-response-rate-is-the-wrong-

metric-for-evaluating-marketing-performance

http://www.youtube.com/watch?v=EFQRvyyv7Fs

http://www.ontologyportal.org/Book.html

http://54.183.42.206:8080/sigma/Browse.jsp?kb=SUMO

http://www.adampease.org/professional/

http://www.ontologyportal.org/

http://www.earley.com/blog/making-it-personal-strategies-creating-meaningful-customer-interactions

http://info.earley.com/roundtable-contextualize-customer-journeys

http://www.amplero.com/research/article/?s=why-offer-response-rate-is-the-wrong-metric-for-evaluating-marketing-performance


Earley Information Science

(EIS)

Information Architects

for the Digital Age

Founded – 1994

Headquarters – Boston, MA

www.earley.com

For more info contact:

[email protected]

[email protected]

Thanks to our Sponsors

Next Roundtable topicJune 22 – Site Search: The Battle for Relevance


Backup


SUMO+Domain Ontology

Military

Geography

Elements

Terrorist Attack TypesCommunicationsPeople

Transnational Issues

Finance

Terrorists

EconomyNAICS

TerroristAttacks

DistributedComputing

BiologicalViruses

WMDECommerce

Services

Government

Transportation

World Airports

Total Terms Total Axioms Total Rules

20977 88257 4730

Relations: 1280

Hotel

Food

Hotel

Dining

Media Domain

Cars

UI/UX

SUMO

Mid-Level

Qualities

Mereotopology

Graph ProcessesMeasure Objects

Structural Ontology

Base Ontology

Set/Class Theory TemporalNumeric


WordNet

• A dictionary for computational linguistics applications

• 100,000 word senses, hand-created

• Mapped by hand to SUMO

• Open source

• Semantic links

• Aid in computation

• Verification of meaning during construction


Formal Ontology

• WordNet has synsets for “earlier” etc

• But nothing in WordNet would allow a computer to assert that the

end of one event precedes the start of another if one event is earlier

than the other

• This is not a criticism of WordNet

time

(<=>

(earlier ?INTERVAL1 ?INTERVAL2)

(before

(EndFn ?INTERVAL1)

(BeginFn ?INTERVAL2)))

Interval 1 Interval 2


Example Rules

(=>

(instance ?DRIVE Driving)

(exists (?VEHICLE)

(and

(instance ?VEHICLE Vehicle)

(patient ?DRIVE ?VEHICLE))))

“If there's an instance of Driving, there's a Vehicle that participates in that action.”

Not just an English definition for humans to read, but a logical definition that can be used in proofs.

predictive analytics, ai and the promise of personalization

Business