data driven sales: building ai that searches, learns, and sells

Post on 15-Jan-2017

2.529 Views

Category:

Business

1 Downloads

Preview:

Click to see full reader

TRANSCRIPT

Data-Driven Sales: Building AI that searches, learns, and sells

Anand KulkarniChief Scientist, Co-Founder

LeadGenius

An Audacious Claim

In ten years, the job ofsalespeople

will be replaced by artificial intelligence.

For those of us who aren’t in sales…

What do salespeople do all day?

They find people/companies who might buy something

old school new school

They analyze which companies want to buy what they’re selling

old school new school

Sales people engage those prospects in commercial conversations (“selling”)

old school new school

What do salespeople do all day?

Salespeople Find Companies The Search Problem

Salespeople Analyze CompaniesThe Intent Problem

Salespeople Talk to PeopleThe Sales Problem

AI that Finds Companies The Search Problem

AI That Understands Buying BehaviorThe Intent Problem

AI that Talks to PeopleThe Email Turing Test

Three Problems of Interest

Let’s talk about each of these problems in turn.

AI that Searches for Customers:The Search Problem

The Company Search Problem

At LeadGenius, we want to figure out every single company in the world who might buy somebody’s product.

We’ll start by solving the slightly more general problem of finding every company in the United States.

After that, we’ll talk about how to decide which ones of those companies want to buy something.

Grabbing data about companies

We crawled data from fifty-five sources,including:• Social Media • Online Directories• Secretary of Sate Listings• SEC filings• IRS nonprofit database

What a company looks like

A problem: how do we tell if two companies might be the same?

Unrelated companies have very, very similar names.

Companies change names. A lot.

Entity resolution: The Fancy Way

A company p is a vector of ~30 properties that we know about it.(Name, address, revenue, industry, founding year, technologies used,…)

Entity resolution: The Fancy Way

A company p is a vector of ~30 properties that we know about it.(Name, address, revenue, industry, founding year, technologies used,…)

two companies are the same if distance (p1, p2) < edistance between companies = probability of same

Entity resolution: The Fancy Way

This works, but…

Super slow!

Requires us to do pairwise comparisons …… potentially across a huge numberof data points and data sources.

Sometimes data falls out of date.

Quiz:What’s an easier way to solve this?

Let’s find a set of properties that are less likely to change often.

Entity Resolution: The Easy Way

Two companies are the same if and only if they have the same “official” physical address.

So… how many businesses are in the US?

21,708,021 US businesses6,049,655 US businesses have >1 person

• Yelp (~47M establishments, some of which are same company)

• LinkedIn (~2M unique companies)• CrunchBase (~650K unique companies)• AngelList (~289K unique companies)

Some queries we can answer• Which U.S. industries have the most distinct organizations listed

in LinkedIn?

Industry CountConstruction 157533Real Estate 114366Information Technology and Services 113292Hospital & Health Care 99552Marketing and Advertising 87820

• Q: How many Fortune 500 companies have websites?• A: 499!

Bonus Problems

• How long is information trustworthy after we retrieve it? (decay functions)

• What’s the optimal frequency to retrieve information? (expectation-maximizations)

• How do we nab information from sites that don’t have cleanly-structured schemas? (watch humans do it)

AI That Understands Buying BehaviorThe Intent Problem

The Problem

Given a set of companies who have brought something from us in the past…… which companies are interested in buying from us in the future?

This is a very hard problem.

Non-generalizable: Whether someone’s buying something depends heavily on the specific industry.

Time-dependent: Whether some company needs a product is always changing.

The Conventional Approach: Machine Learning

From our previous step, we already have a whole set of companies represented as mathematical vectors.

We just need to train up a solid classifier to separate which ones are going to buy from us and which ones aren’t.

How much data do we need?

- confidential -

How it Works

• We train a neural net by showing it a whole bunch (greater than 10,000) labeled examples of companies who have bought our products in the past.

How it Works

• Our system learns a function that separates the objects in space.

How it Works

• For new objects, our classifier can decide which type it is!

There are some good ways we can use them here, too!

companies matchingan ICP

companies not matching an ICP

ICP - “Ideal Customer Profile”

A better strategy: Human Computation

• Pull a probabilistic estimate from our classifier on whether a company is in-market for a product or not.

• If the probability is low – below 80% – we escalate it to a trained person in a 500-person crowd who can make a human-powered determination on whether the company is going to buy or not. They can even add a feature.

• After we make that call, add that data to the training set to make the classifier smarter

• Boosts likelihood of success to human levels… depending on the human.

An even better strategy: monitor behaviors; cluster companies by stage

• Behaviors: For each one of the companies we’ve crawled, see whether they’ve:- Viewed a website that sells that technology- Opened a new job position relevant to the product- Started hiring- Tweeted about problem- Replied positively to an email someone sent

• Stage: Companies need different items at different stages in their lives:- Figure out where they are in a “lifecycle”- Match them against similar companies

AI that Talks to People:The Email Turing Test

How do we write (sales) messages by emailthat are as good as those sent by a human being?

Recent results from deep learning say yes.

Is this even plausible?

Xiaome (Microsoft China) trained on real-world chatbot logs

Support chats trained on data(Deep Mind, Google)

Example: Automated Support Interactions

Describe your problem: browser keeps crashing

Machine: hiHuman: hiMachine: could you please let me know what is the operating system you are using ?Human: linuxMachine: ok this is the issue with the system per- formance , nasser , tools .Human: what is that ?Machine: is it a windows machine ?Machine: we have linux on your machine .

Machine: is it windows xp or windows 7 ?

Human: i am using linuxMachine: let me take a remote of your machine and check if issue has been pushed properly . Human: okMachine: please open terminal and run the be- low commands in one computer :Machine: sudo apt - get update

1) Writing a Message2) Understanding Responses

Problem

Requirements

Given a topic, an author, an inbox, and a recipient, compose a message that:1) has a maximum likelihood of

yielding a positive reply2) looks indistinguishable from an

email that the author sent

This is a hard problem…. the Turing TestIt’s been around since the 40s.Restricting it to sales and email might make it easier

Secret Weapons

* We can choose to ask a human being from our crowd of trained folks for help. * We can mine the inbox for whatever examples we need or want.

Strategy

1) Craft a generalized template by analyzing the sender’s email inbox

2) Collect data at scale to populate that message3) Change content based on what you discover about that

person

Writing Messages

Writing Messages

Going further…- How likely is someone to reply to us based on…- Length?- Tone?- Subject complexity?- Word choice?

Let’s show this to the user and then optimize based on that.

How likely is someone to open this email?

Predicting responses from length

How likely is someone to open this email?

Predicting responses from templatization

Humans in the “crowd” can radically improve our templates automatically

Optimizing Templates

“Wish”, AAAI Human Computation 2014

What did someone say about our email?

Understanding responses

The hard way: sentiment analysis

Understanding responses

Positive sentiment corpus Negative sentiment corpus

Twitter as a Corpus for Sentiment Analysis and Opinion Mining (2011)

Question:What’s the easy way?

Understanding responses

The easy way: human computation

Understanding responses

Scripting responses

From: anand@leadgenius.comTo: sarah@hotlead.comSubj: Quick Question,

SarahHi Sarah,

I saw you guys were hiring for SDRs. We know each other through Michael

James and I wanted to see if we might be able to help you scale your SDR team. I have a few extra SDRs we

can push your way.

Let me know if you’d like to chat further – we’re doing

this for SoldLead8 already. BTW, congrats on your

recent round!Cheers!

AK

Interested?Here’s 3 times that work for

me!

Here’s more information!

Check back later.

Specific question

Automatically schedule a

follow-up mail

Scripting responses into a conversation

AI that Finds Companies The Search Problem

AI That Understands Buying BehaviorThe Intent Problem

AI that Talks to PeopleThe Email Turing Test

Conclusions

• Company search can be attacked with large-scale crawling, human computation, entity resolution, and careful data updates

• Buying intent can be deduced automatically based on classifiers but is done better with human computation

• Email communication is complex, has a lot of interesting subproblems, and is solvable!

anand@leadgenius.com@polybot, @leadgenius

www.leadgenius.com

(We’re hiring!)

That’s it!

top related