computational advertising in social networks
DESCRIPTION
Computational Advertising Workshop @ ICML 2012TRANSCRIPT
Computational Advertising
in Social Networks
Anmol Bhasin Sr. Manager
Analytics Engineering www.linkedin.com
We live in fascinating times. Two new nascent technological disciplines are coming together to transform how the marketers go about their business of reaching consumers, be it businesses or end users. It is time for the practitioners in these disciplines to push the envelope by creating innovative products and sophisticated algorithms to define what the future will hold in this new digitally social era.. Anmol Bhasin
Core Message
Source : www.140proof.com
Our Mission Connect the world’s professionals to make them
more productive and successful.
Our Vision Create economic opportunity for every
professional in the world.
Value proposition To make professionals better in the Jobs that they are already in.
World’s largest professional network
Over 60% of members are now international
6
161+ M
2 4 8
17
32
55
90
2004 2005 2006 2007 2008 2009 2010 LinkedIn Members (Millions)
Company Pages
>2M *
Professional searches in 2011
~4.2B
82% Fortune 100 Companies use LinkedIn to hire
*
*as of March 31, 2012
New Members joining
~2/sec
§ Information Seeking vs Information Consumption
§ Dedicated marketing channels for brand awareness required
§ Hypertargeting
§ Blending organic and sponsored content § Mobile ?
Challenges in Social Network Advertising
§ No Search Queries Q : Home Remodel San Francisco Bay Area
E(pCTR(clicka | qi,uj,C))>> E(pCTR(clicka | u j,C))
Sponsored Search vs Social Advertising
Sponsored Search vs. Social Network Advertising
Source : Marin Software www.marinsoftware.com
Dedicated Marketing Channels
Hypertargeting
E(max[c1,c2,c3.....c n ])> E(max[c1,c2,c3.....c n−1])
Blending Organic and Sponsored
The thing called Mobile..
§ Cannibalizing website page views
§ Small form factor § ~10% views from Mobile but only ~1% monetizable
§ Blending organic and sponsored essential
§ Impression & conversion tracking loop hard to close
The good news..
Hey user.. I know thee
19
https://inmaps.linkedinlabs.com
And your friends..
The good news..
source: h]p://inmaps.linkedinlabs.com
We also know what you read ..
And how much you liked it..
The good news..
Advertising @ LinkedIn
LinkedIn Marketing Solutions
Performance Marketing (LinkedIn Ads)
Organic Marketing
(Company Pages)
Brand Marketing (Display Ads)
Higher Educa-on
Internet Services
Tech B2B
MBA Programs | Masters & Graduate Programs Online Degrees | Execu-ve Leadership
CRM | SoGware/Biz Hardware | ERP | Sales Tools Marke-ng Automa-on | SaaS
Website Hos-ng | Video Conferencing Prin-ng | Phone Systems
Staffing Agencies | Recrui-ng SoGware Corporate Recrui-ng | Job Boards
Staffing & Hiring
Advertiser spectrum
Campaign creation
The basics -‐‑ Ad Ranking § Given
§ Objective
§ Goal: § Increase revenue § Respect daily budgets of Advertisers § Good user experience
Uj,{(c0,b0 ), (c1,b1), (c2,b2 ), (c3,b3)..(cn,bn )},H
argmaxi∈C
(pCTRi*bidi )
Virtual Profiling
Title : Eng Mgr Company : LinkedIn Location : CA,USA Skills : ML, RecSys
Title : Sr. SE Company : Google Location : PA, USA Skills : ML, DM
Title : Eng Dir Company : Linkedin Location : PA, USA Skills : ML, Stats, DM
Title : Sr. SE<1>, Eng Mgr<1>,
Eng Dir<1> Company :
LinkedIn<2>, Google<1>,
Location : CA,USA <2>, PA, USA<1> Skills :
ML<2>, RecSys<1>, Stats<1>, DM<1>
Virtual Profiling
Title : Eng Mgr Company : LinkedIn Location : CA,USA Skills : ML, RecSys
Title : Vice President Company : Twi]er Location : CA,USA Skills : DM, ML, RecSys ……………….
Virtual Profiling
Information Gain
§ Pick Top K overrepresented features from the clicker distribution vs the target segment
A representative projection of the item in the member feature space
CTR Prediction – CF Similarity
Ranker MEMBER FEATURES
Score to pCTR correction pCTRi
§ L2 regularized Logistic Regression (Liblinear, VW, Mahout, ADMM) § Frequency or conditional smoothed oCTR as feature values from
activated features in the Virtual Profile § For new ad creatives back-‐‑off to the advertiser / ad category nodes till
they reach critical impression/click volume (explore/exploit)
AD CREATIVE VIRTUAL PROFILE
Creative features
What about Hypertargeting ?
Done via § Transitions probability § Profile collocation analysis § Co-‐‑Targeted segments § Virtual Profile Similarity § A/B tested for most effective
solutions
RecLS
Recommendations: What are they worth? Think 50%
32
§ > 50% of connections are from recommendations (PYMK)
§ > 50% of job applications are from recommendations (JYMBII)
§ > 50% of group joins are from recommendations (GYML)
Hiring Solutions – Self Serve Jobs Postings
Sponsored Recommendations
Talent Match
Corpus Stats
Job
User Base
Filtered
title geo company
industry description functional area
…
Candidate
General expertise specialties education headline geo experience
Current Position title summary tenure length industry functional area …
Similarity (candidate expertise, job description)
0.56 Similarity
(candidate specialties, job description)
0.2 Transition probability
(candidate industry, job industry)
0.43
Title Similarity
0.8
Similarity (headline, title)
0.7 . . .
derived
Matching Binary Exact matches: geo, industry, … Soft transition probabilities, similarity, … Text
Recommendation Algorithm
Transition probabilities Connectivity yrs of experience to reach title education needed for this title …
37
Feature Engineering – Entity Resolution
§ Companies
• Huge impact on the business and UE
• Ad targeting • TalentMatch • Referrals
‘IBM’ has 8000+ variations - ibm – ireland - ibm research - T J Watson Labs - International Bus. Machines - Deep Blue
K-Ambiguous
38
§ Binary classifier (LR), not ranker § P({position, company entity} is
a match) § Features:
§ Content – name similarity features, industry match, location match, email domain match, company size
§ Social Graph -‐‑ # connections at company entity
§ Behavior -‐‑ # of invitations received from company entity members
§ Company candidate set leveraged from Social graph and cosine similarity
97% Precision at 50% Coverage
Asonam’11, KDD’11
Feature Engineering – Entity Resolution
§ Zip code mapped to Regions § How sticky are those locations? § Huge impact on the business and UE
• Job Seeker, Recruiter
Feature Engineering – Sticky locations
§ Open to relocation ? § Region similarity based on profiles or network § Region transition probability
§ predict individuals propensity to migrate and most likely migration target
§ Impact on job recommendations § 20% lift in views/viewers/applications/applicants
Feature Engineering – Sticky locations
41
The Network effect
What should you transition to & when ?
42
Multiple Objective Optimization
Applicable in multiple contexts § Online Dating
§ Click Shaping
§ Revenue vs CTR optimization tradeoff § Talent Match
Luiz Pizzato, Tomek Rej, Thomas Chung, Irena Koprinska, Kalina Yacef, and Judy Kay. 2010. Reciprocal recommender system for online dating. In Proceedings of the fourth ACM conference on Recommender systems (RecSys 'ʹ10). ACM, New York, NY, USA, 353-‐‑354.
Deepak Agarwal, Bee-‐‑Chung Chen, Pradheep Elango, and Xuanhui Wang. 2011. Click shaping to optimize multiple objectives. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining (KDD 'ʹ11). ACM, New York, NY, USA, 132-‐‑140
Mario Rodriguez, Christian Posse, Ethan Zhang.2012. Multiple Objective Optimization in Recommender Systems. To appear in Proceedings of the Sixth ACM conference on Recommender systems (RecSys 'ʹ12)
Multiple Objective Optimization
Formalism 1. Rank Top K’ > K semantically relevant results
2. Perturb the Top K’ ranking with the parameterized ranking
3. Measure the perturbation via a quality tradeoff
4.
job
TalentMatch Member Z, 0.89, Active
…
JobSeeker Intent
MOO
Member X, 0.98, 0.98, NonSeeker Member Y, 0.91, 0.91, NonSeeker
…
+40% InMail Response Rate
Multiple Objective Optimization § TalentMatch
§ Logistic Regression model
§ JobSeeker Intent § Ordered Logistic Regression model § Active/Passive/NonSeekers § Outputs propensity score
§ MOO (Multiple Objective Optimization) § Grid Search on Objective function § sMOOth for large parameter spaces
Multiple Objective Optimization Formalism 1. Rank top K’ > K semantically rank results
2. Perturb the ranking with a parametric function parameterized by α , β which leads to inclusion of the secondary objective
3. Measure the perturbation using a delta function wrt to the primary objective
4. Create a framework to quantify the tradeoff between the two objectives
Multiple Objective Optimization
Multiple Objective Optimization
Social Referral
Social Referral § Order recommendations by
§ Connection Strength between two users => § Recommendation Strength for the target user => § Combination thereof
σ (ui,uj )R(uj,gk )
Mohammad Amin, Baoshi Yan, Sripad Sriram, Anmol Bhasin, Christian Posse. 2012. Social Referral : Using network connections to deliver recommendations. To appear in Proceedings of the Sixth ACM conference on Recommender systems (RecSys 'ʹ12)
> 2X Conversion
Linkedin Group: Text Analytics
I found this group interesting, and I think you will too Deepak
Linkedin Group: Text Analytics
From: Deepak Agarwal – Engineering Director, LinkedIn
2X conversion
Follow Ecosystem
Recommended Followers Targeting Task : To identify a set (usually Millions) of users likely to follow the given company
Scorer MEMBER PROFILE FEATURES
p( follow | ci,uj )
COMPANY FOLLOWER VIRTUAL PROFILE
Global Company popularity
Other rankings – 1. User’s login probability in next X days 2. User’s PVs in the next X days 3. User’s propensity to follow any company
Weighted Borda count to for Information Fusion & A/B Test combinations h]p://www.colorado.edu/education/DMP/voting_b.html -‐‑ loss of information in plurality votes
53 53 53
A/B Testing Is Option A Be]er Than Option B? Let’s Test
Beware of § novelty effect § Cannibalization § potential biases (time, targeted population)
§ random sampling destroying the network effect
Don’t forget to A/A test first
(“Seven Pitfalls to Avoid when Running Controlled Experiments on the Web”, KDD’09 “Framework and Algorithms for Network Bucket Testing” WWW’12 submission)
Enjoy testing furiously!. Hundreds of tests live on LinkedIn at all times..
0"
1,000"
2,000"
3,000"
4,000"
5,000"
6,000"
7,000"
8,000"
9,000"
0" 5" 10" 15" 20" 25"
job$views$per$5%$bucket$range$5$6/5/11$
job"views"per"5%"bucket"range"?"6/5/11"
0"
1,000"
2,000"
3,000"
4,000"
5,000"
6,000"
7,000"
0" 5" 10" 15" 20" 25"
job$views$6/19/11$
job"views"6/19/11"
A/B allows seemingly subjective questions of design—color, layout, image selection, text—to become incontrovertible ma]ers of data-‐‑driven social science. -‐‑ Dan Siroker, Digital Advisor to Barack Obama’s election campaign -‐‑2008
h]p://www.wired.com/business/2012/04/ff_abtesting/
Supply Demand
Newer Ad products like “fans” & “follows”
Social media frenzy
Newer advertiser acquisition
# of pages with ads
Page view growth , highest in mobile
55
Demand exceeds supply
Real Time Bidding
Fun problems § When not to bid ? § CTR prediction on the publisher § What auction does the exchange run ? § Onsite vs Offsite impression tradeoff
for impression capped campaigns
Onsite/Offsite tradeoff
LinkedIn Ads shown to LinkedIn Member – zillow.com
Other initiatives..
§ Audience Forecasting
§ Bid Landscaping
§ Lookalike Modeling
§ Publisher DNA
§ Auto ad creative generation from landing pages
§ Explore Exploit strategies
And more..
§ New guaranteed display ad product § Impressions guaranteed = 1 § eCPI > $[0-‐‑9]{1}000
New Product!
You
Picture yourself with this New Job:
Applied Researcher / Research Engineer
Credits
Engineering : Abhishek Gupta, Adam Smyczek, Adil Aijaz, Alan Li, Baoshi Yan, Bee-‐‑Chung Chen, Deepak Agarwal, Ethan Zhang, Haishan Liu, Igor Perisic, Jonathan Traupman, Liang Zhang, Lokesh Bajaj, Mario Rodriguez, Mohammad Amin, Parul Jain, Sanjay Dubey, Tarun Kumar, Trevor Walker, Utku Irmak Product : Christian posse, Gyanda Sachdeva, Mike Grishaver, Parker Barrile, Sachit Kamat, Andrew Hill
