the science of brand bidding in search

Microsoft Confidentialhttp://aka.ms/bingadswebinars #bingadswebinars

The Science of Brand Bidding

http://aka.ms/bingadswebinars


Frances Donegan-RyanSEM Pro Marketing ManagerBing Ads, Microsoft@FrancesDR

Lars HirschDirector of Advertiser ScienceBing Ads, Microsoft@larshirsch

MJ DePalmaSEM Pro Community ManagerBing Ads, Microsoft@mjdepalma

Today’s Webinar Speaker Today’s Webinar SpeakerToday’s Webinar Host



Here’s what you’ll learn:• Bing Advertiser Science Team’s overall brand experiment

results• Brand results in the Retail, Finance, and Travel industries• How to design your own Brand term experiment• How to use our Statistical relevance worksheet (T-test template

for download)

Agenda



What’s Advertiser Science?


Advertiser Science

1. Analyze our Bing Ads auction

2. Turn the data into action for our advertisers

3. Share results and methods

MJ DePalma

lars- what are your three team priorities?

“To bid or not to bid on brand terms?”

Data scientists at Bing Ads recently tackled a big question that advertisers wrestle with…


Our brand analysis


1. When a brand term ad was not present on the search results page

2. When a brand term ad was present on the search results page

We compared two scenarios

brandbrand.co

mproduct

Click realityBrand OrganicBrand AdCompetitor OrganicCompetitor Ad

Social Network

No Brand Ad

Wireless Brand

No Brand Ad

Telecom Brand

No Brand Ad

Retail Brand

No Brand Ad

100%

80%

60%

40%

20%

0%

We analyzed 400,000 impressions for Travel on Bing in December 2014

Travel

When no brand ad is present, the brand received 61% of clicks on the search results page.

Travel

Source: Bing Ads Internal Data, December 2014.

People clicked the top organic positions

No brand ad

61%

No brand ad

61%

39%Your competitor

You

Competitors are getting nearly 40% of clicks when no brand ad is

present.

Travel


Brand ad in main line position 1 and top organic listings.

Without a brand ad present, competitors capture nearly half of

the clicksTravel


www.[brand].com [brand].com flights

[brand] flight search

0%20%40%60%80%

100%

73% 57% 51%

27% 43% 48%

Brand Not on ML1 and Competitor Ad Present

Brand organic Competitor ad

Don’t let your competitors steal your customers!

Travel

MJ DePalma

I'm not sure if this should be 45% or the 27% number - see next slide

Gain

Brand ad +

organic61%

27%

That’s 27% more clicks as a result of brand term bidding. Travel



No brand ad

61%

Let’s break it down Travel

Incremental (paid) clicks from the brand ad.

Overlap: paid clicks

43%Brand ad +

organic

18%27%



Total: 88%

No brand ad

61%


Question still left unanswered

What happens with competitors clicks when we do advertise on brand terms?


88%

No brand ad

61%

Brand ad + organic

39% 12%Your competitor

You

Your competitor

You

Competitors are getting nearly 40% of clicks when no brand ad is

present.

Travel



High fives all around for this major travel website! Competitor ads captured

only up to 11% of clicks.Travel


“Travel brand” represents actual results of a top travel industry brand.

[brand] [brand] car rentals www.[brand].com0%20%40%60%80%

100%

48%23%

45%

51%66%

54%

1% 11% 1%Brand on ML1 and Competitor Ad Present

Brand organic Brand ad Competitor ad

We analyzed 3 million impressions for Retail on Bing in December 2014

Retail


Let’s recap


“A great offense wins games but a great defense wins championships.”

1. PLAY OFFENSE Brand term

bidding helps deliver more clicks.

2. PLAY DEFENSE Brand term

bidding means fewer clicks to competitors.

Brand term bidding is a strategy that allows you to play offense and defense at the same time.

Bidding on your brand terms reduces opportunities to capture your customers or their mindshare if they bid on your terms.

1

2

1. Getting a lower “click-yield” or rate of clicks/searches, even with a strong SEO strategy in place.

What do I risk if I don’t bid on my terms?

As we saw, you can risk losing nearly half of clicks if you don’t bid on your terms.

1

1. Losing clicks to competitors attempting to “conquest” the searches for your brand.

What do I risk if I don’t bid on my terms?

Don’t let your competitors steal demand for your brand – your clicks and leads.

As we saw, you can risk losing nearly half of clicks if you don’t bid on your brand terms.

2

Average branded click lift

Financial

Services

+33%

Retail+51%

Travel+44%

Average competitor click loss

Financial

Services

-57%

Retail-43%

Travel-44%


How to design your own brand experiment



It determines if there is a statistically significant difference between two data sets

And you definitely want a statistically significant difference if your experiment is to yield reliable results!

Central to experimentation is the Students t-test



Download these two data sets:1. Organic clicks2. Paid clicks

Set up your t-test to start your brand experiment



• Step 1 - organize your brand keywords into one or more separate campaigns

• Step 2 - Use Bing Ads’ Day of Week targeting to enable/disable brand campaigns every other day

• Step 3 - Compare each day’s total brand clicks for the brand enabled days with the brand disabled days, using t-test

Brand Experiment Type 1 – Days of the Week


MJ DePalma

Not sure why you would measure the differenc in step 3 if you have to put the total clicks into a t-test in step 4?

MJ DePalma

Can you explain Lars?

MJ DePalma

Can you explain Lars?


• You want the t-value to be <0.1 to be statistically significant

• If it’s not – it probably is because you need to gather more data to have a larger sample set

• Or – there’s not a difference large enough to be able to prove running a brand term makes a difference

Interpreting the Results Brand Experiment Type 1 – Days of the Week


MJ DePalma

Broken out on a second slide here "Result interpretation" Also can you give some direction on how they would interpret the results here? need guidance for them

MJ DePalma


MJ DePalma



• Step 1 - Identify pairs of metro areas with similar customer profiles for your business

• Step 2 - Start with as many as you can identify up to 20, so it’s manageable to start (even number for pairs)

• Step 3 - Randomly select metro area from each pair into treatment – (metros are assigned a number)

free True Random Number Generator https://www.random.org/

Brand Experiment Type 2 - Geo


https://www.random.org/


• Step 4 - Measure combined organic + paid clicks by metro area before experiment

• Step 5 – Enable geo targeting on brand campaigns

Brand Experiment Type 2 - Geo



Interpreting the Results Download t-test



1. Have patience 2. Implement3. Wash, rinse & repeat

Safety Considerations



• Decide on a brand bidding experiment type• Have patience; you might run the experiment several times to

reach a statistically relevant pool of data• Use the results to expand your brand keywords with targeted

campaign precision and more data-driven confidence• Play offense and defense if your experiment tells you to!• Finally put to rest the office debate, to bid or not to bid on

your brand terms!

The Science of Brand Bidding Webinar recap



Q A



Frances Donegan-RyanSEM Pro Marketing ManagerBing Ads, Microsoft@FrancesDR

Lars HirschDirector of Advertiser ScienceBing Ads, Microsoft@larshirsch

MJ DePalmaSEM Pro Community ManagerBing Ads, Microsoft@mjdepalma

Today’s Webinar Speaker Today’s Webinar SpeakerToday’s Webinar Host

Thank you! And stay in touch…



Additional Questions?

#AskBingAds



Join us next time…

http://aka.ms/keywordattribution






Thank you!



Appendix



Easy to use online t-test calculator*http://www.physics.csbsju.edu/stats/t-test_bulk_form.html *Credit to College of Saint Benedict Saint John’s University

Read more if you really want to geek-out:https://en.wikipedia.org/wiki/Student%27s_t-test http://www.physics.csbsju.edu/stats/t-test.html

Reading & resources


http://www.physics.csbsju.edu/stats/t-test_bulk_form.html

https://en.wikipedia.org/wiki/Student's_t-test

https://en.wikipedia.org/wiki/Student's_t-test

http://www.physics.csbsju.edu/stats/t-test.html

http://www.physics.csbsju.edu/stats/t-test.html

“travel brand” flight“retail brand”.com

For the retail industry, when no brand ad is present, the brand received 60% of

clicks on the search results page – from organic listings.

Retail


No brand ad

60% People clicked the top organic positions.

Competitors are getting 40% of clicks when no brand ad is

present.

Retail

No brand ad

60%

40%

91%

Brand ad + organic

9% Your competitor

You

Your competitor

You



Clicks increased by 53% when a brand ad was present.Retail

Gain

Brand ad + organic

60%

32%



No brand ad

60%

Yes, some paid clicks you may have received anyway. But you also received 53%

incremental clicks!Retail

Incremental (paid) clicks from the brand ad.Overlap: Paid clicks you may

have received free anyway from

organic listing.

No brand ad

60%49%

Brand ad + organic

11%

32%



Without a brand ad present, this major retail company’s competitors can capture over

half of the clicks.Retail


[bran

d] fur

niture

[bran

d] bri

dal re

gistry

[bran

d] ba

by re

gistry

0%40%80%

57% 34% 69%2% 5%6%40% 61% 25%

Brand Not on ML1 and Competitor Ad Present

Brand organic Competitor organic Competitor ad

Competitor ads captured only up to 11% of clicks

with a brand ad present.Retail

[brand] bridal [brand] home [brand] furniture sale

0%

20%

40%

60%

80%

100%

67% 64% 67%

15% 33% 14%3%1%

5%15% 2% 14%

Brand on ML1 and Competitor Ad Present

Brand organic Brand ad Competitor organic Competitor ad