the marketingexperiments quarterly research journal, q2 2010

8/9/2019 The MarketingExperiments Quarterly Research Journal, Q2 2010

1/132

The MarkengExperiments

Quarterly Research Journal

Volume I, Issue 2 - Q2 2010

Integrate Your Markeng:Hw ne cmpny cmbne

ne n nne mkeng

ncese subscpns by 124% . . . . . . . . . . . . . . . . . . . . . . . p. 6

The Business Case for Tesng:Hw ne mkee cnvnce

he busness ees s

esng n ve 201% gn

n he pcess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . p. 39

1:1 Markeng at Four Levels:Segc wys evey mkee

cn ene n n nne

cnvesn wh cusmes . . . . . . . . . . . . . . . . . . . . . . . . . . p. 80

Technology Blind Spots:Hw humn nsgh evee

hen (n ms msse) 31% gn . . . . . . . . . . . . . . . . . . p. 112


2/132


3/132

The Second Quarter of 2010Volume I, Issue 2

RESEARCHJOURNALT H E MA R K E T I N GEX P E R I M E N T S QUART ERLY

marketingexperiments

Editorial Staff

ACKNOWLEDGEMENTS

We wu ke hnk he Resech Pnes wh hve eme up wh us cnuchese expemens. F me nmn bu Resech Pneshps, pese vs

MkengExpemens.cm/Pnes.

CONTRIBUTORS

Pme MkeyJmmy Es

Sc HwBb Kempe

An Gbe Dz

Bn CBs GnkAm LppAny M

Gn TwnsenTny Dy

Am T. SunJn PweAu Sv

Nhn ThmpsnCey Ten

Editor-in-ChiefFn McGughn

EditorDne Busen

Content ProduconManager

Ausn McCw

DesignerNce Evns

A publicaon of:

MkengExpemens,

MECLABS Gup Cmpny

1300 Msh Lnng Pkwy, Sue 106

Jcksnve Bech, F 32250

(800) [email protected]

MkengExpemens.cm

A ghs eseve. N p hs pubcn my be epuce nsme n ny m by ny mens, eecnc mechnc,

ncung phcpyng, ecng by ny nmn sge n eev sysem, whu wen pemssn m he pubshe,

excep he ncusn be quns.

Cpygh 2010 by MkengExpemens
http://www.marketingexperiments.com/services.html?utm_source=2008_researchpartners&utm_medium=online_formmailto:[email protected]:[email protected]://www.marketingexperiments.com/services.html?utm_source=2008_researchpartners&utm_medium=online_form


4/132

MarkengExperiments Research Journal

Vume 1, Issue 2 Cpygh 2010 MkengExpemens. A ghs eseve2

Table of Contents

Introducon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

By D. Fn McGughn, Dec MECLABS Gup

MARKETING OPTIMIZATION

Integrate Your Markeng: How one company combined oine and online markeng to

increase subscripons by 124% . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6

By Dne Busen, Jmmy Es, Pme Mkey, Ausn McCw, Au Sv, Gn

Twnsen, n Cey Ten

Ask the Scienst: MarkengExperiments Opmizaon Sequence . . . . . . . . . . . . . . . . . . . . . . . .20

By Bb Kempe, Dec Scences

Evidence-based Markeng: Why you need more than just numbers to truly drive ROI . . . . . . .21

By Dne Busen, E

Tricks vs. Tesng: The Bale for Internet Supremacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23By Am Lpp, Resech Mnge, Opmzn n Segy

Flash Banner vs. Headline, Lead Quanty vs. Lead Quality: The ght for online markeng

ROI connues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26

By Am Lpp, Resech Mnge, Opmzn n Segy

Online Markeng Tricks vs. Tesng: The Thrilla on Mozilla . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29


100 Tips for Trade Show Lead Generaon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

By Bn C, CEO InTuch

Ask the Scienst: Price tesng methods and pracces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33By Bb Kempe, Dec Scences

Mulvariate Tesng: Can you radically improve markeng ROI by increasing variables

you test? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

By Bs Gnk, Sen Mnge, Resech n Segy

Cost of Delay: How to win approval for your test and test schedule . . . . . . . . . . . . . . . . . . . . . .37

By Jn Pwe, Resech Anys

The Business Case for Tesng: How one marketer convinced her business leaders to start

tesng and drove a 201% gain in the process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

By Dne Busen, Am Lpp, Pme Mkey, Ausn McCw, Any M

SOCIAL MEDIA MARKETING

Facebook Case Study: From 517 to 33,000 fans in two weeks (plus media coverage) . . . . . . . .54

By Dne Busen, E

Social Media Case Study: Facebook plus integrated markeng helps raise $950,000. . . . . . . . .57

By Dne Busen, E


5/132

Vume 1, Issue 2 Cpygh 2010 MkengExpemens. A ghs eseve. 3

Connecng Social Networks Pays O . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61

By Am T. Sun, Repe MkengShep

Holisc Markeng Opmizaon: Whats more likely to show up on Twier? . . . . . . . . . . . . . .62

By Dne Busen, E

Social Media Markeng Human Factor: Finding the right person for the job . . . . . . . . . . . . . . .65

By Nhn Thmpsn, Resech Anys

Social Media for the COO: How to become the Michael Phelps of implemenng

social media in your organizaon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67

By Sc Hw, Dec Opens

SEARCH OPTIMIZATION

Search Markeng: Tips on mastering the latest innovaons in this mature category . . . . . . . .72

By Dne Busen, E

Google Caeine: Use social media and quality content to get a jolt for your site . . . . . . . . . . . .75

By Nhn Thmpsn, Resech Anys

SITE OPTIMIZATION

1:1 Markeng at Four Levels: Strategic ways every marketer can enter into an online

conversaon with customers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .80

By Dne Busen, Am Dvs, Bb Kempe, Am Lpp, Pme Mkey, Ausn

McCw, Any M

One-to-One Markeng: The true promise of Dynamic Oer-Content Customizaon. . . . . . . . .96

By Bb Kempe, Dec Scences

Web Team Roles, Web Designer: Are you trying to buy bread from the butcher? . . . . . . . . . . .98

By Am Dvs, Dec Techngy

Web Team Roles, Web Developer: What exactly does a developer develop, anyway? . . . . .101

By Am Dvs, Dec Techngy

What Else Can I Test On My E-commerce Or Lead Generaon Website? . . . . . . . . . . . . . . . .104


Markeng Opmizaon Technology: Be careful of shoong yourself (and your test)

in the foot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .108

By Cey Ten, Resech Anys

Google Analycs: New browser-based, data-privacy opt out important, but what consumers

really need is educaon . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111

By Cey Ten, Resech Anys

Technology Blind Spots: How human insight revealed a hidden (and almost missed)

31% gain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .112

By Dne Busen, Bb Kempe, Pme Mkey, Ausn McCw, Any M, Jn Pwe,

Cey Ten

Tbe Cne


6/132

De Rees:

F me hn en yes, weve been seekng scve wh ey wks n g mkeng. Ae pubshng

175 esech sues, we e s enng.

In hs spec pubcn, u secn MkengExpemens Quey Resech Jun, we use gus

scence chenge cen s-ce bes pcces.

Ou nngs wee supsng

MkengExpemens bs scvee 124% cnvesn ncese m yng nne es n

ne cmpgn even hugh hee ws n ec cen beween he w. Hw much mpc es

yu ne vesng hve n yu nne cnvesn? See Inege Yu Mkeng n pge 6.

We cee 1:n mkeng ppch h puce 109% n eque n segmenn. Whe

1:1 mkeng s mj buzzw, yu my be bee seve by 1:n mkeng. Whs he eence?

See One--One Mkeng Fu Leves n pge 80.

Ou esech nyss un hen 31% gn be evng n sme en-veke echngy bn

sps. Wh mgh yu enepse-eve pmzn swe be nveeny keepng m yu? See

Techngy Bn Sps n pge 112.

These e jus ew he nsghs n he u neve-bee-pubshe esech ces eung 12

expemens n hs ques ssue. Yu s n 21 hw- ces cue m he MkengExpemensbg, cnbe e genen vce m Bn C, n sc me gunce m MkengShep.

Thnk yu kng he me e hs ssue. Whe yu my hve eceve cmpmeny cpy hs

Jun, epesens sgncn esech nvesmen. I yu n useu, pese pss ng hs esech

cegues hep us bu cmmuny evence-bse mkees h we cn she es wh s we s

en m.

Snceey,

D. Fn McGughn

Dec MECLABS Gup

P.S. We want to hear from you. Please send your thoughts to the MarkengExperiments content team at

[email protected] or take three minutes to visitMarkengExperiments.com/Survey.
mailto:[email protected]://marketingexperiments.com/Surveyhttp://marketingexperiments.com/Surveymailto:[email protected]


7/132

MARKETING OPTIMIZATION


8/132



Integrate Your Markeng:How one company combined oine

and online markeng to increase

subscripons by 124%

By Daniel Burstein,Jimmy Ellis,Pamela Markey,Ausn McCraw,Arturo Silva, GinaTownsend, andCorey Trent

Nw s ge me be n g mkeng.

Smpy pu, s gwng expsvey.

T w, ee hs ye Peps ece pss

up n he Supe Bw (whch s he Supe

Bw vesng) he s me n 23

yes n v nne mkeng. Peps ws

be cee eepe enshp whs cusmes nne hn cu wh

30-secn Beync hnks s Reesh

Eveyhng cmpgn. In c, Peps hs she

ms ne-h s buge necve

n sc me. An h ecn cme

sgh m he CEO.

An s n jus Peps. As p, he bew

ch m Fese Resech usesnecve mkeng buge ens.

An yeeven wh hs expsve gwh,

mkees e s nvesng vs mjy

he spen ne. The pse Pepss b

mve s spens w-hs s buge

n he n w. Accng he bve

Fese ch, even n 2014 (e expsve

gwh nne), mkees w s spen

ms 80% ne.

S wh cn we en m hs ? I s

cuc unesn he cnnecn beween

ne n nne mkeng. T ge mxmum

ROI m yu mkeng spen, yu mus

nege yu cves n mke he

cnnecn beween yu n mkeng

n web cnvesns.

May 6, 2010
http://www.refresheverything.com/http://www.refresheverything.com/http://www.refresheverything.com/http://www.refresheverything.com/


9/132


T hep yu jus h, wee gng nswe

ve key quesns n hs ce:

1. How much do our oine campaigns impact

our online campaigns?

2. How do we connect oine campaigns to our

online eorts?

3. What negave impact can oine media

have online?

4. How can we set up metrics to monitor and

measure oine campaigns?

5. What impact can our online eorts have

oine?

HOW MUCH DO OUR OFFLINE CAMPAIGNS

IMPACT OUR ONLINE CAMPAIGNS?

T begn evng n hs pc, he eseches

hee MkengExpemens wee ce wh

hs s, unn quesn. Yu my hve

sme ssumpns n hw yu wu hnehs quesn, bu weve un, me e me,

h u ssumpns n nun cn en

be ncec.

S hs e us sme esech wh ne

u Resech Pnes h hs ge ne

mkeng pesence wh ec m

Experiment ID: (Protected)

Locaon: MkengExpemens Resech Lby

Test Protocol Number: TP1321

Research Notes:

Background: We-knwn news pubcn eng hme evey sevces v nne

egsn.

Goal: T ncese he mun hme evey subscpns.

Primary research queson: Whch pge/pcess w genee he ms subscpns?

Test Design: A/B mu-c sp es nng pge n egsn m.

EXPERIMENT #1

Control:

MkengExpemens

eseches h been

esng pge h

ee weeken-

ny en he

puc. Hee s he

pge we wee esng.

I s vey mpn

ne h hs pge h

ey gne hugh

sees ess n

h been mpve

gey. Hweve, we

wnee hw he

mkeng es

ece he success

hs pge.

Mkeng Opmzn | Inege Yu Mke


10/132



The eseches MkengExpemens wne es he hyphess h

een ec m nves mgh cuy ec ve web c cmng

hs pge. Snce hs cmpny h sgncn ec m es, we wnee hw

much ( ny) hse es mpce hs pge.

A hs cmpnys ec m es h unque, cmpgn-specc URLs h

cu be cke n mesue. In hey, eveyhng shu hve been esy n

cen gne he mpc he ne cmpgns.

Bu we wnee hw mny he pepe wh eceve he ec m pece

n w he ecns, wen sgh he webse, n un he wy

he genec puc pge we wee esng (he nk hs pge ws cey pse

hughu he webse).

S we se u esgn es h wu hep us guge he mpc ne es.

Direct Mail Example:

Hee s he cu ec m ceve (Editors note: All examples have been

anonymized to protect the business intelligence of the research partner)


11/132


124% Increase in Total LeadsThe highest performing treatment outperformed the control by 124%

Results:

Conversion Rate Relave Dierence

Cn 20.99% -3.5%

Treatment 1 1.23% 124%

S we ese hs hyphess wh esgn e csey he mgey he ec m peces scve

wu hve ny mpc n cnvesn. Inse he pcue he g, we nsee he pcue m he

ec m pece. We hs y cee eevnce hse cmng m he ec m hs genec

puc pge.

We s nsee sme he hnwng mgey m he ec m pece n he web m. Ths ws

ey sm chnge, bu cee cnnuy n he messgng n ee he sgn-up.

We speccy n hs expemen whe pepe wee ecevng he ec m peces n he mbxes.

Treatment:

The treatment uses an image directly

ed to the direct mail campaign.

The treatment path also includes stylisc

changes similar to the DM campaign.



12/132



What you need to understand:F he w weeks he ec m es, he emensupeme he cn by up 124%. Ths s sgncn ncese n cey emnses he pen

mpc n eec ne es n u Inene cmpgns.

Ae he ec m sens cncue, he esus evee nn-sgncn eence. I ws ny ung he ec m

es h he mpc he emen ws e. Ths unesces he mpc h ne es cn hve nne.

What we discovered:

S wh cn we en hee? Thee s sgncn pen n enyng n cnnecng ne

espnse nne enggemen. In hs pcu es we me evey sm jusmens

he Resech Pnes nne cmpgn cnnec wh he ne cmpgn n cheve

pe-g ncese n espnse.

The key pncpe s h hees sgncn ppuny mpve nne espnse es by

csey cnnecng ne n nne

mkeng cmpgns. Ths mens n nyenyng he vep, bu hen csey

cnnecng he es s hey e semess.

Whe he ec m peces pve

specc ckng URLs, ecpens s ene

up ve he webse n wee kng

eevn nmn. S hs vep s

gng hppen whehe yu wn

n. Is up yu cnnec he peces n

cheve hghe espnse e.

Jus hnk bu . Hw much yu c

gh nw s beng ece by yu ne

cmpgns?

HOW DO WE CONNECT OFFLINE CAMPAIGNS TO OUR ONLINE EFFORTS?

Nw h we hve cncue h hee cn be sgncn mpc beween ne n nne

cmpgns, we ce n even bgge quesn. Hw we mke he cnnecn?

The potenal (posive) impact of oine eorts:

As we hve jus seen, cnnecng ne cmpgns eevn nne web pges sgncny

nceses espnse. Hweve, hs cnnecn s me by me hn jus me ceve. Yu

nee unesn wh mves he vss yu webpge n hen mch h mvnn he ppe sequence.

Les ke k hw hs ees u pmzn mehgy. Hee s he

MkengExpemens Cnvesn Sequence heusc:

Online

Engagement

Oine

Response

KeyPrinciple:


13/132


The ms nuen eemen n hs mu s

mvn. I epesens he c h vss hve

een eves mvn. F exmpe, vs

m yu PPC cmpgns w key hve me

mvn hn smene m n em s.

A vs m n ne cmpgn s jus nhechnne cmng yu webpge wh s unque

mvns. Yu s hve une wh s

mvng he vs, n hen p n h mvn wh eevn messge. The me eevn

yu cn be, he gee espnse yu w see.

Yu g shu be s cnvesn ne, n cnnue gh whee yu e nne.

Creang relevant connuity:

I s mpn mke sue h yu messge s eevn n cmmunce n/ suppe

hughu he ene pcess cnnec he ne n nne cnvesns.

Relevance + Connuity = Conversion

Once we unesn wh s eevn he mvns u ecpens, we hen wn cee

cnnuy css he ene pcess. Tke he vue h yu hve ene n mke sue h n

evey sep he pcess yu e cmmuncng n suppng .

Les k specc exmpe hw we cee eevnce n cnnuy

Example #1

Hee s n exmpe h we hve pue

m sme u wk wh Resech

Pnes. Ths ce cnsn

cmpny n eevsn vesemens

css mupe cbe newks. The

vesemen mny ece pepe pck up phne n c. Hweve,

pve specc URL

hse wh wu ke ccess me

nmn nne.

Les s k wh n , n

hen we w k wh .

C = Probability of conversion

m = Movaon of user (when)

v = Clarity of the value proposion (why)

i = Incenve to take acon

f = Fricon elements of process

a = Anxiety about entering informaon



14/132



But this:

Not this:

In ms cses, s n pm sen ne, -specc c n nne genec hmepge.Thee s n mmee scnnec. F exmpe, by senng pepe m he eevsn

genec hmepge, huge eenn s eque n mny pepe w key bunce m he

hmepge when hey n mmeey n wh ve hem hee n he s pce.

We see hs he me. Cmpnes e spenng mns s ne genee c n

ye hey sen cusmes genec pge h hs n cnnecn wh hey gny sw.

I hee s ne hng h yu cn ke wy m hs ce h yu cn mmeey ppy

yu cmpgns, s hs: be s chnne-specc s pssbe. Segmen yu chnne s much s

pssbe. Cee e-specc, chnne-specc pges h e ecy he messgng n vue

h specc chnnes hve ey seen n been mve by.

Hee s ece nng pge h speccy mches he messge he ve. I even


15/132


pves me ve h cnnecs wh he . An hee s smpe, ce g h mches he

messge.

Ths chnne-specc pge cnnecs ecy he ne cmpgn. The messge hs pge es

ecy he eevsn h ny ve pepe hs nng pge. Thees even ve

h s sm he TV . An he e s smpe n mches he gn e n he TV .

Thee e mny hngs h hs pge s ng gh, bu wh we wn yu see s ey ce

exmpe hw yu cnnec ne me nne cnvesn.

The moment of orientaon:

I s mpn ne h cnusn cn be n mmee enemy mvn. In e keep

yu vss mve m sep sep, yu mus keep hem gune n he eevn vue

evey nsn n mmen enn.

F evey sep n cnvesn pcess ( exmpe, gng m ne nne s sgncn sep),

hee s mmen enn. Ths s whee he use kes he s seven s secns n

es unesn he nswe w cuc quesns Whee m I? n Wh cn I hee?.

In he n hs exmpe, whee he TV sen pepe hme pge, hee s sgncn

scnnec n s h he cusme gue u whee she s n wh she shu be ng.

In he bu hs exmpe, we mmeey gee cusmes wh messge hey e m wh,

n hee s ne smpe bjecve.

I yu cusme cn n qucky gn cy, he cnvesn w neve s n yu w be bck

ccke n bvn.

WHAT NEGATIVE IMPACT CAN OFFLINE MEDIA HAVE ONLINE?

The potenal (negave) impact of oine eorts:

Important Note: Ovekng he mpc ne cves hve n yu nne es pses

sgncn he esng vy.

The sx pmy hes esng vy e Seecn Eec, Smpng Dsn Eec, Muy

Eec, Hsy Eec, Insumenn Eec, n My Eec. (To learn more about validity

threats, please check out ourFundamentals of Online Tesng Training and Cercaon Course.)

T see he cen beween ne n nne cvy, we w cus n he Hsy Eec, whch

s he eec epenen vbe by n exneus vbe ssce wh he pssge me.


Whee m I?

Mmen Oenn VueExchngeCnvesn

Wh cn I hee?

Why shu I ?

Fs Secns Subsequen mnues ( ys) epenng n ses cyce
http://www.marketingexperiments.com/training-items/fundamentals-of-online-testing-0001.html?aff=40http://www.marketingexperiments.com/training-items/fundamentals-of-online-testing-0001.html?aff=40


16/132



The pbem wh Hsy Eec s h cn be s nuen h nves es esus. F

exmpe

Test Design: We pepe hene es usng Gge AWs s he sp-esng pm.

The henes wee chsen by he pcpns cecn cuse m p whch hey

cee. The es ws cnuce seven ys n eceve 55,000 mpessns.

As yu cn see, he ny hng h s een s

he henes. Thee hese s use he w

pe n he henes.

Validity Threat:

Dung he es, he Dene NBC eevsn shw

e spec ce T Cch Pe. Ths

pgm ws vewe by mns nvus.

Thughu hs pgm sex enes wee

eee s pes. Ths nguge ws

use n hee u es henes.




Research Notes:

Background: Onne sex ene egsy sevce pens cncene bu hesuunng es.

Goal: T ncese he cck-hugh e PPC vesemen.

Primary research queson: Whch hene w puce he ms cck-hugh?

Test Design: We pepe hene es usng Gge AWs s he sp-esng pm.

The henes wee chsen by he pcpns cecn cuse m p whch hey

cee. The es ws cnuce seven ys n eceve 55,000 mpessns.

EXPERIMENT #2

Control:

Ch Pe Regsy

Ieny sex enes vng n

yu e. Pec yu ks y.

www.XXXXXXXXXXXXXX.cm

Pes n Yu Ae

Ieny sex enes vng n

yu e. Pec yu ks y.www.XXXXXXXXXXXXXX.cm

Is Yu Ch Se?

Ieny sex enes vng n

yu e. Pec yu ks y.

www.XXXXXXXXXXXXXX.cm

Fn Ch Pes

Ieny sex enes vng n

yu e. Pec yu ks y.www.XXXXXXXXXXXXXX.cm
http://www.marketingexperiments.com/training-items/training-and-certification.htmlhttp://www.marketingexperiments.com/training-items/training-and-certification.html


17/132


Results:

Headline Impressions Clicks CTR

Predators n Yu Ae 21,096 1,423 6.74%

Ch Pe Regsy 14,712 652 4.43%

Fn Ch Predators 18,459 817 4.42%

Is Yu Ch Se? 15,128 437 2.89%

What you need to understand:F he w weeks he ec m es, he emensupeme he cn by up 124%. Ths s sgncn ncese n cey emnses he

pen mpc n eec ne es n u Inene cmpgns.

Immeey, husns pepe begn sechng pes n Gge.

S wh wee he esus u es?

S he quesn s, es he w pe ncese cck hugh s hs eswu nce? I yu wee n we he exen c, yu mgh cncue

h es n push u he new henes yu PPC cmpgns.

Bu wh he w pe ny sw sgncn esus ue he Dene

spec? Wh yu wee bsng yu mkeng ecsn n n nv es esu

ke hs?

The bm ne s h he exen c n hs es jepzes he esus. An

exneus vbe ssce wh he pssge me (he Dene spec)

hws n quesn he vy u expemen.

T be sue he pm henes, n esng wu hve be cnuce eemne he mpc he Dene spec.

Monitoring External Factors:



18/132



Mnng he eec exen cs e hvng n yu nne cmpgns cn n ny

gve yu nsghs n hw be eevn yu vss mvns, bu s pec yu

m mkng nv es cncusns.

n hs ch, we ene unusu esus ung specc pe me, yng gue

u hey wee m n exen suce.

S wh cn yu k n yu wn cmpgns? Thee e w cues h smehng

ne s ecng wh yue ng nne. Fs s he mgnue c. The eses

cue ne mpc s n unexpnbe empy spke ( eucn) n he mun

c vews specc nne cmpgn.

As, keep n eye n he nue he espnse. Ths s pehps me sube cue he

eecs n ne cmpgn. Hweve, yu see ncebe sh (negve psve)

n he kn espnse vss e hvng specc nne cmpgn. (e.g., cnvesn

es, ses, vege puchse muns, bunce es, ec.), n ne cmpgn my be

hvng n nveen mpc.

Graphed results of a 4-week email test with an e-commerce retailer:

(3.000)

(2.000)

(1.000)

0.000

1.000

2.000

3.000

4.000

Saturday,October11,2008

Sunday,October12,2008

Monday,October13,2008

Tuesday,October14,2008

Wednesday,October15,2008

Thursday,October16,2008

Friday,October17,2008















Saturday,November01,2008

Sunday,November02,2008

Monday,November03,2008

Tuesday,November04,2008

Wednesday,November05,2008

Thursday,November06,2008

Friday,November07,2008

Saturday,November08,2008

Sunday,November09,2008

Monday,November10,2008

Tuesday,November11,2008

Wednesday,November12,2008

Thursday,November13,2008

Standardized Conversion Rate

Normalized

Normalized B

Normalized Traffic

Normalized Traffic B

YES

NO

NO

Week-by-week Analysis:

Weeks 1-2: es ves

Week 3: Unusu evn.

Is hee esn excue

he ue?

Week 4: Lng-em esus

n cncusve


19/132




Research Notes:

Background: Ach ecvey pgm ws unnng s vng c he webse

uses sgn up een ecvey pgms.Goal: T ncese he c.

Primary research queson: Hw we quny hs c, knw wh hey e, n wh hey

bugh?

EXAMPLE #2

HOW CAN WE SET UP METRICS TO

MONITOR AND MEASURE OFFLINE

CAMPAIGNS?

Nw h we hve evewe ne

cmpgns n he pssbe mpcs

nne pcesses, es k cse

suy whee we cke hs usng Gge

Anycs

Seng up the metrics:

The s ve senes n een

emgphc mkes unque vny URLs.

We mpemene Gge Anycs ns

h wu ck he URLs n hen ep n

whch s wee vng me evenue.

Yu cn use Gge Anycs Annns

eue ne n ne cmpgns mpc

n Inene c. In hs exmpe, we use

hsc nnns mke nes whencen s e uhe nyss.

Hee e sme he cnsens keep

n mn ckng. Mcses, vny URLs,

n sub-mns cn be ge uses

ckng whe s enng yu bnng.

Bu be wy mkng ckng ems

cmpce hps he use jump

hugh. As ne ee, keep n mn

h hese URLs mgh n necessy be

mesung evey mpc, snce cnsumes w

n ype n neus URLs when hey nce

much smpe URL w ge hem he sme

pce.

Cnse he eecs ve emphszng

he e n n emp ce uses use

spec URLs cupn ces. Ths w cu

yu . F exmpe, yu use 10%

cupn ce n mgzne , yu wn

knw whehe cusmes vse yu se

becuse he mgzne ws sng hey

smpy vue he ncenve.

Yu cn s smpy sk yu cusmes

quesns een pns n he unne.

F exmpe, beyn jus eyng n Gge

Anycs, yu cn s ncpe whee

yu he bu us? e he bm

he checku m, n he hnk yu

pge, n s gn.



20/132



WHAT IMPACT CAN OUR ONLINE EFFORTS

HAVE OFFLINE?

Is pey esy us see he pen

cnnecng u ne cmpgns u nne

cmpgns, bu hw en we cnse whmpc u nne cmpgns cn hve u

ne cmpgns. Hw cn u nne es

ncese he eecveness u ne es?

Onne esng s pweu esuce

scveng he ms eecve messgng

ne cmpgns n es cs-eecve pe-

esng. Usng nne esng pe-es yu

ne ceve cn hve huge mpc n yu

esus.

Onne esng, n ms cses, s s he mscs-eecve wy unesn yu cuen

uence n busness me s we. Yu ge

e-me esus n yu cn un he ess n

n wh he cck bun.

Tesng messages online:

Onne esng ws yu es mny he

key eemens un n ne cmpgns.

Jus bu evey spec yu ne ceve

cn be pe-ese nne, ncung henes,

mges, n he yu nmn. O

cuse, yu cu s hs ne, bu s

usuy me expensve n me cu.

Techngy mkes esng mkeng cmpgns

nne sgncny ess expensve. Onne esng

s puces qucke esus h e bse n

e-me espnses. An nne esng pves

me cnssen smpng n she uns,

n heee pves me scency

ebe .

One cve keep n mn s mpn

m yu uences s csey s pssbe.

The pen vnge nne esng

yu ne cmpgns s ecy ee he

egee smy beween yu nne n

ne uences. I hey e vey sm (ke

m mges), hen he mpc cnnecng

nne n ne cmpgns w be sgncn.

I hey e n sm, hweve, hen he

pen mpc s euce. F exmpe,

ccng he MkengShep 2010 Em

KeyPrinciple:

Tesng headlines:

The Most Accurate Mailing Lists Available!

Get 500 FREE Leads Added to Purchase

Tesng images:

Tesng benet formulaons:

Vs.

Vs.

Vs.

Online tesng allows you to

test many of the key elements

found in oine campaigns.
http://www.sherpastore.com/EmailMKTReport2010.htmlhttp://www.sherpastore.com/EmailMKTReport2010.html


21/132


Mkeng Benchmk Rep, 94% Gen Y (ges 18-

32) uses em whe ny 79% hse ge 64 n up

use em. S yu ese puc hugh em

sens n nene use he sme messge ne,

yu nne smpe mgh hve been yunge hn he

gene ppun yu e mkeng ne n

heee cu skew yu esus.

BOTTOM LINE

Thee s sgncn pen n enyng n

cnnecng ne espnse nne enggemen.

S yu nee eny whee yu ne n nne

cmpgns vep n mke sue yu cnnec he w

n eevn wy.

F cnvesn, yu nee eevnce n cnnuy. The

me eevn yu cmpgn s he uence nhe me semess yu cnnec yu messgng, he

hghe yu cnvesn w be.

Mnng he eec exen cs e hvng n

yu Inene cmpgns cn n ny gve yu nsghs

n hw be eevn yu vss mvns, bu

s pec yu m mkng nv es cncusns.

Onne esng s pweu esuce scveng

he pm messgng ne cmpgns ue s

w cs, spee, n eby. In c, n ms evey

nsnce yu cn use nne esng cs-eecveype-es yu ne cmpgns.

In he en, negn s key. Whe yu cmpny

mkeng epmen my be ve n een

nne n ne vsns, yu cusmes n

see hngs h wy. By ensung yu es

e ghy nege, yu w mpve yu esus

n genee gee ROI n nvesmens h yu

cmpny s ey mkng.

http://www.sherpastore.com/EmailMKTReport2010.htmlhttp://www.sherpastore.com/EmailMKTReport2010.html


22/132



Editors Note: The MarkengExperiments

community is an interacve group with a

great deal of quesons and answers between

marketers and their peers as well as with the

MarkengExperiments sta. Occasionally we

publish these interacons on the blog when

we think there is a parcularly good queson

that our readers can benet from

QUESTION:

H, I cmpee he MEC Em Cecn

cuse whe bck. I mspce he MEC

pmzn mu. I wn she wh

sme nen pepe. Cn yu pese sen

me he mu?

Thnks,

Ken

Cusme Cmmuncns Mnge

Ceven, Oh

ANSWER:

Ahhh, yes. Yue pbby hnkng bu he

Opmzn Sequence, whch ppes

chnnes. See hs ks m

MarkengExperiments Opmizaon

Sequence:

menng h when ppchng n

pmzn nve, yu shu s

pmze he puc cs yu e ensue yu hve he bes puc vbe,

es ne sgncn, escbbe

cusme segmen.

Ony e ng h shu yu pmze he

pesenn cs yu e ensue

yu hve he bes, ms cmpeng e

vue ppsn vbe, es ne

sgncn, enbe cusme segmen.

Yu hs by ppyng he eevn

cnvesn heusc (e.g., nng pge:

C=4m+3v+2(-)-2; n em e: eme=

v(+ ) ( + ) ).

Ony nce yu hve cnme esnbe

eve pmy bh he puc n

pesenn cs yu e shu yu

embk n pmzng he chnne cs

vng s much pbe emn yu

pmze cnvesn pcess s yu cn.

Ths s ne by chnne encn,

seecn n pmzn, usng echnques

such s chnne mppng, p sech

pmzn, SEO, e buepnng, ec.

A he bes,

Bb Kempe

Dec Scences

MECLABS Gup, LLC

Related Resources

Opmzng Yu Lnng Pges

Em Opmzn

Opmzng Oe Pges

Ask the Scienst:MarkengExperiments Opmizaon

Sequence

By Bob Kemper,Director of Sciences

May 24, 2010

Opr = Opmize Product Factor

Oprn = Opmize Presentaon Factor

Ocnn = Opmize Channel Factor
http://www.linkedin.com/groups?gid=41533http://www.linkedin.com/groups?gid=41533http://www.marketingexperiments.com/training-items/email-marketing-0001.htmlhttp://www.marketingexperiments.com/training-items/email-marketing-0001.htmlhttp://www.marketingexperiments.com/whitepapers/MEx-Beyond-Landing-Pages.pdf#page=5http://www.marketingexperiments.com/methodology-marketingexperiments.html#WT_DChttp://www.marketingexperiments.com/methodology-marketingexperiments.html#WT_DChttp://www.marketingexperiments.com/improving-website-conversion/optimizing-your-landing-pages.htmlhttp://www.marketingexperiments.com/email-marketing-strategy/email-optimization-relevance-conversion.htmlhttp://www.marketingexperiments.com/improving-website-conversion/optimizing-landing-pages-increase-148.htmlhttp://www.marketingexperiments.com/improving-website-conversion/optimizing-landing-pages-increase-148.htmlhttp://www.marketingexperiments.com/email-marketing-strategy/email-optimization-relevance-conversion.htmlhttp://www.marketingexperiments.com/improving-website-conversion/optimizing-your-landing-pages.htmlhttp://www.marketingexperiments.com/methodology-marketingexperiments.html#WT_DChttp://www.marketingexperiments.com/methodology-marketingexperiments.html#WT_DChttp://www.marketingexperiments.com/whitepapers/MEx-Beyond-Landing-Pages.pdf#page=5http://www.marketingexperiments.com/training-items/email-marketing-0001.htmlhttp://www.marketingexperiments.com/training-items/email-marketing-0001.htmlhttp://www.linkedin.com/groups?gid=41533http://www.linkedin.com/groups?gid=41533


23/132


Evidence-based Markeng: Why youneed more than just numbers to truly drive

ROI

S mmmgms 50, n 40. Wh hs

vce, he Une Ses Pevenve Tsk Fce

se esm cnvesy quesnng

eveyhng m s mvn wsm.

These ecmmenns, n he cnvesy

h suuns hem, e jus he p he

cmpve-eecveness cebeg. F hse

n m wh he em, yu w he

me n me n he ne uue. The

ee gvenmen s nvesng $1.1 bn

n cmpve eecveness esech n

he ms eecve emens cmmn

cnns.

Does evidence change behavior?

H bu wh wks bes suns g

n hey, bu eseches e nng h

evence s ny p he sy. Cnvncng

he pubc ccep new mec gueneskes me hn numbes. As Chse

Aschwnen expns n he es ssue

Me-McCune, When cmes new

emen guenes bes cnce, bck

pn n he mes, s he nve

pesenn h mes.

S wh hese nsghs n humn nue

men he evence-bse mkee? Whe

he pwe he esng-pmzn cyce

s scveng wh ey wks yu

gnzn, hs knwege ne es n

ve chnge. Beyn p, yu nee ew

g cmmuncn sks. T h en, hee

s sme quck vce un es n

cn

Paint the

picture

Whe ee

s he

eb

ny successu

evence-bse

mkee, mke

sue yu cn

cmmunce

bh he

es n he ees. S bee yu mke ny

pesenn bu he esus yu esng-

pmzn cyce, ke ew seps bck.

Wh s he sy behn he numbes? Wh

s yu ve sy c?

I w key be smehng ng he nes

, We cnuce sees ess hep

mpve u mkeng. Fm hese ess, we

ene wh wks us n wh esn.

Nw we cn ppy h knwege css u

enepse, n by ng s, ve sgncn

ROI.

Mke n mske, he numbes me. Bu

mke sue h hey e ny p he sy,

n he mn cus.

You succeed, we fail

Pepe ge eensve when yu e hem hheye wng. S yue yng cnvnce

ecsn mke chnge eemens

cmpgn h he evepe, yu w hve

ppch segcy. The nguge yu use

pesen hese nngs cn g ng wy

hepng ge hm n yu se.

By Daniel BurstEditor

May 28, 2010

Mkeng Opmzn | Evence-bse Mke
http://www.miller-mccune.com/health/convincing-the-public-to-accept-new-medical-guidelines-11422/http://www.miller-mccune.com/health/convincing-the-public-to-accept-new-medical-guidelines-11422/


24/132



F exmpe, when yu ess shw gn

n e, ce hm (when ppcbe).

Yu hene evee 394% gn.

Hweve, when yu ess shw h

n eemen unepems, she he

bme. The sques h we pu n u

webse unepeme he pmze

emens by 203%.

Accentuate the posive

Negve news ens mke pepe

ee nsecue, unsue, n even nevus.

Yuve bscy jus ppe pbem n

he p.

S when pssbe, n we n he

negves yu hve uncvee wh yu

mkeng expemens. An ecy epesenng hem, pn he psve

cy h yuve scvee wh yu

esech. Whe mges sques hve

been hung cnvesn es, pcues

mes hve ven ube-g

nceses. Awys en n hgh ne.

Be soluon-oriented

Dn jus pesen he . Incue n

cn pn h shws hw pu he

nngs n cn. Weve enehe 27 pces we wn swp u

sques wh mes. Ou esgn em

hs seece new mgey. Once I ge

yu bugey ppv, we cn hve

he chnges ne whn 72 hus. Evey

pbem shu hve sun.

Focus on the boom line

Ms busness-eve ecsn mkes

n ce bu esng. O unsubscbes.

O even cnvesn. They ce bumkng mney.

Mke sue he yu pesen uses

mecs h ey me yu

uence. Whe nemee mecs e

vey hepu yu ung he esng-

pmzn cyce, bm-ne, esus-

ene mecs w wys be bee

hepng yu gn he uhy ve

chnge h yu seek.

Be right

N beb he bvus, bu

yue seekng mke chnges bse

n he ess yu un, mke sue yue

gh. In he ws, n jus ey n

he numbes sp u by yu esngpm. Techngy esn ve esng

success. Pepe .

Appch yu ess wh scenc

mehgy. An unesn hw n

why yu ess e sscy v.

Becuse n he en, he ms beevbe

evence-bse mkee s he ne wh

g wn n he enches n hepe

cee he evence shn.

Related Resources

The Busness Cse Tesng: Hw ne

mkee cnvnce he busness ees

s esng n ve 201% gn n

he pcess

Fcus Gups Vs. Rey: Wu yu buy

puc h esn exs wh peen

mney yu n hve?

Cs Dey: Hw wn ppv

yu es n es scheue

Photo aribuon:pinkmoose
http://www.marketingexperiments.com/blog/analytics-testing/marketing-optimization-testing.htmlhttp://www.marketingexperiments.com/marketing-optimization/the-business-case-for-testing.htmlhttp://www.marketingexperiments.com/marketing-optimization/the-business-case-for-testing.htmlhttp://www.marketingexperiments.com/marketing-optimization/the-business-case-for-testing.htmlhttp://www.marketingexperiments.com/marketing-optimization/the-business-case-for-testing.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/focus-groups.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/focus-groups.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/focus-groups.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/test-resources.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/test-resources.htmlhttp://www.flickr.com/photos/pinkmoose/http://www.flickr.com/photos/pinkmoose/http://www.marketingexperiments.com/blog/analytics-testing/test-resources.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/test-resources.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/focus-groups.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/focus-groups.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/focus-groups.htmlhttp://www.marketingexperiments.com/marketing-optimization/the-business-case-for-testing.htmlhttp://www.marketingexperiments.com/marketing-optimization/the-business-case-for-testing.htmlhttp://www.marketingexperiments.com/marketing-optimization/the-business-case-for-testing.htmlhttp://www.marketingexperiments.com/marketing-optimization/the-business-case-for-testing.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/marketing-optimization-testing.html


25/132


Tricks vs. Tesng: The Bale for

Internet SupremacyNwys he Inene s be y.

MySpce vs. FceBk. We knw hw h

mchup une u.

Gge vs. Yh. Ths nes scheue 12

uns, bu I hnk w be knck u.

An Gge Buzz ne yes e

mch bu e

ug es. S nw,Twe s hs he

Lgh Hevywegh

Be, bu evenuy

eveyne mees

he mch. Anyne

emembe Buse

Dugs knckng u Mke Tysn?

Ths speccu enenmen s ppece

by s-ce guus epc ppns. The

gh Inene mkeng supemcy s

quy cecn bus h w ssymny n Inene guu, bu hw w mpc

he msses n ennce?

W hey w he h ens bny? W

cks quck bucks cnvnce hem hw

n he mnhy buge n unpven,

hghy ue sece wepn?

S whehe s bve-he- nng pges

ve chs, yu nee see he esus

bee yu nves yu w. Thee e

esns ge beeve bes pcces pgm-shng Inene sensn hype

n hpe hese es n gees ccs

genee evenue mke en n yu

bm ne, bu yu n pesny see

he gh hugh yu e meey gmbng.

D yu ey wn h? Bee yu

s pyng he ppe, mke sue yu

yu wn esech s s yu hve hgh

cnence eve mke bes n he ene

gh c.

THE UNDERCARDS

Above The Fold Page vs. Sequence of

Thought

The Bekwn: The F s n new he

gh scene. He gnes m pevus

genen ghes wh hve msy

ee psse n. The F cmes m

he newsppe ng, whee eveyne ugh

he spce n he p h he pge. Fpepe wh n e n ene newsppe

w w, he p h he newsppe

pge s whee yu wu k he ms

mpn n ppu ses. Much he

cnen bew he ws msse.

The F hs new wng n he nene.

Eveywhee I g pepe e kng bu

The F. Hes he pce n he pge whee

ypc vss sceen ens. Anyhng

bew hm kes ex e ge ehe

by mnuy scng usng yu muse

whee. Ahugh ny kes quck wch

he nex nge, mny pepe w n exe

hs e.

S wh e we suppse ? Cm evey

pssbe mge, w, c--cn bve

The F? The nswe s n, n u ess

me n me gn pve .

By Adam Lapp,Research ManaOpmizaon aStrategy

May 28, 2010

Mkeng Opmzn | Tcks vs. Tes


26/132



Wh ve ece n he gme, n 100+

pessn ghs une hs be, Sequence

Thugh enes he ng ke wn he

chmpn. A s gnce, The F ppes

ey. Hes ney nge n he ns

beuu new be. The F ees hs s g

mchup hs sye. He s ypcy hsbes when he cn use hs bwn w n

cnvesns whu much hugh. He wns

yu pess hs buns gh wy whu

even ng he pe-gh esech.

Bu cnsumes e geng sme, me keen.

Hs ccs hve g hm hs , bu The F

hs me hs mch.

The esn why Sequence Thugh

cn ee The F sn becuse hs

ppence. The gh cnvesns s nwn n he pge, bu n he mn he use.

Is wn bee nyne even seps n he

ng. Is wn by unesnng yu vss

mvns n ncpng wh hey w

, wh punches hey w hw bee hey

ve n yu webse.

Whs he hugh pcess? Wh

nmn e hey kng ? D hey

nee nge cpy becuse he puc

eques bg ecsn? O yu nee

uze he p h he pge nuceyuse becuse yue unpven n he ng?

N ny s v unesn he

mvns he use bee yu bu yu

pge, bu s essen es yu wy n

n eecve pge esgn. Use mecs n

uy pmze yu se. Lunch es

ew weeks hen nyze yu vss cns

n e eny pens n n he es

yu se whee mpvemens w puce

he bgges ROI gns.

F sme puc engs, sh pge s

eecve. Bu The F s n ne-sze-s-

ype ghe. Ou esech hs pven

h en vs w sc yu e hem

wn he pge, ehe hugh cmpeng

cnen puc escpns. I yu p

n he mvns n pve hem wh

wh hey e kng , pge engh s n

cnsequence.

ADAM LAPPS UNOFFICIAL SCORECARD:

The Fold 76

Sequence of Thought 102

Prey Page vs. Ugly Page

The Bekwn: A he pmes hese

ys e kng en n chsee wh

mkebe ce. The g s c

enn, pu up vsuy smung

ce n e w vewes he gh n

mxmze py-pe-vew es. Pey Pge hs

s cs n mges n . (D cusme

sevce eps ey k ke he sck phs?

Ww!)

Thees s sh (ng wh sme Fsh).

Whe ex n bck bckgun. I see eshee we cns. Gens ge. An hen

hees h ve h I h g u my wy

cck puse jus s I cu cus n eng

he cu cnen he pge.

The Pey Pge ks g. As shu be

snce yu pbby p sm une

hve esgne n evepe.

Bu es Pey Pge ncese yu

cnvesn e?

Menwhe n he ces he en, Ugy

Pge s whng wy punchng bg.

Unke hs ppnen, Ugy Pge s gzze

veen wh n cue knwege hw

wn gh. Ths cmes m yes n yes

expeence. He knws h cnvesn

esn epen n ks, hvng he bgges

bceps, weng he ms cu shs.

He knws h cnvesn epens sey n

yu segy. An seges e n bu m

he es ens by mpemenng quck

hs h yu ene ne-y mkengcneence. They e bu m esng. Fm

gung u envey wh wks n wh

es n wk.

Bu he pn s n h Ugy Pge s bee

ghe hn Pey Pge. In c, Pey Pge

my evenuy becme much bee ghe

hn Ugy Pge. The mpn hng

unesn s h segy s me mpn


27/132


hn esgn. Yu cu hve he bes-kng

pge n he w, bu hsn been ese,

hsn been pmze bse n wh yu

ene m n esus, hen s nhng

me hn pey pge.

The ghe h w pev s ne h ess

he yu, ne, cpy, n psnng bee

esng mges n c schemes. Ugy Pge

ens hw cmmunce hs vue s

ghe bee he ns hmse wh ncy

be. He euces cn n nxey bee hs

pubcs gves hm mkeve.

The bm ne s h he e esng

sequence s es key seges s, gue

u whch wks he bes, hen n eshec

ye.


Prey Page 88

Ugly Page 96

Lets get ready to rumble

Thughu he week, I be sheng gh n

me Inene mkeng ghs gh hee n

he bg. We hve Fsh Bnne vs. Hene,

Quny vs. Quy, He Mp vs. D Anyss,

Bunce Re vs. Cnvesn, n nyhemn cTcks vs. Tesng.

Related Resources

The Mgc Mecs Tu: Demysyng he

seces behn nyc cks hep yu

ve ROI

A/B Sp Tesng Hw use A/B Sp Tesng

Incese Cnvesn Res, Chenge

Assumpns n Sve Pbems

Muvbe Tesng Hw esng mupe

chnges smuneusy cn sve yu me,

spee up yu pmzn scheue, n

ncese yu ps

Mkeng Opmzn | Tcks vs. Tes
http://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.html


28/132



Editors Note: Research Manager Adam Lapp

is reviewing the bale between common

Internet markeng pracces to help you

determine which opmizaon strategies are

most eecve and give you ideas for new

tests. On Monday, we published Part 1 in this

series. Here is Part 2

Flash Banner vs. HeadlineFlash

Image vs. Headline

The Breakdown: Th ws

neesng, wsn ? Fsh Bnne

enee he ng pumpe up n

ey g n hen,

suen, hs cne guy cme yng

n he ng bngng he bu

n bup h. A vey sppne

shwng he ns. Cs

n mges g nge wh ech

he, he messge sppe

he cnvs, n he puc eng wenyng beween he pes. An jus ke h,

he gh ws ve n he ns neve g

chnce unesn wh ws gng n.

Even hugh ke g n cncep, he

vss n hve chnce unesn

wh yu e, wh yu bes mve s, n

why yue bee ghe, hen whs he

pn? Fsh Bnne n even ge chnce

shw hs pene e hk. He n ge

chnce shw he uence he h

wk he pu n nng. On he nex ghbee bes cu even be ken.

Bu hee ws ceve mkeng pch, sme

hymng, sgn. Fsh Bnne shu hve

h bee shwng. Wh hppene? Hs

bjecve eue hm. Thee shu hve been

cck, puchse, smehng. Bu cn such

bg ecsn ey be me n sh?

The esus sy n.

Des he Fsh Bnne n Abe.cm ey

cnvnce me buy Ceve Sue 5? Des

even cmpe me cck w? We ws

bve The F, n we knw hw h

gh une u.

Snng hee n he me he ng ncey he bes pun--pun ghe

n he w s Hene. As he w ws

neupe by Fsh Bnne, he hene

spke cey he uence eng hem n

jus hee ve sh secns why he ws

he bes chce. He hem he hs wn 14

ghs n w, h he hs ne nn-sp

sx mnhs, n h he cu mch ny

ghe n he ws e.

Immeey he cw s n cheee

Hene be gven he nex gh. Thevss ece h wh Hene s h

evenng ws wh he e cnnue n

n n ex he en.

Hene s he s ex vs w see, s

hs much pen ge mpc. The

ps-men cnsume sees hugh he Dn

Kngs he w. They e nune wh

Flash Banner vs. Headline, Lead

Quanty vs. Lead Quality: The ght f

online markeng ROI connues

y Adam Lapp,esearch Manager,

Opmizaon andtrategy

May 12, 2010
http://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.html#Foldhttp://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.html#Fold


29/132


em, TV cmmecs, n even n-yu-ce

spys he gcey se. They hve me

me n e eepy skepc sespepe

whm hey cn even spek .

The Boom Line: Wh hs n mn, s much

me mpn be specc n nspen

bu yu ghes ens n unque bes

hn shy. Fsh cn be vey useu when

use pppey, bu cn ny ge yu s

. Cey cmmuncng wh yu hve

e n why yu e he bes chce yu

mke s much me enung segy

ncesng evenue.


Flash Banner 55

Headline 89

Lead Quanty vs. Lead Quality

The Breakdown: Snng e e sng

ech he n he eyes, s esy see h he

ece be Le Quny vs. Le Quy

shws n sgns eng up. In he me snce

he s bu, Le Quny hs cke up 12

vbe ecsns. Bu mny hse wee

esse ppnens. He eee ghe

wh n buge ne. He kncke u ghe wh ws -n smene wh

becme . The s mkes cun c

ny bes wh ghs ke hese.

In he men me, Le Quy hs ny

ugh w ghs. Bu ech ws py-pe-vew

speccu. He eee he h n sevenh

ms que ghes n he w. Le

Quy ne mecessy ech gh. He

n jus gh nyby, hs ppnens wee

eque ke Oympc-sye ug ess. They

h be cee membe es wbxng gnzns.

Sue hee ws cn n he pcess,

bu he n wse he me he vewng

uence wh menngess gh cs. Le

Quy benee m he segc ppcn

cn n hs pe-gh equemens,

bu s Oympc-sye ug esng hsh?

Unubey s. He cuy wu hve been

be scheue bu wh he numbe

hee ghe n he w he exe hs

qucns.

Nw vng hee n w een phs, Le

Quny ces wn Le Quy ye gn.

These ghes n seem ge ng. Bu

why? Bh ghes cu bene s much m

ech he.

Decesng, ng wn cn, esus

n ncese Le Quny.

Incesng, ng up cn, cees

nceses Le Quy.

A peec ghe wu be ne-w

cmbnn bh e genen seges.He wu es hs wy n he gh scene

eemne he pppe bnce beween

ncesng vume whe s ncesng Le

Quny. Incesng bh eques ghe

wh vesy n hs skng epee.

A s he be ngs. The ghes e

punches. Jb, css, hk, uppe cu. Ae

sx uns, he sce cs e equ hee

uns pece. Le Quny ppes

be pung u n n n he sevenh s he

uneshes punches, hweve ew e

cnnecng.

As we ne un en, bh Le Quny n

hs em e ng. S much e wh e

esu. Hweve ppes he es hve ve

u vnge ve Le Quy. Bu w!

Le Quy hs jus cnnece bg ne h

equs he es Le Quny. A hs

pence hs p .

The Boom Line: Mkeng wns

es. Scks n pes busness cs nnew cncs. Ths s he ce mec. I cn

be n mpessve sgh. Bu s n he ses

em c Fe, he pumbe, see he

wns buy new ps hs cmmec

wnmwe, e-en h cu hve been

emne by ng ne smpe e he

m.

Mkeng Opmzn | Fsh Bnne vs. He
http://www.marketingexperiments.com/marketing-optimization/transparent-marketing.htmlhttp://www.marketingexperiments.com/blog/marketing-q-a/friction-and-anxiety-in-your-marketing-process-defining-the-difference.htmlhttp://www.marketingexperiments.com/blog/marketing-q-a/friction-and-anxiety-in-your-marketing-process-defining-the-difference.htmlhttp://www.marketingexperiments.com/marketing-optimization/transparent-marketing.html


30/132



Ovesmpe? Sue. Bu hs s hw he be ve

Le Quny vs. Le Quy pys u n mny

gnzns.

S hw yu eemne hw mny es, whch

nmn eques, hw mny seps w ske h

bnce beween Le Quny n Le Quy. We,

hs nswe s smpe. Is esng.

F exmpe, yu hve ne-pge e genen

m, y unnng es h eys he phne numbe

ess eques un he secn pge. Yu e

genen w unubey g up n he s pge,

hs n cke scence. Bu whs he mpc n

he quy cmpee es yu eceve (bh seps

cmpee)? Yu my n h yu eceve ess

cmpee es, bu he es yu ge e exemeyque.

Yu w s be be sce es bse n quy,

ssgnng vss wh cmpee w seps s A es

n vss wh ny cmpee he n sep s B

es. Then yu ses em cn ny mve n B

es e hey hve exhuse me n e wh he

A es.


Lead Quanty 65

Lead Quality 65

Is w.

Related Resources

Tcks vs. Tesng: The Be Inene Supemcy

Le Genen Opmzn: Fnng he gh mun

cn

Opmzng Yu Henes: Hw chngng ew ws

cn hep ( hu) cnvesn

Fve Ds T Tune In Yu Le Genen Pcess

Fsh n Pn: D ps ceve n hme pges

eve ROI hghe bunce es?
http://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/lead-generation-optimization-finding-the-right-amount-of-friction.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/lead-generation-optimization-finding-the-right-amount-of-friction.htmlhttp://www.marketingexperiments.com/improving-website-conversion/optimizing-headlines.htmlhttp://www.marketingexperiments.com/improving-website-conversion/optimizing-headlines.htmlhttp://blog.startwithalead.com/weblog/2010/04/5-dials-to-tune-in-your-lead-generation-process.htmlhttp://www.marketingexperiments.com/blog/research-topics/do-flash-ads-work-html.htmlhttp://www.marketingexperiments.com/blog/research-topics/do-flash-ads-work-html.htmlhttp://www.marketingexperiments.com/blog/research-topics/do-flash-ads-work-html.htmlhttp://www.marketingexperiments.com/blog/research-topics/do-flash-ads-work-html.htmlhttp://blog.startwithalead.com/weblog/2010/04/5-dials-to-tune-in-your-lead-generation-process.htmlhttp://www.marketingexperiments.com/improving-website-conversion/optimizing-headlines.htmlhttp://www.marketingexperiments.com/improving-website-conversion/optimizing-headlines.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/lead-generation-optimization-finding-the-right-amount-of-friction.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/lead-generation-optimization-finding-the-right-amount-of-friction.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.html


31/132


Online Markeng Tricks vs.

Tesng: The Thrilla on Mozilla

Editors Note: Research Manager Adam Lapp is

reviewing the bale between common Internet

markeng pracces to help you determine

which opmizaon strategies are most

eecve and give you ideas for new tests. On

Monday, we published Part 1 in this series. On

Wednesday, Part 2. Well get ready ght fans,

hold on ght to your Mozilla Firefox, AppleSafari, or Google Chrome browser, because

here comes Part 3

Heat Map vs. Data Analysis

Breakdown:

When He Mp enes he ng, hes vey

nmng, n ks ge n he suce.

He wys shws up n shy, beuu

sk be n hs czy musc pyng when

he enes. He Mp ws wz geng

enn n he vues he hngs h gvehm enn he ms. He pns he

cme shes n bnnes n pesses hs

ve wh bg e kss.

A He Mps hecs e ne. Is p

he shw. Hweve, he ges hmse n

ube by mkng unsh cncusns

bu hs ppnens, ke beng be knw

excy wh punch hs ppnen wu n

jus by kng he c hs ches.

Mny mes when He Mp hnks smene

w punch hm n he b cge, hey nse

hw css hs empe.

F hs gh gns D Anyss, he ne

mecessy n he gym ng weghs, ng

cunches, n hng punchng bgs. When

cme me gh, he se n he ng

cnen h hs ppnens sengh ws

he e hk. Why? We becuse he sw e

n yew pyhn n hs e bcep.

T b he n cuy wch ge

D Anysss pevus ghs. I he h,

He Mp wu hve knwn h ms hs

punches cme m he gh. Wh shme

see hm kncke u n me secns.

Smemes He Mp s gh bu hsppnen hugh. Smemes he key

vcy s wen n nes seeve. Bu s vey,

vey ngeus ey n jus smemes.

D Anyss n he he hn s subue

n que. A sk cns He Mp.

Bu une h cm exe s supeme

cnence becuse, unke He Mp, he hs

been suyng hs ppnens m hus.

Hes ene h He Mp hws hmse

e bnce evey me he emps e

hk. He knws h He Mp esn bbn weve, bu nse cuses hs enn

n ne hngn hs cse s he he

pyhn.

Boom Line: D Anyss hs been nng

h, bngng n numeus spng pnes

wh gh ke He Mp. He uses hs nsghs

m hese execses me wh gh

echnques he hnks He Mp my y

pecve meng. Fm he m sues, he

pems egessn nyss eemne

he enshp beween He Mps jbs n

hs uppe cus. He gues u cen

beween he w evey me He Mp

hws w jbs n w hee s hgh

pbby n uppe cu w.

The ght starts, and Heat Map is looking good,

fancy footwork, doing a great job dodging that

le hook. But 1:19 into the rst round, he goes

By Adam Lapp,Research ManaOpmizaon aStrategy

May 14, 2010

Mkeng Opmzn | Onne Mkeng Tcks vs. Tes


32/132



for the double jab and Data Analysis clocks him.

He had been waing for that the whole me.

KO in the rst round. Most of the me Heat

Map is a solid bet, but any smart gambler will

always look past the impressive exterior to see

if his head is really in the ght.


Heat Map 4

Data Analysis 9

Bounce Rate vs. Conversion

Breakdown: Ths s

vey unhx

gh, n I n knw

hw he pmes

s hs ne. Bxngcmmssns m

seve cunes

une wn, n

evenuy hey h

m he Inene

m n unscse

cn.

Bunce Re s jus cusewegh ghe.

Sue, hes g bxe n eveyne knws

wh he s. Hs echnque s bu spee. Is

bu uneshng he s jb n sp secn.Bu Bunce Res gh ec hs me

sses n h wns. F such quck ghe,

hwbck Sug Ry Lews, s cu

unesn why he esn eceve me

vbe ecsns.

S s snge h hevywegh s n hs

. Ech me eeee ses he hn

Cnvesn, smeby hs seus pyy.

Bu Cnvesn hsn ccmpshe eveyhng

n hs wn. He ees n whe em

pepe hm gh: Cy Vue, TheFcn Reuce, n The Anxey Mg.

Unke he em behn Cnvesn, Bunce

Res ns ne n hs pepn s

usuy cuse n ne punch, ne quck h.

Th quck h cn be vey eecve, bu e

hs hwn, Bunce Re ees scy n

mpvsn hpng h he mmenum

he s punch es ecsn. Sue h

s jb s mpn n Bunce Re shu

eveyhng pssbe n , bu hes

cuse eney much n sh-em

g, n secny bjecve.

The gh begn pecby. Bunce Re

uneshe hs s jb n huy n he cw

cheee. Success hey chne epeey!

F he s ew uns, Bunce Re wen w

ngng hmse ve he ng. Cnvesn

cun uch hm. I seeme cen h he

ws he n he scec, bu when he

numbes cme n, Bunce Res success h

n mpc n Cnvesn.

Bm Lne: In he nex ew uns,

Cnvesn begn execung hs gh pn.

He hew ewe punches hn Bunce Re,bu hey wee much me quy shs. He

s h n cpue he cws v. F

sx uns hey subbny chmpne he

sme Bunce Re. Cnvesn knew he ws

he bee ghe, bu hee ws ny ne wy

swy he cwRESULTS.

All of a sudden, Conversion caught Bounce

Rate with a cross and down he went. One, two,

threeAs Bounce Rate was down, Referee ROI

just stood there. Four, ve, sixsll down and

ROI did not move. Seven, eight, nine, tenKnock out! Referee ROI grabbed Conversions

glove and raised his arm in the air.


Focus on what maers. All you need to know is

that conversion won by KO.

Tricks vs. Tesng

I hpe yuve enjye hs hee-p bxng

sees whee weve pe he es n

gees cks gns pven pncpesh hve cme m Tesng. Whehe s

Fsh Bnne n Abve-he-F pge, he

mpn hng emembe s es nse

mpemenng bny bse n smenes

cn mss ecmmenn. Becuse

whe hese pven pncpes hve been vey

successu u Resech Pnes, uness

yu es hem yuse n scve wh e


33/132


wks yu, hey e n bee hn cks.

An ve he s en yes, weve seen

Tcks cme n g. Sme hve wke n sme

hven, especy n he sh em. Bu whu

Tesng, s mpssbe unesn why Tckws successu why wsn.

Becuse Tcks e ppu esn. They cn

be wy successu sme busnesses. The

pmy quesn s, cn hey be successu

yu busness. An hs why Tesng ges he

TKO gns Tcks. Tesng gves yu he busness

negence scve wh ey wks, gh

nw, yu cmpny.

Ths why, he en he y, Tesng w wys

ee Tcks. Bu I p he gh ngy ghnw, becuse he w cn wk gehe. Ths s n

ze sum gme. Wheneve yu he smeby

e yu Ty hs ck Impemen hs cc,

ppy scuny n eemne yuse yu

hnk cn wk yu busness n. I hs

chnce, gve sh. Bu es gns wh yu

knw ey wks see wks bee. O

n. In hs wy, yu w cnnuy mpve yu

mkeng pemnce.

Related Resources

Tcks vs. Tesng: The Be Inene Supemcy

Fsh Bnne vs. Hene, Le Quny vs.

Le Quy: The gh nne mkeng ROI

cnnues

Fce Yu Fes: Why vss ey bunce m

yu se, p 3

Mkeng Opmzn | Onne Mkeng Tcks vs. Tes
http://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/bounce-rate-fears-part-3.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/flash-vs-headline.htmlhttp://www.marketingexperiments.com/blog/analytics-testing/tricks-vs-testing.html


34/132



Le genen emns he p esn

ms cmpnes exhb evens n

eshws. An B2B mkees e

cnsny kng es hey cn use

ve me ROI m he evens buge.

I cme css hs hepu ps by Mke

Thmmesch n 100 Te Shw Le Genen

Ies hs wh checkng u. The wng

s smpng Thmmeschs ps h I hughwee useu:

4. G ewe e shws, bu pu me

e n bh s pepn n

pmns ech emnng shw.

6. Tck es eemne n expn n he

shws wh he bes ROI

9. Ge bh spce cse he hub

c, by bgge cmpe

28. Hve yu ses pepe nve he

pspecs vs yu bh n se up

meengs n vnce

29. Sen n em nvn he shws pe-

egsee enee s hs ye, n

he egsee enee s m s ye

30. Use sc me ech me enees

32. Ps yu e shw scheue n

yu webse wh nk sgn up

ppnmens

45. Gvewy smehng useu yu ge

uence

46. Hve cnes enees n yu bh

Ae eng he s 100, hees ew me

ps I wu :

1. Fw-up qucky e he even. Thnk

bu yu w-up pcess bee he

even hppens n ews.

2. Cee even w-up cnen peces,

kng pns n em empes yu

ses em use vue n cnnue

he cnvesn n eevn wy he

hn pchng eveyby.

3. Devep nuung ck h even

enees cnnecs wh he heme he

cnen he even. Ty hs es

ew mnhs mnmum.

4. See he even s cnvesn (

cnvesn se) n cmpgn. Dn

sp he g. Bnsm wys yu cn

keep he g gng.

Wh he ps wu yu hs s?

Related posts:

Le Genen ps Teshws

Cneences

100 Tips for Trade Show

Lead Generaon

By Brian Carroll,CEO of InTouch

May 11, 2010
http://www.skylinetradeshowtips.com/100-trade-show-lead-generation-ideas/http://www.skylinetradeshowtips.com/100-trade-show-lead-generation-ideas/http://blog.startwithalead.com/weblog/2010/01/lead-generation-tips-for-tradeshows-conferences.htmlhttp://blog.startwithalead.com/weblog/2010/01/lead-generation-tips-for-tradeshows-conferences.htmlhttp://blog.startwithalead.com/weblog/2010/01/lead-generation-tips-for-tradeshows-conferences.htmlhttp://blog.startwithalead.com/weblog/2010/01/lead-generation-tips-for-tradeshows-conferences.htmlhttp://www.skylinetradeshowtips.com/100-trade-show-lead-generation-ideas/http://www.skylinetradeshowtips.com/100-trade-show-lead-generation-ideas/


35/132


Editors Note: The MarkengExperiments

community is an interacve group with a

great deal of quesons and answers between

marketers and their peers as well as with the

MarkengExperiments sta. Occasionally we

publish these interacons on the blog when

we think there is a parcularly good queson

that our readers can benet from

QUESTION:

I m n he pcess ng sh suvey

u cens guge nees n sme new

pucs we w be nucng n he . I

wn n u wh hey wu be wng

py hese nvu pucs.

Am n LegMy gu es me h I s

hee pces, hey w wys seec he es

expensve. Is hee wy sk pce (

nge--pce) quesn h uy ecs

esnbe espnse?

We e ng he A/B/C esng n pce nce

we u he pucs. I m nw he

pn yng eemne whee se

pce esng pupses. Any gunce yu

cu pve wu be gey ppece.

Debh

Vce Pesen Mkeng

Oven Pk, Knss

ANSWER:

Debh, yu sense s cnssen wh u

expeence. Askng cusmes wh hey

wu be wng py s bes n unebe

wy eemne he pm pce new

puc. The w bes ppches n u

expeence e bse n new puc

pces ehe n cmpbes vue.

Comparables: I hee e he sm

pucs n he mke, hen yu cn ge

sense g sng pn bse upn

he nge wh cusmes e ey

pyng he cses cmpeve engs.

Value: I hee s nsucen nmn

vbe bu cmpbes (e.g., becuse

he new pucs e uy bekhugh n

nue), hen yu cn y scve wh

yu e cusmes e cueny pyng

ssy he nee yu new puc us,

wh s cueny csng hem by ng

ssy he nee.

F exmpe, es sy yu mke webcm

sysem bues n cmmec

evepes h wu w hem

emey mn he bung ses he

n vnsm ve he Inene. Yu mgh

bse yu n pe-cme pcng n wh

css he nghme secuy gency

gu n equven e.

Ask the Scienst: Price tesngmethods and pracces

By Bob KempDirector ofSciences

May 5, 2010

Mkeng Opmzn | Ask he Scen
http://www.linkedin.com/groups?gid=41533http://www.linkedin.com/groups?gid=41533http://www.linkedin.com/groups?gid=41533http://www.linkedin.com/groups?gid=41533


36/132



O yu cu bse pcng n bung

nusy n nsunce nusy sscs

bu vege nnu sses ue he

n vnsm n n he es nsue

unsecue bung ses. Then, eemne

whehe ng yu sysem mgh, nn eucng sses, quy euce

hse nsunce es. Ths s eee s

vue-bse pcng.

Yu my be be exc vue m

cusme suveys by skng hem bu he

nue n nnu css he pbems h

yu new puc esses n hw much

cueny css hem ehe sve

mke whu sun.

Agn, hugh, nce hey peceve he skh he nswes mgh nuence uue

puc pcng, he pecve ccucy

mnshes qucky.

I hpe hs s hepu, Debh.

A he bes,

Bb Kempe

Dec Scences

MECLABS Gup, LLC

Related Revsources

Pce esng nne subscpns

Lnng Pge Opmzn Fnng e

pce pns

Oe Pcng Hw es n pmze yu

pcng

Photo aribuon: agentakit
http://www.marketingexperiments.com/blog/internet-marketing-strategy/price-testing-online-subscriptions.htmlhttp://www.marketingexperiments.com/improving-website-conversion/finding-ideal-price-points.htmlhttp://www.marketingexperiments.com/improving-website-conversion/finding-ideal-price-points.htmlhttp://www.marketingexperiments.com/improving-website-conversion/offer-pricing.htmlhttp://www.marketingexperiments.com/improving-website-conversion/offer-pricing.htmlhttp://www.flickr.com/photos/agentakit/http://www.flickr.com/photos/agentakit/http://www.marketingexperiments.com/improving-website-conversion/offer-pricing.htmlhttp://www.marketingexperiments.com/improving-website-conversion/offer-pricing.htmlhttp://www.marketingexperiments.com/improving-website-conversion/finding-ideal-price-points.htmlhttp://www.marketingexperiments.com/improving-website-conversion/finding-ideal-price-points.htmlhttp://www.marketingexperiments.com/blog/internet-marketing-strategy/price-testing-online-subscriptions.html


37/132


Mulvariate Tesng: Can you radically

improve markeng ROI by increasingvariables you test?

As I ws eng ew

LnkeIn scussns bu

muve esng (MVT),

I begn wne 2010

ws gng be he ye

muve.

1,000,000 monkeys cant be

wrong

Muve Tesng

(MVT) s sng en

pce n he pnhen

buzzws ke cu

cmpung, sevce-ene

checue, n synegy. Bu s es he

sme hng s n expemen? Whe I m n

sscn (n I sy he Hy Inn

s ngh), wkng MkengExpemens

wh he nyc kes Bb Kempe (MBA)

n Au Sv Nv (MBA) hs hepe me

unesn he vue scpne ppch

expemen esgn.

Wh I see u hee s h e knwege

s nee ngeus hng. G nenns

behn pweu n evey esy--use

pms ke Omnue Tes&Tge n

Gge Webse Opmze hve genee

mseng sense h s ng s muve

es s ge enugh (seve hune mecmbnns beng ese), es ne he

cmbnns w upem he cn.

Ths nn hs becme he vue ppsn

gwng numbe cmpnes eng

sevces un ehe he bg-nme he

wn (smpe, n en heee ese

se up) MVT s. They e sensby

beng n he echngy, n n n

sysemc ppch expemen esgn

ny pcu UI/UX (use nece/use

expeence) pmzn hey.

Even hugh, s Bb hs pne u

me, s esnbe h n MVT seup wh

bn cmbnns my n ye

ve he cn, my cnenn s h he

sk-weghe busness cs ssse

cusme s w. Theee, e sps he

bugenng MVT shps m sey eng

100% gunee. Jus ke he pveb

mn mnkeys wh ypewes, smewhee

mng husns spy-n-py

emens he MVT ess e expece

puce ne hs bee hn he es.

1 monkey with a sck

One mj cuy wh esng n gene

becmes pnuy bvus wh MVT: he

me emens, he nge he es w un.

F ms cmpnes, wh ks s ke

ge es my eque yes wh c

ge sscy v esus.

By Boris Grinkot,Senior Manager,Research andStrategy

April 26, 2010

Mkeng Opmzn | Muve Tes
http://www.marketingexperiments.com/improving-website-conversion/multivariable-testing.htmlhttp://www.marketingexperiments.com/improving-website-conversion/multivariable-testing.html


38/132



In espnse, ne emegng MVT sevce

me es geng se

by usng pve emnn key

unepemes, n exchnge he

es esus pvng me nmn

beyn enyng he wnne. Such esesus e n s useu s he u-

c behen esgnng subsequen

ess becuse pve emnn

emens mkes cu expe

he psychgc cs n cnsume

peeences espnsbe he es

ucme. The mmee busness benes,

hweve, e me mmee.

S, whee excy s he pbem? As

mkees, e we n he busness

empyng he scenc meh esgn

gceu expemens s u ucy uy

ge mesube esus? I humby sugges

h s mkeng pessns, we shu

nehe be n n be sse wh jus ne

es, n me hw successu s.

The b news n he g news s h

we mus esgn n expemen pn

pmze cnnuy, en m

peceng es esus, n espn

chnges n cusme peeences, mkecnns, n u by segmen

n c. Expese n expemen esgn

n unesnng hw nepe esus

smpy cnn be epce by se--n-

ge- echngy (ye).

Economy of tesng

Th s n sy h MVT pves

ncec esus. The esus e

mhemcy v, even hey

eque ng me bn. A he sme

me, m he busness pn vew,

nvesmen n expemen esgn

expese s expensve. Unesnng

vumes pubshe esech cnsumes

vube me. The 100% gunee sue

suns g.

An s he gunee es w

ppe he spenh mkees

wh e ye eve n he scence

pmzn. The cc ssue n he

ecnmy esng s whehe mehc

esgn expemens s key pve

gee ROI hugh n nepen-

ven sequence es ens hn successu, bu emn nees. Ou

esech supps he me.

2010 my becme he ye muve,

bu I hpe h w s quey se

he sge n upcmng ye ROI-

cnscus esgn expemens.

How do you use mulvariate tesng?

Have you created an experimentaon

plan or do you rely on a series of one-o

tests? Share your triumphs and concernsin the comments secon of this post or

start a conversaon with your peers in the

MarkengExperiments Opmizaon group.

Photo aribuon: kevingessner
http://www.marketingexperiments.com/blog/analytics-testing/multivariate-testing-in-2010.html#responsehttp://www.marketingexperiments.com/linkedinhttp://www.flickr.com/photos/kevingessner/http://www.flickr.com/photos/kevingessner/http://www.marketingexperiments.com/linkedinhttp://www.marketingexperiments.com/blog/analytics-testing/multivariate-testing-in-2010.html#response


39/132


Expeence nne mkees hve he hs

syng ne pn nhe: we jus nhve he esuces wh yue skng.

Mybe he esuces en hee mybe yu

pjec s s n ne. Ehe wy, smeby ese

s eemnng he esny yu pns n

yu messge jus sn beng he.

S, mny mkees scmbe ecsn

mkes enn. They se up bg meengs,

pu gehe bg PwePns, bg Gn chs,

n sy up ngh pccng he peec pch

ge smpe, sheenng espnse.

Thee s me eecve wy ge ppv.

The cost of delay

Yu se yu ecsns mkes me eecvey

by pesenng hem wh smpe, eevn

mec: he pen cs ey.

Decsn mkes nee nmn hey cn

unesn, n he nguge mney (mney

s be me specc) cn be ne yu

ms pweu s. I yu cmpnys webses beeng n yu e ske x , hee

s n bee wy s he cnvesn hn

by shwng hw much b s beng s n

mnhy, weeky, n sme cses y bss.

Remembe, hs s he sme b h heps py

hse he pes sng n n

yus.

The soluon is at your ngerps

I yu e esng n beh yu cmpny,

yu w key hve he nmn yu nee

ge se.

1. Start with your web analycs. Evey es

shu hve n bjecve, n knwng he

bjecve m web nycs pespecve s

yu s sep. Ths cu be se, e,

sgn-up n he pge pcess wn es.

2. Unesn s nnc mpc: Lk ech

ess success mec n sk yuse:

. Hw much s h e wh (ve me)?

b. Wh s he keh h he cmpee

sep w becme e, n hen evenue

yu cmpny?

c. Wh s he vege e vue

pesn cmpeng he e n hs

pge/pcess?

A hese quesns begn cnnec yu web

nycs wh yu nscn/nnc .Hehy cmpnes keep bs n wh ech

he pcesses s wh, n mke busness

ecsns bse n hese busness-eve mecs.

I yu n hve bsc unesnng he

nnc mpc ( bee wse) wh

yu e esng, yu e wkng ngeus

ph.

1. Connect the dots: F exmpe, yu knw

h sep w yu e pcess s seeng

70% p n vss n h he vege

cusme e sze s $150. I yu wee ecve, sy, 10% hse vss h e

ypcy s hnks he negence yu

gn hugh esng, hw much evenue

wu hse ess ve?

2. Ask the big queson: Ae we (s cmpny)

wng sk sng hs much mney by n

geng hs es up n me hs sesns

pek?

Cost of Delay: How to win approval foryour test and test schedule

By Jon PoweResearch An

April 12, 201

Mkeng Opmzn | Cs De


40/132



Yu cnn pssby knw he cmpeng pes

n yu ecsn mkes s, s yu cn cuy

he u nyss hem. Taking this approach,

however, gives them the informaon they need andthe aenon your test deserves to complete the

analysis cmpng he pen ss he cs

pushng hse he pes bck ( cqung he

esuces).

Empwe yuse n yu es scheue by cey

cmmuncng he pbem yu ecsn mkes

shw hw yu esng my sve hese busness-

eve pbems ve se pe me.

Related resources

A/B Sp Tesng

Wecme Messge Sequence Tese

Se Cmpby Tese, Secn 1 (Resech)

Se Cmpby Tese, Secn 2 (Anyss)

Photo aribuon:jjvaca
http://www.marketingexperiments.com/improving-website-conversion/ab-split-testing.htmlhttp://www.marketingexperiments.com/email-marketing-strategy/welcome-message-sequence-tested.htmlhttp://www.marketingexperiments.com/improving-website-conversion/site-compatibility-tested-research.htmlhttp://www.marketingexperiments.com/improving-website-conversion/site-compatibility-tested-analysis.htmlhttp://www.flickr.com/photos/jjvaca/http://www.flickr.com/photos/jjvaca/http://www.marketingexperiments.com/improving-website-conversion/site-compatibility-tested-analysis.htmlhttp://www.marketingexperiments.com/improving-website-conversion/site-compatibility-tested-research.htmlhttp://www.marketingexperiments.com/email-marketing-strategy/welcome-message-sequence-tested.htmlhttp://www.marketingexperiments.com/improving-website-conversion/ab-split-testing.html


41/132


Tyng gn he bugey n execuve

supp cee cuue esng n yu

gnzn? Wee hee hep. Ths ce

w pve yu wh behn-he-scenes

k he sy hw ne mkee ws

be chmpn nne esng whn he

gnzn n he essns yu cn enm he sgncn success h esue.

In s ng, we she esus n nyss

m 14 een nng pge esgns css

mupe expemens n pve yu wh

he segc busness vnges nne

expemenn n hw vecme he

cmmn chenges esng.

What is your biggest internal challenge to

online tesng?

In ecen p he MkengExpemens

uence, we ske bu he bgges nen

chenge nne esng. Ms espnens

nswee me cnsns. Whch, pu

nhe wy, mens hey n hve he

esuces segy n pce evege he

esng-pmzn cyce.

T hep yu se yu gnzns ecsn

mkes n ge he esuces yu nee,

we cumne hs ce wh u-p

success segy ceng cuue

esng css ny gnzn. By ceng

The Business Case for Tesng:How one marketer convinced her

business leaders to start tesng anddrove a 201% gain in the process

By Daniel BursteinAdam Lapp,

Pamela Markey,Ausn McCraw,Andy Mo

cuue esng, yu w ensue h

yu gnzn hs he esuces n

segc vsn mpve ROI wh e-

w, e-me .

T hep yu jus h, we g behn

he scenes he expemens nvve n

esng-pmzn cyce. S es s

by kng he sy u successu

mkee.

I begns wh he s expemen, hnk

s p cncep

What is your biggest internal challenge to online testing?Poll results

Executive Support

Time

Culture

Budget 20%

22%

46%

12%

Mkeng Opmzn | The Busness Cse Tes

April 15, 2010


42/132






Research Notes:

Background: Pves en--en mke suns sm-n-meum-sze busnesses.

Goal: Incese he mun es m n nne m.

Primary research queson: Whch pge w bn he ms m submssns (.e. es)?

Approach: A/B/C/D mu-c sp es h cuse n ncesng ve cmmuncn

he vue ppsn.

EXPERIMENT #1

Control:

Ths ws he pmy nng pge hs Resech Pnes p sech cmpgns.

Ou em nyze hs pge usng cnvesn pmzn mehges, such s he

MkengExpemens Cnvesn Inex heusc
http://www.marketingexperiments.com/methodology-marketingexperiments.htmlhttp://www.marketingexperiments.com/methodology-marketingexperiments.html


43/132


Ae ppyng he heusc bve, he eseches ece es c eesgn he pge

n cee hee pmze es emens. They she mny he sme eemens ncung

pmze henes n sub-henes, cpy, yus, ceby ncs, n cs--cn.

Temen 3 een ncenve n hesub-hene.

Temen 2 ee k n cupspec s.

201% Increase in Total LeadsTreatment 1 increased visit-to-lead conversion by 201.3%

Treatments:

Results:

Treatment 1

Treatment 2

Treatment 3

Conversion Rate Relave Dierence

Cn 4.86% -

Treatment 1 14.65% 201.3%

Temen 2 13.37% 175%

Temen 2 13.10% 170%

Mkeng Opmzn | The Busness Cse Tes


44/132



The Challenge:

S es g bck u sy, n ke k

wh he mkeng em nex. The

mkeng em n see hese esus,bu use hem s p pn gn gee

buge n segc uhy, n ge eepe

n he esng-pmzn cyce.

O cuse, eveyne cu s wh esus

ke hese, seng he esng-pmzn cyce

wu be esy. Bu yu p cncep

es n s wh bg wn (n, s wh

nyhng, smemes kes e pcce

ge gh), we hve ess he pssby

sng hs cnvesn whu sng

esus bck yu up.

Iniang the Conversaon:

The execun-ene chenges yu ce

e key mn cncen yu, bu h s

n wh busness ees ce bu. They

wn knw bm-ne esus. S cee

esng-pmzn cyce, yu e gng

hve w css w gus.

Fs yu mus mve m I knw esng

s mpn n I nee s ng Ive execue sees scency n

sscy v ess. Ths s whee yu

execun chenges y.

Bu uy gn vue yu busness

ees, yu mus g m meey execung

hse ess beng be puy pcm

My esng segy n execun pn s

cnbung he nnc esus my

cmpny. Ths s wh yu busness ees

wn he.

T pesen cse esng n such wy h

yu cn vecme bh hese gus, cus n

hese u key benes:

1. Realisc innovaon Tesng ws yu

scve wh ey wks yu mke

wh e n pecsn, nse

eyng n e pcces.

2. Non-speculave reliability Tesng

pves s h s bse ncusme behv he hn specun.

3. Real-me speed Tesng puces e-

me esus.

4. Precise results Tesng gves mkees

he by mesue he eun n

mkeng es wh pecsn. Tesng

pves mesube

the marketingexperiments quarterly research journal, q2 2010

Documents