next generation marketing measurement

A Database Marketing Agency

Building a next generation marketing measurement system:the frameworkMerkle ThoughT leadership series

Building a next generation marketing measurement system: the framework

Building a next generation marketing measurement system: The framework | © 2008 Merkle

executive summary:marketers struggle more than ever to develop accurate, reliable, and consistent metrics around the performance of marketing spend. relying on incorrect metrics creates a snowball effect that compounds

measurement errors along the entire marketing decision process. decisions on

how much to spend on marketing in total, how much to spend by media, and how

to coordinate a multi-media program are all directly influenced by the metrics

used. for these reasons, the importance of evaluating your current measurement

framework and building a solid foundation can’t be understated. n

Marketers already employ processes to link a purchase with a marketing activity to create metrics on marketing performance. These marketing measurement systems fall into one of two camps, each with its own shortcomings: • Top-downattributionapproachesthatprovidesuchahighlevelviewthattheyfailtobeactionable,or • Bottom-upattributionapproachesthatpresentacollectionofverydetailedinformationbutfailto integrate into a single view.

Bettersolutionsdoexistandcanbeachievedthroughsophisticatedanalyticalandtechnologicaltechniques.MerkleintroducesanextgenerationmeasurementsystembasedonProbabilistic Contribution (PC) scores (see definition on page 5). rather than attempting to link a purchase toamarketingactivity,acontributionscoreiscreated,byintegratingtop-downandbottom-upmeasurement systems, for each factor that potentially influenced the purchase. The key advantage is integration-bridgingmultiplemedia/channelsandpointsoftimeforasinglepurchasetoprovideacomprehensive view of the consumption pattern.

This white paper, the second in a series of four on marketing optimization, addresses the critical elementsneededtobuildanextgenerationmarketingmeasurementframework.Athirdpaperwilldiscuss the data, infrastructure, and software tool requirements. and a fourth and final white paper will outline how to effectively integrate this system throughout the marketing decision making process.

2



five factors fuel marketing’s roi dilemmaWhat makes answering such fundamental questions around the ROI for marketing spend so difficult? Some of the main reasons for this include:

1. Consumers face daily ad bombardment. studies consistently show that consumers are exposed to amind-numbingonetotwothousandadvertisingimpressionsperday.Thesheervolumeof advertising makes drawing a line from messaging to consumer activity all the more difficult.

2. fragmentation of media. The nearly monolithic media channels of previous decades have splinteredintheageoftheinternet,on-demandtelevision,andsatelliteradio.Forinstance,the numberoftelevisionchannelsincreasedoverahundredtimesinthelastthirtyyears.Thegrowing consumer trend of multitasking while accessing multiple media – especially prevalent in young consumers–throwsawrenchintomarketers’abilitytoattributeactiontomarketingmessage.

3. failure to consider the complexity of consumers’ decision process. a purchase is not directly tiedtothemostrecentadvertisingexposure,despitethefactthatmostorganizationsattributesales to marketing spend in this manner today. purchase decisions are exceedingly complex and consumersarelikelytobeinfluencedbymultipleadvertisementsovertimeanddifferenttypesof media.

4. Consumers are in control. Consumers have more choice than ever over when and how they consume media, making it that much more difficult for marketers to even know if a message was received.

5. organizational challenges. Mostmarketinggroupsorganizebymediatypecreatingasiloeffect (e.g.brandmarketing,directmarketing,andinteractive/onlinemarketing).Thesesiloseachdevelop their own goals, metrics, infrastructures, and partners to maximize the impact within the media they can control.

3



Critical elements of marketing measurement Marketingorganizationsrequirethefollowingkeyelementstoeffectivelybuildandimplementnextgeneration marketing measurement:

1.Analyticalacumentocustom-buildasolutiontolinkallmarketingactivitiestoconsistent performance metrics (i.e. measurement framework)

2.Dataavailability:Accesstoacomplete,accurate,andfrequentlyupdatedmarketingdatauniverse

3.Infrastructure(Mediamanagementdatabase)

4.Integrationintobusinessdecisionmaking(belief/buy-inandintegrationthroughsoftwaretools)and integrationacrossalllinesofbusiness.

Terms Used In This PaperStandardizedlanguagewillbeusedinthispapertoproviderelevanceacrossdifferentindustries.Afewnotablecasesarehighlightedbelow.Standardizedtermsaredefinedasfollows:

outcome of interest–Thispaperwilluse“purchase”torepresentagenerictermthatcouldbeappliedto any performance measurement a company may use. Throughout the discussion, purchase can easily bereplacedwithresponse,newcustomer,revenue,profit,LTV,oranyothermetricthatmaybeusedtocalculate desired outcomes. While it is true that some details related to techniques and implementation maychangedependingonthemetricused,thediscussionthroughoutthispaperwillbebroadenoughto encompass any of these metrics.

media/Channel–Adistinctionwillbemadebetweenmediaandchannel,withmediarepresentingcommunication flowing from the company to the consumer, and channel representing communication fromtheconsumertothecompany.Additionally,mediawillbediscussedinthreeclasses;direct,mass,and interactive.

top-down and Bottom-up–Thetop-downandbottom-upapproachesaredesignedtorepresenttwocommonwaysmarketingorganizationsmeasuretheperformanceoftheirmedia.Thetop-downapproachstartswithallmediaspendandthenbreaksthemediaspendintomedia,region,andtimeframe.Thisisthebroadestperspectivecommonlyusedinmediamixanalyses,butrarelytakesverydetailed information into account (individual level data, creative, quality of advertising, etc.). in contrast, thebottom-upapproachstartswiththeverydetailedlevelofinformation.Thisapproachiscommonindirect and interactive media analyses (consumer segments, predictive models, decile analyses, etc).

4



attribution–Theterm“attribution”representsanyprocessdesignedtolinkamarketingactivitywithanoutcomeofinterest(i.e.purchase).Typically,anattributionprocessattemptstolinkthepurchasewithasinglemarketingactivityorcampaign.Moresophisticated,andlesscommon,attributionprocessesapplyprobabilitiesacrossmarketingactivitiestocapturecross-mediainfluencemoreaccurately.

Thispaperoutlinesanewapproachtomarketingmeasurementby:

1.Settingthestagethroughadiscussionoftypicalattributionprocessesandthecurrentchallengesof marketing measurement systems.

2. introducing analytical framework for the next generation marketing measurement system. This paperdescribesanewmetriccalledProbabilistic Contribution score (PC score) used as the key metric to link marketing activities with the outcomes of interest.

3. providing guidelines on the analytical approach required to calculate pC scores across all marketing activities.

Current Challenges facing marketing measurement systems Ifasked,manyorganizationsclaimtohaverobustandreliableprocessestoattributesalestomediaspend.ForresterResearchconductedasurveyin2008thatconcludedthat69%ofmarketersbelievetheyareatleastsomewhateffectiveatmeasuringmarketingROI.IthasbeenMerkle’sexperience,however,thatafterdiggingbeneaththesurface,mostmarketingmeasurementsystemsfallfarshortofrobustorreliable,especiallywhentryingtoforecastmarketingimpactintothefuture.The2008ANA/MMAMarketingAccountabilitySurveyshowedthatonly10%ofmarketersfelttheycouldforecasttheeffectofa10%cutinbudget,andjust14%saidthatseniormanagementintheircompanieshadconfidenceintheirfirms’marketingforecasts.CFO’sareevenmoreskepticalofthenumbers.NinetypercentofCFO’sdon’tusemarketingROImetricstohelpsetmarketingbudgets.“Theydon’tbelievethenumbers,”saidJeffreyMarshall,theretirededitorinchiefofFinancial Executive magazine.

5

Probabilistic Contribution Score:

The Probabilistic Contribution Score (PC Score) is a value assigned to any activity or event (marketing or otherwise) that influences the consumer toward a desired action (i.e. purchase a product). PC Score derives its name from:

• Probabilistic – The sum of all the values for any given purchase equals

• Contribution – Each value represents the relative contribution or influence specific activities or events have on consumers’ decision making process

• Score – The values are designed to be informative within a given purchase or when rolled up to summarized reports



AnotherrecentForresterstudyhighlightssomeofthetopbarriersmarketingorganizationscurrentlyfacetoimprovingmarketingROI(seefigure1).ManyofthechallengesForresteridentifiesaredirectlylinkedtothefourchallengesaddressedbythisframework(analyticsophistication,dataaccessandaccuracy,technology,commonmetricsandbusinessintegration).

Figure 1 marketers face hurdles in staffing, data, technology, and common metrics

Source:ForresterResearch,“DatabaseMarketersEvolveTheirROIMeasurement”

6



two Common approaches to marketing measurement Marketers already employ processes to link a purchase to a marketing activity to create metrics on performance. These marketing measurement systems fall into two camps: • Top-downattributionprocess-providingsuchahighlevelviewthattheyfailtobeactionable,or • Bottom-upattributionprocess-presentingacollectionofverydetailedsystemsthatfailtointegrate into a single view.

Overview of typical top-down attribution processes Thetop-downapproachstartswiththeentiremarketingbudgetandthenlooksathowthemarketingbudgetissplitbetweenmajorcategories(media,regions,products,etc).Theadvantageofthisapproach is that it provides a framework to compare the performance of all marketing spend with commonperformancemetricstooneanother.Thispropertyiscrucialtothebuildingofarobustmarketing measurement system, and is the reason why the approach outlined later in this paper uses thetop-downapproachasthestartingpoint.Thistypeofanalysisisoftentermedmediamixanalysis.

Figure 2 top-down and Bottom-up approaches

7



Certain industries, like consumer product goods and pharmaceutical companies, have a long history of usingmediamixmodelingtohelpdeterminemarketingspend.Theseindustriestendtohavein-directrelationship to their consumers and involve purchasing of less expensive products conducive to a short purchaseconsiderationtime(seefigure3).Theseindustriesdevelopedmediamixmodelingcapabilitiesearlyonpartlybecauseitwasoneofthefewwaystounderstandmarketingperformancewhenthereisnodirecttoconsumerrelationship.Theseindustriesalsobenefitfromstandardizeddatasources(like pos data) which have allowed standardized media mix modeling methodologies to develop. as relationshipsmovemoretowardsdirect-to-consumerandthepurchaseconsiderationincreases,thedataandmethodologyrequiredformediamixmodelingbecomesmorecustomizedforeachcompany.Thiscustomization requires more investment to develop, and therefore it is typically the companies with significantmarketingspendthatcanjustifythemodelingwork.

Figure 3 Choosing the measurement approach

*Note – the company placements above are for illustration purposes only. In some cases a company can be put in multiple areas due to different product offerings. For example, a Dell desktop computer would be in the top right (as shown) whereas a replacement ink cartridge for Dell printerswould be in the lower right.

8

most established top-down measurement systems

non-direCt direCt to ConsumerPu

rC

ha

se C

on

sid

er

ati

on

tim

e



However,therearesomesignificantweaknessestothetop-downapproach,including:

1. it takes too long. it is not uncommon for this type of analysis to take over six months to complete. Thedeliverableisoftenapresentationofhowmediaperformedfortheanalysistimeperiodalong with recommended changes. By the time the recommendations are presented, the marketing landscapemayhavealreadychanged,makingtheanalysisoutdatedbeforeitisevenpresented. Anotherwaytoviewthispitfallisthatmediamixworkisoftenmanagedonaprojectbasis,as opposed to an ongoing system.

2. it fails to take into account the specifics within each media.Thereareanumberofoptionsand choiceswhenspendingdollarswithinamedia.Ifthebudgetforthatmediaisbeingcutback due to historical poor performance, then perhaps the allocation of spend within the mediashouldbechanged,notnecessarilythesizeofthebudget.Forexample,therearemany decisions and analyse within a direct mail program that are not adequately summarized bysimplywhenjusttotaldollarsspent.

3. lack of integration with media execution. Media mix analyses often conclude with recommendationsonhowtoshiftmarketingdollarsbetweenavailablemedia.Itisrareforthese analyses to address the complexities associated with the execution of each media. of particular importanceareinventoryandcostconstraints.Forexample,arecommendationtoincreasespend inspotTVmayrequireintroducingnewdaypartsornetworkswhichtypicallychangesthepriceto purchase the media. as cost per impression increases, the relative effectiveness of that media is likely going to decrease.

Theseweaknessescanbesummarizedbytheprojectvs.processnatureoftheanalyses.Theseanalysesareoftenconductedad-hocwithoutbecomingpartofthemarketingmeasurementssystems.Bynotbeingfullyimplementedintothemarketingculture,thesetypesofresultsrankasmerelyinteresting–butnotrelevantenoughtochangemarketingstrategyandspend.Additionally,becausethetop-downmeasurementapproachfailstotakespecificsintoaccountthereisaneedforthemoredetailedbottom-up approaches.

Overview of bottom-up typical attribution processes Thebottom-upapproachstartswithdetailedinformationwithinamediaandthenestablishesasystemto measure performance of segments or campaigns within the media. The nature of the data and thetypeofmetricscalculatedvarydependingonthemediadiscussed(whichispartoftheproblem).Somemetricsareefficiencybased(costperimpression)whileothersareperformancebased(costperpurchase,ROI).Itismorecommontoseeperformancebasedmetricsavailablewhereitiseasiertoconnect media spend directly to individual level purchases.

9



Directandindirectattributionaretwocommonlyusedpracticesasthefoundationforbottom-uptypemetrics.Abriefoverviewalongwithsomeexamplesandpitfallsarehighlightedbelow.

direct attributionisoftenclassifiedasanattributionprocessthatusessomekindofuniqueidentifierto link a purchase to a specific marketing activity. examples include asking customers to provide a specificcodefromthebackoftheircatalog,aunique1-800numberthatisonlyusedforaparticularrunofatelevisionadvertisement,ortrackingaclickonabanneradbydirectingtoaspecificlandingpage.Whileeachoftheexamplesabovehastheirownmerit,theyultimatelyfailinproducingareliableattributionprocess.Theyfailbecausethedirectattributionisonlyrecordingthelastmarketingexposurebeforethepurchase,butnotnecessarilywhatisdrivingthepurchaseintentinthefirstplace.Acommonexampleisunique1-800’sonTVads.ATVadmaydrivedesiretopurchase,butpeopledon’tusetheunique1-800numberonthead,butrathersearchfortheproductonGoogle,gotoalandingpage,andthenusethatunique1-800number.TheTVadisgivenlesscreditthanitdeserves,and search engine marketing gets more credit than it deserves.

indirect attribution,orbusinessruleattribution,usessomeassumptionstolinkarespondertoamarketingeffort.Forexample,aunique1-800numberisnotusedbutweknowthattheconsumerreceivedacatalogtwoweekspriortothepurchase.Inthiscaseweuseindirectattributiontoassumethe catalog drove the purchase. This process has some advantages and disadvantages over direct attribution.

Indirectattributionallowsamarketertoattributeapurchasewhendirectattributionisnotpossible.However,indirectattributionrulescomedowntolittlemorethananeducatedguess.Howlongshouldthetimewindowbe?Whatifweknowtheconsumerreceivedamailpieceoneweekago?Onemonthago?Sixmonthsago?Thisdecisionwilldrasticallyimpacttheperceivedperformanceofthedirectmailcampaigns. even worse, the pattern feeds on itself (up to a point). if it looks like direct mail performs well then a company will do more direct mail, which leads to an even higher likelihood that a consumer who purchases your product received a mail piece within the time window. This assumption inflates direct mail performance metrics even further. eventually the direct mail performance metrics will drop offasvolumeincreases,leadingtoaperceivedequilibriumrelativetoothermarketingoptions.Howeverthisequilibriumistypicallyfarfromthetrueequilibrium.

a common pitfall for organizations is the sense of sophistication associated with very complex direct andindirectattributionrules.Theattributionrulescontinuallybecomemorerefinedandcomplicated,leadingthemarketertobelievetheyhaveaworld-classmarketingattributionsystemandthereforearedoing very well at linking purchases to marketing activities.

10



Mostoftheproblemsassociatedwithdirectandindirectattributionsystemscanbeplacedintotwogeneralcategories:cross-media/channeltrackingandeffectsovertime.

1. Cross-media/channel tracking–Mostattributionsystemsattempttolinkapurchasetoasingle marketingcommunication,whereasinrealityconsumersdonotjustuseonesourceofinformationto make a purchasing decision.

2. effects over time–multipleadvertisingexposuresmaycontributetotheultimatepurchaseeven withinasinglemedia,butmostattributionsystemswilljustattributethepurchasetoasingle instance.

CurrentmarketingmeasurementsystemsultimatelyleaveCMOswithtwoproblems:

• an inability to see the whole picture.Today’ssystemsforceCMOstolookateitherbroad brushmetricsthatfailtoconsiderspecificsofindividualmediaoratdetailedmetricswithina mediathatfailtotranslateintometricsthatcanbecomparedacrossmedia.

• a disconnect between budget planning and execution.Typically,thebudgetplanning processhastwosteps:first,eachmediaorchannelisallocatedacertainbudgettospendand second,eachmediaorchannelattemptstogainthebestperformanceitcanfromthe allocatedbudget.Thisprocesseffectivelycreatesamajorplanningconstraintwherethe budgetbymediaisconsideredfixed.Eachmediamaybedoingthebestitcangiventhe budget,buttheprocessissub-optimalwhenconsideringallmarketingspendtogether. Theoptimalprocesswouldenableafluidbudgetallocationbymediadependingon performance,opportunity,andcross-mediainteractions.Ofcoursethebudgetallocatedto eachmediacanbechangedovertimebuttheprocesstodosoistypicallyarduous,requiring anefforttoprovetheperformancebenefitofamediawithrespecttoothermedia,which becomesfutilesincetherearenocommonmetricstocompare.

11


Building a next generation marketing measurement system: The framework | © 2008 Merkle 12

introducing a next generation measurement system The next generation measurement system forgoes the classical approach of attempting to link a purchasetoamarketingactivity.Instead,acontributionscoreiscreatedforeachfactorthatpotentiallyinfluencedthatpurchase.Thekeyadvantageoftheprobabilisticcontributionscore(PCScore)isthatitbridgesmultiplemedia/channelsandpointsoftimeforasinglepurchase.Whilethescoreshavelittlebearingonmeasuringmediaforasinglepurchase,thereportingmakesperfectsensewhensummarized.

Determining Probabilistic Contributions Thegoalofthenext-generationsystemistoassignaPCScoreforeverymarketingcommunicationthataconsumermayhavebeenexposedto.ThesumofthePCScoresforanygivenconsumerpurchasewillequal100%.Dependingonthescopeofthemeasurementsystem,PCScorescanbecomputedforthefollowingcategoriesandnon-marketingactivities(thelistbelowisnotexclusive): • Massmedia • Directmedia • Pricing • Promotion(i.e.salesorotherspecialoffers) • Creative,messaging,versioning • Brandawarenessorbaselineeffect • Naturalandeconomicenvironmentalfactors • Competitiveactionsandspend • Legislativeorregulatorychanges • Servicelevelsandcustomersatisfactionratings

The scope of categories a particular organization will want to tackle depends on the purpose, amount ofavailabledata,sophisticationofanalyticalskillset,andorganizationalreadiness.Mediamixmodelingwouldbeanaccuratedescriptionifthisapproachislimitedtomassanddirectmedia.Marketingmixmodelingisamoresuitabledescriptorifpricing,promotion,andenvironmentalfactorsareconsidered.Inourexperience,justusingthemodelabovetoincorporatemassanddirectmediaalonewillproduceasystemmoreadvancedthanthemajorityofmeasurementsystemstoday.Merklewouldrecommendaddinganumberofotherfactorsdeemedtohavethemostimpact.Thesecomponentsusuallyincludeacombinationofpricing,promotion,brand/baselineeffect,andcompetitivespend.

assuming a system is in place to assign the pC score for every purchase, then extracting meaningful informationfromthesystemisamatterofrollingthedatauptotheappropriatelevel.Forexample,ifwejustwanttoknowwhatmediaprovidedthebestROI,wewouldrollupthePCScoresassociatedwitheverypurchaseoveraspecificperiodoftime.Theresultwillshowtotalnumberofunitssoldforeach media used. once the total cost for each media is factored in we can get the roi for each media. Asimilarrolluplogiccanbeappliedtogeographies,customersegments,products,etc.



Probabilistic Contribution Requires an Integrated Approach TheprocesstoassignPCScoresrequiresacombinedtop-downandbottom-upapproach.Theprocessisatwo-stageprocess,withthetop-downapproachlayingtheroughbaselineandthebottom-upapproachrefiningtheweightswherepossible.

Theprocessofintegratingthetop-downandbottom-upapproachesisbestunderstoodthroughthefollowingsimpleexample.Forsimplicity,assumethefollowingeventstakeplace:

1.Themarketerusesonlytwomediatoselltheirproduct,TVanddirectmail.

2.ThiscompanyplacesthreespotTVadvertisementsinaparticularregionatweeks1,3,and8.

3.Additionally,thecompanyexecutesadirectmailcampaignatweek6.

4.Apurchaseismadeatweek9.

5.Thecompanyusesunique1-800numberstotracktheperformanceofeverymarketingactivity.

6.Theconsumerwhomadethepurchasedidnotuseoneoftheunique1-800numbersavailablebut was mailed a dM piece 2 weeks prior to purchase.

Thetop-downandbottom-upapproachesultimatelyproduceverydifferentanswerstothequestionofwhat marketing activity drove that purchase.

Bottom-up approach scenarioThebottom-upapproachwouldlikelyusebusinessrulesaroundthe1-800numbers.Forexample,thefollowingrulescouldbeapplied:

1.Iftheconsumerusesoneoftheunique1-800numberstomakethepurchase,thenattributethat purchasetothemarketingactivityusingthat1-800number.

2.Ifanumberotherthanoneoftheunique1-800numbersisused,thenchecktoseeifthecustomer wasmailedaDirectMailpiecewithinthelastthreemonths.Iftheyweremailedthenattributethe responsetothemailpieceviaindirectattributionprocess.

3.Ifneither1or2apply,thenallocatethepurchasetoabrandawarenessbucket.

Sincetheconsumerdidnotuseaunique1-800numberbutwasmailedwithinthethreemonthtimeperiod leading up to the purchase, we could conclude that the purchase was due to the direct mail piece.



top-down approach scenarioAnotherwaytoapproachthisprocessistousethetop-downormediamixmodelingapproach.Inthisapproach,weestimatetheeffectivemediaexposurethroughanad-stocktransformation(seefigure4). The concept is that a marketing activity can produce purchases over an extended period of time. Forexample,aTVadvertisementcouldcauseaconsumertocallandpurchaseimmediately,orcouldleadtotheconsumerpurchasingweeksormonthslater.Usingad-stocktransformationswecreatetheeffectivemediaexposuresforeachmediaatanygiventime.Figure4showsthisprocess,wheretheredlinerepresentstheeffectiveTVexposureovertimeandthebluelinerepresentstheeffectivedirectmailexposureovertime.Sincethepurchaseoccurredatweek9wecanmultiplytheeffectivemediaexposurebytheircoefficientsestimatedthroughthemediamixmodelstogetprobabilitiesthatthepurchasewasduetoeachpossiblemarketingactivity.Inthiscasewemayconcludethatthepurchasewas60%likelyduetotheTVexposures,10%duetodirectmailexposure,and30%duetoneither(i.e.brandawareness).

Figure 4 top-down or media mix modeling approach

Thetwoapproacheslistedaboveproducenearlyoppositeanswers.Thetop-downconcludesthatTVhadthebiggestinfluenceinthepurchase,whereasthebottom-upconcludesthatthepurchasewasduetothedirectmailpiece.Thetop-downapproachmoreeffectivelyintegratestheinfluenceofmultiplemediaovermultipletimepoints,butfailstotakeintoaccountsomedetail-levelinformation(i.e.ifthat

week

eff

eC

tiV

e e

xP

osu

re

PurChase made



consumerreceivedamailpieceornot).Ontheotherhand,thebottom-upapproachtakesthedetail-levelinformationintoaccountbutusesrigidbusinessrulestoconcludeanallornothinganswer.

an integrated approach to measurementMerklesuggestsanapproachthatutilizesthebestofbothmethodologies(seefigure5).Thetop-downapproachisusedasthebaselinesinceithasthedesirablepropertiesofaconsistentperformancemetricacrossmediaandnaturallytakesintoaccountcross-mediaimpactandimpactsovertime.Buttheresultofthetop-downapproachismodifiedbasedontheknown,detail-levelinformationavailable.Inthegivenexample,wewouldstartwiththetop-downsolutionbutthenadjusttheprobabilitiesbythedetail-levelinformationknown.Forexample,weknowtwoimportantpiecesofinformationabouttheconsumer: first, we know that the consumer did receive a mail piece two weeks prior to purchase, and, we know that the consumer was scored as a decile two name using a likelihood to purchase predictive model.Giventhisinformation,theintegratedoutcomeis50%likelihoodduetoDM,20%duetoTV,and30%duetogeneralbrandawareness.



Figure 5 an integrated approach utilizes the Best of Both methodologies

Thisintegratedapproachretainsthekeyadvantageofboththetop-down(consistentmetric)andbottom-up(useofdetailedinformation)approaches.

week

eff

eC

tiV

e e

xP

osu

re

PurChase made

aPProaCh resultlogiC

Bottom-up Approach

•Businessrules•Atomicdata

Top-Down Approach

•Mediamixmodeling•Aggregateddata

Integrated Approach

•Top-Downenhanced withBottom-up

Ifusedunique1-800numberthendirect attribution(TVorDM)Elseindirectattribution(knownexposuretoDM)Else ….

•Didreceivemailpiecewithinresponse window•Didnotuseunique1-800number•SoattributedtoDMthroughindirect attribution(match-back)

MediaexposuredetailaboveANDCustomerreceivedmailpieceattimepoint6Customerwasdecile2name….….

RespondedtoDM(100%)

LikelyrespondedtoDM(50%)MayhaverespondedtoTV(20%)NeitherTVnorDM(30%,i.e.Brand)

LikelyrespondedtoTV(60%)MayhaverespondedtoDM(10%)NeitherTVnorDM(30%,i.e.Brand)

Qu

ali

ty

Qu

ality

integrated



Guidelines for creating PC Scores The specifics to create the pC scores vary widely depending on the situation, so a description of adetailedprocesstocreatethescoreswillnotbeattemptedhere.Somegeneralguidelinesaredescribedbelow,listedbycategory.

top-down influence (media mix modeling) • Mediamixmodelsshouldbefitatthesmallestgeographiclevelpossible.DMA-levelmodel is ideal.

• Cautionshouldbeusedwithrespecttosmallsamplesizeformodelsatasmallgeographiclevel. BayesianShrinkageorotheradjustmentfactorsshouldbeconsidered.

• Itisimportanttokeepthemediamixmodelsbasedonasrecentdataaspossible.Considerhavinga semi-automatedmodelbuildingprocessifmanymodelsareused(geographicmodels,product models, channel models, etc).

Bottom-up influence (Qualitative data) • TheadjustmentstothePCScoresbasedonqualitativedatamaybeanalyticallydrivenordecided basedonindustryorcompanyexperts.RememberthatcreationofPCScoresispartartandpart science • Theimpactofqualitativedatavariesdependingonthemedia.Knowingthatsomebodyopenedan e-mailandclickedthroughtoyourwebsiteisveryconcreteandsowouldhavemoreimpactthan what commercial ran in a dMa (since we don’t even know if that individual saw the commercial). • Becausethequalitativedatacanchangequickly,tohaveaneasymethodtointegratenewtypesof qualitative data into the models (new creative, for example).

testing and Validation • Becausethereisno‘right’answer,assessingaccuracycanbeachallengingprocess. • Whenpossible,usecontrolledteststovalidateinfluenceofeachmedia.Forexample,adirectmail campaignwitharandomholdoutsamplewillprovidethe‘true’influenceofthatdirectmailpiece since all other factors are controlled for through the randomization.

• Creditabilityisestablishedovertime.Evidencecanbegatheredmultipleways,thebestofwhich beingwhenaninmarkettestdesignproducestheresultsforecastbythemediamixmodels.



summary and Conclusions MarketershaveanunprecedentedopportunitytoincreasetheROIonmarketingprogramsbyimplementinganextgenerationmeasurementsystem.Byutilizingthebestelementsofboththehigh-level,top-downapproachandthedetailed,bottom-upapproach,marketersachieveamuchmoreaccurateviewoftheircustomerspurchasingbehavior.

Thetop-downapproachisusedasthebaselinesinceithasthedesirablepropertiesofaconsistentperformancemetricacrossmediaandnaturallytakesintoaccountcross-mediaimpactandimpactsovertime.Buttheresultofthetop-downapproachismodifiedbasedontheknown,detail-levelinformationgleanedformthebottom-upapproach.Thepowerfulcombinationofthesetwomeasurementsystemsandprobabilisticcontributionscoresprovidesamuchclearerandreliablemarketingmetrics.

our next paper will focus on the data, infrastructure, and software toolset required to operationalize the integrated measurement system. The fourth and final paper will focus on integrating the measurement frameworkintothebusinessdecisionmakingandmarketingexecutionprocesstoensurevalueisdrivenfrom the solution.

How Merkle Can Help Merkleworkswithseveralclientstodeveloptheinfrastructureandanalyticstoenablethequantificationofbrandengagementacrosstheirprospectandcustomerbaseandmakeinformation-baseddecisionsontheirbrandequity.

Merklespecializesininformation-basedmarketingstrategiesandisoneofthenation’sleadingdatabasemarketingfirms.Withaproventrackrecordindevelopingwinningstrategiesbasedoninformationinsightforlargeconsumer-focusedorganizations,Merkleworkswithmanyofthenation’sleadingbusinesses,includingProcter&Gamble,Dell,CapitalOne,GEICO,andDIRECTV.



about the author scott nuernberger is senior director, Quantitative solutions. scott has over eight years of experience in developing and implementing analytical solutions to marketing programs for many different companies,includingGEICO,AEGON,NationwideInsurance,MBNA,Fidelity,andDell.PriortojoiningMerkle, scott worked for american express as a statistician and modeler and taught graduate students statistical methods and experimental design at Cornell university. scott has dual Bs degrees in Brain and Cognitive sciences and statistics from The university of rochester, a Ms degree in statistics from CornellUniversity,andanMBAfromJohnsHopkinsUniversity.