24 Communication World • January–February 2013 www.iabc.com/cw

■ M E A S U R I N G W H A T M A T T E R S

The standardfor social mediaIndustrywide initiatives help communicators

assess program value

by Pauline Draper-Watts

SHU

TTER

STO

CK.

CO

M

www.iabc.com/cw Communication World • January–February 2013 25

Good public relations programs arebuilt on a strong foundation of soundresearch and measurement. However,simply conducting research for its own

sake is not enough. Over the years I have seen alltypes of measurement programs, running thegamut from incomplete and insufficient to effec-tive and groundbreaking. In creasingly, organiza-tions are asking for measurement initiatives—forboth traditional and social media—that adhere toindustry standards and exemplify best practices.Indeed, there are some standards in place, and others are being developed to cope with the ever-changing media landscape.In 2010, delegates at the second European

Summit on Measurement, hosted by the Inter -national Association for the Measurement andEvalu ation of Communi cation (AMEC) and theInstitute for Public Relations (IPR), establishedthe Barcelona Dec laration of Measurement Prin -ciples. The seven principles were proposed by five professional organizations (AMEC, IPR, the Inter national Communi cations ConsultancyOrga nization, the Public Relations Society ofAmer ica, and the Global Alliance for Public Rela -tions and Communication Management) withinput from their members, and set some verystraight forward guidelines for measurement:1. Goal setting and measurement are important.2. Measuring the effect on outcomes is preferredto measuring outputs.

3. The effect on business results can and shouldbe measured where possible.

4. Media measurement requires quantity andquality.

5. Advertising value equivalents (AVEs) are notthe value of public relations.

6. Social media can and should be measured.7. Transparency and replicability are paramountto sound measurement.

Certainly there have been many efforts aimedat establishing common language, standards andbest practices for media measurement. For exam-ple, IPR, with support from other associations,created the Traditional Media Measure mentStandards, to be used for print, broadcast andonline content. These address some of the mostcommonly debated data points in traditionalmedia analysis that are core to PR measurement

and evaluation, andset out best prac-tices for how to cal-culate impressions,what to analyze,how to calculatetone or sentiment,and more.But social media

measurement bringsnew questions. Hence the creation, in 2011, of the Social Media Measurement Standards Coali -tion (#SMMStandards), a cross-industry, united-standards development effort targeted towardsocial media (the sixth of the Barcelona Prin -ciples). #SMMStandards is currently made up of11 industry associations (including IABC) andmore than six leading clients, including Dell, Ford Motor Company, Procter & Gamble, SASand Southwest Airlines. #SMMStandards features a four-phase collab-

orative process around six topics: sources andmethods transparency; reach and impressions;engagement; influence and relevance; opinionand advocacy; and impact and values. Theprocess starts with working-group developmentand continues with discussion. Typically, eachworking group includes three people whoseexperience and expertise qualify them for discus-sion of the topic. The working group then sub-mits proposals to the larger group. This leads toan interim standard for open feedback and com-ment from a wider audience, and then to finalapproval. As such, it is a market-driven processwith opportunity for any member, stakeholderor practitioner to provide input.

Sources and methods transparencyIt is not always clear which factors have gone intoan analysis of a media communication effort.Indeed, two people doing the same analysis mightsee different results based on different content ormethodologies. #SMMStandards has produced atable to guide practitioners in citing the sources oftheir analysis. This table should be a part of everyreport so that every reader has a clear understand-ing of the basis for the findings and knows howthe data were collected and from which channels,how key metrics were calculated, and the depth of analysis and methodology applied. (See sample

Sentiment alone

is insufficient to get at

opinion and advocacy,

which require more

detailed analysis but

typically are much

more valuable to

practitioners.

26 Communication World • January–February 2013 www.iabc.com/cw

■ M E A S U R I N G W H A T M A T T E R S

table below.) The table fits in well with the seventhBarcelona Prin ciple: “Trans parency and replicabil-ity are paramount to sound measurement.” Afterall, not all aggregators and analyses are createdequal, and some apply bespoke techniques or algo-rithms rather than put out a transparent process.

Reach and impressionsAccurate data about impressions—the number ofpeople who may have seen an article or heard aradio segment, for example, about a company orservice—can be hard to source, especially globalimpressions. What’s more, people have differentunderstandings of what’s meant by impressions,reach, OTS, audience size and circulation. Forinstance, when calculating online impressions,some people will look at the number of unique

visitors to a web page per month; others will fac-tor in a multiplier or divisor to that number to try to get the most accurate figure. For example,unique monthly visitors is the number of uniquevisitors throughout a month, but an online newsstory that gets a lot of attention on one day willbe buried on another day. Thus the number ofunique monthly visitors is an inflated figure. Ihave seen people apply a divisor of 2, 3, 4, 15 or30 to statistics like this. (The Institute for PublicRelations recommends dividing the uniquemonthly visitors by the number of days in themonth.) I have also seen inflated figures where amultiplier is used, which results in a large butunrealistic number. This variation highlights theneed for some consistency and reinforces thevalue of the transparency table.

If the objective is to

build the company’s

brand, then ROI might

not be appropriate;

if the objective is to

drive sales, then it is

highly relevant.

Timeframe analyzed January 1, 2012–December 31, 2012

Research lead(s) Pauline Draper-Watts, Edelman Berland

Channels analyzed Twitter (partial), Facebook (brand pages only), LinkedIn, YouTube, blogs, forums

Data/content sources Google search, Radian6, Sysomos, BrandWatch, Twitter API, Facebook API, YouTube

Analysis depth ❑ Automated ❑ Manual ❑✓ Hybrid ❑ All content reviewed ❑✓ Rep. sample

Source languages English, French, German and Mandarin only

Search languages Native-language queries: English, French, German, Mandarin

Sentiment coding ❑ Automated ❑ Manual ❑✓ Hybrid ❑✓ Manual sampling: every 50 posts coded

❑✓3-pt. scale ❑ 5-pt. scale ❑ Other scale ❑✓ At entity level ❑ Paragraph/doc level

Spam/bot filtering ❑ Automated ❑ Manual ❑✓ Hybrid ❑✓ Includes news releases ❑ Excludes news releases

Metrics calculation and sources

Reach Daily unique visitors for specific URLs via Comscore (no multipliers)

Engagement Channel-specific metrics direct from channels

Influence N/A

Opinion/advocacy Human reading and coding

Proprietary methods No proprietary methodology

Search parameters See full search string list in appendix of this report

Practitioners tool: #SMMStandards’ Sources and Methods Transparency TableTo address the seventh Barcelona Principle, “Transparency and replicability are paramount to sound measurement," an #SMMStandards workinggroup produced a table (shown here, with examples in red) to guide practitioners in citing their sources and help readers understand the findings.

www.iabc.com/cw Communication World • January–February 2013 27

EngagementEngagement has sparked considerable discussionwithin the #SMMStandards group. At the lastannual meeting (which takes place in Septemberor October), we considered the variety of factorsthat could constitute engagement:● Engagement is defined as some action beyond exposure to a message and implies an interaction.

● Engagement occurs in response to content—that is, when someone engages with you.

● Any measure of engagement must be tied tothe goals and objectives for your program.

● Engagement occurs both off- and online, andboth must be considered if you intend to inte-grate your metrics with other marketing orcommunication efforts.

● Online or social media engagement includessuch actions as likes, +1, shares, votes, com-ments, links, retweets, Facebook’s “Talkingabout you,” etc.

● Engagement types and levels are unique tospecific channels.

● Engagement actions should be counted by thenumber of interactions, the percentage of peo-ple engaged (by day, week or month) and thepercent of engagement per post.

Another parameter to consider is the level ofeffort involved when people engage with yourcompany: Is it a single click on a link? Are they“liking” your content? Are they providing identi-fying information about themselves? Then there’sthe quality of engagement: Does it resonate withor reflect your messages?

Influence and relevanceInfluence occurs when you are persuaded tochange a behavior or an opinion that would other -wise have stayed static. It happens online andoffline, and both should be measured. Influencecan be exerted on a media outlet or a person, andshould be tied to a specific topic, brand or issue.Considerations include reach, relevance to topic,authority/ability to get the target audience tochange, level of engagement around the outlet/individual and frequency of discussion. Consideralso that influencers may change over time. Afriend of mine tells the story of how she scoreshighly on tools like Klout as an online influencer

on relationships/marriage because she is alwaystalking about “engagement.” In reality, she isdivorced and not contemplating marriage buttalks about engagement in her research and mea -surement. Thus, it’s important to validate any peo-ple you are considering as influential on a topic.

Opinion and advocacySometimes opinion and advocacy are measured bytackling sentiment—for example, an onlinereview from a customer saying, “I just bought Xand love it!” would reflect positive sentiment andendorsement/advocacy. However, sentiment aloneis insufficient to get at opinion and advocacy. Forexample, a review that says “X is expensive butworth every penny” is a positive endorsement, butthe word expensive might be construed as negativeby an automated sentiment detection service. Itrequires more detailed analysis to evaluate opinionand advocacy, but typically is much more valuableto practitioners. Looking at the Net PromoterScore (NPS), which is based on online customerrecommendations about products or services, willget closer to opinion and advocacy; overlaying thiswith primary research will enrich the analytics.

Impact and valueImpact and value will look at the effect of commu-nication initiatives—the impact on the businessand the return to the organization. They are bestreflected through objectives and outcomes. It isimportant to note that they are distinct fromreturn on investment, because they may or maynot include a financial component. If the objectiveis to build the company’s brand, then ROI mightnot be appropriate; if the objective is to drive sales,then it is highly relevant. Impact and value measurement tie in well to

Barcelona Principles 2 and 3, which focus on out-comes and business results, considering surveyresearch for awareness, behavior and attitudechange, purchase consideration, and integrationwith other metrics such as marketing mix models.

These initiatives are a starting point for organi-zations seeking to develop robust and affordablemeasurement and evaluation programs, for bothtraditional and social media. Given the breadth ofthese efforts, there should be no excuse for badmeasurement or evaluation. ●

learn moreFor more guidance and news on measurement standards,check out the resources fromthese groups: ● Institute for Public Relations

www.instituteforpr.org/topic/measurement-and-evaluation

● International Association for the Measurement andEvaluation of Communicationhttp://amecorg.com/important-stuff

● #SMMStandardswww.smmstandards.com

about the authorPauline Draper-Watts is execu-tive vice president at EdelmanBerland, a global, insights-drivenmarket research firm, where sheheads the measurement andanalytics practice. She is also thepast chair for the Institute forPublic Relations Commission onMeasurement and Evaluation,and an active member of theInternational Association for theMeasurement and Evaluation of Communication’s NorthAmerican chapter.