My concern about media audience

measurments

Tamara Silina, February 2013

Thanks to Orson Welles for the Citizen Kane few images I borrowed.

A few months ago, I had a quite unclear understanding of what audience

measurement is. After been involved in managing marketing for a new local radio station, I first looked at how decisions were made by sellers and buyers in the

advertising space market. I was rather disappointed by what I found.

To summarize, I would say there are so many errors, bias and dysfunctions in

traditional medias audience measurement that it seems it is very risky to take decisions based on this information.

Thank you to professionals in this field who answered my questions, who shared their knowledge and opinions with me through my blog and who read back my posts in

despite of my ingenuous ignorance.

And sorry for my poor English.

Tamara

Patrick Chapatte, 28 février 2006, Le Temps.

It is not about creating panic, but to

inform… first question: are we all going to die in horrible suffering?

What I heard from media

What I have heard about media strategies

What purpose do audience measurements serve?

New technical solutions

What we shall have to face

Here you are

5

What I have heard about old traditional media strategies I selected three key ideas about what I was told about old media strategic

analysis of old traditional media groups.

The share of the traditional media (radio, TV, print...) in the ad space market

should decrease because of the inevitable decline of their audience.

Ad expenses should be transferred mainly to digital media and other new

areas of marketing investment.

Their strategy would be to extend as much as possible the period during

which traditional media can still be profitable.

To do this, it would be vital for them not to disturb the factors that keeps their

business especially audience measurement systems.

6

What I have heard about new and small traditional media New entrepreneurs in traditional media business make different analysis:

Today, It is much easier and less expensive to create and operate new

traditional media with new production and broadcasting technologies and

new content sources.

New and small media would be more in synergy with Internet social

networks.

Their potential development would be higher because of their ability to deal

with proximity, community, and generation issues through targeted and

niches strategies.

But, as far as their audience is not measured by the current audience

measurement systems, their development is strongly limited.

7

But what is the use of audience measurements?

8

What I understood about the purpose of audience measurment Audience measurement is mainly used in the advertising space market ,to give

a value to , and promote ad space. It also allows content managers to identify

who is listening, watching, reading, seeing what, in order to adjusts their

contents. But it is mainly the first function that determines the configuration of

audience measurement.

For advertisers and advertising agencies, audience measurement allows,

before a campaign, to identify spaces with the best cost / potential impact on

their target. During and after the campaign, it enables to evaluate some

campaigns performances.

9

In few words, how are audience measurement made? Who take decisions? Associations of media, agencies and advertisers decide

which methods and means to implement. They choose, after bidding, research

agencies.

How is it done? For radio and print, telephone or self-administered surveys are

conducted. They may be continuous or by waves. Television audience

measurement use diaries surveys or people meters panels. For outdoor, they use

surveys or GPS tracking.

Some few systems use automatic individual people meters (radio and TV).

Who pays? These surveys and panels are mainly funded by the media who

want their audience to be measured.

10

Lie to me

11

12

Problems?

In trying to understand these processes, I was very surprised by malfunctions

that I did not expect to find in one of the first markets in the world economy.

I discovered that most transactions are based on approximate data and

techniques that would have horrified brokers and traders in the fifties.

13

17 key problems 1. Systems do not match the needs of data users,

2. Measures that are not cross - media,

3. Poor definition of the population whose audience must be measured,

4. Weak segmentation and profiling,

5. Irrelevant size and structure of samples,

6. Wrong survey techniques,

7. Bias in recruiting,

8. Question that induce untrue answers,

9. Irrelevant delay and frequency in data collection and delivery,

10. Irrelevant presentation of delivered data,

11. Delayed access to the data that make it obsolete,

12. Non-compliance with specifications,

13. Lax controls,

14. Sidelining of the real experts,

15. Dishonest data manipulation,

16. Resistance of main media to change,

17. Resistance of media research institutes to change.

1. Audience measurements don’t match the needs of data users

The first audience measurement function should be to help advertising space

buyers and sellers, to setup space "value".

To optimize and fluidify transactions in a market, the required information for

decision makers should be available by streaming and in real-time, through

consistent tools for consultation, analysis, placement of order and post-

evaluation.

In all structured markets, these functions are integrated into homogeneous,

optimized and controlled systems. In the advertising space market, the

data about programs and commercials monitoring, audience and forecasts,

and the tools to exploit them are provided by heterogeneous suppliers without

global vision of the users real needs, with virtually no control. Each of these

suppliers thinks he is doing its best just by coordinating with others. This type of

'organization' produces a low result.

14

What current systems measure, is mainly the exposure of respondents or

observed members of panels, to what media broadcast. But is this concept of

exposure still enough to define the value of the advertising space?

Don’t you think that, for example, involvement, interest, attention,

appreciation, loyalty to programs, would be taken into account by the

advertisers and agencies if that data was available?

Couldn’t we take into account the relationship between the consumers and

the products?

The audience measurements are designed today without a global vision

and without taking into account the real needs of those who determine the

effectiveness of advertising space and who develop the market space.

15

2. No cross media audience measurement

For a potential buyer, the value of an offer advertising space, is related to his

target audience compared to those of all the alternative media and to their

complementarily, for the target coverage.

Media planner's work is not only finding spaces that cost less by contact, but also

identifying space assemblages of complementary media to optimize the

repetition on a specific share of the target. This is possible by spreading messages

through different types of media. Decision makers need to evaluate at the same

time the cost and complementarily of ad spaces of TV channels, radio stations,

newspapers, etc.. This requirement has been explicitly expressed by international

associations of advertisers and, in 2009, by The Coalition for Innovative Media

Measurement (cimmusorg.startlogic.com).

However, each year, new panels and new audience researches are set

up without they taking into account this requirement. It seems that associations

and groups who control audience measurements do not understand the issue.

16

3. Inadequate definition of what is to be measured

More and more media are cross- national. For a radio or a television station

which broadcasts by satellite or on the Internet, the geographic area of its

potential audience is a linguistic territory. For example, Russian-speaking

channels are viewed in fourteen countries and Francophone channels, in forty

countries.

Many advertisers who work in several countries have already internationalized

their branding positioning according to linguistic and cultural territory. Their ad

space purchasing decisions are based on audience in these areas. To do this,

some of them have to conduct their own multi-country measurements.

International media as those who provide both satellite and cable ,have access

to inconsistent and heterogeneous audience measurement data. This is why

their advertising space is under-estimated.

17

The media audience is fragmented, it is more individual and independent. New

frameworks are needed to understand behavior and segmentation. Despite this,

people who are recruited to participate in surveys and panels are briefly

described and profiled. In fact, in most cases, the minimum criteria is used,

especially for television panels.

Yet the media, advertisers and agencies communicate their targeting criteria

and segmentation. They need segmented data according to these criteria for

their decisions. (http://www.wfanet.org/pdf/med_documents/Media_Charter_FR.pdf. Page 5).

Because they do not have such data, media planners are used to make risky

projections and interpolating.

A paradox: despite the fact that respondents are defined through only basic

criteria, it is generally very difficult and expensive to obtain data extractions using

these criteria.

What could drive those who designed these systems? Did they know that

restricting samples qualification and profiling and reducing access to native

data, would directly affect the media space demand?

18

4. Irrelevant segmentation and profiling tools

5. Irrelevant samples size and structure

All parts of a population are not worth the same as advertising targets. We

could therefore expect that the targets who generate most ad expenses,

would be 'covered' by more accurate and more frequent measurements.

But sample surveys and panels are most of the time structured as reduced

models of population, whose audience has to be measured while they should

over-represent segments of the population that are targeted by most

campaigns.

For this reason, a large part of the data is nearly never used.

With this model logic, to get large enough useful sub-samples, it is necessary

that the size of the entire sample is very large, which ultimately would result in

additional - and hardly justifiable- costs.

How is it possible that those who designed these systems, did not ask

themselves what information was really useful? Why were they so anxious to

have very large samples, so unnecessarily representative of the population that

is useless to data users?

19

6. Survey system which does’t match all segments

Those who decide how audience measurement should be always want to use a

monolithic methodology to cover the entire population, all segments, all profiles,

the same way. This is probably in order to be able to compare audience

behavior of different segments. These doesn't match any need of any data user.

It is common sense that no data collection method can match all profiles. Some

are more adequate to some segments and others to other segments.

Those which can match for average profiles are not effective for higher social

categories. For instance, television people meters are not able to measure the

audience of high purchasing power consumers, who, by the way, are subject to

heavy advertising investments…

20

7. Recruitment methods biases

I was lucky to be in touch with some media audience research teams. One of

the things that disturbs them the most, is the very low representativeness of

people who are recruited for these measurements.

One key factor is the low level of acceptance to participate in this kind of

survey or panel. A large part of the samples seems to be very difficult to

recruit. For phone surveys, 90% refuse and 99% do for panels & people meters.

It is very, very far below standards to get representative samples.

Some profiles - especially in the higher social categories – are just impossible to

recruit. This means that some key targets are just not represented in the

samples, even if it was in the scope of work and even if they appear in the

sample description of the research institute.

Contrary to what is sometimes said, no weighing method can correct these

problems.

Unfortunately, samples are widely made of atypical people who cannot

'represent 'real audience

21

8. Questionnaires that produce wrong answers

For radio and print, most audience measurements are using self-administered

questionnaires or are administered by phone or Internet.

For radio, several researches have shown that a large part of people

answering, often don’t know what radio station they are listening to. And a

larger part of them don’t remember what stations they had listened to the day

before . At the same time, the most used information collected by these

surveys is the last day audience.

It is also shown that respondents tend to say the names of stations with a strong

awareness or reputation, rather than to say that they do not remember. This

greatly benefit the most well known radio stations. Some of the respondents

also say the names of the radio stations for which they have sympathy. Similar

phenomena are observed for newspapers and magazines audience. http://www.fordham.edu/images/undergraduate/communications/audience%20measurement,%20diversity,%20and%201a.pdf

The result is that these survey methods induce wrong answers and more often,

media audiences which are better known ,are overvalued at the expense of

others.

22

9. Dysfunctions due to delays and frequency

Radio and print audience measurement surveys are conducted four or five

times a year in most countries. They need four or six week to be conducted and

data are provided with a delay of several weeks. However, most campaigns last

no more than three weeks.

How could - we justify the quality of the space we sell with such delays? Imagine

a buyer who will receive information on the effectiveness of his campaigns

several months after? Some advertisers have to conduct their own audience

survey to be able to adjust their campaigns.

It is sometimes said by some experts about radio and press audience that

audience vary little. But it is not true, as shown by researches using daily surveys

(Arbitron, Radio audience measurement in New York using PPM).

23

10. Irrelevant data access and delivery

Data supplied by research institutes that produce audience measurements

cannot be accessed through query tools that would cross numerous criteria.

However, such software are available.

According to some professionals, it appears that direct access to data would

make it easy to detect inconsistencies, oversized weighing and too low

representation of some profiles in the samples.

24

11. Information are too old when delivered

In an ordinary market place, the more and the faster information is available,

the stronger is the market active. In most markets, everything is done to get real

time data. That's what I thought until I worked for a radio station.

In most radio stations, like the one I worked for, audience measurements are

made using a telephone survey. About a thousand people are interviewed on

everything they heard the day before. One might say that data every week, is

not so bad and it could stimulate the market. But it is not the case. The data is

processed once per quarter and it takes almost six weeks to have access to the

earliest data.

So by the end of January, I have data from November. I cannot use the data

to justify my new year prices. Advertisers cannot make a decision and they do

not buy my space. Why not treat and deliver data in a timely fit?

25

12. Non-compliance with tenders specifications

The impossibility to reach norms, poorly controlled recruitment

specifications and data adjustments, drive institutes to use solutions that

degrade the quality.

Some profiles, for which recruitment is too difficult or too expensive, are

replaced by people out of the quotas: people of higher socio economic class

are replaced by lower class; people in deep rural areas are replaced by near

city countryside people, or suburban individuals, etc…

Some abusive weighing complements abusive recruitment. Subsamples of

difficult to recruit profiles, are over-weighed well beyond what can be

statistically acceptable. Some people may be counted ten times.

These practices are the main reason for the barriers that are erected to access

individual data from surveys and sorting purposes. They would make them

visible.

We should be wary of audience measurement systems which do not give

access to individual data to any user, even for reason of consistency check.

26

13. Lax controls

In modern markets places such as stock exchange, controlling systems are very

stringent. They focus on the design of systems, their operation, their integrity and

they check absence of abuse in the use of data. Each of these controls is

assigned to a separate institution and whose activity in only controlling and

auditing.

For audience measurements, the practices are much less stringent. These controls

are organized through committees in which members participated in the choice

of the institute or are involved in audience measurement operations.

The assessments are rather assigned to individual experts whose methods may

lack rigor and whose independence is not validated. Often, these experts have

worked in audience measurement leading institutes. In addition, these controls

are very infrequent.

Sometimes experts deliver two versions of their report: one for wide distribution,

with very little criticism, and another for the committee, which it is much more

critical. In these cases, this shows who is the final client of audience

measurements.

27

14. Real experts sidelining

Experts who are requested by associations for the design and control of

audience measurement systems have, most of the time, been involved in

operating this kind of survey or panel. They are not scientists, they are

technicians. They just reproduce what they know, that is to say, the old

systems, and to endorse the currents practices.

However, there are researchers who work and publish on these topics. I

found many papers that addressed the problems I have identified. These

researchers are never asked for advising or technical auditing.

Researchers that I could speak with are pessimistic about change in

audience measurement systems because they think the current

organizations that drive them doesn't match the new required systems and they will not ben revolutionary as they should be today.

28

15. Intentional alteration of data

More serious than the dysfunction which are induced by aberrant specifications,

in some places, data are sometimes intentionally altered, that is to say

deliberately changed. This happened at the request of media, content

producers or agencies.

Owners of media conglomerates explained me how they had to pay to be sure

to get good audience.

Some engineers that programmed software for media audience measurement

explained to me how audience of some channels can be routinely boosted by

a discrete weighting.

Moreover, some media planners have detected daily identical ratios for some

time slots per between the main channels, and for long periods.

All these examples are recent.

These practices may exist because the controls are really insufficient.

29

16. Leading medias resistance to change

Groups and committees that lead measures audiences measurements are

often dominated by mainstream media. They manage to impose themselves

because they have an historical leadership and because they pay more. It is

therefore difficult to oppose their views.

The orientations of these mainstream media are naturally more conservative

because they believe the relation between share of spend (ad expenditure)

and share of voice (share of audience) and current audience measurement

systems protect their share of voice dominance.

The subscription amount for a media to be measured is often used by these

mainstream media as a barrier exclude small media groups.

It is clear that if audience measurement were more cheaper, the leadership of

the mainstream media would be weakened. That’s why they are more fond of

heavy methodologies (with large sample) and they denigrate innovations that

could reduce cost.

30

17. Leadings institute resistance to change

Audience measurements are highly profitable and not risky for leading research

agencies.. In this time of crisis for the marketing research market, they are seen

as a very stable part of the business.

A contract for audience measurements generally last several years, it leaves a

margin over 30% and probability of renewal is high. Few industries have both

such a level and a so low risk.

For television audience measurement, there is a technical barrier to entry which

makes virtually impossible to win tender for a challenger institute.

In some cases, major media are participating in the capital of institutes who

measure their audience. This reinforces their conservatism. In other market

places, such a conflict of interest would be prohibited.

Sometimes, media research institutes are not afraid of conflicts of interest may

sell services to the media. "It is as if a manufacturer of thermometers sold some

aspirin." (E. Leser Slate. Http://www.arretsurimages.net/vite.php?id=13605).

Leading institutes will not add pressure to break the status co. They would fight

against new technologies that could reduce the cost of media research and

open the door to new institutes competitors.

31

New technical solutions

New available systems These are the Arbitron PPM, The GfK MediaWatch, Ipsos MediaCell, INVENTIVE

Technologies and maybe others I have not yet identified.

All of them can be used for both television and radio. They are portable and

individual.

Most of them use watermarking technology. They can measure only the

audience of channels which signal include a special code.

Most are not integrated into overall systems of information and analysis for the

preparation of decisions of buyers and sellers of ad space.

33

Arbitron PPM

This Portable People Meter is a small box like a pager. It use watermarking. Some

versions could provide real time. Versions currently used transfer their data once a

day.

The fact that it is a case to carry always with you may be a handicap. "It's a bit like

the electronic bracelet of custody" as described by one panelist.

This is a closed system (hard and software), highly protected by patents. Thus, it

can evolve very slowly, more slowly than technologies on which it relies, including

telecom, Internet and recognition signals. For this reason, it gradually loses its

advance.

It has been many concession contracts and partnerships with major institutions that

have not used a lot.

www.arbitron.com/portable_people_meters/home.htm.

34

GfK MediaWatch

It is also a passive individual mobile people meter, integrated into a watch.

Its main advantage is that it is constantly kept by the panelist.

It does not provided in real time data because it is not connected. The panelist

must daily or weekly connect on a rack to upload data.

A module can integrate the newpapers and magazines audience

measurement (declarative).

It also use watermarking, in spite the fact Gfk-Telecontrol had pushed far enough

research in recognition of media signals without watermarking.

GfK Telecontrol did not take into account that a watch is a very personal object,

which carry the image and the style of the person who holds it. They developed

quiet ugly and very recognizable watch. This bias is a strong handicap for

MediaWatch panel recrutement.

It is also a closed system (hardware and software) and therefore it can evolve

only slowly while the technological environment is changing fast.

www.telecontrol.ch/mediawatch.html

35

Ipsos MediaCell

This is an app for recognition of sound signals with watermarking which can be

incorporated in smart phones. It transmits in almost real time.

This is the device that would be the ideal if the watermarking algorithm had not

capacity as low recognition level in noisy environment (discussion, engine, etc.).

This is not a hardware closed system like the two previously mentioned systems. It

will benefit other phone features such as localization, audience measurement

behavior of mobile Internet. This is one of the systems most likely to change

quickly.

www.ipsos.com/mediact/sites/ipsos.com.mediact/files/Mediacell.pdf

36

INVENTIVE Technologies

This system also uses panelists smart phone and tablets. It is the only system which

doesn't use watermarking.

The recognition technology for TV channels and radio stations can distinguish

their signal from strong ambient noises. It can measure the audience of any

channels and any radio stations without limitation. It measures audience share, it

provides audience data in real time (2-3 minutes). It includes a module for

measuring print audience (declarative).

It is integrated into a larger system with advertising monitoring tools and media

planning.

www.inventive-technologies.com

37

The use of this technology requires each station / channel that wants to measure

its audience incorporates a numeric code in the audio signal.

It does not measure audience of unmarked channels and stations. Audience

share, which is known to be the first indicator used for content audience analysis

cannot be measured.

It is almost never mentioned that the watermarking signal recognition is

malfunctioning for the radio in noisy environments (such as in cars, then that is one

of the contexts in which the hearing of terrestrial radio is still virtually not

challenged by Internet).

The few independent technical tests conducted on traditional television people

meters (which use watermarking) have an average error of 10%. Tests even rarer

on passive people meters for radio rated more than 40% error. (Joe Pilota

www.imediaconnection.com).

The way marketing research institutes do forcing on watermarking is surprising

when you know its low performances. The reason of this preference could be due

to the fact it allows to force the media to pay to be measured.

The individual passive audience measurement can be performed without

watermarking and improved performance.

Digression on watermarking

38

What to do for my radio?

39

What theory says? ”Decision makers prefer alternatives with less uncertainty. Information reduces

uncertainty. So buyers and sellers prefer bids supported by the latest, the most

relevant, the most exhaustive, the most reliable, the most continuous

information and in larger quantities”

C. Shannon (1948) « A Mathematical Theory of Decisions and Communications ».

That’s why advertising space buyers divest traditional media because the

offer is not supported by this relevant information. Audience measurements

and buying and selling space systems them are handicaps for the

development of this market. We'll have to change that.

40

What is said by the most advanced professionnals? The required system to replace those currently operating is clearly outlined by

the Coalition for Innovative Media Measurement and by experts.

It should measured audience behavior by the same source (single source) and

in real-time, by any channel, for television, radio, Internet, print and outdoor. It

should include real time advertising monitoring.

For space buyers, it should be accessed through a new generation decision-

making software, which would allow to automatically update in real time media

planning.

For space sellers, it would allow to update in real time pricing of space

depending on the audience and demand.

It should be built to easily connect its data to advertising effectiveness measuring

systems.

41

My ideal system?

42

The vision of the ideal system, for my radio, would cover a wider market than

today to match my real targets (linguistic, geographical, lifestyle ...) and other

types of media which match my targets and which could be complementary or

competitors.

Measures would be analyzed by any advertising and editorial target.

It would include a mechanism for the space market (purchase and sale),

powered by…

Audience measurements: Cross media, reliable, targeted, continuous, available

in real time, including audience share of all media likely to compete with me

(no watermarking).

Monitoring: advertising, cross-media, real-time and integrated to audience data.

Structured data and detailed information on the broadcasts.

Tools to assist decision making for both space buyers (optimizing media plans

and impact assessment) and space sellers (optimizing prices).

It would be really controlled. These checks would be continuously performed by

true professionals of control.

The whole system would be optimized, versatile, up to date and inexpensive.

How to start? These new systems will probably not be initiated by existing organizations for

the reasons already mentioned (not cross media, resistant to change, etc).

They probably will be initiated by business clubs, cluster committees and in

countries where new professional structures are still open et par trans-national

media.

Once they will begin to operate, even for small panels, they should trigger

questioning, awareness and widespread rapidly.

43

What we shall have to face

44

What we will have to confront Resistance of traditional mainstream media: who do not want to change

audience measurement systems because they benefit them.

The resistance of some institutes: The current audience measurement systems

are a too important a part of their turnover. Top managers wish to continue this

activity, and for this reason, avoid changes they could not master and / or

which would be less profitable.

The inadequacy of existing professional groups: The new solutions may not be

supported by today professional groups as they are currently organized by

media type and by country. The new structure can form around the new

reference unit: The advertising target. For example, one of these systems could

develop for luxury goods market.

45

Incompetence: A small portion of players has a global vision of the situation

and real needs. Some do not have the skills to diagnose and others prefer no

change and "waiting to see more clearly.”

Denial: some decision makers who actually have the information to understand

the strategic dilemma can not admit the threats it involve and wishes to hide

reality. As they did previously decision makers in other sectors (telecom, micro-

computer, vinyl disk, etc.).

The stereotype of the consensus: it is an often heard sentence said by

professionals and which would shudder in other industries: "The current system is

certainly imperfect, but it is consensus." For some players, the need of a

common rule, of a flat consensus is more important than the right strategy, than

quality and efficiency.

46

Preference for ambiguity: Some buyers and sellers prefer ambiguity which

make impossible the evaluation of the quality of their services. They are

aware of the limitations of their current efficiency, and in some cases,

because their practices are not to the benefit of their clients.

The weight of politics, sometimes: It happened in some countries that

provided audience data were skewed for political reasons.

The attempt to impose watermarking: when the need for change will be

binding for media research institutes, they will highlight systems that reinforce

them in the relations with the media, even if they are not technically the best.

.

47

Thanks