Proper Questionnaire Design

Our questionnaire is aimed at collecting survey data on what people feel about our advertisements and how that affects sales

In this presentation we would consider if our questionnaire is properly equipped to measure the effect of ads on sales or we need to make additions

We would do this by following the development of thought on What constitutes a proper measure of effective advertisement What questions need to be asked in a survey to measure those

metrices The reformulated questionnaires on slide 13 & 14 summarize all

the suggested alterations Given that it is a pub questionnaire, it is difficult to include all the

necessary questions in their ideal form. However, even minor alterations can have significant effect on ad impact measurement

Recognition

Popular till the 1960s

Basic premiseadvertisement directly affects thought which

affects purchase decision making in turn, all in real

time

ProblemTakes no account of time

elapsed between exposure and purchase

Recall

Started in 1930s and was well established by 1960s

Basic premiseusually there is a time gap between ad exposure and

purchase making competitive ad

recall crucial for the ad to have an effect

Problem:Puts too much emphasis on

attention and recall completely ignoring the

effect of perceptions

Favourability

Started around 1980s, major research focus today

Basic Premiseactive attention and recall are not necessary for ad

effectiveness. Faint perceptions of the ad , even unremembered , are more

powerful determinants of its effect

ProblemMeasurement was an issue till the evolution of proper psychometric techniques

1 2 3

In 1961, in response to Vance Packard’s famous polemic ‘The Hidden Persuaders’, Rosser Reeves declared. ‘There are no hidden persuaders, advertising works openly, in the bare and pitiless sunlight’

For this school, advertising equals persuasion. And persuasion works best when paid attention to and recalled at the time of decision making

Followers of this school believe that an advertisement becomes effective only when it is actively considered during making a purchase decision

Models based on this premise are:

Attention Interest DecisionAIDA

Noticed ReadUnderstoo

dDesired

StarchModel

Action

Action

Survey questions to capture Recognition were along the lines of: Have you noticed this ad?

There were two fundamental problems with this approach:1. It couldn't capture the lagged effect of exposure to an

ad on purchase decision, and more fundamentally,2. It only captured the direct effects of exposure to an

ad on memory and thought, completely ignoring the indirect effect on feelings and emotions

Due to the above problems, recognition based metrics of ad effectiveness went out of use in the 1960s

George Gallup extended the concept of ‘recognition’ to ‘spontaneous recall’ to take care of the decay in memory resulting from the time gap between ad exposure and purchase decision

Gordon Brown brought the concept of recall to its maturity by introducing the concept of competitive recall to measure the memory of a specific ad in the middle of a jungle of ad campaigns by competitive brands

Until recently, brand prompted ad recall was the most important metric in measuring an advertisement’s effectiveness in influencing brand attitudes

Supporters still believed that active customer attention and recall are the only two ways to make an ad effective

Survey question to capture spontaneous recall: Have you seen this brand advertised on television recently?

Survey question to capture competitive recall:After showing a list of competitive brands, the question is asked:Which of these brands have you seen advertised on television recently?

• Evidence that recall based metrics might be underestimating ad effectiveness, emerged from the following analyses: 1. In 389 TV advertising experiments, Leonard Lodish

failed to find any strong relationship between TV commercial recall and sales impact of the copy

2. Numerous studies starting from Krugman (1977) to Heath (2001) established the tangible effect of even unattended and unrecalled ads on sales

• The message was clear, to measure ad effectiveness properly, the degree of ad perception, independent of ad attention and recall, has to be measured

Despite having run for 8 years, Stella Artois’ initial press campaign had only achieved claimed ad awareness of 4% in 1990 compared with 29% for rival Castlemaine. Yet Stella’s quality rating was 45% compared to 19% for Castlemaine.

Advertising can thus create strong brand values without necessarily performing well on memory based evaluative measures

• Advertising does not work by creating positive recall alone, but also by generating positive emotions

• Implicit ad processing has been found to be superior to explicit processing in four aspects through psychological research– It has been shown to be substantially more durable,

capacious and independent to attention compared to explicit memory

– implicit memory also stores concepts instead of facts as done by explicit memory

• To measure implicit processing however, metrics were required which could tap into implicit memory

• Typical metrics used to measure implicit processing include:– Proper Recognition to measure implicit exposure: People

shown the ad (visuals, message, sound everything) and asked whether they had seen the ad

– Favourability generated by implicit memory: Rating the brand on an n point semantic scale ranging from ‘very unfavourable’ to ‘very favourable’

• Note that this recognition is different from the pre recall recognition in 2 ways– The latter is time independent while the former was in

real time. – The former asks about conscious exposure whereas the

latter can accommodate any kind of exposure to the ad

Currently our questionnaire has ad exposure questions which measure only ‘recognition’ and ‘recall’ thus underestimating ad effectiveness

We should include questions measuring ‘proper recall’ and ‘favourability’ as well otherwise we might end up underestimating the effect of the ads

• Reeves R (1961) Reality in Advertising. Knopf, New York• Strong, E.K. (1925) Theories of Selling. Journal of Applied

Psychology, 9 (February) 75-86• Starch, D. Principles of Advertising, A.W. Shaw Company,

Chicago• Du Plessis, E (1994) Recognition vs Recall, Journal of

Advertising Research, May/June, p75-91• Brown, G (1986) Modelling Advertising Awareness. The

Statistician 35, p289-299• Lodish, L.M. & Abraham, M. (1995) How TV Advertising

works: a meta-analysis of 389 split world cable TV advertising experiments, Journal of Marketing Research, Vol. 32, Issue 2

• Krugman, H.E. (1977) Memory without recall, exposure without perception. Journal of Advertising Research 17, 7-12

• Heath, R.G. (2000) Low Involvement Processing - a new model of brand and advertising. International Journal of Advertising Vol. 19 No. 3 p287-298