Research Design Research Design

Exploratory and Conclusive Exploratory and Conclusive ResearchResearch

Research Purpose– Exploratory Research

General: To generate insights about a situation

– Conclusive Research Specific: To verify insights and aid in selecting a

course of action

Exploratory ResearchExploratory Research

Develop initial hunches or insightsProvide direction for any further research

neededPurpose:

– To shed light on the nature of the situation and to identify any specific objectives or data needs

Conclusive or Confirmatory Conclusive or Confirmatory ResearchResearch

To verify insights and to aid decision-makers in selecting a specific course of action

More Formal

More Rigorous

Relationship among Research DesignsRelationship among Research Designs

Descriptive ResearchDescriptive Research

Exploratory ResearchExploratory Research

Causal ResearchCausal Research

Selecting the Appropriate Research Type

Is the research purpose specific and

are data requirements clear?

Analyze data/interpret findings

Is there a need for further research

Analyze data/interpret findings

Make recommendations

Conduct a suitable descriptive-research study

Conduct exploratory research with these procedures:

-Key informant technique-Focus group interviews-Secondary-data analysis

-Case study method

Design conclusive research

Does the research purpose call fortesting cause-and-effect

relationships between variables?

Conduct an appropriateexperimental-research study

Yes

No

No

Yes

NoYes

Types of Research DesignsTypes of Research Designs

Exploratory Exploratory ResearchResearch

CausalCausalResearchResearch

DescriptiveDescriptiveResearchResearch

•Formulate problems more preciselyFormulate problems more precisely•Develop HypothesesDevelop Hypotheses•Establish priorities for researchEstablish priorities for research•Eliminate impractical ideasEliminate impractical ideas•Clarify conceptsClarify concepts

•Literature searchLiterature search•Experience surveyExperience survey•Analysis of select casesAnalysis of select cases•Focus groupsFocus groups•InterviewsInterviews•Projective testsProjective tests•EthnographiesEthnographies•Describe segment characteristics Describe segment characteristics

•Estimate proportion of people Estimate proportion of people who behave in a certain way who behave in a certain way•Make specific predictionsMake specific predictions

•Longitudinal studyLongitudinal study•True panelTrue panel•Omnibus panelOmnibus panel•Sample SurveySample Survey•Provide evidence regarding causal Provide evidence regarding causal

relationships by means of: relationships by means of:•Concomitant variationConcomitant variation•Time order in which variables occurTime order in which variables occur•Elimination other explanationsElimination other explanations

•Laboratory experimentLaboratory experiment•Field experimentField experiment

Uses Types

Types of Data for Exploratory Types of Data for Exploratory ResearchResearch

Key informantFocus groupsSecondary dataObservation studiesCase studies

Secondary and Primary DataSecondary and Primary Data

Data collected for a purpose other than the research situation at hand

Data collected specifically for purpose at hand

Secondary Secondary Primary Primary Exs: census,

government, corporate library, scanner data

+’s: quick, cheap, easy, focuses research, benchmark

-’s: doesn’t fit problem, outdated

Exs: focus group, survey, interviews, telemarketing

+’s: tailored to needs, current

-’s: takes time to collect, expensive

***Their strengths and weaknesses are complementary!***Their strengths and weaknesses are complementary! So...any good research project should have both!So...any good research project should have both!

Strength and Weaknesses for the two different Strength and Weaknesses for the two different types of Datatypes of Data

Secondary Data: Small Secondary Data: Small Business ApplicationBusiness Application

Market Research for a small business: You want to start a pool and spa cleaning and repair service

How do you find out about market size and competition?

HOW TO GET STARTED WHEN SEARCHING HOW TO GET STARTED WHEN SEARCHING PUBLISHED SOURCES OF SECONDARY DATAPUBLISHED SOURCES OF SECONDARY DATA

7. Identify authorities in the area and consult them.7. Identify authorities in the area and consult them.

6. Consult the various directory guides.6. Consult the various directory guides.

1.1. Identify what you wish to know and what you already Identify what you wish to know and what you already know about your topic.know about your topic.

2. Develop a list of key terms and names.2. Develop a list of key terms and names.

3.3. Search several of the general guides, directories, and websites Search several of the general guides, directories, and websites for papers and/or reports.for papers and/or reports.

4. Compile the literature you have found. Rework your list of key 4. Compile the literature you have found. Rework your list of key words and authors if necessary.words and authors if necessary.

5. Consult the reference librarian. 5. Consult the reference librarian.

Secondary Data Relevance: Secondary Data Relevance: Measurement UnitsMeasurement Units

Sentinel Corporation produces a line of smoke detectors

U.S. Census of Population and Housing Data can be used to estimate the total residential market potential for their products in different sections of the country

Secondary Data Relevance: Secondary Data Relevance: Measurement Units (Cont’d)Measurement Units (Cont’d)

Sentinel Corporation requires size data expressed in number of rooms per household

U.S. Census of Population and Housing data – Useful to Sentinel Corporation

Digital BabySitterDigital BabySitter

Digital BabySitter.comwebsite

Digital BabySitter (Cont’d)Digital BabySitter (Cont’d)

Specializes in making digital baby monitor devices

Wants to expand beyond the United States– Based on birthrates provided by the United

Nations (www.un.org), the company decided to target China and India

– Obtained information on computer penetration in urban areas and chose urban populations as its target market

Digital BabySitter (Cont’d)Digital BabySitter (Cont’d)

Secondary Data Analysis is not meaningful in China and India because children are either with their extended families or at school

Children are almost never aloneSecondary data is not always relevant!!!Secondary data is not always relevant!!!

Most importantMost important lesson about lesson about Secondary data…Secondary data…

THE SHOCKING TRUTH IS THAT THE SHOCKING TRUTH IS THAT STATISTICS ARE ONLY AS STATISTICS ARE ONLY AS

CREDIBLE AS THE SOURCES THAT CREDIBLE AS THE SOURCES THAT PRODUCE THEM !PRODUCE THEM !

Secondary Data LimitationsSecondary Data Limitations

Accuracy– Who collected the data?– Why was the data collected?– How was the data collected?

Flow Diagram for Conducting a Data SearchFlow Diagram for Conducting a Data Search

Analyze the primary and/or secondary data

Will they be adequate for the data needs?

Are the data sufficiently accurate?

Are the data relevant?

Yes

No

No

No

Yes

Yes

Yes

Identify data needs

Are internal secondary data available?

Are external secondary data available?

No

Analyze the primary and/or secondary data

Are the data sufficiently accurate?

Are the data relevant?

Will they (along with any suitable internal data) be adequate?

No

No

No

Yes

Yes

Yes

Yes

Collect the necessary primary data

No

Analyze the primary and/or secondary data

Internal DataInternal Data

Can often be obtained with less time, effort, and expense than external secondary data

May have relevance to the research being conducted

Examples include:– A firm’s historical record of sales– A public service association’s list of donors– Public opinion polls conducted in the past by a political

candidate’s campaign office

External Data: Government External Data: Government SourcesSources

Collects extensive data about people, firms, markets, and foreign countries; more than any other secondary data source

Data collected is readily available on Internet sites Documents published are in the form of summary

reports based on the raw data collected The raw data is often available for a fee

– Public-Use Microdata files

Syndicated SourcesSyndicated Sources

Syndicated services offered by marketing research firms– Nielsen Retail Index

Fees are required but they are more cost effective than collecting primary data

Focus directly on the needs of decision makers

Updated more frequently than government data

Often allows for customization

Trade AssociationsTrade Associations

Very numerous and diverseMany collect data relevant to and about

their membersAlso collect competitively sensitive data

about members that may not be available to industry outsiders

Competitive Intelligence: Competitive Intelligence: Burger King Corp.Burger King Corp.

Burger King:– Maintains a brand research library and subscribes to

analyst reports that provide a detailed view of competitors' financial and long-term plans

– Gathers syndicated reports that provide sales and cost data and describe the competition's growth plans

– Insights about the restaurant business can be flushed out from interviews with restaurant business leaders, published routinely in these trade journals

Managing Secondary DataManaging Secondary Data

Merely keeping abreast of all the available data without being overwhelmed is a challenge

Effective secondary-data management is necessary in this "information explosion" age

Qualitative ResearchQualitative Research

Qualitative research involves collecting, analyzing, and interpreting data that cannot be meaningfully quantified, that is, summarized in the form of numbers

Qualitative Research (Cont’d)Qualitative Research (Cont’d)

Qualitative Research– Typically involves relatively few respondents

or units– Uses non-structured questioning or observation

techniques– Techniques are most appropriate in situations

calling for exploratory research

On the contrary…On the contrary…

Quantitative Research

– More structure and use larger, more representative respondent samples.

– Logical place is in conclusive research projects

– Calls for very specific data, capable of suggesting a final course of action

– Primary role is to test hunches or hypotheses

Importance of Qualitative Importance of Qualitative ResearchResearch

Works better if the purpose of the research is to understand a problem or to develop “new product concepts”

Can be viewed as generating just the right type of raw material needed to produce a finished product in the form of a relevant quantitative research project

Importance of Qualitative Importance of Qualitative ResearchResearch

More flexible than other types of research More easily capture information crucial to

forming effective strategiesAids in analyzing open-ended questionsRarely but sometimes needed to verify the

accuracy of quantitative research results

Muscular Dystrophy Family Muscular Dystrophy Family Foundation®Foundation®

MDFF wants to start a new program to aid people with serious physical handicaps

They are wondering about what kinds of charitable programs to begin and about what strategies to develop in order to seek

donations from the public – Who should the MDFF talk to?

Officers in well-established public service organizations, such as the United Way, the American Red Cross, and the American Cancer Society.

Focus Group InterviewsFocus Group Interviews

8 to 12 individualsModerator (a well-trained researcher)Informal discussion about research topic

individuals

moderator

discussion

Group CompositionGroup Composition

Generally, focus groups conducted by marketing research practitioners involve between 6 and 12 participants

To be effective, a focus group must be as homogeneous as possible with respect to demographic and socioeconomic characteristics

Moderator TasksModerator Tasks

Guide discussionEnsure key aspects of the topic that are

discussesObserve interactionsRecord dialogue and reactions

Desirable Focus Group Desirable Focus Group Moderator SkillsModerator Skills

Kind but firmPermissiveInvolvedIncomplete understandingEncouragementFlexibilitySensitivity

Conducting Focus GroupsConducting Focus Groups

Focus group sessions typically last 1 1/2 to 2 hours

Focus group interviews are often recorded: audio tapes or videotapes

Advantages of Focus GroupsAdvantages of Focus Groups

Richness of Data Versatility Ability to Study Special Respondents

Disadvantages of Focus Disadvantages of Focus GroupsGroups

Lack of Generalizability– Results cannot be viewed as conclusive

Opportunity for Misuse– Occurs when managers yield to a temptation to

generalize a few key remarks made by participants

Cost– On a cost-per-respondent basis, focus groups are

extremely expensive

Focus Group ApplicationsFocus Group Applications

Understanding ConsumersProduct PlanningAdvertising

Revlon Consumer Products Revlon Consumer Products CorporationCorporation

Revlon conducts focus group interviews of working women between ages 20 and 30 to get them to talk freely about their makeup needs:– Types of cosmetics used– Satisfaction and dissatisfaction with existing

products– Suggestions for improvements

Revlon Consumer Products Revlon Consumer Products Corporation (Cont’d)Corporation (Cont’d)

Focus group interviews likely to generate a range of new‑product ideas appealing to a wide cross-section of working women

No guarantee that ideas for new cosmetic products gathered from specific focus groups (working women 20-30) will appeal to other demographic women groups

Rensselaer Polytechnic Rensselaer Polytechnic Institute Institute

(RPI) in Troy, New York(RPI) in Troy, New York RPI conducted a focus groups to better understand

school’s reputation and profile among influential business leaders

Focus Groups participants commented on– Inconsistent image of the school – Perception of RPI as “that school in Troy!”

"Colorful" and natural customer statements capable of having a significant impact on decision makers can only emerge from a focus group setting

Understanding ConsumersUnderstanding Consumers Pillsbury, a manufacturer of cake mixes, is always

interested in questions such as: – What do consumers like about baking? – What do they dislike? – Why do they bake? – How do they bake? – What words/terms do they use in

describing baking products and their use? Focus groups will be helpful in this situation to get

a feeling for consumers' perceptions, opinions, and behavior concerning cake mixes

3 M3 M Strongly positioned in the home & leisure

business-sponges, scouring pads,scrub sponges Lacked a presence in Wool-Soap-Pads segment Conducted focus groups with consumers around

the country Finding:standard steel wool pads scratched

consumers’ cookware Led to the idea for Scotch-Brite Never Scratch

Soap Pad

Research Techniques -- Research Techniques -- Dual Moderator GroupDual Moderator Group

A focus group interview conducted by two moderators

Useful in focus groups involving discussion of a highly technical topic

Example: A group of electronic engineers discussing new concepts for computer memory chips

Other Qualitative Research Other Qualitative Research TechniquesTechniques

Respondent-Moderator Groups – The real moderator may ask selected

participants to temporarily play the role of moderator to improve group dynamics and productivity

Client-Participant Groups – Client personnel are introduced to respondents

and made part of the group

Impact of Technology on Impact of Technology on Focus GroupsFocus Groups

Electronic Group Interviewing (EGI)Videoconferencing Focus GroupsOnline Focus Groups

Which One to Choose? – In-Depth Which One to Choose? – In-Depth Interview or Focus GroupInterview or Focus Group

Factor In-Depth Interviews Focus GroupsTime Interviews: substantial amount

of time per respondentAnalysis: substantial amountof time for analyzing largevolume of information

Interviews: normally takes 1 ½to 3 hours for the wholegroup.Analysis: group analysis takeslot less time.

Group Dynamics No group interaction. Probingdepends on the Interviewer.No peer influence.

Main strength of focus groups-allows for exchange of ideas.Peer influence may affectresponses.

Topic Sensitivity May be easier to deal withsensitive topics.

Respondents may beembrassed to reveal theirfeelings unless brought out bysome respondents.

Time for the topics In-depth probing of the eachrespondent is possible.

Limited time for eachrespondent.

Geographic constraints Respondents can be drawnfrom geographically-dispersedlocations

Geographic constraints exist asrespondents can be drawn onlyfroma specific location where afocus group facility exists.

Domination of individuals Everybody gets equal time Some individuals tends todominate

Logistics Easier to schedule an interview Recruiting and running severalfocus groups in multiplelocations is cumbersome

Focus Groups and Depth InterviewsFocus Groups and Depth Interviews

Focus Groups

•Group dynamics, expect more creative•Some probing•Relatively inexpensive•Ready industry

Interviews

•Not influenced by others•Max probing, great depth•Expensive•Time consuming•Candid, sensitive topics

•Qualitative•Flexible•Probing•Richness of data•Gets at the “Why” of customers’ behaviors•Generates ideas•Clarifies other project results

Projective Techniques Projective Techniques –– Common FeaturesCommon Features

A fairly ambiguous stimulus is presented to respondents in reacting to or describing the stimulus, the respondents will indirectly reveal their own inner feelings

Word Association TestWord Association Test

A list of words, both relevant and irrelevant, used to understand people’s feelings towards different words– Used to examine the effectiveness of brand

names, new products and services, and key advertising words

Sentence Completion TestSentence Completion Test

Respondents are asked to finish a set of incomplete sentences, often related or neutral to the topic of interest

To uncover feelings about “Buying American”– American automobiles– Restrictions on imports– Every U.S. citizen– Foreign-made products– Unemployment in the U.S

“I think the Nike ads that say, ‘Just do it’ are...”

“I think Nike...”

“When I feel a real need to treat myself, I...” “If it’s lunchtime, and I have a choice

between McDonald’s, Burger King, and Wendy’s, I’ll choose ____ because...”

“My favorite restaurant is ____ because...”

Thematic Apperception TestThematic Apperception Test

Respondents are asked to write a story about one or a series of picture(s) they are shown for a short period of time– Especially useful when dealing with special

groups, such as children

Zaltman’s Metaphor Elicitation Zaltman’s Metaphor Elicitation Technique (ZMET)Technique (ZMET)

ZMET tries to bring to the surface the mental models that drive consumer thinking by analyzing metaphors that consumers might use

DuPont – Lacked an accurate picture of consumer’s true feeling

about pantyhose DuPont used the ZMET

– Zaltman asked women to assemble magazine clips-(collage building)

– Magazine clips stand as metaphor for emotions about pantyhose

ZMETZMET

Deeper probing of the emotions behind the choice of photos revealed true feelings behind the product

The research revealed that pantyhose made women feel sensual and attractive, a difficult thing for a woman to articulate in a focus group or in a survey

Research Realities 3: Customers Drawing Pictures for a Study Sponsored by a Large Florist Retail Chain*

Interviewee #1:Woman, 30-39 years old.

Describe yourself:I am a quiet person, I keep mostly to myself and keep out of trouble that way.

When do you buy flowers?:I love flowers because they beautify the surroundings. They give me a special calm feeling of enjoyment. I give flowers to let someone know you are thinking about them.

Her drawing:

Describe a dream involving flowers:I would send flowers to the ladies at work with a note attached and they would all start crying at one time.

How are flowers and funerals related?:All the funerals I’ve been to have flowers. I guess when you die, you will be in paradise and paradise is full of flowers.

Interviewee #2:Woman, 20-29 years old.

Husband sends her flowers because:I am sweet.

Three words associated to flowers?:I love you.

Compared to other gifts, flowers are:Warmest, most romantic, highest class, sexiest, most fun.

Flowers remind me of:Love!

Describe a dream involving flowers:Someone gets married, they are happy and I am pleased for them and I send flowers to congratulate them. Everyone likes it and they live happily ever after.

How are flowers and funerals related?:I’m opposed to the idea of flowers at funeral. Flowers at funeral are sad. The tradition of sending flowers to a funeral may have come about in order to cover the smell of the body.

Her drawing:

Research Realities 3 continued

Interviewee #3:Man, 40-49 years old.

Describes himself as:…a loving and generous man. He celebrates all occasions with lavish and often costly gifts. He is enthusiastic about giving flowers and says they make him think about happiness and warmth.

He also says:The main reason to send flowers is to apologize for having a fight. They lift the spirits, women would be happy about the flowers.

Describe a dream involving flowers: :I had a dream that I send some flowers and as the person bent down to smell the flowers, a thorn cut her lips. Even though she was bleeding she called to thank me for sending her the lovely roses.

His drawing:

*A million Thank You’s to Sidney Levy for sharing these stories and pictures.

Research Realities 3 continued

Case StudiesCase Studies

In depth examination of a unit of interestPossible units:

– Customer– Website– Store– Salesperson– Firm

Allied Associates CompanyAllied Associates Company National chain of discount stores with 500 retail

outlets across the country Profitability during the past few years has remained

above industry average– Top management needs to identify key elements crucial to

the company’s success and to capitalize on those elements– Question: Where and how should one start looking to

accomplish top management’s objectives ?– CASE STUDY METHOD: Study three best performing

stores and three worst performing stores in depth on numerous dimensions.

ObservationObservation

Human or mechanical observation of what subjects actually do in a particular situation

Record information as events occur or compile evidence of past events

Assesses behavior which can be translated into new products or improvements of current products

Observation (Cont’d)Observation (Cont’d)Rubbermaid routinely sends its employees

to consumers’ homes to observe home-storage practices

Ford has been collecting observational data using video ethnography-”not so much to understand what people do, but to understand who they are and how they live

Discovering New Market Discovering New Market Needs — 3M Shows the WayNeeds — 3M Shows the Way

3M’s Medical-Surgical Market Division- to develop a breakthrough product for the surgical drapes unit

Sales of the product amounted to $100 million annually-but the drapes market was languishing

NEED: To bring in radically new products 3M selected developing countries, where

infectious diseases are a major problem

Discovering New Market Discovering New Market Needs – 3M Shows the Way Needs – 3M Shows the Way

(Cont’d)(Cont’d)Observed how doctors work in the

operating room in different countriesObservational Insight : The hospitals cannot

afford surgical drapes and therefore, require a cheaper and more effective way to prevent infections from starting or spreading that does not depend on antibiotics or surgical

Types of Conclusive Types of Conclusive ResearchResearch

Descriptive– Generates data on the composition and

characteristics of a specific group such as customers, sales people, and market areas

– A descriptive survey data merely suggests causation

Experimental– Generates data to determine causal

relationships

Preference to Buy: Preference to Buy: Gallup Survey Gallup Survey of Chinese Consumersof Chinese Consumers, 1997, 1997

Descriptive ResearchDescriptive Research

Cross-sectional Studies– Data collected at a single period in time from a

cross-sectional sample of the unit of interest that is disbanded after the data collection

Longitudinal Studies– Repeated measurements taken over a long

period of time from a panel group or sample of the unit that is maintained for future measurements

DescriptiveStudies

Cross-Sectional

OmnibusPanel

Longitudinal

SampleSurvey

TruePanel

Classification of Descriptive Studies

Longitudinal StudiesLongitudinal Studies

Repeated measurements taken over a long period of time from a panel group or sample of the unit that is maintained for future measurements– Measurements taken during many different time

periods – Measurements taken from a panel – A PANEL is a sample of units that is maintained for

multiple measurements taken during multiple time periods

Longitudinal Studies (Cont’d)Longitudinal Studies (Cont’d)

Types of Panels– OMNIBUS PANEL: different subjects are

pulled from the panel for each time period Drawbacks: data is not as reliable

– TRUE PANEL: the same subjects are used for the panel every time data is collected

Drawbacks: members sometimes evolve out of the desired study group or are induced by the study to change their practices thus tainting the data

Number of Families in Panel Purchasing Each Brand

Brand Purchase

d

During First Time Period, t1

During Second Time Period, t2

A 200 250

B 300 270

C 350 330

D 150 150

Total 1000 1000

Number of Families in Panel Purchasing Each Brand in Each Period

During Second Time Period t2

During First Time

Period, t1

Bought A

Bought B

Bought C

Bought D

Total

Bought A

175 25 0 0 200

Bought B

0 225 50 25 300

Bought C

0 0 280 70 350

Bought D

75 20 0 55 150

Total 250 270 330 150 1000

Number of Families in Panel Purchasing Each Brand in Each Period

During Second Time Period t2

During First Time

Period, t1

Bought A

Bought B

Bought C

Bought D

Total

Bought A

.875 .125 .000 .000 1.000

Bought B

.000 .750 .167 .083 1.000

Bought C

.000 .000 .800 .200 1.000

Bought D

.500 .133 .000 .367 1.000

Cross-sectional StudiesCross-sectional Studies

Data collected at a single period in time from a cross-sectional sample of the unit of interest that is disbanded after the data collection– Conducted in a single time period– Uses a cross-sectional sample of the unit– Once the study is complete the sample is

disbanded and not used again

ExperimentExperiment

An experiment is a procedure in which a researcher manipulates one (or sometimes more than one) independent or cause variable and collects data on the dependent or effect variable while controlling for other variables that may influence the dependent variable

Experimental ResearchExperimental Research

Manipulates the independent variable or variables before measuring the effect on the dependent variable– The effect of price changes on sales volume of

a particular product can be examined by actually varying the price of the product

The very basis of experimental research lies in the manipulation of independent variables

Sales Productivity ExperimentSales Productivity Experiment

Will an increase in the average number of sales calls per customer from six to eight per year significantly improve sales?

Will decreasing the shelf space allocated to brand X detergent by 25 percent significantly lower its sales?

Randomly sample 100 consumers.

Randomly Assign

50 see package design “A”

50 see package design “B”

Count # your brand purchased in ea group

Example of a Marketing Research Experiment

Conditions For Inferring Conditions For Inferring CausalityCausality

Temporal ordering of variables – X Y not Y X

Evidence of association– X and Y are related ; presence of X presence

of Y; absence of X absence of Y

Control of other causal factors X Y, Z Y

Laboratory vs. Field Laboratory vs. Field ExperimentsExperiments

A laboratory experiment is a research study conducted in a contrived setting in which the effect of all, or nearly all, influential but irrelevant independent variables is kept to a minimum

A field experiment is a research study conducted in a natural setting in which the experimenter manipulates one or more independent variables under conditions controlled as carefully as the situation will permit

Internal ValidityInternal Validity

Internal validity is the extent to which observed results are solely due to the experimental manipulation

Laboratory experiments are generally high on internal validity

Field experiments are generally low on internal validity

External ValidityExternal Validity

External validity is the extent to which observed results are likely to hold beyond the experimental setting

Laboratory experiments are generally low on external validity

Field experiments are generally high on external validity

Types of Extraneous Factors That Can Contaminate Research Results

• History--Specific events external to an experiment, but occurring at the same time, which may affect the criterion or response variable

• Maturation--Processes operating within the test units in an experiment as a function of the passage of time per se

• Testing--Contaminating effect in an experiment due to the fact that the process of experimentation itself affected the observed response

Main testing effect--the impact of a prior observation on a later observation

Interactive testing effect--the condition when a prior measurement affects the test unit’s response to the experimental variable

Types of Extraneous Factors That Can Contaminate Research Results

•Instrument Variation--Any and all changes in the measuring device used in an experiment that might account for differences in two or more measurements

•Statistical Regression--Tendency of extreme cases of a phenomenon to move toward a more central position during the course of an experiment

•Selection Bias--Contaminating influence in an experiment occurring when there is no way of certifying that groups of test units were equivalent at some prior time

•Experimental Mortality--Experimental condition in which test units are lost during the course of an experiment

McDonald's Tests McPizzaMcDonald's Tests McPizza

McDonald's test-marketed McPizza to strengthen the after 4pm adult market– Introduced McPizza with heavy advertising,

emphasizing speedy service for pizza– McPizza received favorable nods in some test

markets and had partial rollout nationally

McDonald's Tests McPizzaMcDonald's Tests McPizza

Pizza Hut, a leading competitor, reacted aggressively to McDonald's move by running a buy-one-get-one-free promotion wherever McPizza was introduced

The sales performance of McPizza did not meet management's expectations

True Experimental DesignsTrue Experimental Designs

The presence of one or more control groups The random assignment of units to various

experimental and control groupsRandom assignment distributes the sample

units chosen for a study to various groups on a strictly objective basis so that the group compositions can be equivalent before an experiment is started

Experimental (Cont’d)Experimental (Cont’d)

Multi-group Design

ControlGroup

All variables stay the same

ExperimentalGroup

Change one variable

Units

Measure the differences.

Frito-Lay Experiments Show Frito-Lay Experiments Show How to Make TV Ads How to Make TV Ads

ProfitableProfitable Frito Lay conducted 23 split-panel experiments in

BEHAVIORSCAN markets to assess the effectiveness of TV advertising on its brands

Brands were classified as big (Doritos) and small (Rold Gold)

Household panel was split into two groups– An experimental group: brand advertising in two

formats- News, Base– A control group: Brand advertising was replaced by

public service advertisements (no advertising group)

Frito-Lay Experiments Show How Frito-Lay Experiments Show How to Make TV Ads Profitable to Make TV Ads Profitable

(Cont’d)(Cont’d) Random assignment of households to advertising

and no-advertising conditions ensured that the effects of promotional activities would not affect the findings

Research showed that TV advertising positively affects sales when accompanied by changes in brand, copy, media strategy, and under low in-store merchandising conditions

One-Group, After-Only DesignOne-Group, After-Only Design

Situation A. A company introduces a new brand of margarine in four test market areas and employs a unique and revolutionary promotional campaign for it

The brand captures at least a 10 percent share in each market within two months after introduction

The company's management concludes that the revolutionary promotional campaign played a major role in the market share achieved by the brand

One-Group, After-Only Design One-Group, After-Only Design (Cont’d)(Cont’d)

Situation B. The president of the United States makes a television speech soliciting public support for legislation favoring prayer in public schools

A telephone survey of those who viewed the presidential speech indicates that 70 percent favor such legislation

The president's speech is therefore considered to have had a significant impact on the U.S. public

One-Group, After-Only Design One-Group, After-Only Design (Cont’d)(Cont’d)

Causal inference from a one-group, after-only design cannot be trusted entirely

Some Popular Standard Test Markets

MidlandTX

Oklahoma City, OK

TulsaOK

WichitaKS

Lexington/Fayette

KY

CharlestonWV

BloomingtonIN

IndianapolisIN

SpringfieldIL

RockfordIL

A Perspective on Various Types of Test Markets

SimulatedTest Market

ControlledTest Market

stop

StandardTest Market

stopstop

promising promisingpromising

not p

rom

isin

g

not p

rom

isin

g

not p

rom

isin

g

NationalRollout