chapter nine primary data collection: experimentation and

Primary Data Collection:

Experimentation andTest Marketing

Chapter Nine

Chapter Nine

Chapter Nine ObjectivesChapter Nine Objectives

• Understand the nature of experiments

• Gain insight into the requirements for proving causation

• Learn about the experimental setting

• Examine experimental validity

• Compare types of experimental designs

• Gain insight into test marketing

• Learn the limitations of experimentation in marketing research

Chapter Nine

Independent Variable: Price, Packaging, Distribution, Product, Advertising

Dependent Variable: Sales, Customer Satisfaction

Treatment: Independent variable manipulated to measure its effect

on the dependent variable

Extraneous Factors: Things you do not control, such as the weather

What Is an Experiment?

Chapter Nine

• Determine whether a change in one variable likely caused an observed change in another

Causal relationships must show three things:1. Concomitant variation2. Appropriate time order of occurrence3. Elimination of other possible causal

factors

Demonstrating Causation

Chapter Nine

• A statistical relationship between variables

• Change in an independent variable occurred before an observed change in the dependent variable

• No other independent variables are causing the change

1 . Concomitant Variation:

2. Appropriate Time Order of Occurrence:

3. Elimination of Other Possible Causal Factors:

Demonstrating Causation

Chapter Nine

• Experiments conducted in a controlled setting

• Tests conducted outside the laboratory in an actual environment, such as a marketplace

Experimental Setting

Chapter Nine

The extent to which competing explanations for the experimental results observed can be ruled out.

The extent to which causal relationships measured in an experiment can be generalized to outside persons, settings, and times.

Experimental Validity

Chapter Nine

“X” = Independent Variable:• Exposure of an individual or group to experimental treatment

• Variable is something the researcher can change • Goal is to test if the change in the independent variable will cause a change in the dependent variable

“O” = Dependent Variable:• A variable the researcher cannot change directly • Test to see if changing the independent variable will result in changes to the dependent variable.

• The dependent variable is “dependent” upon what the researcher does with the independent variable

Experimental Notation

Chapter Nine

History:• Intervention, between beginning and end of

experiment, of outside variables that might change the dependent variable

Maturation:• Changes in subjects occurring during the experiment

not related to the experiment but which might affect subjects’ response to the treatment factor

Instrument Variation:• Changes in measurement instruments (e.g.,

interviews or observers) that might affect measurements

Extraneous Variables

Chapter Nine

Selection Bias:• Systematic differences between the test group and the

control group due to a biased selection process

Mortality:• Loss of test units / subjects during the course of an

experiment

Testing Effect:• An effect that is a by-product of the research process

Regression to the Mean:• Tendency of subjects with extreme behaviour to move

toward the average for that behaviour during the course of the experiment

Extraneous Variables

Chapter Nine

The random assignment of subjects to treatment conditions to ensure equal representation of subject characteristics

Holding constant the value or level of extraneous variables throughout the course of an experiment

Use of experimental design to control extraneous causal factors

Adjusting for effects of extraneous variables by adjusting the value or the dependant variable for each treatment condition

Randomization:

Physical Control:

Design Control:

Statistical Control:

Controlling Extraneous Variables

Chapter Nine

A test in which the researcher has control over and manipulates one or more independent variables

The independent variable that is manipulated in an experiment

The effect of the treatment variable on the dependent variable

Experimental Treatment and Effects

Experimental Design:

Treatment Variable:

Experimental Effect:

Chapter Nine

Test Group• Group is exposed to manipulation (change) of independent variable

Control Group• Group in which the independent variable is not changed

• Group is used for comparison

Experimental Effect• Effect of the treatment variable on the dependent variable

Experimental Terms

Chapter Nine

O = The Measurement of the Dependent VariableX = The Manipulation / Change of Independent VariableE = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable

Selected Experimental Designs

Chapter Nine

One-Shot Case Study Design

X O1

Change the independent variable, then measure the change in the dependent variable to see if there was, in fact, a change in the dependent variable that the researcher might conclude resulted from the change in the independent variable



Chapter Nine

One-Group Pre-Test–Post-test Design

O1 X O2

Same as one-shot design except measure the dependent variable before the change in the independent variables. The researcher is establishing a benchmark from which to gauge the change.



Chapter Nine

Before and After with Control Group Design

Experimental Group: O1 X O2

Control Group: O3 O4

A true experimental design that involves random assignment of subjects or test units to experimental and control groups and pre- and post-measurements of both groups.



Chapter Nine

After-Only with Control Group Design

Experimental Group: X O1

Control Group: O2

Subjects in the experiment are randomly assigned to experiment and control groups. No pre-measurements of the dependent variable are taken.

Quasi-Experiments

• Research in which repeated measurement of an effect “interrupts” previous data patterns

• Interrupted time-series design with a control group

Chapter Nine

1. Standard

2. Scanner

3. Controlled

4. Simulated (STM)

Test Markets: Types

Chapter Nine

Direct Costs:• Advertising expenses• Syndicated research• Coupons, sampling, POP materials• Trade allowances

Indirect Costs:• Diversion of activity from existing products• Possible negative impact of test failure• Possible negative trade reactions• Competition becomes aware of new product idea

Test Markets: Costs

Chapter Nine

1. Define the Objective:• What do you hope to learn?• What are the characteristics of the people /

products of interest?

2. Select a Basic Approach:• Simulated, controlled, scanner, or standard

test?

3. Develop Detailed Test Procedures:• Generate a full marketing plan• Determine positioning approach

Six Steps in a Test Market Study

Chapter Nine

4. Select the Test Market:• Market should not be over tested• Should have little media spillover•Demographics should be similar to your target population•Market should be large enough to provide useful results•Distribution and other patterns should be similar to the nation

5. Execute the Plan:• Determine length of time of test• Finalize the parties involved

6. Analyze the Results:Use qualitative and quantitative techniques when possible

Six Steps in a Test Market Study

Chapter Nine

High Cost:• Is the research affordable? • Will the research be beneficial and help solve problems?• Has a cost and benefit analysis been done?

Security Issues:• Particularly critical with field experiments• The competition might be “tipped-off”• Are the data and findings secure?

Implementation Problems:• People who unwittingly get caught in the experiment• Outside factors unnaturally affecting the experiment• Participants who intentionally try to skew the results

Limitations of Experimental Research

Chapter Nine

chapter nine primary data collection: experimentation and

Documents