Primary Data Collection:
Experimentation andTest Marketing
Chapter Nine
Chapter Nine
Chapter Nine ObjectivesChapter Nine Objectives
• Understand the nature of experiments
• Gain insight into the requirements for proving causation
• Learn about the experimental setting
• Examine experimental validity
• Compare types of experimental designs
• Gain insight into test marketing
• Learn the limitations of experimentation in marketing research
Chapter Nine
Independent Variable: Price, Packaging, Distribution, Product, Advertising
Dependent Variable: Sales, Customer Satisfaction
Treatment: Independent variable manipulated to measure its effect
on the dependent variable
Extraneous Factors: Things you do not control, such as the weather
What Is an Experiment?
Chapter Nine
• Determine whether a change in one variable likely caused an observed change in another
Causal relationships must show three things:1. Concomitant variation2. Appropriate time order of occurrence3. Elimination of other possible causal
factors
Demonstrating Causation
Chapter Nine
• A statistical relationship between variables
• Change in an independent variable occurred before an observed change in the dependent variable
• No other independent variables are causing the change
1 . Concomitant Variation:
2. Appropriate Time Order of Occurrence:
3. Elimination of Other Possible Causal Factors:
Demonstrating Causation
Chapter Nine
• Experiments conducted in a controlled setting
• Tests conducted outside the laboratory in an actual environment, such as a marketplace
Experimental Setting
Chapter Nine
The extent to which competing explanations for the experimental results observed can be ruled out.
The extent to which causal relationships measured in an experiment can be generalized to outside persons, settings, and times.
Experimental Validity
Chapter Nine
“X” = Independent Variable:• Exposure of an individual or group to experimental treatment
• Variable is something the researcher can change • Goal is to test if the change in the independent variable will cause a change in the dependent variable
“O” = Dependent Variable:• A variable the researcher cannot change directly • Test to see if changing the independent variable will result in changes to the dependent variable.
• The dependent variable is “dependent” upon what the researcher does with the independent variable
Experimental Notation
Chapter Nine
History:• Intervention, between beginning and end of
experiment, of outside variables that might change the dependent variable
Maturation:• Changes in subjects occurring during the experiment
not related to the experiment but which might affect subjects’ response to the treatment factor
Instrument Variation:• Changes in measurement instruments (e.g.,
interviews or observers) that might affect measurements
Extraneous Variables
Chapter Nine
Selection Bias:• Systematic differences between the test group and the
control group due to a biased selection process
Mortality:• Loss of test units / subjects during the course of an
experiment
Testing Effect:• An effect that is a by-product of the research process
Regression to the Mean:• Tendency of subjects with extreme behaviour to move
toward the average for that behaviour during the course of the experiment
Extraneous Variables
Chapter Nine
The random assignment of subjects to treatment conditions to ensure equal representation of subject characteristics
Holding constant the value or level of extraneous variables throughout the course of an experiment
Use of experimental design to control extraneous causal factors
Adjusting for effects of extraneous variables by adjusting the value or the dependant variable for each treatment condition
Randomization:
Physical Control:
Design Control:
Statistical Control:
Controlling Extraneous Variables
Chapter Nine
A test in which the researcher has control over and manipulates one or more independent variables
The independent variable that is manipulated in an experiment
The effect of the treatment variable on the dependent variable
Experimental Treatment and Effects
Experimental Design:
Treatment Variable:
Experimental Effect:
Chapter Nine
Test Group• Group is exposed to manipulation (change) of independent variable
Control Group• Group in which the independent variable is not changed
• Group is used for comparison
Experimental Effect• Effect of the treatment variable on the dependent variable
Experimental Terms
Chapter Nine
O = The Measurement of the Dependent VariableX = The Manipulation / Change of Independent VariableE = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
Selected Experimental Designs
Chapter Nine
One-Shot Case Study Design
X O1
Change the independent variable, then measure the change in the dependent variable to see if there was, in fact, a change in the dependent variable that the researcher might conclude resulted from the change in the independent variable
O = The Measurement of the Dependent VariableX = The Manipulation / Change of Independent VariableE = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
Selected Experimental Designs
Chapter Nine
One-Group Pre-Test–Post-test Design
O1 X O2
Same as one-shot design except measure the dependent variable before the change in the independent variables. The researcher is establishing a benchmark from which to gauge the change.
O = The Measurement of the Dependent VariableX = The Manipulation / Change of Independent VariableE = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
Selected Experimental Designs
Chapter Nine
Before and After with Control Group Design
Experimental Group: O1 X O2
Control Group: O3 O4
A true experimental design that involves random assignment of subjects or test units to experimental and control groups and pre- and post-measurements of both groups.
O = The Measurement of the Dependent VariableX = The Manipulation / Change of Independent VariableE = Experimental Effect: Change in Dependent Variable due to Change in the Independent Variable
Selected Experimental Designs
Chapter Nine
After-Only with Control Group Design
Experimental Group: X O1
Control Group: O2
Subjects in the experiment are randomly assigned to experiment and control groups. No pre-measurements of the dependent variable are taken.
Quasi-Experiments
• Research in which repeated measurement of an effect “interrupts” previous data patterns
• Interrupted time-series design with a control group
Chapter Nine
1. Standard
2. Scanner
3. Controlled
4. Simulated (STM)
Test Markets: Types
Chapter Nine
Direct Costs:• Advertising expenses• Syndicated research• Coupons, sampling, POP materials• Trade allowances
Indirect Costs:• Diversion of activity from existing products• Possible negative impact of test failure• Possible negative trade reactions• Competition becomes aware of new product idea
Test Markets: Costs
Chapter Nine
1. Define the Objective:• What do you hope to learn?• What are the characteristics of the people /
products of interest?
2. Select a Basic Approach:• Simulated, controlled, scanner, or standard
test?
3. Develop Detailed Test Procedures:• Generate a full marketing plan• Determine positioning approach
Six Steps in a Test Market Study
Chapter Nine
4. Select the Test Market:• Market should not be over tested• Should have little media spillover•Demographics should be similar to your target population•Market should be large enough to provide useful results•Distribution and other patterns should be similar to the nation
5. Execute the Plan:• Determine length of time of test• Finalize the parties involved
6. Analyze the Results:Use qualitative and quantitative techniques when possible
Six Steps in a Test Market Study
Chapter Nine
High Cost:• Is the research affordable? • Will the research be beneficial and help solve problems?• Has a cost and benefit analysis been done?
Security Issues:• Particularly critical with field experiments• The competition might be “tipped-off”• Are the data and findings secure?
Implementation Problems:• People who unwittingly get caught in the experiment• Outside factors unnaturally affecting the experiment• Participants who intentionally try to skew the results
Limitations of Experimental Research
Chapter Nine