experimental design playing with variables the nature of experiments allow the investigator to...
Post on 22-Dec-2015
222 views
TRANSCRIPT
![Page 1: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/1.jpg)
Experimental Design
Playing with variables
![Page 2: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/2.jpg)
The nature of experiments
allow the investigator to control the research situation so that causal relationships among variables may be evaluated
One variable is manipulated and its effect upon another variable is measured, while other variables are held constant
![Page 3: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/3.jpg)
So… you’ve decided to do an experiment
Decisions… decisions… decisions
![Page 4: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/4.jpg)
Decision 1: Independent Variable?
value is changed or altered independently of other variables
hypothesized to be the causal influence categorical or continuous (?)
Experimental Treatments: alternative manipulations of the
Independent Variable
![Page 5: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/5.jpg)
Experimental and Control Groups Control Group
Experimental Groups
there can be more than one treatment level of the Independent Variable (basic or factorial)
there can be more than one IV
0
5
10
15
20
25
Control Exp 3
IVtreatment
Experimental Groups
![Page 6: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/6.jpg)
Decision 2: Dependent Variable
The criterion or standard by which the results are judged
It is presumed that changes in the Dependent Variable are the result of changes in one or more Independent Variable
the choice of Dependent Variable determines the type of answer that is given to the research question
![Page 7: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/7.jpg)
Decision 3: Test units/unit of analysis The subjects or entities whose
responses to the experimental treatment are being measured
People are the most common test unit in business research
![Page 8: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/8.jpg)
Decision 4: Extraneous variables
A number of extraneous or “other” variables may affect the dependent variable and distort the results
Conditions of constancy: When extraneous variables cannot be
eliminated we strive to hold Extraneous Variables constant for all subjects
![Page 9: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/9.jpg)
But, what about ___________?
Problems… problems…
![Page 10: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/10.jpg)
IMPACT OF THE RESEARCH SITUATION
Demand Characteristics: experimental design procedures that unintentionally hint to subjects about the experimenter’s hypothesis
rumour instructions status and personality of researcher unintentional cues from experimenter experimental procedure itself Setting: Field versus Laboratory
![Page 11: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/11.jpg)
Field versus Laboratory Field experiments: usually used to
fine-tune strategy and determine sales volume
Laboratory: used when control over the experimental setting is more important
![Page 12: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/12.jpg)
Experimental Design effects….
![Page 13: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/13.jpg)
The Hawthorne effectSubjects perform differently just because they know they are are experimental subjects
Western Electric’s Hawthorne Plant 1939 study of light intensity
The Guinea Pig effectexhibit the behaviour that they think is expected
Potential Solutions:
run experiment for a longer period
use a control group
Deception (?)
![Page 14: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/14.jpg)
Experimental Treatment Diffusion
if treatment condition perceived as very desirable relative to the control condition, members of the control group may seek access to the treatment condition
Potential Solutions:-have control group in another site
-of course, this introduces new variables!
![Page 15: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/15.jpg)
John Henry Effect
legend of black railway worker control group overcompensates
Potential Solutions: don’t do threatening experiments don’t set up obviously competitive situations don’t tell control group that they are control group
• conduct in another location somewhere else
• unfortunately, produces new variable of different location, neighbourhood, etc.!
![Page 16: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/16.jpg)
Resentful Demoralization of Control Group Control group artificially demoralized if perceives
experimental group receiving desirable treatment being withheld from it
Potential Solutions? what about giving control group some perk to compensate? don’t tell them they are control group! (but what about
informed consent?)… Use of Placebo… use of blinding…
![Page 17: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/17.jpg)
Getting control….
Design decisions
![Page 18: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/18.jpg)
Physical Control– Holding the value or level of extraneous
variables constant throughout the course of an experiment.
Statistical Control– Adjusting for the effects of confounding
variables by statistically adjusting the value of the dependent variable for each treatment conditions.
Design Control– Use of the experimental design to control
extraneous causal factors.
![Page 19: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/19.jpg)
• Blinding is utilized to control subjects knowledge of whether or not they have been given a particular experimental treatment
• double-blind experiment
• secrecy • but then violate principle of informed consent
• screen out or balance number of placebo reactors in treatment & control groups
Blinding
![Page 20: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/20.jpg)
Sampling
Who and HowAnd How to Screw It up
![Page 21: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/21.jpg)
Terms Sample Population (universe) Population element census
![Page 22: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/22.jpg)
Why use a sample? Cost Speed Sufficiently accurate (decreasing
precision but maintaining accuracy) More accurate than a census (?) Destruction of test units
![Page 23: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/23.jpg)
Stages in the Selection of a Sample
Step 1: Define thethe target population
Step 2: SelectThe Sampling
Frame
Step 3: ProbabilityOR Non-probability?
Step 4: PlanSelection of
samplingunits
Step 5: DetermineSample Size
Step 6: SelectSampling units
Step 7: ConductFieldwork
![Page 24: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/24.jpg)
Step 1: Target Population The specific, complete group
relevant to the research project Who really has the information/data
you need How do you define your target
population
![Page 25: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/25.jpg)
Bases for defining the population of interest include: • Geography• Demographics• Use• Awareness
![Page 26: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/26.jpg)
Operational Definition
A definition that gives meaning to a concept by specifying the activities necessary to measure it. “The population of interest is defined as
all women in the City of Lethbridge who hold the most senior position in their organization.”
What variables need further definition?
![Page 27: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/27.jpg)
Step 2: Sampling Frame
The list of elements from which a sample may be drawn. Also known as: working population. Examples?
![Page 28: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/28.jpg)
Sampling Frame Error:
error that occurs when certain sample elements are not listed or available and are not represented in the sampling frame.
![Page 29: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/29.jpg)
Sampling Units:
A single element or group of elements subject to selection in the sample. Primary sampling unit Secondary sampling unit
![Page 30: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/30.jpg)
Error: Less than perfectly.representative samples.
Random sampling error. Difference between the result of a sample and
the result of a census conducted using identical procedures; a statistical fluctuation that occurs because of chance variation in the selection of the sample.
![Page 31: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/31.jpg)
…Error
Systematic or non-sampling error. Results from some imperfect aspect of
the research design that causes response error or from a mistake in the execution of the research
Examples: Sample bias, mistakes in recording responses, non-responses, mortality etc,.
![Page 32: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/32.jpg)
…Error
Non-response error. The statistical difference between a
survey that includes only those who responded and a survey that also includes those that failed to respond.
![Page 33: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/33.jpg)
Step 3: Choice! Probability Sample:
A sampling technique in which every member of the population will have a known, nonzero probability of being selected
![Page 34: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/34.jpg)
Step 3: Choice! Non-Probability Sample:
Units of the sample are chosen on the basis of personal judgment or convenience
There are no statistical techniques for measuring random sampling error in a non-probability sample. Therefore, generalizability is never statistically appropriate.
![Page 35: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/35.jpg)
Classification of Sampling Methods
SamplingMethods
ProbabilitySamples
SimpleRandom
Cluster
Systematic Stratified
Non-probability
QuotaJudgment
Convenience Snowball
![Page 36: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/36.jpg)
Probability Sampling Methods
Simple Random Sampling the purest form of probability sampling. Assures each element in the population
has an equal chance of being included in the sample
Random number generators
Probability of Selection = Sample Size
Population Size
![Page 37: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/37.jpg)
Advantages minimal knowledge of population needed External validity high; internal validity
high; statistical estimation of error Easy to analyze data
Disadvantages High cost; low frequency of use Requires sampling frame Does not use researchers’ expertise Larger risk of random error than stratified
![Page 38: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/38.jpg)
Systematic Sampling An initial starting point is selected by a
random process, and then every nth number on the list is selected
n=sampling intervalThe number of population elements
between the units selected for the sample
Error: periodicity- the original list has a systematic pattern
?? Is the list of elements randomized??
![Page 39: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/39.jpg)
Advantages Moderate cost; moderate usage External validity high; internal validity
high; statistical estimation of error Simple to draw sample; easy to verify
Disadvantages Periodic ordering Requires sampling frame
![Page 40: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/40.jpg)
Stratified Sampling Sub-samples are randomly drawn from
samples within different strata that are more or less equal on some characteristic
Why? Can reduce random error
More accurately reflect the population by more proportional representation
![Page 41: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/41.jpg)
How?1.Identify variable(s) as an efficient basis
for stratification. Must be known to be related to dependent variable. Usually a categorical variable
2.Complete list of population elements must be obtained
3.Use randomization to take a simple random sample from each stratum
![Page 42: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/42.jpg)
Types of Stratified SamplesProportional Stratified Sample:
The number of sampling units drawn from each stratum is in proportion to the relative population size of that stratum
Disproportional Stratified Sample: The number of sampling units drawn
from each stratum is allocated according to analytical considerations e.g. as variability increases sample size of stratum should increase
![Page 43: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/43.jpg)
Types of Stratified Samples…Optimal allocation stratified sample:
The number of sampling units drawn from each stratum is determined on the basis of both size and variation.
Calculated statistically
![Page 44: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/44.jpg)
Advantages Assures representation of all groups in
sample population needed Characteristics of each stratum can be
estimated and comparisons made Reduces variability from systematic
Disadvantages Requires accurate information on
proportions of each stratum Stratified lists costly to prepare
![Page 45: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/45.jpg)
Cluster Sampling The primary sampling unit is not the
individual element, but a large cluster of elements. Either the cluster is randomly selected or the elements within are randomly selected
Why? Frequently used when no list of population available or because of cost
Ask: is the cluster as heterogeneous as the population? Can we assume it is representative?
![Page 46: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/46.jpg)
Cluster Sampling example You are asked to create a sample of all
Management students who are working in Lethbridge during the summer term
There is no such list available Using stratified sampling, compile a list of
businesses in Lethbridge to identify clusters
Individual workers within these clusters are selected to take part in study
![Page 47: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/47.jpg)
Types of Cluster SamplesArea sample:
Primary sampling unit is a geographical area
Multistage area sample: Involves a combination of two or more
types of probability sampling techniques. Typically, progressively smaller geographical areas are randomly selected in a series of steps
![Page 48: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/48.jpg)
Advantages Low cost/high frequency of use Requires list of all clusters, but only of
individuals within chosen clusters Can estimate characteristics of both cluster and
population For multistage, has strengths of used methods
Disadvantages Larger error for comparable size than other
probability methods Multistage very expensive and validity depends
on other methods used
![Page 49: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/49.jpg)
Classification of Sampling Methods
SamplingMethods
ProbabilitySamples
SimpleRandom
Cluster
Systematic Stratified
Non-probability
QuotaJudgment
Convenience Snowball
![Page 50: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/50.jpg)
Non-Probability Sampling Methods
Convenience Sample The sampling procedure used to obtain
those units or people most conveniently available
Why: speed and cost External validity? Internal validity Is it ever justified?
![Page 51: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/51.jpg)
Advantages Very low cost Extensively used/understood No need for list of population elements
Disadvantages Variability and bias cannot be measured
or controlled Projecting data beyond sample not
justified.
![Page 52: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/52.jpg)
Judgment or Purposive Sample The sampling procedure in which an
experienced research selects the sample based on some appropriate characteristic of sample members… to serve a purpose
![Page 53: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/53.jpg)
Advantages Moderate cost Commonly used/understood Sample will meet a specific objective
Disadvantages Bias! Projecting data beyond sample not
justified.
![Page 54: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/54.jpg)
Quota Sample The sampling procedure that ensure that
a certain characteristic of a population sample will be represented to the exact extent that the investigator desires
![Page 55: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/55.jpg)
Advantages moderate cost Very extensively used/understood No need for list of population elements Introduces some elements of stratification
Disadvantages Variability and bias cannot be measured
or controlled (classification of subjects0 Projecting data beyond sample not
justified.
![Page 56: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/56.jpg)
Snowball sampling The sampling procedure in which the
initial respondents are chosen by probability methods, and then additional respondents are obtained by information provided by the initial respondents
![Page 57: Experimental Design Playing with variables The nature of experiments allow the investigator to control the research situation so that causal relationships](https://reader035.vdocuments.net/reader035/viewer/2022062516/56649d7b5503460f94a5f4b1/html5/thumbnails/57.jpg)
Advantages low cost Useful in specific circumstances Useful for locating rare populations
Disadvantages Bias because sampling units not
independent Projecting data beyond sample not
justified.