types of research design – experiments chapter 8 in babbie & mouton (2001) chapter 8 in babbie...
TRANSCRIPT
![Page 1: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/1.jpg)
Types of research design – experiments
Chapter 8 in Babbie & Mouton (2001)
Introduction to all research designsAll research designs have specific
objectives they strive forHave different strengths and
limitationsHave validity considerations
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 2: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/2.jpg)
Validity considerations
When we say that a knowledge claim (or proposition) is valid, we make a JUDGEMENT about the extent to which relevant evidence supports that claim to be true
Is the interpretation of the evidence given the only possible one, or are there other plausible ones?
"Plausible rival hypotheses" = potential alternative explanations/claims
e.g. New York City's "zero tolerance" crime fighting strategy in the 1980s and 1990s - the reverse of the "broken windows" effect
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 3: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/3.jpg)
The logic of causal social research in the controlled experiment
Explanatory rather than descriptive Different from correlational research - one
variable is manipulated (IV) and the effect of that manipulation observed on a second variable (DV)
If … then ….
E.g. "Animals respond aggressively to crowding" (causal) "People with premarital sexual experience have more
stable marriages" (noncausal)
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 4: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/4.jpg)
Three pairs of components:
Independent and dependent variables
Pre-testing and post-testingExperimental and control groups
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 5: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/5.jpg)
Components
VariablesDependent (DV)Independent (IV)
Pre-testing and post-testingO X O
Experimental and control groupsTo off-set the effects of the experiment
itself; to detect effects of the experiment itself
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 6: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/6.jpg)
The generic experimental design:
R O1 X O2
R O3 O4
The IV is an active variable; it is manipulated
The participants who receive one level of the IV are equivalent in all ways to those who receive other levels of the IV
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 7: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/7.jpg)
Sampling
1. Selecting subjects to participate in the researchCareful sampling to ensure that results
can be generalized from sample to population
The relationship found might only exist in the sample; need to ensure that it exists in the population
Probability sampling techniques
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 8: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/8.jpg)
Sampling
2. How the sample is divided into two or more groups is importantto make the groups similar when they
start offrandomization - equal chancematching - similar to quota sampling
proceduresmatch the groups in terms of the most
relevant variables; e.g. age, sex, and race
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 9: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/9.jpg)
Variations on the standard experimental design
One-shot case study
X O
No real comparison
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 10: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/10.jpg)
A famous one-group posttest-only design
Milgram's study on obedienceObedience to authorityThe willingness of subjects to follow E's orders
to give painful electrical shocks to another subject
A real, important issue here: how could "ordinary" citizens, like many Germans during the Nazi period, do these incredibly cruel and brutal things?
If a person is under allegiance to a legitimate authority, under what conditions will the person defy the authority if s/he is asked to carry out actions clearly incompatible with basic moral standards?
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 11: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/11.jpg)
One-group pre-test post-test design
O1 X O2
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 12: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/12.jpg)
Example
We want to find out whether a family literacy programme enhances the cognitive development of preschool-age children.
Find 20 families with a 4-year old child, enrol the family in a high-quality family literacy programme
Administer a pretest to the 20 children - they score a mean of say 50 on the cognitive test
The family participates in the programme for twelve months
Administer a post-test to the 20 children; now they score 75 on the test - a gain of 25
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 13: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/13.jpg)
Two claims/conclusions:
1 The children gained 25 points on average in terms of their cognitive performance
2 the family literacy programme caused the gain in scores
VALIDITY - rival explanations
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 14: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/14.jpg)
Static-group comparison
X O O
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 15: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/15.jpg)
Evaluating research (experiments)
We know the structure of researchWe understand designsWe know the requirements of "good"
researchThen we can evaluate a studyIs it good? Can we believe its
conclusions?Back to plausible rival hypotheses
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 16: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/16.jpg)
Validity in designs
If the design is not valid, then the conclusions drawn are not supported; it is like not doing research at all
Validity of designs come in two parts:Internal validity
can the design sustain the conclusions?External validity
can the conclusions be generalized to the population?
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 17: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/17.jpg)
Internal validity
Each design is only capable of supporting certain types of conclusions e.g. only experiments can support conclusions about
causality
Says nothing about if the results can be applied to the real world (generalization)
Generally, the more controlled the situation, the higher the internal validity
The conclusions drawn from experimental results may not accurately reflect hat has gone on in the experiment itself
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 18: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/18.jpg)
Sources of internal invalidity
These sources often discussed as part of experiments, but can be applied to all designs (e.g. see reactivity)
HistoryHistorical events may occur that will be
confounded with the IVEspecially in field research (compare the
control in a laboratory, e.g. nonsense syllables in memory studies
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 19: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/19.jpg)
Maturation
Changes over time can be caused by a natural learning process
People naturally grow older, tired, bored, over time
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 20: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/20.jpg)
Testing (reactivity)
People realize they are being studied, and respond the way they think is appropriate
The very act of studying something may change it
In qualitative research, the "on stage" effects
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 21: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/21.jpg)
The Hawthorne studies
Improved performance because of the researcher's presence - people became aware that they were in an experiment, or that they were given special treatment
Especially for people who lack social contacts, e.g. residents of nursing homes, chronic mental patients
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 22: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/22.jpg)
Placebo effect
When a person expects a treatment or experience to change her/him, the person changes, even when the "treatment" is know to be inert or ineffective
Medical research"The bedside manner", or the power
of suggestion
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 23: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/23.jpg)
Experimenter expectancy
Pygmalion effect - self-fulfilling prophecies of e.g. teachers' expectancies about student achievement
Experimenters may prejudge their results - experimenter bias
Double blind experiments: Both the researcher and the research
participant are "blind" to the purpose of the study.
They don't know what treatment the participant is getting
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 24: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/24.jpg)
Instrumentation
Instruments with low reliability lead to inaccurate findings/missing phenomena
e.g. human observers become more skilled over time (from pretest to posttest) and so report more accurate scores at later time points
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 25: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/25.jpg)
Statistical regression to the mean
Studying extreme scores can lead to inflated differences, which would not occur in moderate scorers
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 26: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/26.jpg)
Selection biases
Selection subjects for the study, and assigning them to E-group and C-group
Look out for studies using volunteers
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 27: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/27.jpg)
Attrition
Sometimes called experimental (or subject) mortality
If subjects drop out, it creates a bias to those who did not e.g. comparing the effectiveness of family therapy with
discussion groups for treatment of drug addiction addicts with the worst prognosis more likely to drop out
of the discussion group will make it look like family therapy does less well than
discussion groups, because the "worst cases" were still in the family therapy group
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 28: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/28.jpg)
Diffusion or imitation of treatments
When subject can communicate to each other, pass on some information about the treatment (IV)
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 29: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/29.jpg)
Compensation
In real life, people may feel sorry for C-group who does not get "the treatment" - try to give them something extrae.g. compare usual day care for street
children with an enhanced day treatment condition
service providers may very well complain about inequity, and provide some enhanced service to the children receiving usual careSUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 30: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/30.jpg)
Compensatory rivalryC-group may "work harder" to
compete better with the E-group
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 31: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/31.jpg)
Demoralization
Opposite to compensatory rivalryMay feel deprived, and give up
e.g. giving unemployed high school dropouts a second chance at completing matric via a special education programme
if we assign some of them to a control group, who receive "no treatment", they may very well become profoundly demoralized
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 32: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/32.jpg)
External validity
Can the findings of the study be generalized?
Do they speak only of our sample, or of a wider group?
To what populations, settings, treatment variables (IV's), and measurement variables can the finding be generalized?
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 33: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/33.jpg)
External validity
Mainly questions about three aspects: Research participants Independent variables, or manipulations Dependent variables, or outcomes
Says nothing about the truth of the result that we are generalizing
External validity only has meaning once the internal validity of a study has been established
Internal validity is the basic minimum without which an experiment is uninterpretableSUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 34: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/34.jpg)
External validity
Our interest in answering research questions is rarely restricted to the specific situation studied - our interest is in the variables, not the specific details of a piece of research
But studies differ in many ways, even if they study the same variables: operational definitions of the variables subject population studied procedural details observers settings
Generally bigger samples with valid measures lead to better external validity
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 35: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/35.jpg)
Sources of external invalidity
Subject selection - Selecting a sample which does not represent the population well, will prevent generalization
Interaction between the testing situation and the experimental stimulus
When people have been sensitized to the issues by the pre-test
Respond differently to the questionnaires the second time (post-test)
Operationalization
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 36: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/36.jpg)
Operationalization
We take a variable with wide scope and operationalize it in a narrow fashion
Will we find the same results with a different operationalization of the same variable?
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 37: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/37.jpg)
Field experiments
"natural" - e.g. disaster researchStatic-group comparison type Non-equivalent experimental and
control groups
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...
![Page 38: Types of research design – experiments Chapter 8 in Babbie & Mouton (2001) Chapter 8 in Babbie & Mouton (2001) Introduction to all research designs Introduction](https://reader035.vdocuments.net/reader035/viewer/2022062217/56649e2d5503460f94b1d322/html5/thumbnails/38.jpg)
Strengths and weaknesses
StrengthsControlManipulating the IVSorting out extraneous variables
Weaknesses Articifiality - a generalization problemExpenseLimited range of questions
SUMBER: web.uct.ac.za/.../Types%20of%20research%20d...