Topics Appropriate to Experiments

• Projects with limited and well-defined concepts.

• Projects that are exploratory rather than descriptive.

• Studies of small group interaction.

Components of Experiments

• Independent and dependent variables• Pretesting and posttesting• Experimental and control groups– Stimulus and no stimulus

Experimental andControl Groups

• Must be as similar as possible.• Control or Comparison group represents what

the experimental group would have been like had it not been exposed to the stimulus.

Experiment

Two & Half Men

• Short clip making fun of the experimental design in testing drugs

Selecting Subjects

• Probability sampling• Randomization• Matching

Randomization and Matching

• May not know which variables will be relevant for matching process.

• Most statistics used to analyze results assume randomization.

• Randomization only makes sense if you have a large pool of subjects.

Internal Validity

• Refers to the possibility that the conclusions drawn from experimental results may not accurately reflect what went on in the experiment itself.

• Did something other than the experimental stimulus affect the dependent variable?

Sources of Internal Invalidity

• Historical events may occur during the course of the experiment.

• Maturation of the subjects.• Testing and retesting can influence behavior.• Instrumentation

Sources of Internal Invalidity

• Selection biases.• Experimental mortality - subjects drop out of

the study before it's completed.

External Validity

• Refers to the possibility that conclusions drawn from experimental results may not be generalizable to the “real” world.

• Is the experimental setting unrealistic?• Is pre-testing influencing the subjects (i.e.

cueing them in on what the researcher wants)

Limiting External Invalidity

Solomon Four Group Design - Four groups of subjects, assigned randomly:

• Groups 1 and 2 are the control and experimental group.

• Group 3 does not have the pre-test.• Group 4 is only posttested.

Solomon Four-Group Design

–G1:Pretest-stimulus-posttest–G2:Pretest-posttest–G3:Stimulus-posttest–G4:posttest

Solomon Four-Group Design

Example

• Champney and Edleman. 2010. “Assessing Student Learning Outcomes in United States Government Courses”

• Available on WebCampus (not required reading).

Posttest-only Control Group Design

• Includes Groups 3 and 4 of the Solomon design.

• With proper randomization, only these groups are needed to control problems of internal invalidity and the interaction between testing and stimulus.

• Commonly done in the social sciences

Web-based Experiments

• Increasingly, researchers are using the World Wide Web to conduct experiments.

• Because representative samples are not essential in some experiments, researchers use volunteers who respond to invitations online.

Experimental Method

Strengths:• Isolation of the experimental variable over

time.• Experiments can be replicated several times

using different groups of subjects.

Experimental Method

Weaknesses:• Artificiality of laboratory setting.• Social processes that occur in a lab might not

occur in a more natural social setting.

Example of Experiments in Political Science

• Mendelberg, Tali. 1997. Executing Hortons: Racial Crime in the 1988 Presidential Campaign. The Public Opinion Quarterly 61(1, Spring): 134-157.

• Available on WebCampus (not required reading).

Methodology

• Experiment• 77 white students at U of M• Median age was 18• Treatment group shown the ad, control group

was not• OLS regression (not a T-test)

Findings

• Students shown the Horton ad were more likely to have negative views on race and racial policies

• Students shown the Horton ad did not have different views on crime

The End

• Read Levin and Fox Ch. 7– Will go over homework assignment

• Read Matland, Richard E. 1994. “Putting Scandinavian Equality to the Test: An Experimental Evaluation of Gender Stereotyping of Political Candidates in a Sample of Norwegian Voters.” British Journal of Political Science 24, 2: 273-92.

• Available on WebCampus