Basics of Experimentation (1)Experimental Design:

Which to Choose and Why?

Why Conduct on Experiment?

• Goal of experimentation:– To determine cause-and-effect relations in nature– “Factor A causes factor B”

• What constitutes a good experiment? – Validity of the results: how true the results are

• Internal Validity– Extent to which one can make causal statements

about the relationship between variables– Experiments aim to maximize their internal validity

Variables• Variable

– An event or behavior that can assume two or more values– Ex: temperature; speed of stimuli presentation; medication

• Independent Variable (IV)– Selected and manipulated by the experimenter – Levels: must have at least 2 levels for comparison

• Ex: Cell phone use: no cell phone, hand held phone, hands free phone

• Dependent Variable (DV)– Response or behavior that is measured by the experimenter– Ex: reaction time to pressing the brake pedal; # of words

recalled; symptoms of anxiety

Experimental Designs• How will participants be assigned to the levels of

the IV? • Between-Subjects Design

– Different sets of participants are assigned to only 1 level of the IV

• Within-Subjects Design– Each participant is assigned to all levels of the IV

Level 1BillJohnSally

Level 2JackSamLisa

Level 3EricPeterSandy

Level 1LarryBobJenny

Level 2LarryBobJenny

Level 3LarryBobJenny

Example• The effects of cell phone use while driving on the

time for the driver to realize the car in front of him/her is stopping.

– Independent variable • cell phone use• 3 levels: no cell phone, hand held phone, hands

free phone

– Dependent variable• Time it takes participants to press the brake when

the car in front of them brakes.

Between-Subjects Design

• Considered “safer”– No chances of contamination from one treatment

condition to the other.– The same participant never gets more than one

treatment.

• Experimenter must check for as few differences as possible to exist between participants across all treatment conditions before testing. – Reduces chances of confounded experiment– Maximizes cause-and-effect relations

Between-Subjects Design

No Cell Phone Condition

Participant 1

Participant 2

Participant 3

Participant 4

Participant 5

Participant 6

Hands Free Phone Condition

Participant 13

Participant 14

Participant 15

Participant 16

Participant 17

Participant 18

Hand Held Phone Condition

Participant 7

Participant 8

Participant 9

Participant 10

Participant 11

Participant 12

Total of 18 participants needed in this experiment

Two ways to increase internal validity

• 1- Matching– Select a variable(s) other than the IV that could mostly likely

affect performance on DV.– Ex: vision, hours of video game playing, reaction time to

pressing a pedal, age– Create triads of participants who are equal on all these

variable.– Randomly assign each member of the triads to a level of IV.

• Problems:– Cannot match participants for everything– Difficult to know which is the most important variable to

match

Two ways to increase internal validity

• 2- Randomization– Each participants has an equal chance of being in

any of the conditions of the experiment.

– Does not guarantee groups will always be equal.

– Preferred method when basis of matching is unsure.

Between-Subjects Design

Advantages

• Simpler to conduct.

• No chances of contamination across treatment conditions.

Disadvantages

• Requires more participants.

• Results may be confounded if groups are not equated by randomization.

• Difficult to determine which factors are important to match participants.

Within-Subjects Design

• All subjects receive all levels of the IV• The performance of each subject is compared

across the different experimental conditions.• Reduces the effects of individual differences

across conditions.

• Carryover Effects– Performance on one condition affects performance on

a follow-up condition(s).– Reduces internal validity of experiment.

Within-Subjects Design

No Cell Phone Condition

Participant 1

Participant 2

Participant 3

Participant 4

Participant 5

Participant 6

Hands Free Phone Condition

Participant 1

Participant 2

Participant 3

Participant 4

Participant 5

Participant 6

Hand Held Phone Condition

Participant 1

Participant 2

Participant 3

Participant 4

Participant 5

Participant 6

Total of 6 participants are needed in this experiment

To reduce carryover effects

Counterbalancing– Technique used to counter the effects of presenting

conditions in a particular order/sequence.

Requirements:– Each condition must the presented to each

participant an equal number of times.– Each condition must occur an equal number of

times in each treatment order. – Each condition must precede and follow each of the

other treatments and equal number of times.

How to know the number of sequences

• How do you know the number of sequences that you need to run to meet these requirements?

• Formula: n! (n factorial)– n = number of conditions; n = 3– 3! = 3 x 2 x 1 = 6 sequences– 5 conditions? 6 conditions?

• As the number of conditions increase, the number of sequences becomes too large.

• Alternative procedure: Balanced Latin Square

• Balanced Latin Square– Each condition precedes and follows every other condition an

equal number of times. – Preferred method of counterbalancing– For even number of conditions

• first row formula = 1, 2, n, 3, n -1, 4, n - 2…

Order of conditions

1st 2nd 3rd 4th

Participant 1 A B D C

Participant 2 B C A D

Participant 3 C D B A

Participant 4 D A C B

Need to run participants in multiples of n conditions, or 4

Order of conditions

1st 2nd 3rd

Participant 1

A B C

Participant 2

B C A

Participant 3

C A B

• Odd number of conditions - uses 2 Latin Squares

• The 2nd Latin square is the mirror image of the 1st.

• Need to run subjects in multiples of 2n, or 2 x3 = 6.

Order of conditions

1st 2nd 3rd

Participant 4

C B A

Participant 5

A C B

Participant 6

B A C

Order of conditions

1st 2nd 3rd

Participant 1

No cell phone

Hand held Hands free

Participant 2

Hand held Hands free No cell phone

Participant 3

Hands free No cell phone

Hand held

Order of conditions

1st 2nd 3rd

Participant 4

Hands free Hand held No cell phone

Participant 5

No cell phone

Hands free Hand held

Participant 6

Hand held No cell phone

Hands free

• Odd number of conditions - uses 2 Latin Squares

Within-Subjects Design

Advantages

• Requires fewer participants.

• Reduces individual differences.

Disadvantages

• Potential for carryover effects if conditions are not counterbalanced.

Which Design Works Best?

• It depends!

• If you have a large sample size and predict IV will have a large effect: – Use a between-subjects design.

• If you have a limited sample size and predict IV will have a small chance of carryover effects:– Use a within-subjects design.

Cross-sectional designs• Study individuals of different ages at the same

time.– Ex: effect of age on cognitive abilities (memory)– Test 7- yr olds, 25 yr olds and 60 yr olds

• Advantage– A wide variety of ages can be studied at the same time– Can collect data quickly, maybe all at once.

• Disadvantage– Individuals are born at different times and raised with

different generations; different educational systems– Cohort effect: era in which individuals are born

affects how they respond in a study.

Longitudinal Design• Same participants are studied repeatedly over time

as they age. – Ex: test the same participants every 3 years: 9 yr old,

12 yr old, 15 yr old…

• Advantage: no cohort effect

• Disadvantage– Attrition rate increases: participants drop out of study– Expensive and time consuming– Participants who remain may not be a representative

sample

Sequential Design

• Combination of longitudinal and cross-sectional designs

• Test 5 yr olds, 15 yr olds and 45 yr olds ever 3 years

• Advantages– Allow to examine age effects without taking as much

times as a longitudinal study.

• Disadvantages– Expensive and time consuming– Attrition rates