Design

Experimental Control

Experimental control allows causal inference (IV caused observed change in DV)

Experiment has internal validity when it fulfills 3 conditions for causal inference

1) covariation

2) time-order relationship

3) elimination of plausible alternatives

• Controlling extraneous variables

• 1) elimination

• 2) holding conditions constant

• 3) randomization/balancing

• 4) counterbalance

Specify variables to be controlled

If possible eliminate the extraneous variable

Eg noise

a) As a confound; group A measured during high traffic Group B low traffic noises

b) Nuisance variable (may not be a confound). Random noises from heating system.

1) Elimination

2) Hold conditions constant

Minimize variability

• Time of day

• Lighting

• Instructions

• Stimuli

• Procedure….

Loftus and Burns ( 1982)

• Two groups both saw a film of a bank robbery. Only the ending differed.

• Group A violent ending• Group B nonviolent• Both groups asked questions about events that

happened prior to end scenes• Eg the number on a t-shirt worn by a bystander• Correct recall group A 4% Group B 28%• Same film, same instructions, same questions, same

room… • Did not control same temperature or weather…• Only factors thought to impact DV

3) Randomization/Balance

• Especially useful if unsure what extraneous variables may be operating

Between Subjects Design

only choice if• a) subject variable eg smoker and non-

smoker• b) if manipulation of IV makes repeats

impossible or undesirable (deception or carryover effects)

the number of groups = the number of levels of IV

disadvantages:• many subjects needed• individual variation and selection effects

statistical tests • compare variability between groups to variability

within groupssources of variability are • a) the IV• b) confounds –systematic• c) error – unsystematic (individual variability)

Design problems

• The equivalent group

Equivalent Groups

• - try to compensate for selection effect

• - groups are equal to each other in important ways

• - the number of groups = the number of levels of IV

Random Assignment

a)Every participant has equal chance of being in each group, the individual variation is spread through the groups evenly

this works well with big N

b) Block Randomizationuse random number table to assign orderif have 5 groups then use numbers 1-5 list the numbers in the order they appear – must finish sequence

before repeating a number

c) Matchingif small N then a few individuals assigned by chance can have a

big impacttest participants on a variable and pair scores – each group gets

similar scores• -you need a priori reason to match on a variable• -it adds logistical complexity• -may give away hypothesis ( bias and reactivity problem)

Example

weights

156 167 183 170 145

143 152 145 181 162

175 159 169 174 161

order

143 145 145 152 156 159 161 162 167 169 170 174 175 181 183

MatchingGroup 1 Group 2 Group 3

145 143 145

152 159 156

161 167 162

169 170 174

181 175 183

161.8 162.8 164

Block randomization

Group 1 Group 2 Group 3

143 156 175

167 159 152

183 169 145

170 181 174

161 162 145

164.8 165.4 158.2

156 167 183 170 145143 152 145 181 162175 159 169 174 161

2 1 1 1 31 3 3 3 23 2 2 2 1

Balancing

• Cannot control characteristics of participants.

• Try to evenly spread the individual differences between the levels of IV

• Random assignment

• Eg if in the Loftus and Burns study groups differed in attention or memory then problem

Between SubjectsDesign problems

• The equivalent group

• Solution – randomize or balance

Within Subjects Design (repeated measures)

• Each participant exposed to each level of the IV

• Fewer people needed (economical)

• Individual variability removed as source of error (more power in testing)

Great for rare events/species/diseases

BUT sequence or order effects can be problematic

Progressive effects

Practice improves performance

Fatigue worsens performance

Carryover effects

Doing task A has bigger impact on task B than the reverse

Uneven impact

Within SubjectsDesign problem

• Sequence effects

4) Counterbalance

a) complete counterbalancing – use all possible sequences of orders at least once

good if few conditions (3 or less) (n! possible)

3 groups gives ? possible combinations

4 groups ? possible….

4) Counterbalance

a) complete counterbalancing – use all possible sequences of orders at least once

good if few conditions (3 or less) (n! possible)

3 groups gives 6 possible combinations

4 groups 24 possible….

b) partial counterbalancing

- take random sample of all possible sequences , reduces systematic bias

c) Latin squares

every condition appears equally often in every sequential position

- if balanced Latin square then each condition precedes and follows every other once

Latin Squares

order

participant 1 2 3 4

1 1 2 3 4

2 2 3 4 1

3 3

4 4

Latin Squares

order

participant 1 2 3 4

1 1 2 3 4

2 2 3 4 1

3 34

4 4

Latin Squares

order

participant 1 2 3 4

1 1 2 3 4

2 2 3 4 1

3 34 1 2

4 41

Latin Squares

order

participant 1 2 3 4

1 1 2 3 4

2 2 3 4 1

3 34 1 2

4 41 2 3

Balanced square

order

participant 1 2 3 4

1 1 2 4 3

2 2 3

3 3

4 4

Rule is first row 1,2,n, 3, n-1, 4,n-2 ,5….Second row add one

Balanced square

order

participant 1 2 3 4

1 1 2 4 3

2 2 31

3 3

4 4

Rule is first row 1,2,n, 3, n-1, 4,n-2 ,5….Second row add one

Balanced square

order

participant 1 2 3 4

1 1 2 4 3

2 2 31 4

3 3

4 4

Rule is first row 1,2,n, 3, n-1, 4,n-2 ,5….Second row add one

Balanced square

order

participant 1 2 3 4

1 1 2 4 3

2 2 31 4

3 34 2 1

4 41 3 2

Rule is first row 1,2,n, 3, n-1, 4,n-2 ,5….Second row add one

Within SubjectsDesign problem

• Sequence effects

• Solution - counterbalance

Experimental Control

Dependant Variable

• validity

• reliability

• multiple measures

Independent Variable

Vary in a systematic way• Control confounds related to IV

EliminateHold constantBalance (groups)Counterbalance (order)RandomizePlan for experimenter bias

Participant Effects

• Random assignment

• Pilot measures for social desirability

• Consider floor/ceiling

• Yes/no bias

Single group

A single group threat includes history, maturation, testing, instrumentation, mortality and regression to mean threats.

Multiple Groups

• These multiple group threats are called a selection bias or selection threat.

• These include selection history, selection maturation, selection testing, selection instrumentation, selection mortality and selection regression threats

The design includes two measures as denoted by two "Os" prior to the program.

This design can rule out selection maturation threat and a selection regression threat. It will help to make sure that the two groups are comparable before the treatment

Double pretest

Switching Replication Design

Good at solving the social threats to internal validity

compensatory rivalry,compensatory equalization, resentful demoralization.

Both groups get same program so no inequity

• control group – assumes extraneous variables operate on both experimental and control equally

• more than one control group can be used to assess different variables

Before training training After

experimental O X O

control O O

Before training training After

O X O

Before training Training After

experimental O X O

Control 1 O O

Control2 O

Single Group

Multiple Groups

Solomon 4 group design

testing threat The design consists of four groups of randomly assigned. Two of them receive the treatment as denoted by " X" and the other two do not.

Before training Training After

experimental O X O

Control 1 O O

Control2 X O

Control3 O

Determine extraneous variables

Will not influenceDV

ignore

Continue experiment

Might influence DV

Can be controlledCannot be controlled

Randomize Cannot randomize

Continue experiment

Continue experiment

Abandon experiment