Experiment Design 5:Experiment Design 5:Variables & LevelsVariables & Levels

Martin, Ch 8, 9,10

Recap

Different kinds of variables– Independent, dependent, confounding, control, and

random Different kinds of validity

– Internal, construct, statistical, external– Each associated with a question

Randomization– Random sampling: generalization– Random assignment: causation

Picking a design

Choosing how to assign participants to levels of an independent variable– Between vs. within

Choosing how many levels of an independent variable

Choosing how many independent variables

Between vs. Within designs

Condition 1:– Fred

– Ginger

– Mary Condition 2:

– Ed

– Mabel

– George

Condition 1:– Fred

– Ginger

– Mary Condition 2:

– Fred

– Ginger

– Mary

5

8

6

6

9

7

5

8

6

6

9

7

Within vs. Between Subjects

Cost– Between: More participants– Within: More time per participant

Confounding variables– Between: Group differences possible

• Use randomization, many subjects, matching

– Within: Order effects possible• Use counterbalancing

Transfer effects (order effects) Definition:

– When taking part in earlier trials changes performance in the later trials

Types– Learning– Fatigue– Range or context effect

Problem:– Makes within-subjects designs difficult to

interpret

Counterbalancing

Adjust condition order to unconfound transfer effects with condition effects– A,B,C– A,C,B– B,A,C– B,C,A– C,A,B– C,B,A

Counter-balancing either within- or between- subjects

Between:– Joe: A,B– Mary: B,A

Within:– Joe: ABBA– Mary: ABBA

Things to worry about in counter-balancing

If within-subjects counter-balancing:– Linear transfer effects?

• Is the transfer from the 1st position to the 2nd position the same as the transfer from 2nd to 3rd position?

– E.g., sometimes most learning happens in 1st trials

Always worry about asymmetrical transfer– Does A influence B more than B influences A?

Asymmetrical transfer

Time 1 Time 2

% trigrams remembered

Noisy

Quiet

Effect of noise depends on order People stick with the strategy they pick first

– Or mix strategies

Quiet

Noisy

Partial counterbalancing: Latin Square

Every condition appears in every position equally:– Joe: A B C– Mary: C A B– John: B C A

Matching Try to reduce between-group differences E.g., rank hearing as Good, Fair, Poor Unmatched, could get

– Noisy: Poor1, Poor2, Fair1– Quiet: Good1, Good2, Fair2

Matched, get:– Noisy: Poor1, Fair2, Good1– Quiet: Poor2, Fair1, Good2

Matching

Match variable(s) and DV’s should be strongly correlated

Caveat: Match test should not affect DV– e.g., use existing match variable (SAT-M)

Note: Within-subjects designs “match” automatically

Number of levels

How many different groups or conditions that change just one independent variable– Two:

• Experimental vs. control

• Massed vs. Distributed practice

– More:• Drug vs. Placebo vs. No pill

• # of times an item is studied: 1,2,4,8, or 16 times

Inter- and extra-polating

0204060

0 1 2 3 4# study repetitions

0

50

100

0 1 2 3 4# study repetitions

0204060

0 1 2 3 4# study repetitions

? ?

inside outside

Floor & Ceiling Effects

0

20

40

60

80

100

2 times 4 times

# of study repetitions

0

20

40

60

80

100

2 times 4 times

# of study repetitions

Single Variable vs.Multiple Variables

Single Variable:– Only one independent variable– Cannot look at interactions

Multiple Variables:– Two or more independent variables– If use factorial design, can look at interactions– Can require a lot of participants (between) or

time (within)

Interactions

Author Editor

% errorsdetected

100

0

Proofreader

Who finds more errors, author or editor? How to spot the interaction graphically?

PrepLevel Manuscript Draft

Interactions

Two independent variables interact when the effect of one depends on the level of the other

Independent vs. Control vs. Random– What if PrepLevel had been a control variable?– What if PrepLevel had been a random variable?– Make it an independent variable if there is

reason to believe its effect might depend

Factorial design Do all combinations of factors (cells)

– E.g., Language learning

German Male FemaleOldYoung

French Male FemaleOldYoung

A factor can be within or between

Converging Operations(≠converging series)

Using more than one method to test the same hypothesis– E.g., using experimental and observational

methods– E.g., using cross-sectional and longitudinal

designs

Baseline procedure

Example 1: Clinical– No drug, drug, no drug, drug,...

Example 2: Education– Regular class, new format, regular

class, new format,..