Download - Experiment Design 5: Variables & Levels
Experiment Design 5:Experiment Design 5:Variables & LevelsVariables & Levels
Martin, Ch 8, 9,10
Recap
Different kinds of variables– Independent, dependent, confounding, control, and
random Different kinds of validity
– Internal, construct, statistical, external– Each associated with a question
Randomization– Random sampling: generalization– Random assignment: causation
Picking a design
Choosing how to assign participants to levels of an independent variable– Between vs. within
Choosing how many levels of an independent variable
Choosing how many independent variables
Between vs. Within designs
Condition 1:– Fred
– Ginger
– Mary Condition 2:
– Ed
– Mabel
– George
Condition 1:– Fred
– Ginger
– Mary Condition 2:
– Fred
– Ginger
– Mary
5
8
6
6
9
7
5
8
6
6
9
7
Within vs. Between Subjects
Cost– Between: More participants– Within: More time per participant
Confounding variables– Between: Group differences possible
• Use randomization, many subjects, matching
– Within: Order effects possible• Use counterbalancing
Transfer effects (order effects) Definition:
– When taking part in earlier trials changes performance in the later trials
Types– Learning– Fatigue– Range or context effect
Problem:– Makes within-subjects designs difficult to
interpret
Counterbalancing
Adjust condition order to unconfound transfer effects with condition effects– A,B,C– A,C,B– B,A,C– B,C,A– C,A,B– C,B,A
Counter-balancing either within- or between- subjects
Between:– Joe: A,B– Mary: B,A
Within:– Joe: ABBA– Mary: ABBA
Things to worry about in counter-balancing
If within-subjects counter-balancing:– Linear transfer effects?
• Is the transfer from the 1st position to the 2nd position the same as the transfer from 2nd to 3rd position?
– E.g., sometimes most learning happens in 1st trials
Always worry about asymmetrical transfer– Does A influence B more than B influences A?
Asymmetrical transfer
Time 1 Time 2
% trigrams remembered
Noisy
Quiet
Effect of noise depends on order People stick with the strategy they pick first
– Or mix strategies
Quiet
Noisy
Partial counterbalancing: Latin Square
Every condition appears in every position equally:– Joe: A B C– Mary: C A B– John: B C A
Matching Try to reduce between-group differences E.g., rank hearing as Good, Fair, Poor Unmatched, could get
– Noisy: Poor1, Poor2, Fair1– Quiet: Good1, Good2, Fair2
Matched, get:– Noisy: Poor1, Fair2, Good1– Quiet: Poor2, Fair1, Good2
Matching
Match variable(s) and DV’s should be strongly correlated
Caveat: Match test should not affect DV– e.g., use existing match variable (SAT-M)
Note: Within-subjects designs “match” automatically
Number of levels
How many different groups or conditions that change just one independent variable– Two:
• Experimental vs. control
• Massed vs. Distributed practice
– More:• Drug vs. Placebo vs. No pill
• # of times an item is studied: 1,2,4,8, or 16 times
Inter- and extra-polating
0204060
0 1 2 3 4# study repetitions
0
50
100
0 1 2 3 4# study repetitions
0204060
0 1 2 3 4# study repetitions
? ?
inside outside
Floor & Ceiling Effects
0
20
40
60
80
100
2 times 4 times
# of study repetitions
0
20
40
60
80
100
2 times 4 times
# of study repetitions
Single Variable vs.Multiple Variables
Single Variable:– Only one independent variable– Cannot look at interactions
Multiple Variables:– Two or more independent variables– If use factorial design, can look at interactions– Can require a lot of participants (between) or
time (within)
Interactions
Author Editor
% errorsdetected
100
0
Proofreader
Who finds more errors, author or editor? How to spot the interaction graphically?
PrepLevel Manuscript Draft
Interactions
Two independent variables interact when the effect of one depends on the level of the other
Independent vs. Control vs. Random– What if PrepLevel had been a control variable?– What if PrepLevel had been a random variable?– Make it an independent variable if there is
reason to believe its effect might depend
Factorial design Do all combinations of factors (cells)
– E.g., Language learning
German Male FemaleOldYoung
French Male FemaleOldYoung
A factor can be within or between
Converging Operations(≠converging series)
Using more than one method to test the same hypothesis– E.g., using experimental and observational
methods– E.g., using cross-sectional and longitudinal
designs
Baseline procedure
Example 1: Clinical– No drug, drug, no drug, drug,...
Example 2: Education– Regular class, new format, regular
class, new format,..