experiment design 5: variables & levels
Post on 30-Dec-2015
Embed Size (px)
DESCRIPTIONExperiment Design 5: Variables & Levels. Martin, Ch 8, 9,10. Recap. Different kinds of variables Independent, dependent, confounding, control, and random Different kinds of validity Internal, construct, statistical, external Each associated with a question Randomization - PowerPoint PPT Presentation
Experiment Design 5:Variables & LevelsMartin, Ch 8, 9,10
RecapDifferent kinds of variablesIndependent, dependent, confounding, control, and randomDifferent kinds of validity Internal, construct, statistical, externalEach associated with a questionRandomizationRandom sampling: generalizationRandom assignment: causation
Picking a designChoosing how to assign participants to levels of an independent variableBetween vs. withinChoosing how many levels of an independent variableChoosing how many independent variables
Between vs. Within designsCondition 1:FredGingerMaryCondition 2:EdMabelGeorgeCondition 1:FredGingerMaryCondition 2:FredGingerMary586697586697
Within vs. Between SubjectsCostBetween: More participantsWithin: More time per participantConfounding variablesBetween: Group differences possibleUse randomization, many subjects, matching Within: Order effects possibleUse counterbalancing
Transfer effects (order effects)Definition:When taking part in earlier trials changes performance in the later trials TypesLearningFatigueRange or context effectProblem:Makes within-subjects designs difficult to interpret
CounterbalancingAdjust condition order to unconfound transfer effects with condition effectsA,B,CA,C,BB,A,CB,C,AC,A,BC,B,A
Counter-balancing either within- or between- subjectsBetween:Joe: A,BMary: B,AWithin:Joe: ABBAMary: ABBA
Things to worry about in counter-balancingIf within-subjects counter-balancing:Linear transfer effects?Is the transfer from the 1st position to the 2nd position the same as the transfer from 2nd to 3rd position?E.g., sometimes most learning happens in 1st trialsAlways worry about asymmetrical transferDoes A influence B more than B influences A?
Asymmetrical transferEffect of noise depends on orderPeople stick with the strategy they pick firstOr mix strategiesTime 1Time 2% trigrams remembered
Partial counterbalancing: Latin SquareEvery condition appears in every position equally:Joe: ABCMary:CABJohn:BCA
MatchingTry to reduce between-group differencesE.g., rank hearing as Good, Fair, PoorUnmatched, could getNoisy: Poor1, Poor2, Fair1Quiet: Good1, Good2, Fair2Matched, get:Noisy: Poor1, Fair2, Good1Quiet: Poor2, Fair1, Good2
MatchingMatch variable(s) and DVs should be strongly correlatedCaveat: Match test should not affect DVe.g., use existing match variable (SAT-M)Note: Within-subjects designs match automatically
Number of levelsHow many different groups or conditions that change just one independent variableTwo:Experimental vs. controlMassed vs. Distributed practiceMore:Drug vs. Placebo vs. No pill# of times an item is studied: 1,2,4,8, or 16 times
Inter- and extra-polatinginsideoutside
Floor & Ceiling Effects
Single Variable vs.Multiple VariablesSingle Variable:Only one independent variableCannot look at interactionsMultiple Variables:Two or more independent variablesIf use factorial design, can look at interactionsCan require a lot of participants (between) or time (within)
InteractionsWho finds more errors, author or editor?How to spot the interaction graphically?AuthorEditor% errorsdetected1000Proofreader
InteractionsTwo independent variables interact when the effect of one depends on the level of the otherIndependent vs. Control vs. RandomWhat if PrepLevel had been a control variable?What if PrepLevel had been a random variable?Make it an independent variable if there is reason to believe its effect might depend
Factorial designDo all combinations of factors (cells)E.g., Language learningA factor can be within or between
Converging Operations(converging series)Using more than one method to test the same hypothesisE.g., using experimental and observational methodsE.g., using cross-sectional and longitudinal designs
Baseline procedure Example 1: ClinicalNo drug, drug, no drug, drug,...
Example 2: EducationRegular class, new format, regular class, new format,..
Ch7 within vs. between counter balancing (complete vs. partial) transfer (symmetrical, etc.) range effects, context effects matchingHow many subjects in each condition?How many subjects total?How long to run each subject?
Give example numbers with effect to illustrate statistical power.Count #of times A is first, B is first, C is firstCount # of times A comes before B and B comes before ACh8 levels (two vs. more) problems of extrapolation ceiling & floor effects (give graph examples)Ch9 factorial design , interactions, mixed factorial design converging operations (not quite the same as in book)Answer 1: It depends
Answer 2: Crossing linesIf Preparation had been a control variable, we might have concluded that (a) editors were a bad idea (draft only), or (b) its always good to give your work to an editor (manuscript only)
If Preparation had been a random variable, we would have underestimated the value of editors in proofing manuscripts