experiment design 5: variables & levels

Download Experiment Design 5: Variables & Levels

Post on 30-Dec-2015




0 download

Embed Size (px)


Experiment Design 5: Variables & Levels. Martin, Ch 8, 9,10. Recap. Different kinds of variables Independent, dependent, confounding, control, and random Different kinds of validity Internal, construct, statistical, external Each associated with a question Randomization - PowerPoint PPT Presentation


  • Experiment Design 5:Variables & LevelsMartin, Ch 8, 9,10

  • RecapDifferent kinds of variablesIndependent, dependent, confounding, control, and randomDifferent kinds of validity Internal, construct, statistical, externalEach associated with a questionRandomizationRandom sampling: generalizationRandom assignment: causation

  • Picking a designChoosing how to assign participants to levels of an independent variableBetween vs. withinChoosing how many levels of an independent variableChoosing how many independent variables

  • Between vs. Within designsCondition 1:FredGingerMaryCondition 2:EdMabelGeorgeCondition 1:FredGingerMaryCondition 2:FredGingerMary586697586697

  • Within vs. Between SubjectsCostBetween: More participantsWithin: More time per participantConfounding variablesBetween: Group differences possibleUse randomization, many subjects, matching Within: Order effects possibleUse counterbalancing

  • Transfer effects (order effects)Definition:When taking part in earlier trials changes performance in the later trials TypesLearningFatigueRange or context effectProblem:Makes within-subjects designs difficult to interpret

  • CounterbalancingAdjust condition order to unconfound transfer effects with condition effectsA,B,CA,C,BB,A,CB,C,AC,A,BC,B,A

  • Counter-balancing either within- or between- subjectsBetween:Joe: A,BMary: B,AWithin:Joe: ABBAMary: ABBA

  • Things to worry about in counter-balancingIf within-subjects counter-balancing:Linear transfer effects?Is the transfer from the 1st position to the 2nd position the same as the transfer from 2nd to 3rd position?E.g., sometimes most learning happens in 1st trialsAlways worry about asymmetrical transferDoes A influence B more than B influences A?

  • Asymmetrical transferEffect of noise depends on orderPeople stick with the strategy they pick firstOr mix strategiesTime 1Time 2% trigrams remembered

  • Partial counterbalancing: Latin SquareEvery condition appears in every position equally:Joe: ABCMary:CABJohn:BCA

  • MatchingTry to reduce between-group differencesE.g., rank hearing as Good, Fair, PoorUnmatched, could getNoisy: Poor1, Poor2, Fair1Quiet: Good1, Good2, Fair2Matched, get:Noisy: Poor1, Fair2, Good1Quiet: Poor2, Fair1, Good2

  • MatchingMatch variable(s) and DVs should be strongly correlatedCaveat: Match test should not affect DVe.g., use existing match variable (SAT-M)Note: Within-subjects designs match automatically

  • Number of levelsHow many different groups or conditions that change just one independent variableTwo:Experimental vs. controlMassed vs. Distributed practiceMore:Drug vs. Placebo vs. No pill# of times an item is studied: 1,2,4,8, or 16 times

  • Inter- and extra-polatinginsideoutside

  • Floor & Ceiling Effects

  • Single Variable vs.Multiple VariablesSingle Variable:Only one independent variableCannot look at interactionsMultiple Variables:Two or more independent variablesIf use factorial design, can look at interactionsCan require a lot of participants (between) or time (within)

  • InteractionsWho finds more errors, author or editor?How to spot the interaction graphically?AuthorEditor% errorsdetected1000Proofreader

  • InteractionsTwo independent variables interact when the effect of one depends on the level of the otherIndependent vs. Control vs. RandomWhat if PrepLevel had been a control variable?What if PrepLevel had been a random variable?Make it an independent variable if there is reason to believe its effect might depend

  • Factorial designDo all combinations of factors (cells)E.g., Language learningA factor can be within or between

  • Converging Operations(converging series)Using more than one method to test the same hypothesisE.g., using experimental and observational methodsE.g., using cross-sectional and longitudinal designs

  • Baseline procedure Example 1: ClinicalNo drug, drug, no drug, drug,...

    Example 2: EducationRegular class, new format, regular class, new format,..

    Ch7 within vs. between counter balancing (complete vs. partial) transfer (symmetrical, etc.) range effects, context effects matchingHow many subjects in each condition?How many subjects total?How long to run each subject?

    Give example numbers with effect to illustrate statistical power.Count #of times A is first, B is first, C is firstCount # of times A comes before B and B comes before ACh8 levels (two vs. more) problems of extrapolation ceiling & floor effects (give graph examples)Ch9 factorial design , interactions, mixed factorial design converging operations (not quite the same as in book)Answer 1: It depends

    Answer 2: Crossing linesIf Preparation had been a control variable, we might have concluded that (a) editors were a bad idea (draft only), or (b) its always good to give your work to an editor (manuscript only)

    If Preparation had been a random variable, we would have underestimated the value of editors in proofing manuscripts


View more >