# experiment design 5: variables & levels

Post on 30-Dec-2015

21 views

Embed Size (px)

DESCRIPTION

Experiment Design 5: Variables & Levels. Martin, Ch 8, 9,10. Recap. Different kinds of variables Independent, dependent, confounding, control, and random Different kinds of validity Internal, construct, statistical, external Each associated with a question Randomization - PowerPoint PPT PresentationTRANSCRIPT

Experiment Design 5:Variables & LevelsMartin, Ch 8, 9,10

RecapDifferent kinds of variablesIndependent, dependent, confounding, control, and randomDifferent kinds of validity Internal, construct, statistical, externalEach associated with a questionRandomizationRandom sampling: generalizationRandom assignment: causation

Picking a designChoosing how to assign participants to levels of an independent variableBetween vs. withinChoosing how many levels of an independent variableChoosing how many independent variables

Between vs. Within designsCondition 1:FredGingerMaryCondition 2:EdMabelGeorgeCondition 1:FredGingerMaryCondition 2:FredGingerMary586697586697

Within vs. Between SubjectsCostBetween: More participantsWithin: More time per participantConfounding variablesBetween: Group differences possibleUse randomization, many subjects, matching Within: Order effects possibleUse counterbalancing

Transfer effects (order effects)Definition:When taking part in earlier trials changes performance in the later trials TypesLearningFatigueRange or context effectProblem:Makes within-subjects designs difficult to interpret

CounterbalancingAdjust condition order to unconfound transfer effects with condition effectsA,B,CA,C,BB,A,CB,C,AC,A,BC,B,A

Counter-balancing either within- or between- subjectsBetween:Joe: A,BMary: B,AWithin:Joe: ABBAMary: ABBA

Things to worry about in counter-balancingIf within-subjects counter-balancing:Linear transfer effects?Is the transfer from the 1st position to the 2nd position the same as the transfer from 2nd to 3rd position?E.g., sometimes most learning happens in 1st trialsAlways worry about asymmetrical transferDoes A influence B more than B influences A?

Asymmetrical transferEffect of noise depends on orderPeople stick with the strategy they pick firstOr mix strategiesTime 1Time 2% trigrams remembered

Partial counterbalancing: Latin SquareEvery condition appears in every position equally:Joe: ABCMary:CABJohn:BCA

MatchingTry to reduce between-group differencesE.g., rank hearing as Good, Fair, PoorUnmatched, could getNoisy: Poor1, Poor2, Fair1Quiet: Good1, Good2, Fair2Matched, get:Noisy: Poor1, Fair2, Good1Quiet: Poor2, Fair1, Good2

MatchingMatch variable(s) and DVs should be strongly correlatedCaveat: Match test should not affect DVe.g., use existing match variable (SAT-M)Note: Within-subjects designs match automatically

Number of levelsHow many different groups or conditions that change just one independent variableTwo:Experimental vs. controlMassed vs. Distributed practiceMore:Drug vs. Placebo vs. No pill# of times an item is studied: 1,2,4,8, or 16 times

Inter- and extra-polatinginsideoutside

Floor & Ceiling Effects

Single Variable vs.Multiple VariablesSingle Variable:Only one independent variableCannot look at interactionsMultiple Variables:Two or more independent variablesIf use factorial design, can look at interactionsCan require a lot of participants (between) or time (within)

InteractionsWho finds more errors, author or editor?How to spot the interaction graphically?AuthorEditor% errorsdetected1000Proofreader

InteractionsTwo independent variables interact when the effect of one depends on the level of the otherIndependent vs. Control vs. RandomWhat if PrepLevel had been a control variable?What if PrepLevel had been a random variable?Make it an independent variable if there is reason to believe its effect might depend

Factorial designDo all combinations of factors (cells)E.g., Language learningA factor can be within or between

Converging Operations(converging series)Using more than one method to test the same hypothesisE.g., using experimental and observational methodsE.g., using cross-sectional and longitudinal designs

Baseline procedure Example 1: ClinicalNo drug, drug, no drug, drug,...

Example 2: EducationRegular class, new format, regular class, new format,..

Ch7 within vs. between counter balancing (complete vs. partial) transfer (symmetrical, etc.) range effects, context effects matchingHow many subjects in each condition?How many subjects total?How long to run each subject?

Give example numbers with effect to illustrate statistical power.Count #of times A is first, B is first, C is firstCount # of times A comes before B and B comes before ACh8 levels (two vs. more) problems of extrapolation ceiling & floor effects (give graph examples)Ch9 factorial design , interactions, mixed factorial design converging operations (not quite the same as in book)Answer 1: It depends

Answer 2: Crossing linesIf Preparation had been a control variable, we might have concluded that (a) editors were a bad idea (draft only), or (b) its always good to give your work to an editor (manuscript only)

If Preparation had been a random variable, we would have underestimated the value of editors in proofing manuscripts