![Page 1: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/1.jpg)
Multiple Group ExperimentsMultiple Group Experiments
Experimental Psychology
Arlo Clark‐FoosArlo Clark Foos
![Page 2: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/2.jpg)
Two Group ExperimentsTwo Group Experiments
• Design: Between‐Subjects vs Within‐SubjectsDesign: Between Subjects vs. Within Subjects
( ) β ( )• α (Type I) vs. β (Type II)
• Logic of Significance Testing: Rare occurrences
• Directional Testing…
![Page 3: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/3.jpg)
Multiple GroupsMultiple Groups
• Two GroupsTwo Groups– Experimental + Control
Treatment + Placebo– Treatment + Placebo
• Multiple Groups:– 2 x Experimental + Control
– 3 Experimental
![Page 4: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/4.jpg)
Which design works best?Which design works best?
![Page 5: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/5.jpg)
Multiple Groups ExampleMultiple Groups Example
• IV: Type of Role ModelIV: Type of Role Model
• DV: Likelihood of purchasing product
None Peer Celebrity ParentNone Peer Celebrity Parent
![Page 6: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/6.jpg)
Participant AssignmentParticipant Assignment
• Independent Samples (Between‐Subjects)Independent Samples (Between Subjects)– Completely Random Assignment
Controls for unknowns– Controls for unknowns
![Page 7: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/7.jpg)
Participant AssignmentParticipant Assignment
• Correlated SamplesCorrelated Samples– Nonrandom Assignment
• Matched Sets
• Repeated Measures
• Natural Sets
• Goal of all 3 is to reduce error!
![Page 8: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/8.jpg)
Analyzing Data from Multiple Groups
• Two: t testTwo: t test
l i l l i Of i ( O )• Multiple: Analysis Of VAriance (ANOVA)– Also used for designs with multiple IVs
– For now, One IV One Way ANOVA
– We still have 2 types of One Way ANOVA• Independent Samples (Between‐Subjects)
• Repeated Measures (Within‐Subjects)
![Page 9: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/9.jpg)
A Familiar ExampleA Familiar Example
• Old Two Group Example: Room LightingOld Two Group Example: Room Lighting– Brightly Lit vs. Dimly Lit
• Sig difference between groupsSig. difference between groups
• Are there differences between those levels?
![Page 10: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/10.jpg)
Lighting Conditions ExampleLighting Conditions Example
• Operational Definition(s)Operational Definition(s)– What constitutes Bright, Moderate, & Dim?
• Experimental Design– Repeated Measures?
– Independent Samples?
• How do we analyze the data?
![Page 11: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/11.jpg)
Analysis of VarianceAnalysis of Variance
• Rationale of ANOVARationale of ANOVA
– Between Groups Variability• Variability in DV that is due to IV
– Within Groups Variability• Variability in DV that is due to something other than IV (individual differences measurement error extraneous(individual differences, measurement error, extraneous variables)
![Page 12: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/12.jpg)
Analysis of VarianceAnalysis of Variance
h h h i ifi ff h• When the IV has a significant effect on the DV, the F ratio will be large.
• When the IV has no effect or only a small effect, the F ratio will be small (near 1).
![Page 13: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/13.jpg)
Interpreting OutputInterpreting Output
• Source TableSource Table– Specifies the source of variability
• Sum of Squares• Sum of Squares– A sum of squared deviations
• Mean Square– Divide SS by df to put on equal footing
![Page 14: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/14.jpg)
Determining SignificanceDetermining Significance
• If computing F ratio by handIf computing F ratio by hand…– Just as with t test, check a table
• Use both types of df to look up critical valueUse both types of df to look up critical value
• If computing F ratio by computer…L k f f th f ll i d– Look for one of the following words
• PROB, SIGNIFICANCE, P‐VALUE
![Page 15: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/15.jpg)
Interpreting DifferencesInterpreting Differences
Mean St. Dev.
Bright Light 86.6 6.43
Moderate 81.9 6.97
Dim Light 58.4 5.78
Sum of Squares DF Mean Square
Between Groups 1141.75 2 570.85
Within Groups 2546 25 21 121 25Within Groups 2546.25 21 121.25
Total 3688.00 23
F ratio = 4.71 PROB = .02
![Page 16: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/16.jpg)
Interpreting DifferencesInterpreting Differences
• What do learn from a significant difference?What do learn from a significant difference?– A difference between groups
Where is the difference?– Where is the difference?
W d t t t f diff b t h f– We need to test for differences between each of our groups…after we compute the F statistic
• post hoc tests (after the fact tests)
![Page 17: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/17.jpg)
Post Hoc TestsPost Hoc Tests
• Many types of Post Hoc testsMany types of Post Hoc tests– Tukey’s HSD, Bonferonni, Duncan, Dunn, Fisher’s LSD Dunnet Scheffe Student‐Neuman‐Keuls etcLSD, Dunnet, Scheffe, Student Neuman Keuls, etc.
![Page 18: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/18.jpg)
Brief Digression on ProbabilityBrief Digression on Probability
• Let’s say we put the numbers 1‐6 on slips ofLet s say we put the numbers 1 6 on slips of paper and put them all in a hat…
• Probability of 1 AND 3:• Probability of 1 AND 3:
• Probability of 1 OR 3:
![Page 19: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/19.jpg)
Brief Digression on ProbabilityBrief Digression on Probability
• Planned Comparisons (a priori)Planned Comparisons (a priori)
fl i f• Inflation of α
• Impact on Post Hoc testing
– More stringent α in many post hoc tests
![Page 20: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/20.jpg)
Translating Stats Into WordsTranslating Stats Into Words
• “Lighting conditions affected learning”g g g– Not enough information
“Th ff t f diff t li hti t d t ’ l i“The effect of different lighting on students’ learning was significant F(2, 21) = 4.71, p = .02. Tukey tests indicated (p < .05) that learning in brightly lit rooms (M 86 6 SD 6 43) as better than learning in diml(M = 86.6, SD = 6.43) was better than learning in dimly lit rooms (M = 58.4, SD = 5.74). Learning in brightly lit and moderately lit rooms did not differ from each other”other.
Lay person should be able to understand it.
![Page 21: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/21.jpg)
Repeated Measures ANOVARepeated Measures ANOVA
• Example: What type of words do peopleExample: What type of words do people remember the best?
• IV: Type of Word (Noun Adjective Adverb)• IV: Type of Word (Noun, Adjective, Adverb)
• DV: Recognition or Recall
• Groups: Same participants studying all three p p p y gword types
![Page 22: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/22.jpg)
A Note on Computer OutputA Note on Computer Output
What does it mean when p = .000?
Let’s look at the normal distribution.Tails are asymptotic = They never touch the baselineTails are asymptotic They never touch the baselinep is NEVER equal to 0 (probability of chance is NEVER 0)
Just say that p < .001
![Page 23: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/23.jpg)
Translating Stats Into WordsTranslating Stats Into Words
• “People remember some word types better than eop e e e be so e o d types bette t aothers”– Still need post hoc tests
“The effect of word type on participants’ later recall was significant F(2 14) = 19 71 p < 001 Tukey testssignificant, F(2, 14) = 19.71, p < .001. Tukey tests showed (p < .01) that people remembered nouns better (M = 63.25, SD = 11.73) than either adjectives ( ) d b ((M = 48.38, SD = 9.46) or adverbs (M = 48.88, SD = 9.55). Recall did not differ between adjectives and adverbs.”
![Page 24: Multiple Group Experiments - University of Michiganacfoos/Courses/465/09 - One Way ANOVA.pdf · Two Group Experiments • Design: Between‐Subjects vs. Within‐Subjects • α (Type](https://reader030.vdocuments.net/reader030/viewer/2022041223/5e0dea1939eb7972db6ed5f5/html5/thumbnails/24.jpg)
What comes next?What comes next?
• Now we know how to design and analyze dataNow we know how to design and analyze data from multiple groups– Where can we go from here?– Where can we go from here?
– Multiple IVs