chapter 6 class version
DESCRIPTION
TRANSCRIPT
![Page 1: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/1.jpg)
Chapter 6: RESEARCH VALIDITY
9/27/2012
![Page 2: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/2.jpg)
Roadmap
• Research study announcement• Exam 1 update• Reflection Assignment #1• Quick review & Ch. 5 wrap-up• Begin Chapter 6: Research Validity
![Page 3: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/3.jpg)
Reflection Assignment #1
• Due Tuesday (next class)• Hard copy due at beginning of class– STAPLE YOUR BUSINESS. I’m so serious.
• Cover page: Assignment document (rubric for grading)
![Page 4: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/4.jpg)
Quick Review
Random SamplingSimple Random
Stratified Random
Cluster Random
Systematic Sampling
Nonrandom SamplingConvenience
Quota
Purposive
Snowball
![Page 5: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/5.jpg)
Quick Review
• Random selection vs. Random assignment
• Difference = purpose
![Page 6: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/6.jpg)
How do we determine sample size?
• If population <100, measure them all– Special term for this?
• In general, get as big a sample as possible
• Sample size calculator: G*Power
• Depends on lots of factors
![Page 7: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/7.jpg)
RESEARCH VALIDITYChapter 6
![Page 8: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/8.jpg)
MORE Validity—yay!
• Remember: validity has to do with drawing accurate inferences
• So far: validity in the context of measuring variables
• Now: validity in the context of setting up studies
![Page 9: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/9.jpg)
Research Validity
• Refers to the truthfulness of inferences made from a research study
• Think of validity on a continuum rather than all-or-none
• 4 major types of research validity
• Must prioritize
![Page 10: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/10.jpg)
Types of Research Validity
• Statistical Conclusion Validity• Construct Validity• Internal Validity• External Validity
![Page 11: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/11.jpg)
Statistical Conclusion Validity
• Validity with which we can infer that the IV and DV covary– Covary = vary together
• The validity of the inferences we make from our analyses
![Page 12: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/12.jpg)
Stats Refresher: “Statistically Significant”
• p <.05
• What does it mean?– The observed relationship is probably
NOT due to chance alone
• Sometimes our stats are just wrong• Chance• Too little power (sample size)• Type 1 / Type 2 error
![Page 13: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/13.jpg)
Construct Validity
• Refresh: construct = ?
• Validity of the inferences we make about constructs based on how we measure them
• What does this sound like?– The chapter 5 validity topics!
![Page 14: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/14.jpg)
• Which constructs/operationalizations do we need to consider for construct validity?
• All of them!– IV– DV– Population– Setting
![Page 15: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/15.jpg)
How do we assess Construct Validity?
• Content validity
• Criterion validity
• Convergent validity
![Page 16: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/16.jpg)
ACTIVITY
![Page 17: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/17.jpg)
Group Activity: 5-7 minutes• You’re applying for a grant to fund a
research project
• Identify research idea– IV - operationalize– DV - operationalize– Hypothesis
• Explain how you will gather evidence of construct validity in your measurements
![Page 18: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/18.jpg)
Threats to Construct Validity
• Factors that impact how well our operationalizations actually represent constructs
• Pg 171, Table 6.2 – long list of threats
• We will focus on two major ones:– Participant reactivity to the experimental
situation– Experimenter effects
![Page 19: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/19.jpg)
Reactivity to the Experimental Situation
(From the participant angle)
• Participants’ motives and perceptions
• Demand characteristics
• Positive self-presentation
![Page 20: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/20.jpg)
Instruction set #1
We want to see how well you are able to learn the following sets of letters. Letters will appear in groups of 3 to 7, and each letter will appear on the screen for 1 second. Following the presentation of the letters, …
![Page 21: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/21.jpg)
Instruction Set #2
In the following task, you will be presented with groups of letters ranging from 3 to 7 letters. Each letter will appear on the screen for one second. Your task is to…
![Page 22: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/22.jpg)
Experimenter effects
• Researcher actions and characteristics that influence the responses made by the research participant
• Expectancies– Clever Hans
• Attributes– Biosocial– Psychosocial– Situational factors
![Page 23: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/23.jpg)
Clever Hans
I Math
![Page 24: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/24.jpg)
Internal Validity
• The extent to which we can accurately infer that the independent and dependent variables are causally related
Observed Effect (DV)
Independent Variable
![Page 25: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/25.jpg)
Causally Related
Independent Variable
Observed Effect (DV)
Cause must precede effect
Cause and effect are related (covary)
No other explanation is plausible
No other explanation is plausible
![Page 26: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/26.jpg)
Special Considerations
• Extraneous variables
• Confounding extraneous variables
![Page 27: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/27.jpg)
Threats to Internal Validity
• History • Maturation • Instrumentation • Testing • Regression Artifact • Attrition • Selection • Additive and Interactive Effects
![Page 28: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/28.jpg)
History
• Any event occurring after the study begins that could produce the observed outcome
• Differential history: only one group experiences history event
![Page 29: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/29.jpg)
Maturation
• Changes in biological and psychological conditions that occur with the passage of time – Factors within the individual
• Example: Head Start program and achievement over a school year
![Page 30: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/30.jpg)
Instrumentation
• Changes in the assessment/measurement of the dependent variable
• Example: multiple observers and interviewers
![Page 31: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/31.jpg)
Testing
• Changes in a person’s score on the second administration of a test a result of previously having taken the test
• Example: pre-test and post-test on memory task
![Page 32: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/32.jpg)
Regression Artifact
• A.k.a. regression toward the mean
• The tendency for extreme scores to become less extreme on a second assessment
![Page 33: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/33.jpg)
Attrition
• Participant drop-out– Don’t show up for appointment– Decide to discontinue study
• Differential attrition is especially problematic
![Page 34: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/34.jpg)
Selection
• The choice of participants for the various treatment groups based on different criteria – NOT random assignment
![Page 35: Chapter 6 class version](https://reader033.vdocuments.net/reader033/viewer/2022061209/548c1ad6b479597a708b4696/html5/thumbnails/35.jpg)
Additive & Interactive Effects
• The combined effect of several threats to internal validity