psychology 2020 measurement & observing behavior

11

Psychology 2020 Psychology 2020 Measurement &Measurement &

Observing BehaviorObserving Behavior

Unit 2Unit 2

2

Measurement ReliabilityMeasurement Reliability

► Measurements consist of two components. Measurements consist of two components. Both must be inferred because they can’t be Both must be inferred because they can’t be directly observeddirectly observed True scoreTrue score

► The real score of the variable that exists hypothetically The real score of the variable that exists hypothetically and is considered a constantand is considered a constant

Measurement errorMeasurement error► The portion of the measurement result that varies from The portion of the measurement result that varies from

measurement to measurementmeasurement to measurement

► The higher the correlation (The higher the correlation (rr near 1) near 1) between two measurements, the more between two measurements, the more reliable the measurementreliable the measurement

3

Assessing the Reliability of Assessing the Reliability of Psychological MeasuresPsychological Measures

► Test-Retest ReliabilityTest-Retest Reliability Take the measurement at two different times and Take the measurement at two different times and

correlate the results (correlate the results (r r =.8 to be considered =.8 to be considered reliable).reliable).

► Internal Consistency Reliability Internal Consistency Reliability Split test in half an correlate the scores on the first Split test in half an correlate the scores on the first

half with the scores on the second halfhalf with the scores on the second half Correlate every item score with every other item Correlate every item score with every other item

score (Cronbach’s alpha)score (Cronbach’s alpha) Correlate every item score with the total scoreCorrelate every item score with the total score

► Interrater ReliabilityInterrater Reliability Correlate rater scores with each otherCorrelate rater scores with each other

4

Validity in ResearchValidity in Research

► Validity refers to “truth” and accuracy of Validity refers to “truth” and accuracy of measurementmeasurement INTERNALINTERNAL validity (extent to which IV validity (extent to which IV

has influence on DV)has influence on DV) EXTERNALEXTERNAL validity (extent to which validity (extent to which

results obtained can be generalized)results obtained can be generalized) CONSTRUCTCONSTRUCT validity validity

►Several typesSeveral types

5

Measurement ValidityMeasurement Validity

► If a measurement tool is valid, it is If a measurement tool is valid, it is really measuring the construct it really measuring the construct it purports to measure rather than some purports to measure rather than some other characteristicother characteristic A valid measurement tool is said to have A valid measurement tool is said to have

““Construct ValidityConstruct Validity””

6

Methods to Assess Construct Methods to Assess Construct ValidityValidity

►Face Validity (does it appear valid)Face Validity (does it appear valid) An IQ test that only asked questions about An IQ test that only asked questions about

a person’s income, address and clothing a person’s income, address and clothing preferences would have little face validitypreferences would have little face validity

►Criterion-Oriented Validity (does it Criterion-Oriented Validity (does it allow accurate predictions)allow accurate predictions) A safe-driving test that passed people A safe-driving test that passed people

who continued to have driving accidents who continued to have driving accidents would lack criterion-oriented validitywould lack criterion-oriented validity

7

Forms of Criterion-Oriented Forms of Criterion-Oriented ValidityValidity

► Predictive Validity- Predictive Validity- ability to predict ability to predict something it should theoretically be able to something it should theoretically be able to predictpredict.. A measure of math ability accurately predicts who A measure of math ability accurately predicts who

will be a successful engineerwill be a successful engineer

► Concurrent Validity- Concurrent Validity- ability to distinguish ability to distinguish between groups that it should theoretically between groups that it should theoretically be able to distinguish betweenbe able to distinguish between A measure of anxiety distinguishes individuals A measure of anxiety distinguishes individuals

with an anxiety disorder from those with major with an anxiety disorder from those with major depressiondepression

8

Forms of Criterion-Oriented Forms of Criterion-Oriented ValidityValidity

► Convergent Validity- Convergent Validity- results from the results from the measure are similar to other measures of measure are similar to other measures of the same constructthe same construct The block assembly IQ test yields high IQ scores The block assembly IQ test yields high IQ scores

with the same people who got high IQ scores on with the same people who got high IQ scores on a spatial problem solving IQ test.a spatial problem solving IQ test.

► Discriminant Validity- Discriminant Validity- results from the results from the measure are NOT similar to measures of measure are NOT similar to measures of theoretically different constructs.theoretically different constructs. An abstract-reasoning test yields different results An abstract-reasoning test yields different results

that a rote-memory testthat a rote-memory test

9

Measurement ReactivityMeasurement Reactivity

►Reactivity in measurement is when the Reactivity in measurement is when the very act of measuring a variable very act of measuring a variable changeschanges that variable that variable Example: recording the amount of food Example: recording the amount of food

eaten each a day causes the person to eat eaten each a day causes the person to eat less less

►Using Using unobtrusiveunobtrusive measures reduces measures reduces the probability of measurement the probability of measurement reactivityreactivity

10

Measurement ScalesMeasurement Scales

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

11


►NominalNominal No numerical properties, only named No numerical properties, only named

categoriescategories Examples: male or female, married or singleExamples: male or female, married or single

►OrdinalOrdinal Has numerical properties but only for the Has numerical properties but only for the

purpose of ordering the categories from first purpose of ordering the categories from first to lastto last

Examples: Academic grades, places in a Examples: Academic grades, places in a race, ranking schools or sports teamsrace, ranking schools or sports teams

12


► IntervalInterval Has numerical properties with equal distance Has numerical properties with equal distance

between each number on the scale but no zero between each number on the scale but no zero point (representing a total absence of the thing point (representing a total absence of the thing being measured).being measured).

Example: Rating beauty on a scale of 1-10Example: Rating beauty on a scale of 1-10► RatioRatio

Same as the interval scale (equal intervals Same as the interval scale (equal intervals between numbers) except there is a zero point.between numbers) except there is a zero point.

Examples: time measures, response rate Examples: time measures, response rate measures, other physical measurements. measures, other physical measurements.

13

Name the Measurement Name the Measurement ScaleScale

• A food critic rates restaurants according to the A food critic rates restaurants according to the quality of food, service, and atmosphere. She quality of food, service, and atmosphere. She assigns 4 forks for excellent, 3 for good, 2 for fair, assigns 4 forks for excellent, 3 for good, 2 for fair, and 1 fork for poorand 1 fork for poor

• Researchers have identified styles of leadership as Researchers have identified styles of leadership as relationship-orientedrelationship-oriented or or task-orientedtask-oriented

• Rating satisfaction on a scale of 1 to 100Rating satisfaction on a scale of 1 to 100

• A measurement of weightA measurement of weight

14

Name the Measurement Name the Measurement ScaleScale

• Getting a count of the number of automobiles on the Getting a count of the number of automobiles on the highway that were made in Europe, Japan and the highway that were made in Europe, Japan and the USA (totals for each group)USA (totals for each group)

• The score you receive on the Beck Depression scale The score you receive on the Beck Depression scale (1-18 is not depressed, above 18 is depressed)(1-18 is not depressed, above 18 is depressed)

• Deciding if a house style is Victorian, Tudor, or Deciding if a house style is Victorian, Tudor, or Greek RevivalGreek Revival

• Measuring how long it takes someone to drink a beerMeasuring how long it takes someone to drink a beer

15

QuickTime™ and aTIFF (Uncompressed) decompressor


16



17



18



20



22

General Observational General Observational ApproachesApproaches

►Quantitative approachesQuantitative approaches Emphasize numerical properties of the Emphasize numerical properties of the

thing being observedthing being observed Observational results can be statistically Observational results can be statistically

analyzedanalyzed ExamplesExamples

►Hours spent studyingHours spent studying►Ranking of task difficultyRanking of task difficulty►Percentage of on-task behavior Percentage of on-task behavior

23

General Observational General Observational ApproachesApproaches

► Qualitative approachesQualitative approaches Emphasizes the nonnumerical aspects of Emphasizes the nonnumerical aspects of

the thing being observedthe thing being observed Usually consists of descriptive paragraphsUsually consists of descriptive paragraphs Statistical analysis of data in this form is Statistical analysis of data in this form is

not possiblenot possible ExamplesExamples

►DiariesDiaries►Film documentariesFilm documentaries►Written summaries of main themes and/or Written summaries of main themes and/or

styles observedstyles observed

24

Naturalistic ObservationNaturalistic Observation

► Observing and recording behavior in its Observing and recording behavior in its natural settingnatural setting Quantitative or qualitativeQuantitative or qualitative

► Confirmation through multiple Confirmation through multiple observationsobservations

25

Issues in Naturalistic Issues in Naturalistic ObservationObservation

►ParticipationParticipation►ConcealmentConcealment

Accuracy v. ethicsAccuracy v. ethics►Scope of observationScope of observation

ComplexityComplexity►Limits of observationLimits of observation

Public v. privatePublic v. private Uncontrollable factorsUncontrollable factors

26

Systematic ObservationSystematic Observation

►Careful observation of specific Careful observation of specific behaviors in specific settings using behaviors in specific settings using structured, quantifiable methods structured, quantifiable methods (coding systems)(coding systems)

►Statistical data analysisStatistical data analysis

27

Methodological IssuesMethodological Issues

► Technical equipment (video tape, Technical equipment (video tape, computers, clocks, etc.)computers, clocks, etc.)

► ReactivityReactivity► ReliabilityReliability► SamplingSampling

28

VideoVideo

29

Case StudiesCase Studies

► Case studies are usually extensive Case studies are usually extensive descriptions of one individual or descriptions of one individual or organization over a period of time.organization over a period of time.

► Case studies are one form of Case studies are one form of naturalistic naturalistic observationobservation research research

► Case studies are conducted when an Case studies are conducted when an individual is unusual, has a rare disorder or individual is unusual, has a rare disorder or noteworthy conditionnoteworthy condition

► Case studies provide valuable information Case studies provide valuable information that is not available in other forms of that is not available in other forms of researchresearch

30

Archival ResearchArchival Research

►Archival research uses data that was Archival research uses data that was previously collected and compiled previously collected and compiled (usually for other reasons) to answer (usually for other reasons) to answer new questionsnew questions

►Two problems exist for doing archival Two problems exist for doing archival researchresearch The necessary records or documents may The necessary records or documents may

be difficult to obtainbe difficult to obtain The accuracy of information collected by The accuracy of information collected by

someone else is always suspectsomeone else is always suspect

psychology 2020 measurement & observing behavior

Documents