ValidityValidity

Face ValidityFace Validity

The extent to which items on a test appear The extent to which items on a test appear to be meaningful and relevant to the to be meaningful and relevant to the construct being measuredconstruct being measured

Content-Related Evidence Content-Related Evidence for Validityfor Validity

The evidence that the content of a test The evidence that the content of a test represents the conceptual domain that the represents the conceptual domain that the test is designed to covertest is designed to cover

The extent to which test items or tasks The extent to which test items or tasks cover a representative sample of the cover a representative sample of the behavioural or conceptual domain they are behavioural or conceptual domain they are supposed to measuresupposed to measure

Concepts Related to Concepts Related to Content ValidityContent Validity

Construct UnderrepresentationConstruct Underrepresentation Failure to capture important components of Failure to capture important components of

the constructthe construct Construct-Irrelevant VarianceConstruct-Irrelevant Variance

Scores are influenced by factors irrelevant to Scores are influenced by factors irrelevant to the construct the construct

Criterion-Related EvidenceCriterion-Related Evidencefor Validityfor Validity

The evidence that a test score The evidence that a test score corresponds to an accurate measure of corresponds to an accurate measure of interest, called the “criterion”interest, called the “criterion”

The criterion is a direct & independent The criterion is a direct & independent measure of that which the test is supposed measure of that which the test is supposed to predictto predict

Validity CoefficientValidity Coefficient

Correlation between test & criterionCorrelation between test & criterion Rarely greater than .6Rarely greater than .6 Considered satisfactory if in the range Considered satisfactory if in the range

of .3 to .4of .3 to .4

Concerns About Evaluating Concerns About Evaluating Validity CoefficientsValidity Coefficients

Changes in cause of relationshipsChanges in cause of relationships Meaning of criterionMeaning of criterion Subject populationSubject population Sample sizeSample size Confusion of criterion with predictorConfusion of criterion with predictor Restricted range of predictor & criterionRestricted range of predictor & criterion Evidence for validity generalizationEvidence for validity generalization Differential predictionDifferential prediction

Construct-Related EvidenceConstruct-Related Evidencefor Validityfor Validity

A A CONSTRUCTCONSTRUCT is a theoretical entity used to is a theoretical entity used to explain and organize response consistenciesexplain and organize response consistencies

A process used to establish the meaning of a A process used to establish the meaning of a test through a series of studiestest through a series of studies

Researcher simultaneously defines a construct Researcher simultaneously defines a construct & develops a measure of that construct& develops a measure of that construct

Then looks at how the measure relates to other Then looks at how the measure relates to other measures, behaviours, etc.measures, behaviours, etc.

Types of Construct-Related Types of Construct-Related EvidenceEvidence

Convergent evidenceConvergent evidence Evidence that the test correlates highly with Evidence that the test correlates highly with

other tests, behaviours that supposedly other tests, behaviours that supposedly measure the same or related constructmeasure the same or related construct

Discriminant evidenceDiscriminant evidence Evidence that the test is unique, that it Evidence that the test is unique, that it

measures something different from what other measures something different from what other available tests measureavailable tests measure

Ways of Establishing Ways of Establishing Convergent ValidityConvergent Validity

Correlation with measure designed to Correlation with measure designed to measure same or related constructmeasure same or related construct

Age differentiationAge differentiation Experimental interventionsExperimental interventions Known groupsKnown groups

Class ExerciseClass Exercise

Using the four methods (correlation with other Using the four methods (correlation with other measures, age differentiation, experimental measures, age differentiation, experimental intervention, known groups), think of at least 2 intervention, known groups), think of at least 2 ways to validate each of the following:ways to validate each of the following: A measure of reading abilityA measure of reading ability A measure of “business sense”A measure of “business sense” A measure of musical aptitudeA measure of musical aptitude A measure of “social consciousness” or “social A measure of “social consciousness” or “social

responsibility”responsibility” A measure of political knowledgeA measure of political knowledge

Relationship Between Relationship Between Reliability & ValidityReliability & Validity

A test should not A test should not correlate more highly correlate more highly with any other with any other variable than it variable than it correlates with itselfcorrelates with itself

Maximum validity Maximum validity coefficient (Rcoefficient (R12max12max) ) between 2 variables between 2 variables is equal to the square is equal to the square root of the product of root of the product of their reliabilitiestheir reliabilities

221112 RRR