measurement “if you can’t measure it, you can’t manage it.” bob donath, consultant
TRANSCRIPT
Measurement
“If you can’t measure it, you can’t manage it.”
Bob Donath, Consultant
Measurement
Selecting measurable phenomena
Developing a set of mapping rules
Applying the mapping ruleto each phenomenon
Measurement
What is Measured?
Object-things that is experienced and also those that are not very concrete
Properties – Characteristics of the object
Levels of Measurement
OrdinalOrdinal
intervalinterval
RatioRatio
NominalNominal ClassificationClassification
OrderOrderClassificationClassification
OrderOrderClassificationClassification DistanceDistance
Natural OriginNatural Origin
OrderOrderClassificationClassification DistanceDistance
Sources of Error in Measurement
Respondent
InstrumentMeasurer
Situation
Characteristics of a Good Measurement
CriteriaCriteria
Validity
Practicality ReliabilityReliability
ValidityThe ability of a scale to measure what was intended to be measured.
ReliabilityThe degree to which measures are free from random error and therefore yield consistent results.
PracticalityRelates to the economic factors, convenience and interpretability
Characteristics of a Good Measurement
Validity and Reliability
Reliability and Validity on Target
Old Rifle New Rifle New Rifle Sun glare
Low Reliability High Reliability High Reliability Low Validity Validity ? Low Validity(Target A) (Target B) (Target C)
The Goal Of Measurement Validity
Validity
Validity
ContentCriterionRelated
Construct
Concurrent Predictive DiscriminantConvergentFace
Content Validity
Type What is measured? Technique Content Does the measure adequately Judgment measure the concept? Panel Evaluation Content Validity Ratio Face Do experts validate that the Instrument measures what its Name suggests it measures?
Criterion related Validity
Type What is measured? Technique Criterion Does the measure differentiate Correlation based in a manner that helps to predict a criterion variable? Concurrent Does the measure differentiate in a manner that helps to predict a criterion variable currently? Predictive Does the measure differentiate in a manner as to help predict a future criterion?
Quality of Criterion
Relevant (Proper measure) Sales performance measured using the total sales in a
year
Free from bias (Equal opportunity) Must be adjusted according to area, competition and
potential
Reliable (Stable or Reproducible) Monthly sales versus yearly sales
Available (Ease of collection) Cost Difficulty in securing the data
Construct Validity
Type What is measured? Technique Construct Does the instrument tap the Judgment concept as theorized? Correlation of proposed
with established ones Convergent-discriminant
technique Factor Analysis Convergent Do two instruments measuring the same concept correlate highly? Discriminant Does the measure have a low correlation with a variable that is supposed to be unrelated to this variable
Reliability
Reliability
Stability EquivalenceInternal
Consistency
ParallelForm
Inter rater Inter-itemSplitHalf
Test-retest
Reliability (Stability)
Type What is measured? Technique Stability Reliability of a test or instrument Correlation inferred from examinee scores. Same test is administered again after some time to see the stability of the 2 scores Test-retest Same test is administered over a short
interval of less than 6 months
Problems in Test-retest
Time-delays between measurement leads to situational factor changes
Insufficient time between measurements permit the respondents to remember previous answers
Respondent’s discernment of a disguised purpose may introduce bias – opinion related to the purpose but not assessed
Topic sensitivity occurs when the respondent seeks to learn more about the topic before retest
Introduction of other variables – unrelated to the research
Reliability (Equivalence)
Type What is measured? Technique Equivalence Degree to which alternative forms of the Correlation same measure produce same or similar results. Administered simultaneously or with a delay
Parallel forms Same measurement only difference in terms of sequence and wording Inter rater To what extent 2 raters agree on the rating of the same object/characteristics
Reliability (Internal Consistency)
Type What is measured? Technique Internal Degree to which instrument items are Cronbach Alpha Consistency Homogeneous and reflect the same Split half
correlation Underlying constructs. KR20 Inter item Measures the consistency of responses Consistency in a given measurement tool Split- half The measurement tool is split into two halves and the correlation measured
Improving Reliability
Minimize external sources of variation Standardize the condition under which
measurement occurs Use only trained and experienced investigators
and motivated persons Broaden the sample of measurement questions Taken into consideration only extreme
responses then eliminate questions that do not discriminate
Practicality
Economy InterpretabilityConvenience