Measurement and Scaling Concepts

© 1–1

What Do I Measure?

• Measurement• The process of describing some property of a

phenomenon, usually by assigning numbers in a reliable and valid way.

• Concept• A generalized idea about a class of objects, attributes,

occurrences, or processes

13–2

EXHIBIT 13.2 Are There Any Validity Issues with this Measurement?

13–3

Operational Definitions

• Operationalization• The process of identifying scales that correspond to

variance in a concept involved in a research process.

• Scales• A device providing a range of values that correspond

to different characteristics or amounts of a characteristic exhibited in observing a concept.

• Correspondence rules• Indicate the way that a certain value on a scale

corresponds to some true value of a concept.

13–4

Operational Definitions (cont’d)

• Variable• Anything that varies or changes from one instance to

another; can exhibit differences in value, usually in magnitude or strength, or in direction.

• Capture different values of a concept.

• Constructs• Concepts measured with multiple variables.

13–5

EXHIBIT 13.3 Susceptibility to Interpersonal Influence: An Operational Definition

13–6

13–7

Concept behind scaling

• Scales are developed based on three aspects:– Order : Numbers (categories) are ordered– Distance : Difference between numbers (categories)

are ordered and quantifiable. For our purposes and most of practical purposes we assume this difference is same.

– Origin : The series has a unique origin which is indicated by the number zero.

Levels of Scale Measurement

• Nominal• No order, distance and origin.• Assigns a value to an object for identification or

classification purposes.• Most elementary level of measurement.

• Ordinal• There is order, but no distance and origin• Ranking scales allowing things to be arranged based

on how much of some concept they possible.• Have nominal properties.

13–8

Levels of Scale Measurement (cont’d)

• Interval• There are order and distance, but no origin• Capture information about differences in quantities of

a concept.• Have both nominal and ordinal properties.

• Ratio• There are order, distance and origin.• Highest form of measurement.• Have all the properties of interval scales with the

additional attribute of representing absolute quantities.

• Absolute zero.

13–9

EXHIBIT 13.4 Nominal, Ordinal, Interval, and Ratio Scales Provide Different Information

13–10

EXHIBIT 13.5 Facts About the Four Levels of Scales

13–11

Mathematical and Statistical Analysis of Scales

• Discrete Measures• Measures that can take on only one of a finite

number of values.

• Continuous Measures• Measures that reflect the intensity of a concept by

assigning values that can take on any value along some scale range.

13–12

Index Measures

• Attributes• Single characteristics or fundamental features that

pertain to an object, person, or issue.

• Index Measures• Assign a value based on how much of the concept

being measured is associated with an observation.• Indexes often are formed by putting several variables

together.

• Composite Measures• Assign a value to an observation based on a

mathematical derivation of multiple variables.

13–13

Computing Scale Values

• Summated Scale• A scale created by simply summing (adding together)

the response to each item making up the composite measure.

• Reverse Coding• Means that the value assigned for a response is

treated oppositely from the other items.

13–14

EXHIBIT 13.6 Computing a Composite Scale

13–15

Three Criteria for Good Measurement

13–16

SensitivitySensitivity

ReliabilityReliability ValidityValidity

Good Measurement

Good Measurement

Reliability

• Reliability• The degree to which measures are free from random

error and therefore yield consistent results.• An indicator of a measure’s internal consistency.

• Internal Consistency• Represents a measure’s homogeneity or the extent to

which each indicator of a concept converges on some common meaning.

• Measured by correlating scores on subsets of items making up a scale.

13–17

Internal Consistency

• Split-half Method• Assessing internal consistency by checking the

results of one-half of a set of scaled items against the results from the other half.

• Coefficient alpha (α)• The most commonly applied estimate of a multiple

item scale’s reliability.• Represents the average of all possible split-half

reliabilities for a construct.

13–18

Test-Retest Reliability

• Test-retest Method• Administering the same scale or measure to the same

respondents at two separate points in time to test for stability.

• Represents a measure’s repeatability.

• Problems:• The pre-measure, or first measure, may sensitize the

respondents and subsequently influence the results of the second measure.

• Time effects that produce changes in attitude or other maturation of the subjects.

13–19

Validity

• Validity• The accuracy of a measure or the extent to which a score

truthfully represents a concept.• Does a scale measure what was intended to be

measured?

• Establishing Validity:• Is there a consensus that the scale measures what it is

supposed to measure?• Does the measure correlate with other measures of the

same concept? • Does the behavior expected from the measure predict

actual observed behavior?

13–20

Validity (cont’d)

• Face Validity• A scale’s content logically appears to reflect what was intended

to be measured.

• Content Validity• The degree that a measure covers the breadth of the domain of

interest.

• Criterion Validity• The ability of a measure to correlate with other standard

measures of similar constructs or established criteria.

• Construct Validity• Exists when a measure reliably measures and truthfully

represents a unique concept.

13–21

Validity (cont’d)

• Convergent Validity• Another way of expressing internal consistency;

highly reliable scales contain convergent validity.

• Discriminant Validity• Represents how unique or distinct is a measure; a

scale should not correlate too highly with a measure of a different construct.

13–22

EXHIBIT 13.7 Reliability and Validity on Target

13–23

Sensitivity

• Sensitivity• A measurement instrument’s ability to accurately

measure variability in stimuli or responses.• Generally increased by adding more response points

or adding scale items.

13–24

Case 13.1

1. Both methods use consumer based data.

2. The survey method is more direct, while the weighted average method uses reported consumer complaints as a large part of the data base.

3. Surveying relies on the use of single “overall” rating to determine a rank order of airlines as to quality. The AQR produces a weighted average number that, when compared for each airline, can be treated as a continuously scaled variable.

4. The survey method examines perceptual, qualitative, more subjective aspects of the service experience. The weighted average method examines performance, quantitative, and more objective aspects of the service experience

13–25

Positive aspects of survey method:

+ Easily understood by management.

+ Represents broad based opinion of actual consumers of airline services.

+ Looks at the perceptual, subjective aspects of service quality.

 

Negative aspects of survey method:

- Cumbersome and costly to accomplish.

- Respondents lack of experience with all airlines taints the value of the opinion expressed.

- Most probably long time lags between survey efforts.

13–26

Positive aspects of weighted average method:

+ Because of standardized factors, the results are very comparable from airline to airline and from time period to time period.

+ Easily repeatable, uses inexpensive publicly available data, and is quick to accomplish.

+ Responds to consumer concerns via the inclusion of several customer complaint factors.

+ Looks at performance oriented factors that are more objective.

 

Negative aspects of weighted average method:

- Uses only limited data points.

- Performance data that is used is self-reported by the airlines under Federal mandates.

- Weights assigned to each factor can be questioned as to their true representativeness of consumer opinion.

- The direction of impact (+/-) for each factor can be argued as well.13–27