ReliabilityReliability

Presented By: Mary Presented By: Mary Markowski, Stu Ziaks, Markowski, Stu Ziaks,

Jules MorozovaJules Morozova

IntroductionIntroduction Reliability, what does it mean to you?Reliability, what does it mean to you? Reliability = consistencyReliability = consistency Ways reliability affects everyday teaching:Ways reliability affects everyday teaching:

Classroom assessments may lack sufficient reliabilityClassroom assessments may lack sufficient reliability Parents may want to discuss adequacy of standardized Parents may want to discuss adequacy of standardized

teststests Important for teachers to realize a given child’s test Important for teachers to realize a given child’s test

performances will varyperformances will vary

Due to variation and subjectivity of scoring, Due to variation and subjectivity of scoring, individual score and average group score individual score and average group score always reflect x amount of measurement always reflect x amount of measurement errorerror

3 Types of Reliability3 Types of Reliability

1. Stability reliability (test re-test)1. Stability reliability (test re-test) Consistency of test results over timeConsistency of test results over time

2. Alternate form reliability 2. Alternate form reliability Two points of testing with same test, over timeTwo points of testing with same test, over time

3. Internal consistency reliability3. Internal consistency reliability Consistency of function of items in an Consistency of function of items in an

assessmentassessment These forms of reliability are NOT These forms of reliability are NOT

interchangeable!interchangeable!

Stability ReliabilityStability Reliability Consistency of test results over timeConsistency of test results over time Aka: Test Re-test reliabilityAka: Test Re-test reliability Test students with same assessment as two Test students with same assessment as two

different points in timedifferent points in time Must make sure no significant relative information Must make sure no significant relative information

is introduced to students that would affect re-testis introduced to students that would affect re-test Find correlation coefficient between Find correlation coefficient between

individual student scores on both tests (r= individual student scores on both tests (r= 1.0, -1.0)1.0, -1.0) 0 meaning no correlation, 1 being perfect 0 meaning no correlation, 1 being perfect

correlation, -1 meaning opposite relationshipcorrelation, -1 meaning opposite relationship

Stability ReliabilityStability Reliability

Key point for teachers: always a Key point for teachers: always a level of instability between level of instability between students’ performances on two students’ performances on two different testing occasions, even different testing occasions, even with same testwith same test Confounding variables: mood, health, Confounding variables: mood, health,

personal situations, of students, can personal situations, of students, can affect performanceaffect performance

Alternate-Form ReliabilityAlternate-Form Reliability Assess whether two or more ‘allegedly’ Assess whether two or more ‘allegedly’

equivalent assessments are in fact equivalent.equivalent assessments are in fact equivalent. Introduced when teachers must use different Introduced when teachers must use different

two forms of a test (ie: GED, GRE, SAT)two forms of a test (ie: GED, GRE, SAT) Teachers have little need for this type, yet Teachers have little need for this type, yet

should know about the consistency of should know about the consistency of alternate examination forms used with their alternate examination forms used with their students outside of classstudents outside of class

Evidence of this consistency can be gathered Evidence of this consistency can be gathered by correlation coefficients of scores of an by correlation coefficients of scores of an individual taking both testsindividual taking both tests

Internal Consistency Internal Consistency ReliabilityReliability

Different from other formsDifferent from other forms Does NOT focus on consistency of student scoresDoes NOT focus on consistency of student scores Focuses on extent to which Focuses on extent to which items items in an in an

assessment are functioning consistentlyassessment are functioning consistently More common form of reliability (one test)More common form of reliability (one test) If test items are all truly measuring “math If test items are all truly measuring “math

achievement”, each test item = equalachievement”, each test item = equal Ie: problem solving test of 20 questions: skilled Ie: problem solving test of 20 questions: skilled

should get most correct, unskilled lessshould get most correct, unskilled less More homogeny in responses obtained, More homogeny in responses obtained,

higher internal consistencyhigher internal consistency

Internal Consistency Internal Consistency ReliabilityReliability

Two way to obtain:Two way to obtain: 1. Kuder-Richardson (K-R) for right vs. 1. Kuder-Richardson (K-R) for right vs.

wrong assessments (m.c.)wrong assessments (m.c.) 2. Cronbach’s alpha for subjective 2. Cronbach’s alpha for subjective

assessments (essay)assessments (essay) More items on assessment, more More items on assessment, more

reliable it tends to bereliable it tends to be Like dipping into soupLike dipping into soup

Teachers: Remember About Teachers: Remember About ReliabilityReliability

3 types NOT interchangeable3 types NOT interchangeable Stability reliability= Alternate Form Stability reliability= Alternate Form

= Internal consistency= Internal consistency

Standard Error of Standard Error of MeasurementMeasurement

Deals with individual performanceDeals with individual performance The consistency of an individual’s The consistency of an individual’s

performanceperformance Need to estimate this variability, like +/- Need to estimate this variability, like +/-

error in pollserror in polls Inverse relationship between reliability and Inverse relationship between reliability and

standard errorstandard error Standard Error reminds teachers of Standard Error reminds teachers of

imprecision of scores earned by students on imprecision of scores earned by students on commercial or classroom tests are NOT all commercial or classroom tests are NOT all that exactthat exact

What Teachers Really What Teachers Really Need to KnowNeed to Know

Reliability is central concept in Reliability is central concept in measurement and assessmentmeasurement and assessment

If assessment fails to yield consistent If assessment fails to yield consistent results, impossible to make accurate results, impossible to make accurate inferences about student scoresinferences about student scores

The more important the test, the more The more important the test, the more important the reliabilityimportant the reliability

Knowledge of reliability key to Knowledge of reliability key to understanding of assessment as well as understanding of assessment as well as explanation of assessments to parents, explanation of assessments to parents, educators, and communityeducators, and community