reliability presented by: mary markowski, stu ziaks, jules morozova
Post on 28-Dec-2015
216 Views
Preview:
TRANSCRIPT
ReliabilityReliability
Presented By: Mary Presented By: Mary Markowski, Stu Ziaks, Markowski, Stu Ziaks,
Jules MorozovaJules Morozova
IntroductionIntroduction Reliability, what does it mean to you?Reliability, what does it mean to you? Reliability = consistencyReliability = consistency Ways reliability affects everyday teaching:Ways reliability affects everyday teaching:
Classroom assessments may lack sufficient reliabilityClassroom assessments may lack sufficient reliability Parents may want to discuss adequacy of standardized Parents may want to discuss adequacy of standardized
teststests Important for teachers to realize a given child’s test Important for teachers to realize a given child’s test
performances will varyperformances will vary
Due to variation and subjectivity of scoring, Due to variation and subjectivity of scoring, individual score and average group score individual score and average group score always reflect x amount of measurement always reflect x amount of measurement errorerror
3 Types of Reliability3 Types of Reliability
1. Stability reliability (test re-test)1. Stability reliability (test re-test) Consistency of test results over timeConsistency of test results over time
2. Alternate form reliability 2. Alternate form reliability Two points of testing with same test, over timeTwo points of testing with same test, over time
3. Internal consistency reliability3. Internal consistency reliability Consistency of function of items in an Consistency of function of items in an
assessmentassessment These forms of reliability are NOT These forms of reliability are NOT
interchangeable!interchangeable!
Stability ReliabilityStability Reliability Consistency of test results over timeConsistency of test results over time Aka: Test Re-test reliabilityAka: Test Re-test reliability Test students with same assessment as two Test students with same assessment as two
different points in timedifferent points in time Must make sure no significant relative information Must make sure no significant relative information
is introduced to students that would affect re-testis introduced to students that would affect re-test Find correlation coefficient between Find correlation coefficient between
individual student scores on both tests (r= individual student scores on both tests (r= 1.0, -1.0)1.0, -1.0) 0 meaning no correlation, 1 being perfect 0 meaning no correlation, 1 being perfect
correlation, -1 meaning opposite relationshipcorrelation, -1 meaning opposite relationship
Stability ReliabilityStability Reliability
Key point for teachers: always a Key point for teachers: always a level of instability between level of instability between students’ performances on two students’ performances on two different testing occasions, even different testing occasions, even with same testwith same test Confounding variables: mood, health, Confounding variables: mood, health,
personal situations, of students, can personal situations, of students, can affect performanceaffect performance
Alternate-Form ReliabilityAlternate-Form Reliability Assess whether two or more ‘allegedly’ Assess whether two or more ‘allegedly’
equivalent assessments are in fact equivalent.equivalent assessments are in fact equivalent. Introduced when teachers must use different Introduced when teachers must use different
two forms of a test (ie: GED, GRE, SAT)two forms of a test (ie: GED, GRE, SAT) Teachers have little need for this type, yet Teachers have little need for this type, yet
should know about the consistency of should know about the consistency of alternate examination forms used with their alternate examination forms used with their students outside of classstudents outside of class
Evidence of this consistency can be gathered Evidence of this consistency can be gathered by correlation coefficients of scores of an by correlation coefficients of scores of an individual taking both testsindividual taking both tests
Internal Consistency Internal Consistency ReliabilityReliability
Different from other formsDifferent from other forms Does NOT focus on consistency of student scoresDoes NOT focus on consistency of student scores Focuses on extent to which Focuses on extent to which items items in an in an
assessment are functioning consistentlyassessment are functioning consistently More common form of reliability (one test)More common form of reliability (one test) If test items are all truly measuring “math If test items are all truly measuring “math
achievement”, each test item = equalachievement”, each test item = equal Ie: problem solving test of 20 questions: skilled Ie: problem solving test of 20 questions: skilled
should get most correct, unskilled lessshould get most correct, unskilled less More homogeny in responses obtained, More homogeny in responses obtained,
higher internal consistencyhigher internal consistency
Internal Consistency Internal Consistency ReliabilityReliability
Two way to obtain:Two way to obtain: 1. Kuder-Richardson (K-R) for right vs. 1. Kuder-Richardson (K-R) for right vs.
wrong assessments (m.c.)wrong assessments (m.c.) 2. Cronbach’s alpha for subjective 2. Cronbach’s alpha for subjective
assessments (essay)assessments (essay) More items on assessment, more More items on assessment, more
reliable it tends to bereliable it tends to be Like dipping into soupLike dipping into soup
Teachers: Remember About Teachers: Remember About ReliabilityReliability
3 types NOT interchangeable3 types NOT interchangeable Stability reliability= Alternate Form Stability reliability= Alternate Form
= Internal consistency= Internal consistency
Standard Error of Standard Error of MeasurementMeasurement
Deals with individual performanceDeals with individual performance The consistency of an individual’s The consistency of an individual’s
performanceperformance Need to estimate this variability, like +/- Need to estimate this variability, like +/-
error in pollserror in polls Inverse relationship between reliability and Inverse relationship between reliability and
standard errorstandard error Standard Error reminds teachers of Standard Error reminds teachers of
imprecision of scores earned by students on imprecision of scores earned by students on commercial or classroom tests are NOT all commercial or classroom tests are NOT all that exactthat exact
What Teachers Really What Teachers Really Need to KnowNeed to Know
Reliability is central concept in Reliability is central concept in measurement and assessmentmeasurement and assessment
If assessment fails to yield consistent If assessment fails to yield consistent results, impossible to make accurate results, impossible to make accurate inferences about student scoresinferences about student scores
The more important the test, the more The more important the test, the more important the reliabilityimportant the reliability
Knowledge of reliability key to Knowledge of reliability key to understanding of assessment as well as understanding of assessment as well as explanation of assessments to parents, explanation of assessments to parents, educators, and communityeducators, and community
top related