mini project i-- evaluation of a standardized test by marcia luebbe
TRANSCRIPT
Mini Project I--Mini Project I--Evaluation of a Evaluation of a
Standardized TestStandardized Test
Mini Project I--Mini Project I--Evaluation of a Evaluation of a
Standardized TestStandardized Test
ByBy
Marcia LuebbeMarcia Luebbe
California Achievement Tests, Fifth Edition
• Published by CTBS Macmillan/McGraw Hill
• 1992• $90.30 for 30 third-graders, using
consumable books• Requires 330 minutes to administer
Description of Purpose and Nature of Test
• “Designed to measure achievement in the basic skills taught in schools throughout the nation”
• Applicable for Kindergarten through Twelfth grades
Content/Appropriateness• Basic skills K-12• Complete Battery: Reading, Language,
Spelling, Math, Study Skills, Science, Social Studies
• Basic Battery: Reading, Language, Spelling, Math, Study Skills
• Survey• Performance Component
Content/Appropriateness
• Review by Bruce G. Rogers, UNI, Cedar Falls, IA• “Well-developed achievement series whose
authors are responsive to suggestions for improvement
• “Suitable for schools that emphasize the ‘three R’s’ across the curriculum”
• Grade 9 assessment may have questionable appropriateness
• 12 to 25 constructed response items per subject
Content and Appropriateness
• Review by Victor Willson, Texas A&M, College Station, TX
• “As good as any norm-reference multi-level basic skills test”
• K-1 tends to be give somewhat poor results• “Good, interpretable test”
Technical Evaluation• Types:
– Objective Mastery– Grade Equivalent (also Grade Mean
Equivalent)– National Percentile (also Median National
Percentile)– Number-Correct Score– National Stanine (also Mean National
Stanine)– Scale Score (also Mean Scale Score)– Normal Curve Equivalent (also Mean
Normal Curve Equivalent)
Standardization Sample
• Spring 1991– 115,888 students– K-12, geographic groupings– 734 Schools responded to questionnaire– 261 Public Schools, 112
Private/Parochial Schools, all volunteer
• Fall 1991– 109,825– K-12– 265 Public Schools, 96 Private/Parochial
Schools
Factors Considered• Microcomputers in home• Average annual teaching salary• Mobility• Ethnic• ELL• Working parents• Single-parent homes• Number of children in family• %-age of students receiving ADC• Parent Jobs• Parent Education
Judges--Three Groups• Bias--41 reviewers, ethnic balance• Content--20 content and
curriculum specialists• Teachers--17 at appropriate grade
levels
Reliability• KR20• Split-half coefficients• Conclusion: .94 - .98 on total
battery, spring, fall, and winter
Validity
• Item-by-item bias• Ethnic
– Black– Hispanic– Other
• Gender• Results: “little evidence of biases”
Summary• Anthony J. Nitko, University of
Pittsburgh– Strong assessment tool– Be careful about educational
decisions for individuals– CHECK YOUR CURRICULUM FOR
ALIGNMENT. The CAT5 matches most 1980’s curriculums, changes may (and should) have been made to curriculum since 1980.
Summary• Robert F. McMorris, EdPsych/Stats,
State Univ. of New York, and others– “New-questions” sentiment– Great effort to follow curriculum– CTB provides terrific assistance– Fairly small number of items per
objective
Summary• Marcia Luebbe, Curriculum
Director and Educator, Pierce Public Schools– Good tool– Easy for teachers to interpret– Some concern with support personnel