Mini Project I--Mini Project I--Evaluation of a Evaluation of a

Standardized TestStandardized Test

Mini Project I--Mini Project I--Evaluation of a Evaluation of a

Standardized TestStandardized Test

ByBy

Marcia LuebbeMarcia Luebbe

California Achievement Tests, Fifth Edition

• Published by CTBS Macmillan/McGraw Hill

• 1992• $90.30 for 30 third-graders, using

consumable books• Requires 330 minutes to administer

Description of Purpose and Nature of Test

• “Designed to measure achievement in the basic skills taught in schools throughout the nation”

• Applicable for Kindergarten through Twelfth grades

Content/Appropriateness• Basic skills K-12• Complete Battery: Reading, Language,

Spelling, Math, Study Skills, Science, Social Studies

• Basic Battery: Reading, Language, Spelling, Math, Study Skills

• Survey• Performance Component

Content/Appropriateness

• Review by Bruce G. Rogers, UNI, Cedar Falls, IA• “Well-developed achievement series whose

authors are responsive to suggestions for improvement

• “Suitable for schools that emphasize the ‘three R’s’ across the curriculum”

• Grade 9 assessment may have questionable appropriateness

• 12 to 25 constructed response items per subject

Content and Appropriateness

• Review by Victor Willson, Texas A&M, College Station, TX

• “As good as any norm-reference multi-level basic skills test”

• K-1 tends to be give somewhat poor results• “Good, interpretable test”

Technical Evaluation• Types:

– Objective Mastery– Grade Equivalent (also Grade Mean

Equivalent)– National Percentile (also Median National

Percentile)– Number-Correct Score– National Stanine (also Mean National

Stanine)– Scale Score (also Mean Scale Score)– Normal Curve Equivalent (also Mean

Normal Curve Equivalent)

Standardization Sample

• Spring 1991– 115,888 students– K-12, geographic groupings– 734 Schools responded to questionnaire– 261 Public Schools, 112

Private/Parochial Schools, all volunteer

• Fall 1991– 109,825– K-12– 265 Public Schools, 96 Private/Parochial

Schools

Factors Considered• Microcomputers in home• Average annual teaching salary• Mobility• Ethnic• ELL• Working parents• Single-parent homes• Number of children in family• %-age of students receiving ADC• Parent Jobs• Parent Education

Judges--Three Groups• Bias--41 reviewers, ethnic balance• Content--20 content and

curriculum specialists• Teachers--17 at appropriate grade

levels

Reliability• KR20• Split-half coefficients• Conclusion: .94 - .98 on total

battery, spring, fall, and winter

Validity

• Item-by-item bias• Ethnic

– Black– Hispanic– Other

• Gender• Results: “little evidence of biases”

Summary• Anthony J. Nitko, University of

Pittsburgh– Strong assessment tool– Be careful about educational

decisions for individuals– CHECK YOUR CURRICULUM FOR

ALIGNMENT. The CAT5 matches most 1980’s curriculums, changes may (and should) have been made to curriculum since 1980.

Summary• Robert F. McMorris, EdPsych/Stats,

State Univ. of New York, and others– “New-questions” sentiment– Great effort to follow curriculum– CTB provides terrific assistance– Fairly small number of items per

objective

Summary• Marcia Luebbe, Curriculum

Director and Educator, Pierce Public Schools– Good tool– Easy for teachers to interpret– Some concern with support personnel