3YID, 148,-889

TITLE

INSTITUTION cPUB DATEROTE

,AVAILABS.E FROM

c

EDRS PRICEDESCRIPTOA

,

IDENTIFIERS,

ABSTRACTDisc signs of any test must focus on reliability,

content validity, anWinterpretation of results, In this critique ofOn Fuxther Examination--the report of the'AdvisOfi Panel on the°ScholaStic Aptitude Teit -(SAT) Store Decline- -the National EducationAssociation (NEA) 'finds no fault with the reliability of the SAT. Itscontent validity, however, is suspect. The NEA itaittains that the SATis really'in achievement test, and gudstions its relevance io'highmchoorcurricula. The SAT is constructed primarily bypsychometricians and psychologiSts who may not 14' sufficientlyfamiliar With_ secondary curricula. to develop a test with-cgntentvalidity. The College Entrance.Einsination Board Achievement Tests,which draw heavily on 'the expettise ofteathers for content, do notparallel the SAT score decline: The Panel'iso,criticized'for

, ,

dismissing the issue of SAT bias by noting' :that racial differencesoccur on cost ,standardized tests. Questions are raised 'about thedesirability of an unchanging standard and about what- a predictive -.test should measure. From the teachers' perspeCtive, the Skescoredecliie -raises 'important issues 'concerning: the quality of .

linstruction, innovative teaching Method's, the back-to-basicstiovement, literacy, automatic prqpotion, exit examinations at the endof high school, reduction in homework,riwiividual'diffirenceslearning and -maturation, standardized ,teacher examinations, an.dtea her organizations. (EVH)

DOCIMENT 411011E. .

t. -TI 006 8454,

: .

On,)Further Examination of_ "On filither'Examination..,QVick Referenc: Testing/SAT. .'National Education Association, Washington, D.C.,77

32p.'; For related documents see TM 006 750-7761 Forcomplete report, see, TM 006 516Instruction and Professional Development InformationCenter, National Education Association, 1201:16th-Street N.W., Washington, D.C. 20036 ($0.50)

MF-$0.83 Plus Postage.. HC Not'Available from EDRS:Achie'Vemeni Tests; College Bouna Students; *CollegeEntrance Exami'nation's; Educational Problemi;Educational Trends; Performance Factors; PredictiveAbility (Testing); *Scores; Secondary. EAucatiOn;*Teacher.Associatibns; *Teacher 'Attitudes; Test Bias;Testing Problems;. Test Reliability; *Test ValidityNational Edieation Asiociationl *Scholastic Aptitude

,

Test; *Teit Score Decline

t

4

Documents acquired by ERIC include many infornial unpublished materials not available from other sources. ERIC ,makes every.effort, to obtain the best copy available..Neverthetess, items of marginal reproducibility are often encountered and this affects thequality of the inicrofiche and ,4ardcopy reproductions ERIC makes available via the ERIC Document Reproduction Service (EDRS).EDRS is not responsible for the quality of the original document. Reproductions supplied by EDRS are the best that dan be made fromthe original.

QuickReference:..

TESTING/SAT

On `Further'4-Examinatiottof"qn-Tuitherm D.

4-Examination.'9;U4.0EPARTMENT OF HEALTH,

EDUCATION & WELFARE ,NATIONAL INSTITUTE OF 4. .1

EDUCATION

'HIS DOCUMENT HAS. BEEN REPRO.DUCED EXACTLY AS RECEIVED FROMTHE PERSON OR ORGANIZATION ORIGINATING IT POINTS OF VIEW OR OPINIONSSTATED DO NOT NECESSARILY REPRE._SENT OFFICIAL NATIONAL INSTITUTE OFEDUCATION,POSITION OR POLICY

"PERMISSION TO REPRODUCE THISMATERIAL IN MICROFICHE ONLYHAS BEEN GRANTED BY

-C' NA-ioN 41 e-ix)0 il-r i NU -A.-,So0 04-1-10/0

(:::) TO THE EDUCATIONAL RESOURCESINFORMATION CENTER (ERIC) ANDUSE RSOF THE ERIC SYSTEM."

1".".". t

'A National Educatton Pubikation

.4

Copyright,0 1977National 1.clucation Assuctation of the,Linited States

, -

On 'Further'Examinatio4o1"On GFurtherExaminatime

This paper was written as one NSAresponse to the Report of te, Advi-story Panel on the Scholastiv AptitudeTest Score Decline. We ex nd our ap- .e'predation ,lo Roger W . Shuy andNancy M. Yanofsky, f the Centerfor Applied Lingu us, who wereconsultants to the taff on & prepara-tion. The Ass tion's initial responseto the panel eport came from Presi-dent John yor, whose editorial fromthe Nove ber-Receniber 1977 issue ofToday's duration is reprinted in thisbookie

T e-report of the Advisory Panelhe Scholastic Aptitude Test Score

D line has been entitled On Further.amination. This title is appropriate

n two ways. First; the panel has re=ported to the public the results of itsextensive "further examination" ofthe problem. Second, the examinationis by no means finished. Many' ques-tions remahl unanswered, questionsthat warrantand demand"furtherexamination."

The panel,is to be commended foits lack of indictment. Many questionswere, raised, many analyses under-taken, many theories entertainecr,^any discussionsitheld, ,dome expertsconsulted; but even with all of theabove, no single reason emerged asTHE REASON ftr the test score decline.The report reflects oh why. the Storeshave declined but by no means ex-plains all, the reasons for this decline.

4

I

"'T

447

The College Entrance ExaminationBoard (CEEB) and the EducationalTesting Service (E\TS) convened theadvisory pat* two years ago. Itscharge was to examine the ScholasticAptitude `Test (SAT) score declineover the last 14 years. More specifical-ly, the panel was charged with deter-mining the reason(s) for the 49-pointdrop in the Verbal SAT scoies and the32-point drop in the 'MathematicalSAT scores.

As-the panel's inquiry, progressed, itbecame quite clear that it would notbe able to identify or explain all thereasons foi the Reclining test scores.It has done, however, a reasonablejob of presenting its findings aboutwhat has 'happened per the last 14years. It has reflected on the 'SATitself, other standardized tests, thepopulations taking the tests, educa-tional changes in school. practices andstaffing ,patterns,, and a variety ofsocial forces.

In attempting to discuss why -thetest scores have declined, the panelstates, "Fotirteen years of uknter-fupted decline in the SAT sorescreate the illusion that there is somesingle force or closely related set offorces at work here. This isn't thecase. The'decline has developed in twodistinct gages..." (p. 13).

The first stage of the decline tookplace ,betwfen 1963 and 1970 and isdescribed as the period of "composi-tional change." During this time anincreasing number Of students becametest takers apd thus potential collegestudents. Thts larger group includedmore women, more minorities, mote

'representatives of all socioeconomic.leimls, and high school students with /lower .acadernic ranking. This period /

also marked the beginning of manymore opporttunities for students to goto college, which can be attributed tofactors such as the advent of openadmissions, the establishment of two-year colleges,'and increased scholar-ship support for minorities.

The test score'` decline .cannot beattributed solely to this compositionalchange. in the test-taking group, how-ever. The panel has emphasized thatother factors in schools aed'in societymust also be considered in order tounderstand the decline., Hence, the"-second stage, from. 1970 to 1977, isregarded as the period of '''pervasivechange," -a period characterized byuncertainty' and unrest ip our society.The remainder of this sMtement willaddress itself to the panel's analysis ofwhat has contributed t6,, "pervasive'change." The panel report' notes thatthis change has a tendency to becomeso elusive that a "now you seeit,.nnwyou don't" phenomenon often occurs."So much has happened'" that

. have affected' this record that thenp way of 'telling-what did; the onlyevidence is circumstantial, leavinghard to distinguish cause from coin-cidence" (p. 25). .

In contrast with 'these carefullyqualified ,firidings,. the panel reportwas interpreted, rather loosely, by thepress. A representative sample ofnewss_articles, editorials, and letters tothe editor reveals that the, public hasnot been given a..aceurate picture of

the report. The Washington Star, forexample, editorialized, that the panel.was "ahhost too judicious in itsnuances, -distinctions and imponder-

.ables to account for the *ling off ofscores." Perhaps the press is the bodyreferred tot:in the report itself when.the .panel.tiotes that tliree false as-

,

in.

ef

6

sumptions underline most theories forSAT score declines, the first of whichis that there is.a simple, clear answerto the problem. The ,,Slar takes theNEA President to task for his criti-

. cism s f the SAT, observing that ifteachers develop. their own tests, itwould b "a classic case of the in-mates ru ning the asylum." In thatthe panel report does not implicateteachers in any major way, such flam-boyant verbiage by the ,star, is com-pletely uncalled for.

Education, as every teacher..knows,suffers today from the attitudes of'thegeneral public, which thinks it knowsall there is to, know about 'education,.that special t ought; study, or eXperi-.ence with edu tioa is. not needed toqualify one to make grandiose- state-ments abo education: .Sinceeducators appear, to be under siege,careless interpratation of an importantreport such as On Further Examina-tion can be damaging to -the profes,sion: It is 'fOr this reason- that thispaper has been prepared.

On Further Examination concludeswith the observation that readers whoturn to the summary of the report fora quick and easy picture of the panel'sview are themselves guilty of the sortof practice which may have contrib-uted to SAT score decline's. This paperis nt a substitute for the report itself,which is a well7written.andimportantdocument.* Ratherr,it will analyze;

- .' comment °ill and set the report in thecontext of current American educa-tional practice. At will note in particu-lar _what the panel has to say aboutthe SAT,. about teaching, and aboutselected aspects of our society. It will

On FurThe; Exammaln, Beim' of the AlyisorY Panel onthe Scholastic Aptitude Test Score Decline, may be orderedfrom College Board 'Publications Orders, fox 2815. Prince-ton, NJ 06540. The price is S4.

7

v

'''''."'S:I'q

, .

\

..

I. conclulle.witIkt brief disc ssion about

the future,-Including a set of questionsfor even ".fliither exatnin tion."

. 1 About the S T

In order to, adequat y discuss any,test, its is important t focus on thefollowing f three issue reliability ofmeasure dent, conte validity, andinterpreta ion of resu ts. Statedanoth-er way, the construct on of any testinginstrument must co sist.of three com-ponents: /what Is' asured, how, it ismeasured,. and wh t the Tesults mean.

1. Reliability

The panel h great admiration forthe reliability f the SAT. Reliabilityin resting is, the 'degree of consistencybetween two pleasures of the samething. Assuring reliability is probablywhat the CEEB and iliegTS do best.

test tAktestsexample, the SAT tes compafea

s capacities with those of hispeers .and with students who took thetest before. The test is also correlatedwith the student's grades in high school

it is claimed, 'is adtually'iniprov-ing asp a predictor of successin thefirst year of college: LikeWise, dif-ferent \forms of the test can be mea-sured against each other for reliability.From this, the panel concludes that "

o the. SAT -is: not out of line with teach-ing practice in the secondary schools.

The NBA ha; no question about theefficienC34 and integrity of the'ETS todo what it does best. And the panel isno oubt correct in its admiration forth nttention.to reliability. which theS' T contains. But tttis is not re.41154-sqing very much because any test Canbel technically reliable and still totallyfail; to measure what it intends tomeasure. The attractiveness of relia-bility is attached to the fact that it can

be quantified, thus giving the air ofscientific respectability.

2\ Validity .

e panel report deals very littlewith \flie issue of content validity de-spite \he fact that this is the centralissue to, thok involved in standardizedtesting today. Recent federal hearingsrelated to the language asstssmqnt ofbilingual children in New York, Chi-cago, Philadelphia,' and many otherschool systems have stressed this basic .

issue: "Does the test measure what. itsays it measures?" Singe the SATclaims not to be an achievement test..but rather a predic6e measure-for-success-in-college , instrument, contentvalidity becomes a cloudy notion. TheSAT is ,,called an intelligence test bysome, since the difference lietWeenmeasurements of intelligence andmeasurements of aptitude are slight atbest. The basic (and relatively minor)difference between intelligence andaptitude tests is that, aptitude teatsmeasure specific factors rather thangenera) ones while -intelligence testsmeasure general factors rather thanspecific ones.

Thus the-SAT measures verbal andmathematical factors rather than alltfie thing one might want' to call iv-telligence (provided we could severagree on whaLthey are). During the 0history of, the development, of intelli-gence tests% the term intelligence wasthought 'to be unchanging and innate.To avoid the implications, of innate':ness, test developers have Used the teamaptitude. Since these tests are usedprimarily to predict fchool success,the term scholastic .was added: One .

could make the case that ithe SAT isreally an intelligence test and that, fromwhat little is known about, intelligence

J

7 9.

A

or how to measure it, the SAT is in-valid. If one Stakes the Position. thatthe SAT is an aptitude test, however,\\the test is subject to analysis of whysuch group aptitude .tests predict .

\ school achievement. S

In theory it is possible to distin-guish between aptitude and achieve-ment,.One is an innate property andthe o \her is accomplishment. Oncecontent validity hak been determined,it ispossible to measure 'achievement.How jo get at aptitude, however, isnot so clear. The SAT selected themathematical and verbal areas as alocus for measurement. Exactly howthis decision -separates achievementpossibilities from aptitude is still un-clear. All children have knowledge,memories, and feelings about the sub-'jects behig measured. Furthermore,distinguishing ,amotraVAT verbalaptitude items and those 'items in ver-bal abilities on achievement tests isvirtually impossible. Distinguishingbetween aptitude and achievement,though possible in Itheory, is impos-sible in practice. What is more, abilityappears to increase as achievementlevel rises. Why is it, then, that groupaptitudo4eVoin predict achievement?It is 'highly probable,that they arereally achievement tests..

It has been necessary to foflow'.thisreasoning in order to address the issueof content validity. Once verbal achieve-

r -mint and mathematical achievementare recognized, as integral to any as-sessmenrof the SAT, we mustask thequestion, How do we know that what.we are measuring really rnattprs? Thepantl did not address thiVquestion,although to its credit it did-recognizeit as ,a legitimate one: "Wvf have ac-cepted, for purposes of this inquiry

10

1....0-9,and report, t e traditional value baseof the SAT its validity..." (p. 11).

Perhaps one of tIle most significantstatements of the {panel repoit grewout 61' a discussion of why scores onthe Achievemerit Tests (which mayalso be elected by SAT takers) did notparallel SAT, score declines. Scores ontests in English Composition, French,Spanish,' Biology, Chemistry, andPhysici have increased while other sub-jects reveal only small declines. Stu-dents whose scores went up in .these.'are showed 'cidelines in'SAT ,verbal'scores and increases,in SAT , mathe-matics scores. The panel lamented' thatit had not beenable to analyze thisphenomenon fully and'concluded, "It..is conceivably important thatst.he Col-lege Board and ETS make much largeruse 'of gutside committees in connec-

-doll with the Achievement Tests thanwith t)te SAT; the counsel sought is inthe one case from experts in the partic-ular diiciplines, in the other more frbmpsychometricians and psychologists"(p. 23; italics-added).

This criticism' of the SAT, thoughcautiods and. indirect; appears to beOne of the central issues in accountingfor the score decline. The SAT is ac-tually a disgUised achievement testwhich, in.order to be Valid, musebet.....

. ter address what really matters in thestwo content fields which it me ures.

Like most Objective tests, it 1 a dis-crete point measure. That is, certainiteths are selected from an inventoryof all' pogibleknowledge in order torepreient that Jarger knowledge in as .few questions as possible4 Only-thosewho know the content of the gelds \

being measured have the key to con-tent validity. Test makers create itenis 71

to test only after content validity has ?-liten established. .

0

3. Cultural Bias

The panel the 'charges of, Cultural bias by observing that a defin-

itive arialysis of cultural bias is virtu-. ally 'impossible. It appears to the NEA

that definitive analysis of cultural biasis o more difficult than definitiveanalysis of mrly, oth'er. educationalconcepts. Test ,questions_of any sortinvolve matching the assumptions,`values, and presuppositions of the testdeicer against those of the test taker.An understanding of basic principles ..

of language and culture-make the '. identification of potential cultural

,.* bias 'available to those who care tofind it. However, even publishers pf 4

standardized rests have publicly, ad-mitted cultural bias and some havemade serious-efforts to combat It. BythN we do net accuse the SAT of suchbias but, rather, disagree with thepanel about its possible identification.Nor do we understand what the panel'might mean when it/Observes, "T'hesesameklirference§ show up on most- .

other standardized tests, and yet this,proves nothing" (p. 16). Such an ob-servation follows the logic of saying ....;

that because everyone steals, stealingis all right. The panel is quite right,however,, when'it indicts society as awhole for such bias. Having said this,it is odd to conclude that only the

-SAT has been' preserved from. such ,wbias.

. -About Teaching

In an attempt to further examinethe "peivasive change" contributingto test score decline, the panel reportdiscusses ten issues which directly con-

, cern schools and teachers. Froth theteacher' perspective, the reported de-clintrin SAT scores reveals a numberof important issues.

12

1

1. Quality of Instruction

The panel clearly recognizes thatthe primary intent of the SAT is topredict success in the first year of col-,lege. Furthermore, the panel expliCitlynotes that the SAT is not intended toreflect the adeqtacy of the student'straining. Thus ttie.report does notatithorize or condone criticism of schoolsor individual teachers on 'the-basis ofSAT scores (p. 5). No evidence hasbeen presented that would lead to theconclusion that the quality of schoolinstruction.is what is being measured

' on the SAT (p. W. The 'report doesnot deal directly with the.' issue ofquality of instruction. :43

2., Innovative Teaching

';The panel Ades not Warne:innova-tive teaching for the decline. in SATscores. It observes, in ,tar.,4"We findno evidence of any causal relation-ship betwjeen what are commonlyreferred to as 'experimental teachingmethodsi and the SAT score decline"(p. 41). The panel carefully examinedthe results of two different investiga-tions into this topic and/notes thatthere is neither consistent nor sbbstan-tial association between student achieve-ment and overall level of innovationacross the grades. It actually advocatesthe search for new ways of addresjingthe learning needs of children: "Wedo not read the SAT score 'decline asan Lnstruction that eckication in thiscountry must or should be more rigid,more seleCtive, more rejective, more **

uniform. Instead, the instruction isthat education, 'especially secondaryeducation, must.become still more di-

:r- versified, more variedbut withoutbeing watered down" (p. 31). Thepanel's insistence that innovation not

cpme to d grinding halt is encouraging,and highlyllesirable. ,

3.* hack to Basics

The panel does not wish their reportor-the SAT score declines to be thought.to suggest a Itack-to-basics movement:"In our view; 'returning to the basics'would be wrong unless it included, fullreappraisal' of what the right basicsare-taking account of children's dif-ferent rates and modes of learningand their different interests-and plansfor the future. The need is not to re-vert to uniform drills and exery.6ise

(commended only by a traditional ped-- agogy., but to move ahead to a largeremphasis .on the fundamentals of

' lerning that can be identified asstrengthening the base on which allstudents can build" (pp. 26-27)r Thereport doeknot suggest a return to the

.,,""good old ways," but it cltes encour-age a thorough reexamination of whatthe basics really are.

4. LiteracyThe closest the panel comes criti-

cizing the schools occurs in what the,Vane! calls its "firmest- conclusion":that thoughtful and critical reading isnot being demanded and carried outt,and that careful writing apparentlyhas about gpne out of style. Althoughthe panel 'Makes tbis observationrather forceful the( fault for-the sit-uation is not laid solely at the feet ofteachers. The panel argues, in contrast,that the testing industry itself, which.introduced quantifiable data, even inthe liberal arts, has contributed to thesituation which the declining SATscores illustrate, The NEA is in gen-.eral agreement with the panel that read-ing and writing are critical skills which.

14

t

.. .

S../educators should address with addedzesand knowledge: 1.

.5. Automatic Pr_9mdtione

Although automatic promotion isthought to be a contributing. factor inSAT score declines, teacheis are not'held totally responsible for this in thepanel report. The report says, id fact,that teachers do 'not : lik\t this ideaeither. Thus the panel disagrees with.those critics of the schools who ob-serve that automatic promotion is theinvention and fault of teachers. The,mangy factors which have'conspired tolead to automatic promotion,(many of .

which reflect the goals of-society as' awhole) must be equally addressed, Itis interesting to note that the paneldid not take into consideritiOn theavailable research on late retention,.much of which indicates'that retaininga child can also haVe negative effectson learning.

...

..; 'k;5. Exit Examinations ...,, ,.

The panel flatly denies that evidencefrom SAT score declines suggests theneed for the development of so-calledexit examinations at or'near the endof high school: "...we question' anequally 'automatic' answer that suchpromotion-should depend entirely onscores on still othef standardized tests" _(p. 30 The underlying assumptionsof either minimal competency exami-nations or exit examinations are ques-tionable at best. Once we establishminimal competeicies we tend' to getjust thatminimal competence: Onewould hope for considerably, morethan this To adopt yet another singlemeasure of "competencey appearsboth unfounded and unnecessary. ,

\ %.

1 '

, =15=.,,

o sr

7. Reduction in Homework -f,The question of reduced homework

was not discussed in any depth by eqe

panel but it is interesting to not thatthe responsibility 'for the pre l tiltedreduction in homework is not /placedsolely on teachers. Although eke panelobserves that SAT score declines seemto be related to this factor, ',it alsoconcludes thit it is impossible,to tell ,

whether reductions in t-,siech assign-ments are parent or teacher' induced.In fact, the panel notes that there isno quantifiable evidence that home-work has in fact been reduced.

:

8. Individual Diffetences ,,

t- The. crtic41.issue of individual dif-'ferences wai not seriously addressedin the report. bifferencei iiilearningcurves and maturation were givenonly cursory - attention. One wouldlike to assunie4hat the failure, todis-

.

cuss this critical- issue was an over-sight, but perhaps-it was merely re-flecting the state of the art in the fieldof educationafresearch. From the per.;spective of the teacher, however, it isapparent that two students of :equalaptitude may be in a different learninggrowth position at the point at whichthe SA, Is taken, grossly differentiat-inging th ;scores. Sikh conditions shg-gest only the ob,f/ious:,-that research be 0undertaken to see how -well the SATpredicts second, third, and final yeariof college and beyond. The fact thattheAAT predicts only the first yearseems very suspicious; for that year is_a ,major" transition year for youngpeople: It should be noted -41.1aet thepanel did little more than casually'mention the'important matter of indi-vidual differences, an issue that de-served careful attention.

In

9. Standardized Teacher Examinations

The phel argues against reliande onstandardized twher egamfnations to

-insure that schools have only qualifiedteachers. Despite the fact that approx.,imately half of the state legislaturetare considering legislation to require .0"

such exams as a condition to the ,hif-ing of elementary and secondary

e

teachers, the panel concludes, "Our .increased understanding of the work-ings of standardized student examina-tions

.0

in such situations does nokhowevera, commend exclusive relianceon the use of comparable examina-

r

lions for teachers" (p. 33).' Thus, thepanel observes, the declining SATscores should not be used as evidencefor the movement toward or foi legis-lating teacher examinations.

10, 'Teacher Organizations

The panel in no way accepts the-suggestion that teacher organizationssuch .as the 'NEA., or developments,such as collective bargaining ..havotcontributed to the decline' in SAT 'scopes. It states, in fact; that thesesuggestions "offee-little in themselves TN.as far.--as explaining :the decline in thetest scores is concerned [and] ,weimply nothing ,one way or the otherhere abotitAhe broader values theyreflect".(0-.'442). The panel in no waycasts aspersion on our existence!

,About Society -

A great deal of what the press hasstressed from the report concerns theblame which society as a whole mustshare for the decline in SAT scores.The home environment 'comes in forserious questioning along with issues

'involving equal education, televisio4,and motivation.

4.1.=11=NIMINI1111111111

As usual, however, since the panelwas careful not to assign blame wherethe evidence was not clear, little morethan reflectionor hunch are,offered:

..."...the SAT shouId\nolAbecomethesole thermometer for measuring thehealth of schools, family, and student"

4(P40)The'panel expressqd belief that the

traditional relationship, between the-.school and the family has wdakened,

seriously. Broken homes and changing'family relationships, couPleds withchanges in educational practice, have, co 'buted to a strain on the parent-teacher r ati'direct or quable to suggetributes topanel expres

nship'"(p. 34). Since notifiable evidence is avail-t that this situation con-AT score declines, thes this opinion only

the most ghome-schoolsubstantiallyfor leariiinipis rkot expectcharige.shigle-

The panel c

eral terms. Improved'lotions can contribute

a better environmentowever, the professiond to bring about thisandedly,early stales that there is ,

nothing In the SAT score decline to"warrant generalization about what ishappening to the abilities at lorgesofyouth as a whole" (p. -21). Thus anyportrayal, of the "disintegration ofyouth," so popular' in the current

. literature, cannot be taken as a relleb-' tion of or a contribution to such decline.' In fact,' the panel does not feel that

the SAT is a broad measure of thegeneral tuality of youth: "It' tells usnothing Gout young people's honestyand integrity, about,whethefthey careabout each other, or about a.lot ofother things that matter more 'thantest scoreS"- (p. 40). This cautiousunderstanding of the weakness-ef

suchscores, including the SAT, puti suchscores in pioper perspective for sttt-

1

r,.1- 18

dents, parents, administrators, *coun-selors, or whoever,else tends to -oiler;rely on such measures.

.Much has been made of the so-calleddistractions which characterizedAmerican lift in the past decade , orso. Usually nominated ar,elhe war inSoutheast Asia, the cljalt, riots,corruption in high .Places, assassina-tions, etc. Television is also includedas a distraction, but the panel hasnoted that this medium could be usedeffectively for instructional purposes.There 4 no rejection of televisioncinthe report; only a warning about itsmisuse.

Like many good things) televisioncan be and often is misused. Thepanel notes that educators -have theresponsibility for making effective useof its potential rather than rejecting itout of hand.

Since much emphasis is placed onchanges in society that have takenplace in this country, at least since1970, it is noted that the panel seesthese changes'reflected in our youth:"...students entering college duringthat period had gorie through five orsix years of national disillusionment,especially for young people, virtuallyunparalled in Ameridan history" (p.37).

This "decade ,of distraction"should be -kept in mind while observ-ing the test score decline. In manyways this puts 'ari additional burdenon teachers who are asked to providestability while other aspects of life arein chaos. The task of the teacher isseen by the panel to be far more coM-plex than it is often assumed- to be.Social changes must be considered

\, when discussing any issue related toeducational effectiveness.

t

19 k' Il

=

'AhoUt the Future

, Wes wish 'to make an additionalir*1: _comment on the title Ort Further Ex-,

amination. As noted earlier,.the Panel--,, investigated phst events that led tb the

SAT score decline and repotted Its -. current reflections abohtt these events.

All that.remains iS-ito point to the, -

future, itnd the panel did not4-tieglect .this opportunity.

reviewing the tfast 14 years ofjest score, decline-it became increasing-ly evident to 'the panel that no singlecause, could be identified. Eventhough The panel acknowledges thatthe quality of educatidn r is ,,not ,yet

. wheie it shotild be, it does not despair:Rather, it urges. educatbrs to moveahead to the future. Perhaps_the ques-tion of why' SAT scores have declined .is the wrong- question,tp ask. Perhapsit is impossibletizi4nswer adequately.But whatever ilse!the report offers, iturges those conceited with Americaneducation to make, an even "furtherexamination":

So thte is no one .cause of',:theSAT score decline; (least as far aswe can discert, suspect nosingle pattern of causes. LeaNing istoo much a part, of Life to haveexpected anything' else.

It would be too bad,, further-more, if our 'concentration bn the .

implications pf a declihe in the sta-tistical 'averages oh a ,e.t of stan-dardized examinations should seemto ignore how incomplete a Measurethis is of either educational orbroader human purpose. While we'ask why the scores on college en-trance examinations have gonedown, T.S. Eliot's probing' goesmuch deeper: "Where is the learn-

,. ing we have, lost in information?

. 20

Where is the understAding we have,lost4n.knpwledge? Where is the life`.we have lost in living?"

Vet in the panels view of it all;'the facf of the hard askingof bothkinds of questions offers neWpromise of new answ s. We, find

- nothing in the re rd we havereviewed to-di ourage the ,convic-'tion that learning' in America can bemade all that is hoped for it. What,:is clearest is the reflection,. in thereactions to these test scores and tothelpoet's lament alike, of renewedpurpOke to implement these hopes:The future continues to seem-4, a-good idea. (p. 48) ."

If still further examination is,. given 0'to thii subject, some or all of thefollowing questions might be asked:

1. 4ere an unchanging standard?It is unlikely that eighth-grade

teachers would think it appropriate togive a test to one eighth-grade classin1970 and to another.eighth-grade classseven yfars later and expect the dif-ference ih scores to say anything use;ful. 'What would such a difference inscores mean? That the teacher is

'betteIN or vorse? That the studentshive gostten smarter or dumber? Thatsocietal values have changed? Thatour knowledge-base is different? Canwe, in fact, comparechildren of oneset of circtrastances with those of

- another? The CEEB expects teachersto believe apt there is a .singlOuntchanging standard which can be mea;sured and compared across time. Isthis a realistic assumption?

12. What lhould a predic4.tive testpredict?

Should not the predictive power of

t, '

th T be measured beyOnd the first,of college? Why stop there? Why

has it not seemed useful for the. S4T toPredict ,sollege graduation or spmeother measure of life success?

3. How is validity as4suied? 'Who should be the arbiters of the

validity of test Otiestions relating toverbal and matheMatics i6i1ity3 Whyhave content expetts been used morein she development of achievementtests- than the SAT?' How can validitybe determined for a test which is saidto measure aptitude but appears to be'measuring achievement?

4. How is culitird I bias determined?

Why is it that the panel declined todiscuss the issue of cultural bias inlight of the fact that other test makersnot -only have admitted such bias 'butate attempting to cOrrect for it? Whatmight She, panel. have done to assessbias? Howmig t the SAT aivid it?

5. What' instru tional changes are in-dicated gs fesult of the panel'sassessment of the SAT score de-.cline?

'The panel has strongly recommendedthat teachers not abandon innovation..They should not go back to "basics."In light of these recommendations,how should instruction change? Shouldwe be concerned with developing min-hal competency examinations? Shouldwe increase homework?

.6. Should the SAT°gtlempt to Jnea-sure thoughtful pjd critical reading.and "careful writing"?

Can critical readi actually bemeasured in d time , pressured en-vironment? Who can adequately deter-mine just what is thoughtful andcritical reading? Can it be measured

22

by readatAlify formulae? How can atest such as theSAT measure "carefulWriting"? Who should determine thecriteria for just what careful writingreally is? 0

4

De'cliningSATScpre8

The recent report of the CollegeEntrance Examination Board (CEEB)on declining Scholastic Aptitude Test(SAT) scores among high school stu-dents has done an excellent job inhighlighting some of the crushingproblems facing students, parehts,and teachersall of -whom have ahigh stake in the future of the Ala-

., tion's public schools. The ,ationalEducation Association shares concernwith CEEB over the seriousness of

c \

these prOlgems. No one is more awareof them than teachers themselves.

Teachers believe in high standards ,for their students. 'We also know thatfor teachers to' teacli, for learning totake place, students nust be eValu-'aced. But we believe strongly thatlearning- milst be evaludted in a varietyof Wags.

Each year, nevertheless, 1 milliongriatating seniorsabout a quarterof their age cohortiwill have pre-pared for the rite of passage fromhigh schoql 'to coltge by taking4the

.1 . ,For the past 14 years, SAT scores

kave dropped steatlily, after remain-

fig relatively'stable during the preced-ing decade. Verbal scores have dropped )11

49 points, from 478 in 1963 to 429 in1977; scores, -32 points,from 502 to 470. The question is,Why?

This fall the College Entrance/ Ex-,amination Board published the find-ings of a 'two -year - study commis-

24

,

.sioned to investigate. the: causes of thedeclining stores. The study wascon-

'ducted by an advisory panel of 21.people chaired by former Secretary ofLabor Wil lard-\Wirtz. It was spon-'sored and funded try CEEB and 'by

-4, the Educational Testing Service (ETS)the General Motors of the testing iindustrY\ and sole producer of theSAT. EB sponsors the SARI'; ETSdevelops 'a 0 administers it.)

The final ieport, entitled On FurtheiExamination,\ks impressive, broad-Sfanging and wrien in immaculate .prose. Its findings resent a compre-hensive analysis of soCial,,It

, tional change from 1963 to 1977.

May of the conclusions reflect condi:tjons in schools° and society thatteachers have 'known firsthand for

st 'fears. .

Despite its studied fairness, thereport presents several problem's. The

-panel points with pride to the SAT,which was designed in 1941 as an"unchanging measurement" to predictstudent, performance in college. .(Although specific test items change from

'year to year, the SAT remains essen-daily the same.) As the report itself

'documents, society and schools havechanged drastically _since 1963 and,in my opinion, certainly even more

, since 1941but not the SAT. Is it nottime to look at the test as Well as thetest results? ,,

Before discussing the other tkrob:Jems with the report, let's look at thereport findings.

The reportjtates that the decline,has developed In "two distinct stages,characterized by significantly differentbalances of...different causal factors."

In the first stage, 1963' tc about1970, the report says, as much asthree-quarters of the decline was caused

25

o

? ,by the "notable extension andiexpa.n-,sion of educational opporttmity in theUnited States" 'during that period.This statement Jefers to sharp in-creases in the proportions of studentstaking the test-who came from groupsthat have always registered §uatatitiallylower -than- average scores .on the teststudents from families with lowersocioeconomic status, minorities, andwomen (whose avetase scores on the.mathematical portion of the test, butnot on the verbal, ha4 been lowerthan mien's).

In the second stage of the decline,after _about 1970, there were fewer 'changes In the demographic distribu-tion off-the SAT takers. During this 14g.time, the decline. in scores sweptacross the board: 'scores of tradition-aally -higher-scoring' stud'entA dropped,raid those of traditionithy lower-scoringstudents fell even lower.

The -Panel 'attributes' three.fourthsof the decline in this period to theimpact of "veivasive" sdcial,forces,-\slime effects it cannot, documentprecisely. It does, however, cite .5-sets of developments that may haveaffected this decline:

Characterizing the period of sharp- .

est decline in score's (1972-75) and theyears immediately before this declineas a "decade of distraction," thepanel says that "there is simply noway of knowing -'how much thetraurnalietween 1967 and 1975 of 'co=incident divisive war,...political asses-sination,...burning cities, and the core:,,ruptiOn of national leadership affectedthe motivation bf the young people...and whether there was consequenteffect on heir college entrance exam- .

illation scores." The report cqnjec-tures that "this probe* made quite adifference.%!

246

o.

The panel alip points to changes-during this peri6d in the role of thefamily in the educational process, not-ing particularly the' increase in thenumber of children living in ilomei inwhich two parents are riot present.Although lacking definitive evidenceon this point, the panel concludes thatthe effect of these changes is negative.

Noting- that "Vy age 16 mostchildren have spent between 10,000and °15,000 hours watching television,more time than they have spent inchool," the panel surmises that this

-t-'11.&-- activity distracts students from doinghomework, competes wit schoolinin other ways, and has contribute tothe decline in SAT score averages: .

"There has been a ,significantdispersal of learning vivifies and ent-phasis in the schools, reflected partic-ularly in the, adding of, many electivecourses -and reduction in the number

\ of courses , that ,all student ...arerequired to take." The pan1 says,however, that a "broadside condem-nation" of electives' is not the solu-tion; it recommends instead "restoringthe tradition of critical reading andcareful writing."

1 4The report cites , clearly observ-able evidence of diminished seriousnessof purpose and attention to masteryof skills and knowledge...in theschools, the home, -and' society gener-ally." It points to the condoning ofexcessive absenteeism, grade inflation,

.automatic promotion from .grade tograde, less homework, and easier text:books. *- .

"Thefe has been an apparentmarked diminution," ttie-panel finds,"in young people's learning motivation,at least-as it appears to be related, di-rectly and indirectly, to their perfor-mance on college entrance examinations" :

1'

The report does not fault teachers.In fact, the panel shows some under-standing of their difficult situation.At the CEEB press confrence to an-nounde the fiddings, Harold Howe II,panel member and fornier U.S. 'Cormmissioner of Education, said there isevidence that the job of teaching istougher today than ever before. Thereport's discussion of teachers is

,mainly descriptive', noting on the One7 "Aland a drop in the average years of7.. experience of', elementary school

teachers, but pointing on the other toan increase in teachers' average edu-cational levels... It also refers to adecrease in pupil-teacher ratios andcites an increase in teaching salaries.

The panel concludes that teacher?and school adiVinistrators' responsi-bility for what has happened, "centersin their having made more concessionsbecause of changing circumstancesand demands...than has, been goodfor anybodiNnvolved. But _ this be-comes a hard question of hOw much'choice they have had and of-how the°demands .pf a changing student clien-tele are best met."

How are the demands of a changingstudent clientele best met as theymake the transition from schoolto college? By a standardized multiple-.choite test, designed in 1.941 -andremainittg virtually unchanged, rigor!opsly guardinjithe gates to the nation'scolleges? The NEA sloes not think so. .

While the nation endures a "decade ofdistraction '° the SAT, sails confidentlyinto the past.

In fairiess to the panel, if must be,said that they were not unaware ofthis issue. While they defendatheSATrigorously on its merits as a predictorof academic college performance, theyrecognize that high school grades are

28,

even more valid predictors. They alsoacknowledge .current broadek\criticismsof the SAT related to common mis-uses and abuses of the scores. In'January 1977, CEEB issued a set ofguidelines for avoiding misuse of thescores as measures of the effectivenessof elementary and secondary educa-tion in general. The scores %re, never-thelessi often used erroneously asexcludve measurements of individual 'or institutional quality. The reportnotes that when this occurs, "a veryreal 'relevancy' issue arises."

,A far. more serious consequence oftest misuser is that it may elicit the cryfor "back to basics." The report itselfis provoking this, kind of inisuse.When the public and state legislatureslook at it, they may see only oneissue, lower test scorek. They m4y notheed the panel's thoughtful advice toschools not to become."more rigid,...more rejective, more uniforin." Theymay not pay attention to the panel's

' observation that "the record may, sug-geSt as much about' youth's inherentresiliency and the resourcefulness ofthe formal educational process underunusual circumstances-as about 'deter-ioration in either personal or institu-tional fibers. We have wonderedsometimes in the course of our inquirywhy the score declines haven't .beenlarger."

The panel believes 'that there is "al-most certainly" some causal relationship between the shift from traditional .

courses to eleclives and the decline inthe SAT verbal scores. But despite itswarning against "oversimplistic inter-pretatioilof this finding," the knee-jerk response to declining test scores%may be back to basics.

A most important part is that virtu-ally no one is addressing the question

29

of the relationship betweeh'the SATtest and curriculum as it exists in16,500 school districts in the UnitedStates. The problem does not lie inidentifying and explaining causes forthe drop in SAT scores; the committeedid. that and did it well. A fundamen-

_ -tat value question must be answered:Should a SAT test which hasn'tchanged significantly in 36 years -beallowed to become a major determi-nant of school curriculum?

If legislatures and school boardsrush to lunfounded judgments abouttheir own curriculums (based on SATscores), they will be giving assent tothe SAT as the basis for a nationalstandard.

In the final analysis, it might be in'the best interests of public educationto have some standards national inscope, but who sets those standardsshould be a subject- of thoughtfuldecision. As things stand, curriculumis a primary province of thousands ofloal school boards. Who's to say thata few experts in the testing industryare betty able to determine curricu-lum needs?, 1.

And so we come full circle to theSAT test itself. "The panel according-ly commends further inquiry by theBoard and ETS into the function oftests at this critical passage point....The pUrpose of such inquiry would beto identify and put in; appropriate

7 priority whatever can be distilled fromcurrent national concerns about the

-society's educational values, aud then-to evaluate the traditional tests in thelight of that determination."

There is just one more -problem.Theie ir 'no 'time, frame, no target&ate, no 'deadline: The need for ur-gency at the tranquil ETS campus inPrinceton, New Jersey, is not the

30

0.

4..

same ai in the high schools of Bald-.. more Td Indianapolis. While ETS,moAcei toivIrd further inquiry, hun-.iirpds df thousands of students'. are

7being measured on archaic scales andfound wanting. 0

John Ryor ,.

President, NEA., .

1 Today's EducationNovember-December 1977 .

o .

4

.,

C

,

1

...

It $V. RI) 11.6 >i I ``,1()N11 Fl 1st Nt_

Documents such as this are a partof our effort to help teachers teach.The size and The' complexity of theteaching professiOn are reflected inthe range and the diversity of these'publications. -4 ,

t

The realities of the classroom andthe problems which &Instantly Con-front thg professional teacher areoften deeply (rooted in larger social,pdlitical,. gild; economic issues. Forthis reason a number of our publica-tions, as well some other programactivities, are directed at very specificgroups within the profession.

The Natiohal Education Associa-tidn works through its Instruction andProfessional Development (IPD) unito provide members with information,',service, and resources related to theAssociatidn's -goal area of Profession-al Excellence.'Fot more, information,

,write to the IPD Information Center,National Education Association120116th W6thStreet, ashingten, DC200361.

. Jdhn D. Sullivan, DirectorNE-A Instruction andProfessional Development.

0 I,

.3-

3