ED 253 126

AUTHORTITLE

PUB DATENOTE

PUB mg

EDRS PRICEDESCRIPTORS

DOCUMENT RESUME

HE 017 927

Kroc, Richard J.Comparing Citation Rates with Other MeasureS ofScholarly Productivity.Oct 84

' .28p.; Pgper presented at the Joint Meeting of theAmerican Educational. Research Association Division Jand the Associationipor the Study of Higher Education(San Francisco, CA, October*28-30, 1984).Reports Research/Technical (143)Speeches/Conference Papers (150)

MF01/PCO2 Plu0Posta0.*College FaJUlty; Conferences; *Evaluation Criteria;Financial Support; -Higher Education; InstitutionalCharacteristicS; *Productivity; Reputation;*Scholarship; *Schools of Education *Wrip iting forPublication

IDENTIFIERS *Faculty Publishing

'ABSTRACTThe construct validity of scholarly productivity was

investigated, with attention to definifions and measurementapproaches. A nonrepresentative sample of 51.schools of education wasselected to include very produgtive schools. Measures were made ofthe following variables: citations, rankings froth previous studies,publications, conference participation, funding, and generalinstitutional characteristics. Publication counts were estimatedlromdata in the Educational Resources I4lformation Center (ERIC) system.Each school of educption's participation in the American EducationalResearch Association conference. was assessed for 1981 and 1982.Information on grants awarded to schools of education during1978-Y982 was obtained from the Smiths9nian Science InformationExchange. Finally, the Higher Education General Information Surveyprovided data on _five variables: percentage of aoctoraes granted,total research expenditures, total overall expenditures, governmentgrants and contracts per full-time equivaldncy, and, averagesalary. Conclusions include the following: program size did nosignificantlydetermine prestige; surveys on reputation did notappear to be reliable; and. citation counts seemed to b the bestmeasure of scholarly work. (SW)

.,

1

***************************************************1**********Ao*****Reproductions Supplied by EDRS are the best that can be made

from ihe original document.******************.************,*****t***********************v**********

0.1

to-

-

Comparing Citation Rates with.Other Measures

"PERMISSION TO REPRODUCE TINS

MATERIAL.HAS BEEN GRANTED. BY

TO THE EDUCATIONAL RESOURCES

.INFORMATION CENTER (ERIC)"

of Scholarly Productivity *

. U.S. DEPARTMENT OF EDUCATIONNATIONAL INSI IIUT( ,I.DOCAT ION

t OUCAT IONAL Itl SOIATC.IS 4NrpliNIA I IONCLNITIt (Intel .

(..411,441.'etonlimt hat, tUtUo toptOthtflUti aA

Richard J. K r o c mcmvtul hum the, tmus011 et oftiointe*on

ofiti,nallmodMIt(11 Imbues kive boon made to 11114110VO

telpot 1)4:1) itmlity.

P us .11 owmoos *Anted it 11:*do(*.mom do not ectrwtin, roptoent offIc141NIE

UniversAy of Colorado

pn511n/n of putt V.

In.assessments of excellence in higher education, uestions

of scholarly productivity are often pivotal.. Research and

publication are the driving forces in the modern university,

evaluators of university programs are attuned ;() this fact.

From i6divAual.promotion and tenure decisions to overall...

institution-al prestige, scholarly research is a fUndament.I1141 ,

issue.

A variety' of procedures hive been used 'to measure

productivity. Some researchers have surveyed faculty or

and

\

administrators to obtain ranjdngs of universities and

departments. ,Others have-used the, quantity or quality of

publications to rate programs, sometimes including presenttAans

of papers at conferences. Another measure is the type and .

amount of funding obtained by a ,department, and, at -thy

university level, characteristics such as faculty salaries and

research expenditures have been ] -inked to faculty productivity.-.

Fln14h aelly, citation rates, a measure of peer recognition, have

been used to- assess scholarly accomplishment. In some way,

of these measures has' been associated.wi.th productt sty.

.This study ..addressed tke construct validitx oft cholarly

productivity: Mow this concept has :been defined, and how

.:S

Paper p sented at the Joint Meeting of the American Educational ResearchAssoc.]. ion DivIston J and the Association for the Study of) Higher Education,San Fr n4pcd, C.41ifornia, October-2830, 1984.

Sr

-.)111yr

At*

ti

*2

various ways of measuring it compared. In particular, data

pertaining to schools of ed.ucation were examined. The

. perspective was that of an administrator or evaluator who must

,collect and assess information regarding productivity,

discovering in the 'process what data are available for making

.valid, useful comparisons among 'schools of educatico, whatti

problems occur in gathering this data, what relationships 'exist

amoog measures, and what interpretations can be made. These

issues are of considerable practical significance, reflecting

processes that often have a direct impact 611 the future of a

program, a school or a career'.

Interrelationships among variousfmeasures of productivity

have been explored indifferent ways by several researchers.

Hagstrom (1971) determined that a single, unrotave'd factor was

sufficient to account for most of the variance among 188

university.science and mathematics departMents on ten variables

related to institutional quality. Astin and Solmon discovered

two factors,'"scholarly excellence of faculty" and. "commitment

to teaching, ". in their factor analysis of a questionnaire on

undergraduate excellence (Astin &.Solmon, 1981; Solmon & Astin,

19.81).' They' also found evidence thatoyerall institutional -

V.-

prestige is an important aspect.of'individual departmental

ratings. Another study ,(Andersen, Narjn, & McAllister, 1978)

used regression techniques to predict prestige rankings from

various publication and citation data, finding that variables

related. to .size, as well as to. quality; are predictive of

preWge. Endler, Rushton and R.oediger. (1975) correlated both

.

4

,

-)t

4

publicition counts and 'citation data with other measu.res,

concluding that, citation rates.may be Wetter than publication'

counts as a\ea'sure of quality. :Sash (1983) produced composite

rankings of schools of education using conference presentations

and publication counts. 1r

ThiS Atudy..examined. relationships among a series of

N.

measuc.es of productivity for a single sample of schools'usincl

coAputerized data bases easily accessible5to any researcher. In

particular, the results of citation analysis, a method not often

used in many fields, including education 4Kroc, 1984), was co.-.

pared with 'other

Sample

V .

Method

A non-representative sample of 51 s.chb9ls of education was

chosen from rankings in.other studies tO include as many of the

most productive schools. as possiblei (see Table 1). For each-.

school, faculty lists were obtainetfrom 1981 course catalogs-or

-by writing directly to the university. Measures were made of

the following variables:' citations, rankings from previous

studies, publications, conference participation, funding, and-.

general institutionbl characteristics. Data or the 4600 ..

, V i afaculty included in the study were- agg'regated at the school

r

..,

,

level... .

.,

Citations41.

The number of citations for each faculty member in the 1981

Social Science Citation Index (SSCI) b4as counted. The SSCI

fists, by author, citations of that author's, work during a given

. .

.11

-*

4

year. k.set of rules was developed .to standardize the counting.

procedure, and two Judges were trained, enabling an assessment,.

v

to b made of the relipility of citation counts? 'Me obtaineda

inte rater-4prrelation coefficient of .r *.90 ipdicated that.it

cttations can be re.l.iably, counted.4.

Th'e,tC1 proved to b'ie a:r1ch s:oirce data,-fully.

ref rencing 160 education *journals, as yell, assmany other

journals in related fields, on an 'annual basis. A more thorough, ,

discu.ssi'on of SSCI citation analysis occurs elsewhere (Kroc,

k

4

198/1).

Rankings from Previous Studies

Rankings of schools of education from nine studies were

recorded. Five of these researA 'efforts (Blau & Margulies,

1975; Cartter, 977; Ladd.& 1.ipset, 1979; Sieber, 1'966;

Walberg,'1972) were surveys of, either education faculty, AERA

memb.ers, or deans of. schools of education. Theotherjour

studies prodcued ranks based on publication counts°(West, 1978),

AERA -presentations (Dole, 1981; Schubert, 1979) or both (Eash,

1983).

Publications,

Two data bases, Research in Education (RIE) and the Current.

Index to Journals in Education (CIJE) werl.e searched using the

ERIC system. The high cost of obtaining.this information 'for

each facultymember ensured that there were limitations on the

amount of data, that could be 'collected and on the. number of

fculty surveyed, resulting in a decisionto sample 25% of. the

4600 faculty. Based, on previously obtained citation data,

N.! ':;,

5

faculty from each university were, split iQto four strata, and an

optimal allocation 'formula (tokiran,11953, p. 93) was used to

determine the propo'rtion of ot2servations within each strata

which wbuld minimize the standard errorof the mean publi6tion. ,

'counts.

Conferenceayarticipatison ,

Conference participation was measured by counting the

number of presentations listed in the Annual meeting programs1

at two confere7,s,..-the American Educational Research kssocia- .

tion (AERA) conferencv in 1981 and 1982. Two consecutive 'AERA. . t

conferences were surveyed to control for bias due o the meeting..- A .

ocation: The 1981 conference was htld in 1.0*Angeles, while

the 1982 location was Boston.\ All types of AERA presentations

were given equal weights, except discussants, who were not

counted.

Eundi n1

Only one accessible source of funding data was found, the

Smithsonian Sciene InfOrmation Exchange (SSIE), an agency that

compiled and stored for computer access information on.grants

issu'ed between 1978 and 1982. . SSIE received project

descriptions from over 1;300 organizations: federal, state and.

local government agencies; nonprofit assviations.and founda-.

tions,; colleges and iniversities; alid,tto a liMi/ed extent,

private industry and foreign research organizations. Approxi-'

mately,90% of the information was provided by agencies of the

federal government. Unfortunately, the SSIE was phased out late

in 1981, a victim of federal budget cuts. Nowever4, the National

4

14

6

-0

Technical ijnfOrmation Servise'(NTIS) has assumed 'atRortion of

SSIP's former. services, but does not yet gather djta from the

agencies which are mOstI11kely to furid educational research.,1

The disSolution of the SSIE was not a problem in this study,

'since funding data was. available through 198o

Thee organization of the SSIE data-base madlg it possible to

ts

enter the program' and university name at a computer terminal and

receive a printout containing abstracts of the iovernmenti grants

obtainwed by that education program between 1978 and 1982. Each

.

abstract listed the geant's,nvestigators, title, sponsoring ,

organization, performing organization, funding period, and

dollar amount., 'Unfortunately, this' information was not always

complete; in fact, 30% of the abstracts did not list a dollar

amount for the grant. Still, it was possible to count the

number -of grants obtained by each program, As well as to get

some indication of the money involved.

Institutional Characteristics

All universities that 1eceive federal funding must 'report

certain information annually to the government.' This data is

documented in the Higher Education General Information Survey

(HEGIS) and is ;available 011 magnetic tape.' Variables which were

accessible and of interest for this study were dollar amount of -

government grants and contracts per FTE (1978-79), total

research expenditures (1979-80), total expenditures (1979-80),

percentage of doctorates granted (1979-80), ,and average faculty

salaries (1980-81). All of these measures were at the univer-

sityilevel, not at the school or,department level.

..4

'Analysis r. ,

7

Two sets .of rankings of schools of education were produced

fir each variable, one based on total.counts for each school'apd

the other on .total ,.counts. divided by the number of faculty in

each school. Cpmparisons of the six sets of variables, using

both unit's of analysis, were'mode by examining correlation

$ coefficients. The patterns of 'these coeffic) ts provided data

on how.the variables in this study were inter-re ated as well

as how they Compared with measures of productivity fron' other

studies.

RESULTS

Relationship Between.Cqation Counts

and Rankings from Other Studies

As discussed, a number of studies of education schools, have

been done. The ranking' resulting from these studies were

correlated with oni another as well as with citation (Jaya,

producing the colificients 4a Table On the diagonals are the

number of schools which were ranked in the study that is listed

at the. top of the column. For example., the ranks for 23

programs which' were available from tht Dole (1981.) study,

correlated'.26 with rankings from the Ladd and Lipset (1979)

study.. Five sets of rankings (Cartter, 1977; Ladd & Lipset,

1979; Sieber, 1966; 8.1au,and Margulies, 1975; an6:Walberg, 1962)

were based on surveys which asked resp6ndents to list the top

schbols. of education. The correlatAons among these: five studies

showed considerable variation, indicating how surv6 riesponses

vr,

8 . I I.

it

. can differ. For examp bath, artter. (1977) and Ladd and.

lipset (1379) surVeyed e cationjaculty memberS.in.,Ottempts to

discover the most di-Stingu shed sCifools of education. Yet their-.4 .

. .

.

rankings correlated only Prestige'rank,ingsf then ..may not

be very consistent, even when, similar questionsA1 re asked of the

same grbup of respondents.

Studies that obtained less subjective measures of quality

fared somewhat better when compared with one-another. Schubert.,

(1979), Dole (1981)-and Eash (1983) used AERA presentations as

measures for ranking programs. Schubetrt cjounted total presen7

.tatons, Dole :os,ed preseniatiOns per faculty, and Eash i-eported

both-. The ,ranks of Dole and Eash show a correlation of ..70,

.v4ile those of Schubert, an(C'Eash correlate .83, an imdication

that AERA presentation counts are somewhat consjstent..4.

-Similarly, Eas.h (1983) and West,, (`197.8) ranked programs

according to the )otal number of articles appearingk

eading

'education journals. The correlation between these studies,

r = .49, woald probably have been highfrr had more of the same

journals been surveyed. Nonetheless, there' is evidence .of

moderate reliability in ranking by counts of journal articles.

ReSults of studies using total faculty counts showed low.

'correlations with those calculating rgsults on a per faculty

basis. for example, the AERA presentation ranks of.Dole and.

Schubert correlate only .19. In general, ratings using the

faculty member as the unit of analysis correlate more highly

with-prestige rankings; as shown by comparing Walberg's 11972

rankings with those of Eash. When asked,to rank schoo,ls':of

r,

Ir.

9

education-, survey,respondentsseeff to think more in terms of. ,

research h-producOvity pr faculty memberthan they do of total

productivity.

Meah-citation'eanks correlated moderately to stronly \vith

all .survey rankings except Ladd and LiOset's and with '4.11 other

rarkkillgs which used individual faculty as the uni,t,of analysis.. .

The highest correlations were with the Carttvr survey it

.83) and with Eash's rankings of AERA preseneationsper FTE

a

0(r =.65). Mean citation rank, then, showed a relationship .

with the ranking,methods used sin other studies.

Although'mean citation rank correlated well with' other

types of ratings, total citation counts were not as strongly'

relatedto other measures. Percentage of faculty with 7,,ero

citations, on the other hand, followed a pattern identical to

that of the an citation rate, indicating that they both may be,.

tapping the same dimension of productivity.I

- Publ i cat ion vCounts

Publications were estimated from,data in the ERIC system.

Two other studies (West, 1978; Eash 1983) used publication

counts as a method for ranking programs. Table 3 shows inter--

correlations,-among the ranks produced from thest various methods

of measuring publications, while Table 4 displays correlations

between publication ranks., citation measures,:and rankings from.

other studies..

The consistency between .Sash's" coups of articles in the 14

journals and the publication measures fn this study was shown by

the strong relationship between Eash'.s total article's and the4

''' "77- 77.t."*.:::; "7. 7.!--7"7 . .

4

10

-*total .ERIC publication rank (r .75) as well as by the-7

correlation betwe'en'the, two FTE counts. (r = .63)1, West's

r- method of rankng (based on total publication counts in Li

journal's) did, not correlate as well. with either of the ,other

measures'. = .33 with total 1ERIC publication rank, and r

.49,with Eash's total counts) .

. The correlations between publication .rankings and other

p

ty()es of radVings showed a wide range of values. In general,

though, the five survey results had more in common with per-

faculty measure% of publications than with total counts: four

aut of /the five studies had higher correlation's with per faculty

publication rates.,

Citation ranks were not strongly related to publication

counts, 'as shown by the coefficients in the first three columqs

of Table 4. Mean citation rate,correlatedonly .34 with ash's

FTC publication 'ranks and .39 with per faculty ERIC publication

ranks, while total citation rank' showed a similar degree of

relationship with the three measures of total publications. A

stronger correlation, though, was found between the percentage

of faculty with no citations bnd per faculty publication ranks.

It may be that those faculty who are more often cited have qu'ite

variable publication pAtterns, th'us lowering any correlation

between. citation and publication rates, while tho4e who are not

cited tend homogenously not to publish, thereby increasing th.es.

relationship between publication rates and the percentage with

no citations.

It should be note that the publication,ocounts i .this

p.

.

a

So.:Se

4 11'4

,

study were basect on combining entries from both the CIJE andk

RIE. -These publicaion counts, tlieb, include bath journal

articles and "fugitive" literature such as technical reports and

evaluations.

Conference FlaitriciallV

Pa

Cacti school of "Oucation's participation in the American

Educational ReSearch'AssociOtion, conference was assessed for two

years, 1981 and 1982., Three other- studies used AERA participa-.

ti on as a means for ranking programs: Schubert (1979) used data

from 1975 to 1979 to'obtain counts of total presentations by '

each school of education., Dole (1981) used Schubert's figures,

divirigaeach total by the number of faculty to produce' rankings,

with faculty as the unit of analysis. 'Sash (1983) counted AERA

presentations over. a sAten-year period, 1975-1981, com Aling

both total and FTE rankings of schools. Table 5 shows e

intercorrelations among these three studies and with the ranks

from this, study. These coefficients Indicat-ed greater consis -.

tency among AERA studies than among studies of publication

rates. This seems reasonabl*, given the ease with which AERA

participation'can he judged. from the program meeting notes, and

thefact'that each publication study used counts from a differ-

ent set of journals.-,100

Table 6 indicates correlations between the _various "AERA

'measures and three groups of rankings, representing citation

rates, publiption counts. anal prestige surveys. Citation

'measures correlated better with AERA presentatlons than. with.

publication.rates. Perhaps authors who are cited-more often

,

At

t.

have a greater ii1cl n at ton to submj t: th 4" WQrk to, the -n atci 'Dna.]

exposure of fIr.ed by Ae/RA conferencs.,

COrrelations between. l'AEW data ',And, publication ranks showed.',

\ a pattern of moderate relationships, whil prestige su7rveys didF(.)

not correlate consistently with AERA results, as alight. be of

expected.

Fundi ng

A computer search .of the Smithsonian Science Information .;;...

Exch-arige (SSIE) eliciet lists of abstracts of grants awarded to

schools of edVcation between 1978 and 1982. From this intorma-

ti on, the number of grants "for each education' program .as well as

the doljar amount was extracted.

Correlations between funding data and other i n d i c e s a r e .;'

P,jndicatielPf In Table 7. Higher. corpse] apt i ons are apparent for the

,0:hu,mbe"r ,qpier4.s (first two rows of the table) as opposed to

j ASS

- -; ,

d.,011,af ianiatiqXs..:.(1 a st two rows). (s,- mentioned previously, there

was:di.ff$.6ultiy- obtainir0 complete information on the amount

f -the :award s.;if.,;..../`... .

Clearly4i,t-aj on rates correlate better than publication4.,-..

couhts or' XE.Wpres'entati ons with- funding data. If. the case ismade"Olat more spoducti ve fa cul ty ° aoe better ab.l e to obtain

grants, thien',.the'se riptrelations. enhance the validity of. .

citation eovnt5 'as an,

i nde,,R af,.Ischolarly work, particu.larly in. .

comParisOiri,;:'.w.it publication data and AERA. conference 'Partici-

patj any 1),p that more cited scholars write stronger grant 41'

\\1

proposal-5, or<<thast'thel`r reputations influence the decisions

of the grantinCagdncies.'

\

at 4

.

13

Institutiohal Characteristics. .

The Higher1

Education General Information Survey (HEGIS) was

,the :source for data on five variables: percentage of doctorates

k,

'granted, total research expendFtures1 total overall'expendi-

tures, gownment grants, and contracts per FTE? and average,

faculty. salary. ;Alf_of these, variables were measured fort the

entire univerSitY,pot at the school of education level, and

reflect'either 1978 -1979 or the 1979-1980 academic year. fe

Table 8 shOws-correlations between institutional charac-

teristics and four sets of variables consideredin this study:

citation rates,, publication data, AERA presentation ranks and

funding information. These. coefficients show' that data reflec-

tive.of the university As a. whole was related 'to department-

level indicators of prodyctivity. This was.tHhe for the schools

of education in this Study across all measures. Perhaps

universities which command greater resources., and hence can

afford larger salaries and expenditures, are able to attract

more productive faculty.

4s with the SSLE funding data, institutional variables,

especially 1,..xpendi.tures, grants and contracts; and salarieS,

tended td correlate best with'citation rates. Although the halo

effect majf, have been an issue with these data, this does not

explain why citation rates are the strongest correlates. A more

likely explanation maybe that these institutional variables

a. direct or indirect influence_on departmental productivity, and

1'

the numper of citations were a better reflection of this.00

O

4

D.

s

N

r.,

14

Conclusion

a

Few would doubt the importance of scholarly roductivitY

university, life; yet obtaining measures suitable :r making

comparisons amongwunivers,ities iliay be difficult pnd.controver-

sial. This, study. provided comprehensive data on several

relevant variables for A selecte4-d sample of schools of educa-

tison. Aelationsdlips among these measures .suggest'several points.

related to construct validity.

First, the tendency for prestige survey results to

correlate more highly with meansorathec than with totals on

other measures was an indication that the program size was not a

silnificant.determinant of pre-Se: quality was more. impo'rtant

than quantity.4

Second, surveys had an-annoying Cendency to s how inconsis-

ient relationships with one another. Although the small number

of,zschools ranked in some studies, as well as differences in

when the assessments were made, may have contributed to Ulis

Troblem, the reliability of surveys on reputation was4ot

evident in_these data.

Third, 'the moderate Correlations found among the various

measures indicated.considerable shared variance and, perhaps,AL \

justification for. thinking of schonanly productd vity vi,n schools

of education as a''unitary concept.

Finally, citation analysis, a reliable process with a1

compelling and Logical basis, produced data whilich .showed a

stronger re1ationship than did any other measures with funding

4.

4.

15

and' institutional variables. The evidence in this. study implied

that *citation counts are most central to the concept of.

fprodu.aivity: Although all measures are somewhat flawed,

particularly when used to evaluate indiNidua) careers, the

citation rate may* be the best single measure 'of scholarly work.

e

I

rr

d.

ti

14.

:

v

REFERENCES .

. fa

Anderson, R. C. Narin, & McAllister, P.7- Publitation,ratings at Journal of.t.he American Societz.for,WorMatio Science, 1178, M-1)-10:101.

Astin A. W , & Solmon, L. C. Are reputational ratings neededto measure quality. Change, 1981, 13, 14-19.

Blau; P. M.J, & Margulies, R. 7. The reputations of America.pprofessional schools. Change, 1975, 6(10),'42 -47.

.The Cartter Report on the leading schools of education, raw, and.

business. Change, 1`577, 9(2,), 44-48..

Cochran, W. G. 'Samplingrtechniques. New York: John Wiley and' Sons, 1953.. .

Faculty size in institutional proa'uctivity. Schoolof Edueition,.bn-FieT'ilIy ofPennsylvaTra,1§81.

Eash, M. .Educational research productivity of institutions ofhigher education. American Educational Pesearch Journal,1983, 2Q(1),

Endler, N. S., Rushton, J. P., & Roediger, H. L. Productivity. and scholarly. impact (citations) of Briitish, Canadiah, andU.S. departments of'-psychology (1975). American

tsno9.91q, 197a,.33, '1,064 - 1,082,

Hagstrom, W, 0. Inputs; outputs, and the prestige of univer-sity science departments. Sociology of Education, 1971,,44, 375-397.

Kroc, R. J. Using citation analysts to assess scholarlyproductivity. Educational Researcher, 1984, 13(6),17-22.

Ladd, E. C. & Upset, S. M. The 1977 survey of the Americanprofessoriate. Chronicle oftilhfs Education, 17(18),7-8.

93Schubert, W.'H. Contributions to AERA annual pro.9rams as an

(

indicator of i'ns itutjonal productivity. EAucationalResearcher, 1979 8(7), 13-17.

,Sieber,, S. D. The organization of educational researcW. NewYork: Columbia 'University, BureaTWTApplied SocialResearch, 1966. .

am.

Solmon, L. C., & Astin, A. W. Departments %Jithout distin-guished graduate programs. Chan.91, 1981, 23-28.

..

4

NR

r

.Walberg, H. J. University distinction in educational research:An exploratory turvey. educational Researcher, 1972,1(1), 15-16.

West, C. K.. Productivity ratings of instituti,QA based onpublication in the journals of the American EducationResearch. Association: 1970-1976. Educational Researcher1978, 7(2), 13-14.

ti

'4."?sil.

If

4

4

w.

.

4

..1

Ar

Table

List .of Universities

University of ArizonaArizona State bniveFsityBoston UniversityUniversity of California at BerkeleyUniversity of talifornia at Los AngelesUniversity of California at Santa BarbaraCat lic)Universityn varsity of Chicago

City University of NewyorkUniversity of ColoradoColumbia UniversityUniversity of ConnecticutCornep. UniversityUniversity of FloridaGeorgia State UniversityHarvard University

,,

University of HoustonUniversity of Illinois: Champaign ,UrbanaUniversity of Illinois: Chicago-CircleIndiana University,University of IowaUniversity of KansasUniversity of MichiganMichigan State UniversityUniversity,of MinnesotaUniversity of MissouriUniversity of Montana

. University of NebraskaNew York UniversityUniversity of North Carolina: chip*i HillNorthwestern Un,,ivetsityOhio' State University:University of Oregontniversity of PennsylvaniaPennsylvania State UniversityUniversity of PittsburghPurdue University..UEINWEsity of RochesterRutgers University

. University of Southern CaliforniaStanford UniveriitySyracuse UniveriityTemple University(University of TexasPeabody-Vanderbilt' University 4University of Virginia.Virginia Polytechnic InstituteWashington University

; Unfvertity. of WashingtonUniversity of Wisconsin

Table 2

Correlations Among Other Studies Producng Rankings of Education Program* and Citation Path

a

4. 'V iiII Ili t

^ ...I4 :s. .-.S.

*V

'0 ....l 0

w 43 ..C a .1.. o--

t- a .04-1 .. 44 .4 a '.0. .. - .a.a 0 ..I

41 - i ' i P:4

O.0

0,.1% i i , ..1. I a

Mean Citation*Mean Citation RankTotal Citation Plank

ercent'o1 raculty 141tillo CitationCarttar 119771Slam and Margulies 119751Ladd and Lipsett (19791:Siabor-11966)Skilful'', 119721

Schubert 119791pole 119011

M*Al 11979)t ask (190)1 AEA Totalg ash 119m1) ADA Per r?EEash419011 Total Solicit,*Kash 419011 Article* Per rTE

1.

. . .

4

-.7S n,S8noS1 .69

nS1

. i -,.

-.77* .79'

.47

rp*S1

BEST COPY AVAILABLE

-.79 -.40.8) .11

.54 .17

.06 .19n.14 .51

n.12

2 0

M1ek

M

2

1 I

a-.22 .J7 -.45 .04 -.11 -.10 '.01 , .09.07 :19 .47 -.06 .,40 .2$ .21 .65 -,;34*. 4 1 - 14 .17 .41 .7111 '.65 .04 .31 .42.11 .75 4 4 .11 .SS

*.S9 .09 .43.

.14 ..61 ' 11.01 .51, ...05 ..71 Ofq .41 :411'

.47 .06 .05 -1.21 .01 *.. ***06 Ao.n.1S .49 .26 q.0.24 .17 .7

' n110 . 9 6 $4 ,:27 .52°1 :65 .t 0,2 .21 .'.'

.19 Ob,))1!. osaA; V 70 .03

44e 1' 'al) 1.20

.427

os24At2*.

r

a

Table 3

Intarcocrolatidls at Publication Measure, or

Alt

EI

2

2

C

2

a

2A

a.7.; «,0

r4 4. .4

.0

I

Total ERIC Publication Rank .33 .76

Par Eacplty ERIC Publication Rank .47 .39 .63

Mast (1978) Total Publication Rank. ..49 .30 ,

'Bash (19931 Total Publication Rank .19

Lash (19931 FTE Publication Rank

x ,

. I

Table 4

Correlations &Mouton Publication 14saauta and (ahoy ..11enkin9a S.

J;

*-k

J

et A 4-4v-1 00

I.- .4

etr1. ......

,

'.......

S 11 a ta ..-...

.......

Iri$7-r7 ,

Crb 1 00at0-4

V 1..... p4..01.11 14 1 I 01

/ 417.1 M

4$ 0 Ni

C;t

i C.4 1 I N -.4

USill

-0 3g

Total ERIC Publication Rank -IlPer Faculty ERIC Publicationul

!lank .34

Moat (1978) Total Publication fsank .25

caontash 103) Total PublitiRank

19..06

-84tilt (1983) FTC Publicationsank .39

.09.

.45

.24

.08

.54

.31

-.12

.20

...49

.21

.11

.11

.05

-.40

.51

..-.06

.10

-.111

-.48

-.22

51

.47..

.24 .

17A .

.38

.00...

.31

'.21"

.04

.37

`.27\---....

.. 55-.

.66 .-.111,..

.48 .60

4

.31 .68

.83 4 .36

..03

.60

.31

,126

.36

.

.000.000-*---

22

.

.42 -.27. Lz

-.84 7 .51

.46 .20

. .02

.. L 1: 7 . 4; ;i

-44

. 1

:

.411

:

v

1

, ..1.1

J

-4.

.$

4Table $

inturcorrelatione.Among ARM Presentation Measures

ARRA Rank Per Faculty

V.

s,

.ah-.,r;--.---.=s. , v

oY.

u. .... 1t

a : 1...

. a- .4 a

I e !4 1. g

Aig 1

ARM Total Rank

Sash AERA Rank Par FETEA,

Rash ARRA Total Rank

Dole AERA Rank Per Faculty

Schubert ARM Total Rank

' #..A

234

.66 '.014

. '.79

.09

.71 .26

.20 .83

.70 .09

.11 .83

.19

I

,

i

."1

p

. I

I

r--

BEST COPY AVAILABLF

Table 6

4

Correltinp. between IWRA.FropentoltionAleiimireio and Other Rankinga

3.4

.n.

wl

AERA Rank Per Faculty .56 .20 .64 :10 .60 -.20 '..35 .42 ..55 -.06 .26 .37

ARIA Total Rank 06 .57 .34 .63 .41 .39 *.19 .55 .01 ...29 .46 ,41 .

.Each AERA Rank Per PTE. ,.65 ,04 49 -.35 .51 -.27 .59 .20 .67 -.05 -,.04 .52

Each -AERA Total Rank .21 .65 .42, .S .01 .42 -.44 446 --.31.,-.26 .17 .27

-Dol *ERA flank Per Faculty : .40 -.10 -.SS 426' :36 .0) .60 .31 .53 #01 .26 .SE

Schubort AERA-Total Rank ,.06 -.41 -.11 .v...60 -.36 -.55 -.13 -.60, .03 .21 --..311 ...49

.66

.53

.72

.25

.65 _.,=

.57 -

et.

Table 7

Correlations Retween Funding Data and Other Measures of Productivityfor 51 Schools of Education

t0z

4.1

Id614

0

01 U)

4,4w P

'VP.O

.14

-N1 4

c a

1 .0

tl .4-I-0

0

OS

.C.0 .1-1

-.10

-.0Id

04U

4.)-.I Al -41

0-14 -el 4-I U . 44

P.

.0 :1:13

IXor-i

4u r-11 P.

- AI.44-4

C.)-1 H 4.1

0

, ........ .4, , 7.1

5Ncr.co CO

11 U 4-1 1,-.4iti.... 4.....

isS4.0

E-4 04ra

111

P.>40 r5a.4a. g1 r-.VI gg g A

g 114

4.1

Total Numberof Grants -.55 -.71 -.49 -.44 -.35

1

Number of GrantsPer Faculty -.65 -'.47 -.77 .01 -.45

Total Numberof Dollars -.38 -.18 -.22 -i,21

Number of DollarsPer Faculty -.55 -.25 -.45 .02 -.40

-.43 -.09 -.38 -.34 -.54 -.08 -:39 06 -.48

-.08 -.3 !-.18 -.53 ..30 .r..46 -.09 -.34

.4.09 ,31 .22 -.39 -..11 -.48 08 .0.21

.22 ...25 .23 ,-.57 -.39 -.56 .-.182.-.47

P.

:34

a.

Tata* 8 /

I.

Correlations between Institutional, characteristics and ileastarlts of Productivity for M Schools of Plue.ltion

2 .

13 ac

II al iI 1

..1

2 ../3 4. a

V / 8 §.. a0

g . a J3 3 .1 . I

r . . 1. . ....

1 1 3 ./ 4 2 4,

x., 2

2 Om *4

1 i ... II E 1.. e e5 ...3 &

. ...... Z' -:;-;

$-

(.10 3 t u 4:14

.-4....

0.4,0....

.i140

1-6

44. 6

La I44WI

. .

Percentage ofDoctora t41111 7030

Total Nesearch .11Egpenditi.lio -.SS -.30v1- .06 -.21 -.OS -.32

3

1

-.14 -.44 -.0.! -.4, -.1.1 7 *Ai

.. ' Total 'Expenditures -.60 '-21,. . ..(.,0 1 .1 - .20 0% :,,. t.,,,.''.,' f". Government Grants and,. .' - ..

Cant cacts..Por FTC ov-i-Slit -.2411 ' -63 ..411 ..16Avorage Faculty ,

Salary ' 6..41i . -.56 ...II -...10 ,-.43 .l0 ..-.S240

, e ..-

0 r-.4 -`10"4.4.4-4 - "

VPIP

t42§Ssi4t....

4.

. s

#;u44

, . , ,n.

, . . .

. .

;

I

1a

.23 241, ''. .i7

7. 4 -.33 wall ...34 ...III *. .36 .36-..4.I . .12 *,..14 as , .**.35 -.02 . .33 .44

*0,13 ,...39 .Al .7.30 .02 -.34 -...04 .lit) .41

-.44 ".3 . i . .40 -.4'1 .44 ......1111 ...At! ..42. .113,;

.43.

.

+3.

. ."

cf. . .

. .

: -."

'

, .