the counseling psychologist - university of memphis

http://tcp.sagepub.com

The Counseling Psychologist

DOI: 10.1177/0011000005277814 2005; 33; 709 The Counseling Psychologist

Walter C. Buboltz, Jr., Steve M. Jenkins, Adrian Thomas, Lori D. Lindley, Jonathan P. Schwartz and James M. Loveland Research Productivity in Counseling Psychology: An Update

http://tcp.sagepub.com/cgi/content/abstract/33/5/709 The online version of this article can be found at:

Published by:

http://www.sagepublications.com

On behalf of:

Division of Counseling Psychology of the American Psychological Association

can be found at:The Counseling Psychologist Additional services and information for

http://tcp.sagepub.com/cgi/alerts Email Alerts:

http://tcp.sagepub.com/subscriptions Subscriptions:

http://www.sagepub.com/journalsReprints.navReprints:

http://www.sagepub.com/journalsPermissions.navPermissions:

http://tcp.sagepub.com/cgi/content/refs/33/5/709SAGE Journals Online and HighWire Press platforms):

(this article cites 7 articles hosted on the Citations

© 2005 Division 17 of Counseling Psychologist Association. All rights reserved. Not for commercial use or unauthorized distribution. at Ebsco Host temp on January 17, 2008 http://tcp.sagepub.comDownloaded from

http://www.div17.org

http://tcp.sagepub.com/cgi/alerts

http://tcp.sagepub.com/subscriptions

http://www.sagepub.com/journalsReprints.nav

http://www.sagepub.com/journalsPermissions.nav

http://tcp.sagepub.com/cgi/content/refs/33/5/709


10.1177/0011000005277814THE COUNSELING PSYCHOLOGIST / September 2005Buboltz et al. / RESEARCH PRODUCTIVITY• Professional Forum

Research Productivity in Counseling Psychology:An Update

Walter C. Buboltz Jr.Louisiana Tech University

Steve M. JenkinsWagner College

Adrian ThomasAuburn University

Lori D. LindleyJonathan P. SchwartzJames M. Loveland

Louisiana Tech University

This article is an update and review of institutional research productivity in counselingpsychology. Institutional research productivity is assessed by totaling credits for articlespublished from 1993 to 2002 in the following journals: Journal of Counseling Psychol-ogy, The Counseling Psychologist, Journal of Consulting and Clinical Psychology, Jour-nal of Vocational Behavior, Journal of Counseling and Development, Journal of Multi-cultural Counseling and Development, and Cultural Diversity and Ethnic MinorityPsychology. Results show that the rankings of many programs have remained fairly sta-ble over the years, while others have substantially changed. Additionally, two factorswere found to represent the research productivity of institutions in the field. These resultsare discussed in terms of the identity of counseling psychology and research productivity.

When the field of counseling psychology was developed in the 1940s, itsinitial role was generally seen as one that would fill a perceived gap betweenindustrial/organizational psychology and clinical psychology (Gelso &Fretz, 2001). A major emphasis of this new branch of psychology was tostudy and treat normal personality, functioning, and development. Thisemphasis continues to be integral to counseling psychology’s identity today.However, over the past 60 years, counseling psychology has evolved into adistinct discipline that contains diverse areas of specialization and that con-tinues to grow and further define its role within psychology. In fact, Division17 (Society for Counseling Psychology) of the American Psychological

709

Please address all correspondence to the first author at Department of Psychology, LouisianaTech University, PO Box 10048, Ruston, LA 71272.

THE COUNSELING PSYCHOLOGIST, Vol. 33 No. 5, September 2005 709-728DOI: 10.1177/0011000005277814© 2005 by the Society of Counseling Psychology



Association has 9 sections and 11 special interest groups, including sectionson ethnic and racial diversity, counseling health psychology, and teachingand research on qualitative methods (Division 17 of the American Psycholog-ical Association, 2003).

Training programs in counseling psychology reflect the evolution of thefield, with most institutions ascribing to the scientist-practitioner model.With this model, students are afforded the training and knowledge to becomeproficient practitioners and researchers. As the field continues to grow anddiversify, its identity, and thus its outlets for research, will broaden accord-ingly. This broadening may give clues to the evolving identity of counselingpsychology.

By examining the primary journals in which counseling psychologistshave published, we may be able to identify trends in the areas of research thatare being pursued by the major academic counseling psychology trainingprograms. Since each journal has a mission and publishes research that isconsistent with that mission, changes observed in the number of publicationsindicate that researchers and training programs are either focusing moreclosely on the area covered by the journal or are focusing on areas not cov-ered by the journal and therefore publishing in other journals. These potentialchanges in the research productivity of institutions for the journals over theyears may indicate that other areas of counseling psychology or psychologyin general are emerging as part of the identity of counseling psychology. Forexample, if a noticeable decrease was noted in the publication numbers forthe Journal of Counseling Psychology across years, while an increase wasnoted in the Journal of Vocational Behavior, then the indication may be thatcareer issues are becoming more salient than the areas covered by Journal ofCounseling Psychology. We can further extrapolate that if new areas areemerging in training programs through research and productivity, these areasare being passed to trainees during their doctoral education. Through the pro-cess of examining research productivity one can better understand theidentity of counseling psychology and gain insight into the future direction ofthe field.

The quality of graduate training programs has traditionally been assessedin two ways: reputation (Roose & Anderson, 1970) or research productivity(Howard, Cole, & Maxwell, 1987). Over the years debate regarding whichmethod is the most fruitful and accurate has been heard. Cox and Catt (1977)argue against the use of reputation ratings in determining the quality of grad-uate programs in the field of psychology. They argue that use of reputationmay be fraught with numerous errors that can be misleading. Therefore therehas been a long history of tracking institutional research productivity of pro-grams by the frequency of publications in key journals. The quality of eacharticle was not considered because this is beset with potential pitfalls. The

710 THE COUNSELING PSYCHOLOGIST / September 2005



overall quality was considered to be at a minimum level as each article hasundergone a review by an editorial board and editors for its relevance,quality, and contribution.

However, relying solely on research productivity data or reputation maybe misleading as to the quality of graduate training. The use of research pro-ductivity may not reflect the actual training and quality of graduates from aprogram. Several other markers may be used to assess the quality of graduatetraining in counseling psychology. For example, one could look at internshipand employment placement, scores on the National Licensing Examination,and other potential outcome variables. Examining every marker to determinethe quality of a graduate program was beyond the scope of this study. Thisstudy focused on research productivity as one of the many potentialimportant aspects of program quality and identity.

Additionally, in the current environment of accountability, national rank-ings (e.g., National Research Council), and cost structures, programs that donot add to the value and prestige of a university are overlooked and in somecases phased out. This point was made clear at the midwinter (2004) meetingof the Council of Counseling Psychology Training Programs as many coun-seling psychology doctoral programs would not be included in national rank-ings because they are housed in colleges of education and not in departmentsof psychology. The current study describes one way that programs can showtheir standing in the field and provide administrators with data about the pro-ductivity and value of the program. Additionally, the ranking of programsbased on research productivity can provide a means by which programs canincrease their reputation and value to a university.

Initially, counseling psychology research productivity was defined by theinstitutional contributions to the Journal of Counseling Psychology (Cox &Catt, 1977) and was later expanded to five representative journals (cf.Howard, 1983). The most recent studies on productivity in counseling psy-chology that considered multiple journals as representing the field were byDelgado and Howard (1994) covering 1983-1992 and by Diegelman,Uffelman, Wagner, and Diegelman (2005) covering 1993-2002. However,note that Diegelman et al. employed only four of the journals from Delgadoand Howard, omitting Journal of Consulting and Clinical Psychology(JCCP). The journals included in Diegelman et al. were determined based onthe frequency of endorsement by training directors of counseling psychologyprograms. Diegelman et al. only included the top four most frequentlyendorsed journals in which counseling psychology faculty and students pub-lish, as perceived by directors of training. The authors state that by limitingthe review to four journals that they believed most represented the fieldwould lead to a better capture of the research productivity of faculty and stu-dents in counseling psychology. Although we agree with this rationale to

Buboltz et al. / RESEARCH PRODUCTIVITY 711



some degree, this may not be the case. Due to the changing nature of the fieldand the development of other substantive areas of research not covered bythese journals, many potentially important areas may have been left out oftheir analysis.

Thus, to correct for this potential problem, this study included two addi-tional journals to better encompass the identity of counseling psychology andreflect its commitment to diversity: the Journal of Multicultural Counselingand Development and Cultural Diversity and Ethnic Minority Psychology.This study also updates and extends the current research productivity incounseling psychology by including two additional journals that are believedto represent the field. The inclusion of these two journals may help alleviatethe potential shortcomings of previous reviews (Delgado & Howard, 1994;Diegelman et al., 2005) that focused on journals that have been traditionallybelieved to represent counseling psychology. Additionally, because the Jour-nal of Counseling Psychology has been used as a primary journal for trackingresearch productivity over the past 35 years in counseling psychology, thisjournal is also examined individually for comparison purposes andexamination of productivity changes over the years.

The Diegelman et al. (2005) study used four of the original journals of theDelgado and Howard (1994) study, postulating JCCP no longer representedcurrent counseling psychology research. This belief was based on responsesfrom training directors rather than a representative sample of counseling psy-chologists. Since JCCP was not included in their analysis it cannot be empiri-cally determined if JCCP represents counseling psychology. Because of thispotential shortcoming, this study included all the original journals as well asthe additional two. Inclusion of the all the original journals allows for replica-tion and testing of the Delgado and Howard model to determine if that modelis still current. Additionally, by examining the contributions to representativejournals some insight can be gained into the identity of counseling psychol-ogy. By examining changes or lack thereof in the number of publications injournals that represent counseling psychology we can get an idea of thefield’s identity based on the missions of the journals.

Finally, by examining the productivity across the seven journals we canbetter understand the research productivity of programs based on the currentstate of the field. Although the purpose was not to rank programs in terms oftheir overall quality of training these data may be used to gain some insight onone aspect of the quality of training. Institutions that are more active inresearch and publication may afford students more opportunities to learnresearch and publication skills. Additionally, the data of this study can beused by programs to increase their reputation and to show their value to theirinstitutions.




METHOD

To be consistent and to allow for comparisons with the previous produc-tivity studies, the same methodology was employed in this study to obtainresearch productivity ratings. Howard (1983) identified the five top journalsin which counseling psychologists publish (noted above). However, in addi-tion to the original five journals two other journals were also included in thissecondary analysis to better represent the field. The original five journalswere included in the current study to allow for accurate comparisons and totest the model proposed by Delgado and Howard (1994). Additionally, fourof the journals were found to represent the field based on responses fromtraining directors in the Diegelman et al. (2005) study. The inclusion of thetwo new journals reflects the changing nature of the field and reflects opin-ions of editors and editorial board members. Additionally, as was noted byBuboltz, Williams, and Miller (1999), the content areas of psychology havebeen increasing and thus may not be reflected in the more traditionalcounseling psychology journals.

Credit for authorship was assigned in the following manner: Articlesreceived a total authorship institution credit of 1.0. Thus, the institution of anindividual responsible for a single-authored article was credited with 1.0points. The institution of the first author of a coauthored article received .67points, and the second author’s institution received .33 points. For articleswith three authors, the breakdown follows: .50 for first author’s institution,.30 for second author’s institution, and .20 for third author’s institution. Simi-lar divisions of credit for articles having more than three authors wereobtained by using Howard et al.’s (1987) formula for determining productiv-ity credit, on the basis of an article’s number of authors and each author’sordinal position on the article.

The period covered by the current analysis ranged from 1993 to 2002,inclusively. For each of the seven journals, all articles were included in theanalysis except for letters to the editor, introductions to special sections, bookreviews, presidential addresses, and test reviews. Comments, brief reports,and reactions were included in the analysis. All authors (including students)were given the appropriate institutional credit depending on their ordinalrank. Note that institutions that do not have doctoral programs in counselingpsychology were included in these analyses to help determine the overallstanding of counseling psychology programs in the field. A counseling psy-chology program was not teased apart from institutional affiliation becausewe were partially interested in publication trends in the journals to get someinsight about the identity of the field. The exclusion of non–counseling psy-chology program faculty would tend to overestimate the contribution of fac-




ulty members while excluding the contribution to the field of counseling psy-chologists in other employment settings (e.g., counseling centers or master’sprogram in counseling) that are clearly involved in training. Additionally,allowing other individuals who were not formally part of a counseling psy-chology program to contribute to an institution’s rank would eliminatecollaborative efforts despite the fact that the article was published in a journalexamined in the study.

The number of articles reviewed for each journal follow: Journal of Coun-seling Psychology, 496 articles; JCCP, 1,100 articles; Journal of Counselingand Development, 601 articles; Journal of Vocational Behavior, 445 articles;The Counseling Psychologist, 344 articles; Journal of Multicultural Coun-seling and Development, 205 articles; and Cultural Diversity and EthnicMinority Psychology, 135 articles.

RESULTS

All results for this study employed the same methodology (except for theadditional journals included in the second analysis) and statistical analyses asthose employed by Delgado and Howard (1994). The same procedures andanalyses were used to allow for accurate comparisons between the two stud-ies. A confirmatory factor analysis was conducted on the five previous targetjournals to determine the degree of relationship between the latent variable(institutional productivity) and the observed variables (five measurements ofthe latent variable). A second factor analysis was conducted for all seven ofthe journals that were identified to currently represent the field. Due to thepossible impact of faculty size on research productivity, productivity rank-ings were also adjusted for the number of faculty members in each counsel-ing psychology program. The faculty size used was the current number ofcore counseling psychology faculty members listed for each program. Gath-ering information about the amount of time that each faculty member con-tributes to the program or about contributions of noncore or adjunct facultymembers was impossible. This adjustment was made by simply dividing theoverall productivity index for each university by the number of core facultymembers at the time of preparation of the current manuscript. However, notethat this adjustment was only made for counseling psychology programs asinformation was not available for other programs that contributed to thejournals.

Results of the first confirmatory factor analysis showed that the data didnot satisfactorily fit a one-factor model, �2(5, N = 1,133) = 13.32, p = .02.Removal of JCCP resulted in a one-factor model to which the data showedadequate fit, �2(2, N = 1,133) = .258, p = .879. Note, however, that �2 is




dependent on sample size, and Delgado and Howard’s (1994) reliance on it astheir only measure of fit may be viewed as a limitation of their work. As such,the current study obtained additional measures of the fit of the data to themodel. For the current one-factor model based on the four journals, the com-parative fit index (CFI) was 1.0, the goodness-of-fit index (GFI) was .99, andthe root mean square residual was .01. According to generally accepted prac-tice, CFIs and GFIs exceeding .95 and root mean square residuals of less than.08 demonstrate good fit.

Table 1 presents the productivity rating on the four journals for the institu-tions that ranked in the top 40 for overall productivity for 1993-2002. Thefour journals were included in the table for comparison purposes. The firstfour columns present the productivity for each institution across the fourjournals. The fifth column (“overall productivity index”) is the factor scorederived from the confirmatory factor analysis on the productivity scores forthe four journals, not including JCCP. The sixth column shows the currentrank without faculty size adjustment. The seventh column shows the produc-tivity index and rank adjusted for faculty size. The eighth column shows therank from the Delgado and Howard (1994) study. However, note thatDelgado and Howard used all five journals in their factor analysis.

Examination of Table 1 shows that the University of Missouri–Columbiaranks first in the present study and the previous study. Additionally, the Uni-versity of Maryland–College Park ranks second in the present and previousstudies. Overall, examination of Table 1 shows that 7 of the institutions thatwere ranked in the top 10 in 1994 are still ranked in the top 10. However, notethat a few institutions that were ranked outside the top 20 in 1994 are nowranked within the top 20. Additionally, 12 institutions that were not ranked inDelgado and Howard (1994) are ranked in the top 40 in the current study.

Comparing the institutional contributions of each journal in the presentstudy for top 10 ranked institutions with the institutional contributions inDelgado and Howard (1994) reveals interesting results. First, most of theleading institutions have increased their contributions in the Journal ofCounseling Psychology. For JCCP, contributions from the top 10 rankedinstitutions have uniformly decreased. Finally, for the other three journals,contributions have been fairly stable with only minor changes incontributions between the two studies.

When rankings were adjusted for faculty size, the top three institutionsshowed little change in rank. However, noticeable changes were observed forother institutions. Four institutions previously outside the top 10 in thenonadjusted rankings moved into the top 10. Note also that several institu-tions changed little in their rank (e.g., University of Mississippi moved from16 to 17), while others showed large changes (e.g., University of North Texasmoved from 19 to 33). However, these numbers should be interpreted with




716

TA

BL

E 1

:P

rodu

ctiv

ity

Rat

ings

on

the

Four

Jou

rnal

s fo

r th

e 40

Top

Ins

titu

tion

s R

anke

d on

Ove

rall

Pro

duct

ivit

y Fa

ctor

Sco

res

Ove

rall

Ran

kFa

cult

yR

ank

ofR

ank

ofP

rodu

ctiv

ity

Pre

sent

Adj

uste

dD

ieg

1994

Inst

itut

ion

JCP

JCD

JVB

TC

PIn

dex

Stud

yIn

dex

(Ran

k)St

udy

Stud

y

Uni

vers

ity o

f M

isso

uri–

Col

umbi

a41

.71

23.1

48.

0014

.65

15.0

391

1.11

(2)

11

Uni

vers

ity o

f M

aryl

and–

Col

lege

Par

k44

.53

10.3

711

.04

14.6

014

.622

21.

88 (

1)2

2U

nive

rsity

of

Akr

on10

.51

7.80

13.9

99.

646.

907

31.

02 (

4)3

18U

nive

rsity

at A

lban

y, S

tate

Uni

vers

ity13

.77

7.33

6.00

9.69

6.65

84

0.77

(9)

44

of N

ew Y

ork

Ari

zona

Sta

te U

nive

rsity

9.50

9.46

5.95

8.95

5.84

45

0.61

(15

)7

9Io

wa

Stat

e U

nive

rsity

12.6

62.

5014

.63

4.34

5.15

06

0.76

(10

)6

29U

nive

rsity

of

Iow

a6.

875.

561.

999.

174.

662

70.

81 (

7)15

5O

hio

Stat

e U

nive

rsity

5.99

9.22

12.4

15.

094.

589

80.

48 (

20)

83

Vir

gini

a C

omm

onw

ealth

Uni

vers

ity14

.29

0.94

2.84

5.84

4.58

29

0.68

(11

)13

10U

nive

rsity

of

Illin

ois,

Urb

ana-

Cha

mpa

ign

15.5

13.

0010

.54

2.08

4.57

310

0.67

(12

)9

12U

nive

rsity

of

Not

re D

ame

4.50

0.63

2.83

11.1

44.

480

110.

67 (

13)

2111

Uni

vers

ity o

f M

inne

sota

–Min

neap

olis

0.67

4.80

13.0

08.

014.

210

121.

10 (

3)5

8U

nive

rsity

of

Cal

ifor

nia–

Sant

a B

arba

ra7.

670.

334.

507.

173.

919

13N

/A16

7Pe

nnsy

lvan

ia S

tate

Uni

vers

ity7.

334.

159.

463.

823.

649

140.

95 (

5)10

14So

uthe

rn I

llino

is U

nive

rsity

–Car

bond

ale

9.33

1.70

11.2

92.

723.

609

150.

62 (

14)

1415

Uni

vers

ity o

f So

uthe

rn M

issi

ssip

pi5.

339.

050.

006.

243.

572

160.

53 (

17)

18N

RM

ichi

gan

Stat

e U

nive

rsity

8.76

6.65

6.12

3.06

3.56

417

0.92

(6)

1230

Uni

vers

ity o

f W

isco

nsin

–Mad

ison

7.01

6.52

0.00

5.53

3.40

618

0.39

(23

)22

36U

nive

rsity

of

Nor

th T

exas

4.53

5.09

2.47

6.17

3.27

619

0.31

(33

)24

6B

osto

n C

olle

ge8.

002.

661.

335.

173.

236

200.

37 (

24)

27N

R



717

Col

umbi

a U

nive

rsity

Tea

cher

s C

olle

ge4.

183.

502.

016.

833.

226

210.

48 (

18)

2532

Uni

vers

ity o

f Fl

orid

a4.

832.

642.

506.

553.

222

220.

48 (

19)

1717

Bal

l Sta

te U

nive

rsity

5.57

4.20

1.33

5.84

3.15

823

0.27

(36

)11

NR

Loy

ola

Uni

vers

ity o

f C

hica

go7.

781.

505.

013.

722.

993

240.

78 (

8)26

NR

Uni

vers

ity o

f W

isco

nsin

–Milw

auke

e5.

831.

504.

534.

502.

841

250.

37 (

25)

30N

RTe

xas

Tech

Uni

vers

ity6.

304.

652.

003.

942.

790

260.

36 (

26)

28N

RU

nive

rsity

of

Ore

gon

8.50

1.50

0.33

3.74

2.65

427

0.55

(16

)29

31In

dian

a U

nive

rsity

1.67

6.03

1.34

5.18

2.41

428

0.25

(37

)19

NR

Uni

vers

ity o

f So

uthe

rn C

alif

orni

a5.

831.

610.

004.

502.

391

290.

31 (

32)

3133

Ford

ham

Uni

vers

ity2.

872.

000.

005.

972.

364

300.

41 (

22)

NR

NR

Col

orad

o St

ate

Uni

vers

ity7.

001.

183.

511.

882.

076

310.

18 (

42)

3520

Uni

vers

ity o

f N

ebra

ska

4.53

2.86

1.20

3.50

2.07

532

0.43

(21

)36

16U

nive

rsity

of

Uta

h2.

841.

202.

004.

502.

013

330.

35 (

27)

NR

39W

ashi

ngto

n St

ate

Uni

vers

ity2.

000.

504.

004.

321.

938

340.

29 (

34)

39N

RU

nive

rsity

of

Mem

phis

3.33

2.67

1.67

3.62

1.92

635

0.33

(28

)N

RN

RU

nive

rsity

of

Okl

ahom

a0.

844.

832.

604.

041.

910

360.

29 (

35)

33N

RU

nive

rsity

of

Kan

sas

3.07

1.95

3.67

3.26

1.89

737

0.33

(30

)N

R24

Uni

vers

ity o

f G

eorg

ia1.

5012

.04

6.04

0.20

1.83

438

0.24

(39

)23

25U

nive

rsity

of

Cal

ifor

nia–

Los

Ang

eles

4.65

0.50

1.00

3.17

1.74

539

N/A

NR

23Il

linoi

s St

ate

Uni

vers

ity7.

671.

671.

000.

831.

652

40N

/A40

NR

NO

TE

:JC

P=

Jour

nalo

fCou

nsel

ing

Psy

chol

ogy;

JCD

=Jo

urna

lofC

ouns

elin

gan

dD

evel

opm

ent;

JVB

=Jo

urna

lofV

ocat

iona

lBeh

avio

r;T

CP

=T

heC

ouns

elin

gP

sych

olog

ist;

Die

g=

Die

gelm

anet

al.(

2005

);U

=un

iver

sity

;NR

=no

tran

ked;

N/A

=no

doct

oral

prog

ram

inco

unse

ling

psyc

holo

gy.H

ighe

rpro

duct

ivity

num

bers

indi

cate

grea

terp

rodu

ctiv

ity;l

ower

num

eric

alra

nkin

gsin

dica

tegr

eate

rpro

duct

ivity

.The

over

allp

rodu

ctiv

ityin

dex

repr

esen

tsth

esu

mac

ross

the

four

jour

nals

ofth

e pr

oduc

t of

each

inst

itutio

n’s

prod

uctiv

ity to

tal a

nd th

at jo

urna

l’s lo

adin

g on

the

prod

uctiv

ity f

acto

r.



caution, as an institution with 5 faculty members that produces a total of 10articles a year would rank equally with an institution with 20 faculty mem-bers that produces a total of 40 articles a year. However, in overall productiv-ity, universities with larger faculty numbers are clearly contributing more tothe field even when the data are corrected for faculty size.

Results of the second confirmatory factor analysis showed that a two-factor model represented a satisfactory fit for the data from the seven jour-nals. Again, as �2 is dependent on sample size, more appropriate fit indiceswere computed. These were computed on the entire two-factor model (i.e.,we did not compute different fit indices for each factor separately). The esti-mates of fit for the data to the two-factor model follow: .96 for the CFI, .97 forthe GFI, and .08 for the root mean square residual. These numbers are againtaken as evidence that the data provided sufficient fit to the two-factor model.

The first factor was represented by the same four journals that were high-lighted in Table 1. Results for the first factor have previously been discussed.The second factor was represented by JCCP, Cultural Diversity and EthnicMinority Psychology, and Journal of Multicultural Counseling andDevelopment.

Table 2 presents the productivity rating on the three journals of factor 2 forthe institutions that ranked in the top 40 for overall productivity during 1993-2002. Note that Table 2 presents data only for the institutions that have coun-seling psychology doctoral programs. However, the rankings reflect rankswhen including all institutions. Examination of Table 2 shows that out of thetop 40 institutions only 18 have doctoral programs in counseling psychology.This may indicate that the field as represented by doctoral programs in coun-seling psychology is not publishing in the areas covered by this cluster ofjournals as much as it publishes in the more traditional areas of counselingpsychology (i.e., those in factor 1).

The top two ranked institutions were the University of California–LosAngeles and the University of Hawaii–Manoa, both of which do not havedoctoral programs in counseling psychology. The next two ranked institu-tions were the University of Maryland–College Park and Columbia Univer-sity Teachers College, both of which have doctoral programs in counselingpsychology. Also note that examination of the contributions to each journalon which the rankings were based tends to decrease dramatically for institu-tions with counseling psychology programs once past the top 12 counselingpsychology programs. These results tend to indicate that the areas of diver-sity and multiculturalism as well as other areas which are part of counselingpsychology may not be as dominant in the research areas of faculty at thedoctoral programs. Comparisons among counseling psychology doctoralprograms after adjusting for the size of faculty show some changes in therankings; however, the top institutions tend to remain high.





TABLE 2: Productivity Ratings for the Second Factor Analysis for the 40 Top Institu-tions Ranked on Overall Productivity Factor Scores

FacultyOverall Adjusted

Productivity Present IndexInstitution JCCP CDEMP JMCD Index Rank (Rank)

University of Maryland– 2.22 4.20 5.21 5.93 3 .74 (2)College Park

Columbia University 4.54 3.39 4.6 5.10 4 .73 (3)Teachers College

University of Wisconsin– 5.68 2.14 5.52 4.53 6 .5 (6)Madison

Pennsylvania State 7.24 2.89 3.46 4.32 7 1.08 (1)University

Ohio State University 10.8 1.79 1.92 2.89 10 .29 (13)University of Minnesota– 3.0 2.33 1.16 2.49 15 .36 (7)

MinneapolisUniversity of Missouri– 3.34 0 5.13 2.36 16 .17 (24)

ColumbiaFordham University 0 0 5.08 2.12 18 .35 (8)University of Southern 11.40 1.12 1.35 2.09 19 .26 (15)

CaliforniaMichigan State University 4.33 0 4.27 2.03 20 .51 (5)Temple University 15.57 0.21 2.23 1.98 21 .33 (9)University of Memphis 10.95 0.38 2.47 1.94 23 .32 (10)University of Louisville 1.0 0 4.28 1.82 29 .30 (11)University of Illinois, 1.0 1.42 1.53 1.76 31 .25 (16)

Urbana-ChampagneArizona State University 5.2 0 3.56 1.75 32 .18 (22)Oklahoma State 0.5 1.47 1.0 1.52 34 .15 (30)Western Michigan 0 1.0 1.88 1.49 35 .17 (26)

UniversityHoward University 0 0 3.64 1.45 37 .29 (12)Southern Illinois University– 1 1 1.38 1.33 41 .22 (19)

CarbondaleUniversity of Texas–Austin 8.74 0.96 0.35 1.31 42 .22 (20)Washington State University 1.37 0 3.0 1.25 47 .18 (21)West Virginia University 2.33 1.0 1.0 1.24 48 .16 (29)Seton Hall 0 1.28 0.79 1.23 49 .25 (17)University of Florida 2.56 0 2.79 1.22 50 .18 (23)Virginia Commonwealth 7.46 1.07 0 1.16 51 .17 (25)

UniversityUniversity of Georgia 5.41 0 2.20 1.13 52 .13 (32)University of Colorado 0.52 0.08 2.10 0.84 60 –.02 (66)University of Oregon 2.01 0 2.0 0.82 61 .17 (27)Stanford University 4.52 0.89 0 0.82 62 .27 (14)Georgia State University 2.78 0 1.88 0.82 63 .10 (37)

(continued)



The intercorrelations among the seven journals and each journal’s loadingon the overall productivity factor score are presented in Table 3. Examinationof the table shows that most intercorrelations and productivity factor scoresare high, except for the intercorrelations with JCCP and the other four moretraditional journals that represent counseling psychology. The lowintercorrelations between JCCP and the four original journals are not sur-prising given the fact that JCCP is not part of the first factor. Additionally, thecorrelation between the two factors was .36 (p < .01). Thus, despite the differ-ent rank order of the institutions, the two factors are still moderatelycorrelated.

Table 4 highlights more than 30 years of productivity research that isbased on the Journal of Counseling Psychology. Several studies in the pasthave used the Journal of Counseling Psychology as the sole criterion or partof the criterion for research productivity (Cox & Catt, 1977; Delgado &Howard, 1994; Howard, 1983). Examination of the validity coefficients fromthe various factor analyses conducted have ranged from .68 (Howard, 1983)to .93 for the present study. Note that the first four columns of rankings inTable 4 are directly comparable since all the studies used similar methodolo-gies for assigning author credit and are based on institutional productivitycredits.


University of Oklahoma 0 0 2.25 0.81 64 .12 (35)University of Nebraska 0.30 0 2.20 0.80 65 .16 (28)Texas A&M University 0.84 1.0 0.26 0.80 67 .13 (33)University of Pittsburgh 13.79 0 0 0.66 78 .22 (18)University of Akron 0 1.0 0 0.62 82 .09 (39)University of Iowa 3.93 0.16 1.0 0.62 89 .10 (36)University of North Texas 1.0 0.64 0.40 0.56 95 .05 (42)Loyola University of 5.64 0 0.74 0.47 102 .12 (34)

ChicagoBoston College 0 0 1.47 0.45 104 .05 (43)Lehigh University 0 0 1.46 0.44 106 .09 (40)

NOTE: JCCP = Journal of Consulting and Clinical Psychology; CDEMP = Cultural Diversityand Ethnic Minority Psychology; JMCD = Journal of Multicultural Counseling and Develop-ment; N/A = no doctoral program in counseling psychology. Higher productivity numbers indi-cate greater productivity; lower numerical rankings indicate greater productivity. The overallproductivity index represents the sum across the three journals of the product of each institution’sproductivity total and that journal’s loading on the productivity factor.

TABLE 2 (continued)

FacultyOverall Adjusted

Productivity Present IndexInstitution JCCP CDEMP JMCD Index Rank (Rank)



Examination of Table 4 shows that the University of Maryland–CollegePark has ranked number one across all studies. Additionally, the Universityof Missouri–Columbia and Southern Illinois University–Carbondale haveboth consistently ranked in the top 10 across all studies. Four schools thatwere not ranked in the top 10 in Delgado and Howard (1994) are now rankedin the top 10, with three (Iowa State University, University of Akron, and Ari-zona State University) rising in the rankings. Finally, seven schools that werenot ranked in the top 20 in the Delgado and Howard or Howard et al. (1987)studies are ranked in the top 20 in the present study.

DISCUSSION

This study examined the institutional productivity rankings for 1993-2002 as represented in seven leading counseling psychology journals. Theresults indicate that two factors best represent the field based on publicationin these journals.

The results of the first factor were generally consistent across most jour-nals, with 7 of the top 10 institutions in the current study having appeared inthe top 10 in Delgado and Howard’s (1994) study. Additionally, the results ofthe current study are consistent with Diegelman et al. (2005), with 8 commoninstitutions in the top 10. Note that the rankings in Diegelman et al. are based


TABLE 3: Intercorrelations Among Journals

Journal JCP JCD JVB TCP JCCP CDEMP JMCD

JCP — — — — — — —JCD .62** — — — — — —JVB .60** .48** — — — — —TCP .78** .61** .60** — — — —JCCP .20** .17** .24** .21** — — —CDEMP .22* .12** .15** .19** .34** — —JMCD .45** .43** .30** .44** .27** .49** —Productivity .93** .74** .72** .93** — — —

Factor 1Productivity — — — — .51** .92** .75**

Factor 2

NOTE: JCP = Journal of Counseling Psychology; JCD = Journal of Counseling and Develop-ment; JVB = Journal of Vocational Behavior; TCP = The Counseling Psychologist; JCCP = Jour-nal of Consulting and Clinical Psychology; CDEMP = Cultural Diversity and Ethnic MinorityPsychology; JMCD = Journal of Multicultural Counseling and Development; Productivity Fac-tor 1 = productivity score for Factor 1; Productivity Factor 2 = productivity score for Factor 2.*p < .05. **p < .01.




TABLE 4: Productivity Rankings in Journal of Counseling Psychology for Five StudiesDuring 1970-2002

Howard,Delgado Cole, &

Present & Howard Maxwell Howard Cox & CattStudy 1983-1992 (1987) (1983) (1977)

Institution 1993-2002 (1994) 1976-1985 1976-1982 1970-1975

University of Maryland 1 1 1 1 1University of Missouri 2 3 3 4 4University of Illinois, 3 7 17 26 16

Urbana-ChampaignVirginia Commonwealth 4 4 10 11 NR

UniversityUniversity of Albany 5 2 5 14 NRIowa State University 6 23 8 6 29University of Akron 7 NR 14 NR NRArizona State University 8 25 NR NR NRSouthern Illinois University– 9 5 4 5 5

CarbondaleMichigan State University 10 18 40 NR 7University of Oregon 11 11 NR NR NRBoston College 12 NR NR NR NRLoyola University of Chicago 13 40 NR NR NRIllinois State University 14.5 NR NR NR NRUniversity of California– 14.5 6 9 10 NR

Santa BarbaraPennsylvania State 16 15 11 25 13

UniversityUniversity of Wisconsin– 17 34.5 NR NR 8

MadisonColorado State University 18 22 14 9 12University of Iowa 19 12 13 18 18Texas Tech University 20 NR NR 27 NROhio State University 21 10 2 2 3University of Southern 22.5 24 NR NR NR

CaliforniaUniversity of Wisconsin– 22.5 NR NR NR NR

MilwaukeeBall State University 24 NR NR NR NRUniversity of Southern 25 NR NR 29 NR

MississippiUniversity of Florida 26 21 32 33 NRUniversity of California– 27 32 NR NR NR

Los AngelesUniversity of Nebraska 28.5 8 6 7 NRUniversity of North Texas 28.5 13 21 38 NRUniversity of Notre Dame 30 9 12 13 NR

(continued)



on summation of the contributions to each of the four journals, while the cur-rent study used an overall productivity score based on factor analysis. Thedifferences in deriving the overall productivity scores between the currentstudy and Diegelman et al. lead to several differences in the rankings. Thesedifferences are primarily due to the fact that institutions that published morein journals with higher factor loadings using the factor analysis approachwould increase an institution’s overall productivity index more than wouldbe obtained by just adding contributions for each journal. The difference inrankings based on the two methodologies can be seen in Table 1.

In contrast to the Delgado and Howard (1994) study, the results of thisstudy indicated that JCCP does not constitute a large proportion of theresearch productivity of counseling psychology in the first factor. This find-ing may indicate that a proportion of the research is no longer appropriate forJCCP. Note that JCCP was significant in the second factor. Based on whatwere previously considered traditional journals in counseling psychology,this may reflect a change in the identity or focus of counseling psychology.The articles that in the past would have been published in JCCP are nowbeing published in other journals that have a different mission and focus,such as multiculturalism. This is consistent with the focus on emphasizingthe unique and dynamic identity of counseling psychology (Fouad et al.,2004; Sue, 2003).


Columbia University 31 NR NR NR NRTeachers College

University of Memphis 32 NR NR NR NRUniversity of Kansas 33 28.5 18 12 15Fordham University 34 NR NR NR NRUniversity of Utah 35 20 16 19 NRWashington State University 36 36.5 NR 40 NRIndiana University 37 NR 19 15 11University of Georgia 38 NR NR NR NRUniversity of Oklahoma 39 NR NR 37 NRUniversity of Minnesota– 40 17 7 8 2

Minneapolis

NOTE: NR = not ranked. Lower numerical rankings indicated greater productivity.

TABLE 4 (continued)

Howard,Delgado Cole, &

Present & Howard Maxwell Howard Cox & CattStudy 1983-1992 (1987) (1983) (1977)

Institution 1993-2002 (1994) 1976-1985 1976-1982 1970-1975



Results also showed that a few institutions have made great strides interms of research productivity and have moved up in the rankings. Forinstance, the University of Akron has moved from 18th (Delgado & Howard,1994) to 3rd; Iowa State University moved from 29th to 6th; and the Univer-sity of Southern Mississippi moved from not being ranked to 16th. The over-all change in rankings of the various programs in the current study is similarto the reported changes identified by Diegelman et al. (2005). Examination ofthe contributions for each journal across the various studies showed minimalchanges in the contributions to each journal, except for JCCP.

Results for faculty-adjusted rankings show that the top institutionschanged modestly in the rankings. However, several institutions with smallerfaculty moved up in the rankings. As was noted, these results should be takenwith a degree of caution as institutions with fewer faculty may receive ahigher ranking, despite an overall lower number of contributions.

Results of the productivity ratings based solely on the Journal of Counsel-ing Psychology show that 4 of the 5 top schools in the present study were alsoin the top 5 in the Delgado and Howard (1994) study. Also 6 of the top 10 inthe present study were in the top 10 in the previous study. These results lendeven more credence to the stability of the ratings over time. Despite the rela-tive stability, a few noteworthy changes have occurred in productivity ratingsbased on contributions to the Journal of Counseling Psychology. Specifi-cally, Iowa State, Arizona State, and Michigan State gained 17 places, 17places, and 8 places, respectively, and the University of Akron went frombeing unranked to being ranked 7th. These institutions and programs shouldbe commended for their achievement, and others may want to explore whatmeasures and methods were adopted or operationalized to lead to thisincrease in research productivity.

Examination of the institutional research productivity serves as onemethod of evaluating the work of graduate programs and may also lend someinsight into the identity of counseling psychology. As can be seen, a changein the research contributed to JCCP has occurred over the past 10 years, withfewer contributions seen. This may indicate that a shift is occurring or hasoccurred in the identity of counseling psychology as members of the fieldpublish in other journals that have a different focus and mission. At the sametime, however, the contributions to the other four journals have remainedfairly constant, which may indicate that a large portion of the identity ofcounseling psychology has remained fairly stable. Another possible reasonfor the decrease in contributions to JCCP is that counseling psychology hasseparated itself from the field of clinical psychology, so the idea that the twofields may be merging is not as true as believed (Beutler & Fisher, 1994).

The addition of other journals into this study further illuminates theresearch productivity and identity of counseling psychology. The results of




the second factor indicate that some programs are publishing in journals thathave a strong multicultural and diversity focus. However, as was previouslynoted, many of the leading institutions in the second factor do not have coun-seling psychology doctoral programs. For instance, institutions such asColumbia University Teachers College and University of Wisconsin–Madison are highly ranked on factor 2 yet are not ranked among the top 10institutions on factor 1. These results may reflect that doctoral programs maystill be focusing on more traditional areas of counseling psychology based onthe interests of faculty members and are only just beginning to branch out tonewer journals that have a more multicultural/diversity mission. Senior fac-ulty that already have an established area of research may be unlikely tochange their interests and research focus. This may also reflect the particularinterests of faculty members at various programs. Although these journalsrepresent an area of counseling psychology, the area of multiculturalism/diversity may also have broad implications for other disciplines (e.g., coun-selor education, clinical programs, and minority programs) that use thesejournals as outlets for professional work. This hypothesis is supported by thefact that the Journal of Multicultural Counseling and Development is thejournal for the Association of Multicultural Counseling and Development ofthe American Counseling Association and that Cultural Diversity and EthnicMinority Psychology is the official outlet for Division 45 of the AmericanPsychological Association. This correlation reflects the fact that these jour-nals are not core counseling psychology journals as they are published byother divisions and organizations.

The fact that several counseling psychology programs are highly rankedon factor 2 and that 18 of the top 40 institutions have counseling psychologyprograms may indicate that multicultural/diversity is beginning to be recog-nized as an important research and training area. However, multicultural/diversity research is not supplanting traditional research areas at these lead-ing institutions. Given that multicultural/diversity issues are only one com-ponent of doctoral training in counseling psychology, it makes sense thatmulticultural/diversity research should comprise a smaller portion of theresearch at leading counseling psychology research institutions. The resultsalso may indicate that some doctoral programs in counseling psychologymay be developing the area of multicultural/diversity issues as a specialty. Interms of actual training on multicultural/diversity issues, the data suggestthat students in counseling psychology programs may be getting more expo-sure to multicultural/diversity issues through research but do not lend anyinsight to formalized training in multicultural/diversity issues.

In terms of research productivity, the top institutions in this study repre-sent a select group that emerged out of large number of schools that were rep-resented in the seven journals. The use of research productivity rankings is




just one of many ways of gauging a program’s productivity and quality andmay not reflect actual training of students. Research productivity does notinherently gauge the overall quality of a program. Research productivitydoes not lend any evidence to the training model used by programs (scientist-practitioner) as many students can be well trained in the research processwithout faculty or students having published articles. Research productivityor lack of research productivity by a faculty member does not provide anyspecific evidence about the training provided to students by each facultymember.

Traditionally, counseling psychology has relied on productivity or reputa-tion as a measure of program quality (Delgado & Howard, 1994; Howardet al., 1987; Roose & Anderson, 1970). This approach may be misleading asit ignores various aspects that would indicate the quality of training. Trainingof counseling psychologists involves multiple skills and knowledge thatmust be assessed to determine the true quality of the training. Although notone method would gauge a program’s quality, the use of multiple methodsand outcomes would better indicate overall program quality. For instance,some of the outcomes that could be employed are graduates’ accomplish-ments, job and internship placements, assessment of clinical skills after grad-uation, and scores on the national licensing examination. Additionally,examination of specific skills germane to the field, such as multiculturalcompetencies, could be formally assessed during the program or after com-pletion. The use of multiple methods and outcomes is consistent with Ameri-can Psychological Association accreditation criteria that examine outcomesof training. Despite this consistency, American Psychological Associationaccreditation alone does not allow for comparison between programs due tothe individual nature of accreditation and the fact that American Psychologi-cal Association accreditation requires that each program devise its own train-ing goals and outcomes which are evaluated by the American PsychologicalAssociation to determine accreditation. At the current time no national stan-dards with which counseling psychology programs can be compared exist. Ifthe field is truly interested in the overall quality of programs, then futureresearch must consider myriad outcomes that represent program quality. Nooverall assessment for programs and graduate quality has been identified. Itmay behoove the field to develop a valid and reliable assessment for programand student quality.

Several implications can be drawn from the data. First, the data providethe field with a gauge of the most productive universities that may be consid-ered the leading institutions in the development of the field. Second, they pro-vide students with information about which universities are producingresearch and publications, which can indicate potential research involve-ment. By examining the focus and mission of particular journals, students




can learn about the research focus of universities and their faculty and decideabout their fit with the program. In conjunction, this study provides informa-tion on not only the traditional areas of counseling psychology but also on asecond factor that is more related to diversity and multiculturalism, whichwas not present in the previous studies. The changes in rankings can be usedby program members and administrators to examine the effectiveness of pro-grammatic and institutional initiatives to increase the productivity and effec-tiveness of counseling psychology research programs. Finally, although con-troversial, studies of this nature can be used by programs and faculty to arguefor resources and maybe even their existence in the current market.

Studies of this nature must address the question of quantity versus qualityof scholarship depicted in the results. Many would argue that studies thatexamine productivity in refereed journals with high rejection rates also givecredence to the quality of the scholarship due to the rigorous review that thesearticle have undergone. However, the actual impact of the published articlesand their ideas on the field is not quantified in a study of this nature. Authorsand their institutions that contributed few articles to these journals, but arti-cles that were of high impact, would be ranked lower in the overall productiv-ity rankings. Unfortunately, the examination of impact was beyond the scopeof this study.

In terms of the future, two avenues can be explored. First, for productivity,future research may include more journals that are believed to representcounseling psychology. Through the use of factor analysis, a greater under-standing of the diversity of publications in the field will lend insight into thechanging nature of the field. Factor scores could also be used to develop amore sensitive gauge of productivity across different domains. The secondavenue would be examination of program quality. To date no large-scale sys-tematic investigation of this question has been undertaken. Although articlesof this nature attempt to address quality, many other aspects of program qual-ity previously mentioned are unexplored. An undertaking of this naturewould require considerable openness and intellectual honesty as the individ-uals would have to reveal specific information about their program and stu-dent performance. However, these traits would be necessary to meaningfullycompare programs in terms of overall quality.

REFERENCES

Beutler, L., & Fisher, D. (1994). Combined specialty training in counseling, clinical, and schoolpsychology: An idea whose time has returned. Professional Psychology, 25, 62-69.

Buboltz, W., Miller, M., & Williams, D. (1999). Content analysis research in the Journal ofCounseling Psychology. Journal of Counseling Psychology, 46, 496-503.




Cox, W. M., & Catt, V. (1977). Productivity ratings of graduate programs in psychology based onpublications in the journals of the American Psychological Association. American Psycholo-gist, 32, 793-813.

Delgado, E. A., & Howard, G. S. (1994). Changes in research productivity in counseling psy-chology: Revisiting Howard (1983) a decade later. Journal of Counseling Psychology, 41,69-73.

Diegelman, N. M., Uffelman, R. A., Wagner, K. S., & Diegelman, S. A. (2005). Current institu-tional trends in research productivity in counseling psychology journals. The CounselingPsychologist, 33, 327-339.

Division 17 of the American Psychological Association. (2003). Retreived September 18, 2003,from http://www.div17.org

Fouad, N. A., McPherson, R. H., Gerstein, L., Blustein, D. L., Elman, N., Helledy, K. I., et al.(2004). Houston, 2001: Context and legacy. The Counseling Psychologist, 32, 15-77.

Gelso, C., & Fretz, B. (2001). Counseling psychology (2nd ed.). Fort Worth, TX: HarcourtCollege.

Howard, G. S. (1983). Research productivity in counseling psychology: An update and general-ization study. Journal of Counseling Psychology, 30, 600-602.

Howard, G. S., Cole, D. A., & Maxwell, S. E. (1987). Research productivity in psychology basedon publication in the journals of the American Psychological Association. American Psy-chologist, 42, 975-986.

Roose, K. D., & Anderson, C. J. (1970). A rating of graduate programs. Washington, DC:American Council on Education. Retrieved September 18, 2003, from http://www.div17.org/sections/sigs.htm

Sue, D. W. (2003). From the President. Division 17 Newsletter, 25.




the counseling psychologist - university of memphis

Documents