II II:~~·.~· · I II· ...

August 1, 1988

'Ibis ptq;a;al grew out of discussions of the Quality Irxiicators Group, an ad ~ ccmaittee o "tosed of the followin; in:li.vidnal s:

David J. Berq I Assistant Vice President, Manaqement Plazmirq ani Infcn:mation Sel:Vices

Dal:Win 0. !fsl'ml., Research Associate, Academic Affairs Jeanne T. Illptal, Special Assistant to the Vice Presidents for

stD3ent Affairs ani Acadeni C Affairs Rcnald P. Ma.trcss, Assistant Director, Infcn:mation System Ser.Tices Keith N. McFarlarxl, Act::irq Dean, General College Jack c. Mexwi.n, Qlair, Department of Educational Psychology James B. Preus, C0Ct:1::iinatcr, student SUpport Sel:Vices Bruce Vandal, 01air, Minnesota stD3ent Association Forum Jolm wa1 J ace, Assistant Vice President for Aca,demi c Affairs flo h. wiger, Coordinator, Office for Minn'ity ani Special student

Affairs

TAmE OF a:NrEN'1S

Page

Executive SUIIIlmy i

Intrcductian 1

overview at lSIIIJM in Quality Ass!SSIW'lt 4 P\npcae at Quality .bal& ...... xt 4 Oetiniticns of Quality in lJndergraduata Edl.1cation 5 ASIISESIMilt Bl!lc1o;rcun:i in the OIMIJ.opi&lt Of Quality Indicators 7 Acccunt:ztbillty ani Quality Assa s••It Indicators 8 Quality Asses~It Ettarts at othar Imicatcrs 10 'lha Role of stan::1ardized Te51:in1 13 camect:.i.aJs am I.ccal ccncems about 'Ordel:graduate amcation 15 Guidel.ines in Oevelopin;J Quality Assessment Indicators 16

Ratiaal.e for -Pl:oposed In:ti.cators 18

Irxlicator 1: Praparatial Rsquirements (Inp.lt) 20 Irxlicator 2: AcacJemic Pot:entiaJ. at F.nt.erin; StlD!nts (Inp.lt) 21 Irxlicator 3: sti.D!Ints of COlor (!np.tt) 21 Indicator 4: Advisin; Rllscuxt:es (Inp.lt) 22 Irxlicator 5: Cl•MtOCiil Facilities ani st:a1y Space (Inp.lt) 23 Indicator 6: Rat:a1ti.al Rates for F.nt.erin; Freshmen (Precess) 24 Irxlicator 7: Instruct:ial of I.owar Divisicn students (Process) 26 Indicator 8: t1ndergraduate Olrriculum (PJ::ocass) 27 Irxlicator 9: Ose of SCAlnd Educatia1al Pl:act:ic:es ani Prin::iples 28

(PJ::ocass) In:Ucator 10: Class Size Experiences of students (Process) 29 Irxlicator 11: Grades Received (Process) 30 Irxli.cator 12: 'l'ra.i.nir¥; of 'l'eacbiD;J Assistants (Precess) 31 Irxli.cator 13: Mcnitorin;J of st\X3ent Experiences (Process) 31 Irxlicator 14: student Course Evaluaticns for Large Enrollment 32

CCUrses (Process) Indicator 15: students' Part.icipaticl' in Key university Activities 34

(Pu:.cns) Indicator 16: Graduates' PertODII!lnCI8 at Graduate Reccmi ExaJnj nation 35

(outccma) Indicator 17: mtpl.oymnt Experiences of Graduates (outo:De) 36 Irxlicator 18: Pcst:bac:cal.au Edl.1catiaal Experiences (outcaDe) 37

References 38

SeVeral current forces Sl¥}geSt that the university should initiate a process to develop a set of quality in:lic:ators for urxlergraduate education. First, several university task force z:eports in the last five years have Called for substantial imprcvement:s in un:mqraduate education an:i a.ccc:mpanyin; data to docnmant: tba effects of these bp:cvements. Sec:crd, the university needs to respc:ni to rer:pasts trail its anltit:usncies ab:lut the edllcatiaal quality on its QIII'IPJSes. 'lbird, tbe still-qrowinq naticmal assess!Mnt mvement: in ~ educatial is net a fad that wUl diMt"Pf'T, l:Jut represents serious c:a1C8ml ab:lut quality that require px:UitJt inst:itutiaal attention. Finally, siD:e w need to mve faz:waxd in 1:hink:in;J ab:lut the goals an:i objectives far the baa:alanreate degl: es provided by the institution, di solSSions ab:lut quality in:licators can infoz:m an:i be infoz:nei by those disoJssions.

What is present:ad in this dccnnent: is a snmmaz:y of the issues that bear on the daval.q::ll&nt of quality in:lic:atcrs. '!he dccunent is in'teRBi to help us develop an approach that can be ~ by all-students, staff, faculty, administ::rator, regents, an:i cur ncrnmiversity constituencies. '!he document at this point is intm'ffi for ni 591"ion pgrcoses only rather than as a set of in:1icators to be accept:ad or rejected. '!he illustrative in:lic:ators z:epresant: cur collective wisdall ab:lut pot:ent:ially useful ways to define quality in unr.i£graduate educatial at tbe '1Win Cities amplS.

Defininq quality in urDm;radllate educatial histaric:ally has . con:::ent:z:ated on three different approaches. '1hsy focus at different parts of the educ:atiatal. experience. 'lhe first approach concetltrates at relevant resc:urces or "inputs." 'lhe seccn:i approach fOOJSes on educational "processes," .irlclu::lin; tbe o:::nt:ent: and nature of student experiences as well as institutional pract:ic:::es an:i px:ocednres designed to influenCe quality. P:t:ocess variables are especiaJJy iDpntant: to constituencies interested in whether or net the university shaa the intention to achieve certain desireable outcanes of un:mqraduate educatiat. '!he third approach eq;:hasizes the results or "c::ut:c:cmes" of educational experiences. All three variables should be .irlcluded in a ·e:::c::ouprehenive set of quality in:licators, although recent: di soJSSions ab:lut assessment in higher educatiat have focused at the need far more c::ut.cx:me assessment:. Imkin; at outccme measures has value ally if you can silllll.tanecusly describe the inplts an:i processes neoessaz:y to achieve part::i.a1lar out:o::mes.

'lhe 18 px:oposed in:lic:atcrs, classified by type, are as folla.NS:

. Irxlicator . Indicator . Irxlic:ator . In:U.c:ator . Irxlic:ator . Irxlic:ator . Irxlic:ator . Irxlic:ator

1: 2: 3: 4: 5: 6: 7: 8:

Preparatial Requi:raDents (Inplt) Acadenic Potential of Ent:erin; students (Input) students of COlor (Inplt) Advisin; Resources (Inplt) Classroc::m Facilities an:i Study Space (Input) Retention Rates for Ent:erin; Freshmen (Process) Instruct:ion of Lower Division Students (Process) t1n::lez:graduate OJrriculum (Process)

i

• Iniicatcr 9: Use of Sound Educational Practices and Principles (PLocess)

• Inlicatcr 10: Class size Experiences of Students (Precess) • Inlicatcr 11: Grades Received (Process) • Iniicatcr l2: 'lninin;J of '1'eac::hiD1 Assistants (Process) • Inlicatcr l3: Mcnitorin; of student Expe.rien:::es (Process) • Inlicator 14: St1Dmt COUrse Evaluaticr1s for Iarge Enrollment

Courses (P:tocess) Indicator 15: St:udents' Part::icipat.1a in tr..y university Activities

(Pl:OCIIIEIS) • Indicator 16: Graduat:M' Pertoz:mance at GndL1ate Recxlrd Examination

(outcaaa) • ID:ti.cator 17: ~oymant Experiencas of Graduates (out:cxme) • Indicator l8: Post:t.ac1:alanreata Educatic:nsl Experiences (out:cane)

We ptoposa that the followin;J t:imetable be followad to lliMl us forward in devel.c:pinq a set of quality in:licators:

August 1988: tti scnssicn of ptqosal cmr.n:J central officers and zepz:asantatives of·appz:oprlate student and faculty CXIIIDitteas.

Pl.am.i.n; process is est:ablisha:i for widespread di sa.111S:ial in the Fall.

sept:ali:ler - Dl•c:ussial and cxnsensws bdJdin; al:olt indicators TNOrth oct:cblr 1988: pmiiUinq.

Novellblr 1988: Slllall tam111 established to specify pJ:'OC'«iJres for c::al.a.1l.a~ irxiicators and to estimate c::osts of collectin; and report:.in; data.

Jaruaey 1989: Revised ptoposal that includes ocst estimates an:i specifications is di SO'ssed.

Fellrual:y 1989: Final set of in::licators adcpt:ad.

March 1989 and Mac:hani.sma estah1 ished for collectin; and report:.in; tbm:ea:tter: cpllty irxiicators.

ii

IN!'lgXTCI'IQN

When A Cgmdt:rpent to FOC!JS was presented to the Board of Regents on Februal:y 8, 1985, it proposed several implementation rec:x:mnermtions to help the university "mcve with a sense of pn:pose through this period of transition ani into the future" (p.l) • Two of the three foci for chan;e were related to the university's ~te education mission: (a) to recruit high ability undergraduates who can best benefit fran the university's pLogzCiiiiS: and (b) to .illp:cve the quality of our undergraduate pro:Jram. 'Ihe pLqa;al grar alt ot the University's pl.arJn.ilr; pLtX"eSSes in the late 1970s and early 19808 and rafiect:ed real and widespread ccnceJ:nS about the quality ot ~ta edllcatial provided at the University of Minnesota, especially at tbe Twin Cities canplS.

we have enccuntared many difficulties in winni.rr; support for A Ccmnitment to ~, nat the least of which was skept,icism about the University's real cx:mait:mant to high quality ~te education. CCI'lstituencies both within and outside the academy ~ to support efforts to improve undergraduata educatial, b.tt expected to see chan;es occur that confil:med the uni~ity' s intant:i.cn to improve the quality of undergraduate edllcatiat. As an instit:lltial, we had a wealth of suggestialS about what to improve and hew to iDp:cMa it, l:llt get "1:Joc};ed da.m11 in liiCVin;1 quickly to lll2lka significant, visible iDp:ovement:s. we pLGp.?Sed that we wanted to improve quality by reducin;J the lUIIbers of ~tes (but retaini.n;. the same fundinq laval that had been based em enrcl.lJDents) without specifyi.n; the nature and extent of ~ tha~ wculd result fran decreased enrollments.

cax:urrentl.y, the University was ~ enccmaged by the gcveJ:nOr ani his staff and by legislators to deveJ.q) a set of indicators to be used in evaluat.in; the institut:icn's relative suo::ess in its attell'pt to ''DJve fm:ward with a sense of p.n:pose." Numerous di scussialS within the University durinq the past three years explarad possible strateqies for devel.opin; indicators of quality, b.tt without c:cnsensus to move forNard. We waited to see what wculd happen. !tJdl has happened since then, sane of which has been ur;lleasant and datKralizin; to administrators, faculty, staff and students. It is new time tor us to initiate a process to develop a set of quality indicators tor ~te education.

A c:olleague, Professor Mazy Corcoran, cn::e remarked in a discussion about ~te educatian that 'tyal can tell a lot about an organization by ident:i.fyin;J the activities ani :fl.D:tialS for which it keeps good data." What struck me then and new is the paucity of infcmDatia'l about the quality ot undergraduate educatian at the University of Minnesota. We readily have avaUable extensive data that SUIIIIBrlzes qUantitative aspects of undergraduate educatian (e.q., total nmt>er ot student credit halrs), but relatively little averal.l institut:i.anal data about the quality of what we pLtJYide to students. certainly, there are bits and pieces of infcmDation that one ccul.d use in mald.n; jn:}:Jn'euts about quality, although no one has been mtivated to pull them together and identify specific problems of quality that need to be addressed.

'Ihis proposal, which presents the content, framework and rationale for evaluat.in; eadi of 18 quality indicators, first ImJSt. be discussed widely and

~ Quality Irxiicatcrs for t1ndergraduate Education ~ 2 .

emcrsed l:ly ~opriata ~ of faculty, stments ani administrators on canpJS. Faculty ani staff DllSt be i.nvolved in devel.opin:J the irxticators if we expect these same in:lividnal s to value ani use the irxticators. '!he seccni step, which ccW.d taka place ccncurrently with the first step, is to di solSS the pz:op::sed set of i.n:ii.catcrs with the university's extel::na.l c:awti-c.ncies-l.eqisl.atms, educators in otbar systems in Minnesota (iD:ludiD;J tba x-12 systaa), and c:oncecm citizens. 'lha thim step is to select Sll1ll taaaB. ot in:lividnal s, o "{QMd ot st:a:1ents, faculty ani aalCWii c and pmfMaia1al st:aU, to d8valq» a spacitic pz:qoosal for caJ.c::u.1atiD1, coJ.l.act:1n) ani suamerizin; infaratial for each in:licator.

We at tba univamity ot Mimesot:a are not ala1a in our efforts to define educatiaal. cpsllty ani dsYalop in:1:ic:::atars for each of several dimensions of quality. otbar large xesearch instituti.ons are c::ax:emed about the quality of un:3eJ:gJ:aduat education ani the need for increased attention to assessing teacbin;J ani lem::n!D;J. 'lha Alliance for tJmergraduate Education, a twe1 ve­institut.ial caJSOJ:tium (i.e., university of callfomia, Bel:Xeley~ university of cal.ifomia, Ias An381es1 university of D.lincis; university of Maeylan:i, COllege Park; university of Mic:::higan; university of Minnesota; university of North C1l:Ol.1na at OVIptl Hill; 'lbe auo State university; '!he Pennsylvania state university; uniwz:sity ot Texas at Austin; university of Washin;ton; ani university ot Wiscxnlin, MacH sen) has spent cxnsidarable time disnlSSi.rJ1 the illpl.ic:aticl1 f~ large resam:d1 institutions. At the recent thiJ:d anrusJ. AAHE Ass meut FoJ::um, four of these instituticns, includ:ing the univamity ot Minnesota (Bernal, 1988a), part.icipatad in a panel disaJSSial ot tbai.r c:u:a:1:ant assessnwlt activities. Appraximataly 150 in:1:ividJal s at:t:ami the pmal. di scussial, wbic:h is an in:lication of the. intm:ast in tba tcpic ot assessment in large research institutions. SUbsequent sectia1s of this doc'IIDPI1t suamarize natiaal. trends ani qive eoampl.es of quality assessment initiatives in other states ani at other colleges ani universities. ·

One of the diffic:ulties in ptoposin:J a set of qua.Uty assessment irxli.cators is that the tcms "a=esenent" ani "quality" can be defined ani un:!erstood in many different ways. SUbsequent secticns of this ptopceal. di sruss the <'X'I!'I'licated iss'les in tryirq to arrive at a definition of quality in uniergx'aduate education. Far puxposes ot this di scuss:ial, we will accept Boyer ani Ewtil. 's (1988) definition ot ass•sS"ftPtt'lt as:

Arrf pz:ocas:s ot gatheri.n; cx:n::reta avidance al:xlut the impact ani tun::tiaUn; ot undarc;Jraduata educaticm. 'lha tam can apply to processes that prcvide infemaation al:xlut iniividual students, al:xlut curria.1la or {JL09l:ams, about instituticns or about entire systems of instituticns. 'lbe tam en::a c•tesses a ran;& of pz:ocednres :includin} testin;J, survey methcds, perfoman:e measures or feedback to in:lividual. students, resultin:J in both quantitative ani qua.Utative infcmnation. (p.3)

In M:imesota, efforts to develop quality in:1:icatcrs in the K-12 system are JrCVirq forward ani can serve as an illustration to guide the dsvelopnent of iniii:atcrs for the postsec:cn:!ary level. 'lha I.sqisl.ative Qmnission on

Proposed Quality Wicatcrs for umergraduate Fducation Page 3

PUblic Fducation is currently en;aqed in a process of ~ the criteria to be used in jua;in;J educational quality in Minnesota. 'Ihe Crmnissi.al hired a consultant to provide them with data al:x1It educational goals, possible i.rxlicatcrs of educational quality, ani info:cnation systems to be used in report:in:; data. A survey of educators, potential EmPloyers and citizens is baiD; ccrx1Llct:ed to detemine their views on the al:love three topics. Here m:a tba twal.va possible i.rxlicatcrs fran the ''Fducational Goals and Irxticators SUl:Vay" that are beiD;J used to collect data:

• Tests which lMP'sur& mi.nilllliD oc::upd:ency at various levels

• Tests which O'Jit'2l%8 students • perfcmmmce to national st:arx3ards or noxms

• Percent of students who enroll in college or other post-high school educat.ial

• Percetrt: of sbx!ents who participate in extracurricular activities ..

• surveys to lMP'sur& the satisfaction of consumers (students, parents, eupl.oyers)

• Eviden:e of· a plan for the achievement of stated educational goals

• Professialal qualificatials of staff

• Eviden:e of regular sbDant mnitorin; ani feeiback

• Evidence of high expectatials for student perfo:cuance

'Ihe future health of the university depends em many factors, not the least of which is deaa'lstrated amcern for and brprovemant of un:!ergraduate educaticm em the Twin Cities cauplS. Institutional quality is just that-a c:haracterist:i of the whole rather than a quality possessed by a few of its ~. We lll.lSt datalsb::ate a <XIIIIlitmant to inprovin;J the quality of undergraduate edllcatiat. Develcpinj ani usinj a set of quality irxlicators will help us reach that goal..

'!his pzuposal has tw parts. 'Ihe first part SUJIIDarizes eight topics central to an un::1erst:arx:t of the current status of quality assessment in higher education. Additionally, this overriew produced guidelines for constructing the 18 proposed quality i.rxlicators foun:i in this proposal. 'Ihe secon:i part

Pl:uposed Quality Indicators for ~ta Fducation Paqe 4

is a detailed rat:ionale for usin; each of 18 quality assessment inii.cators on the '!Win Cities campJS of the university of Minnesota.

oyERVm1 OF rs.c;rms IN cmt.m ASSESSMENt

'Ibis di salSSial CXMD:S eight interralatad tcpics that serve as general ~ far tba danl.qiiEnt of tba l8 undargraduata quality irxii.cators. '1ba fizst tqdc is a pc4*0=tiva em tba ultimata puz:pose of quality a· asw•lt activita.. '1ba saxad tqd.c is an ovarview of others • attempts to define and lll""'l%'8 quality in undargraduata a:hlcatia1. 'lbe third topic loalas at tba devalopawut of c;pslity in:U..c::ators within the recent context of assescrent initiatives in higher edllcatial. '1ba fourth topic relates quality assas"''Mit initiatives to cxn::exns for increased acx:cuntability of colleges and universities. 'lbe f:itt:h sacti.a1 qivas exanples of quality asses"R'Wlt efforts at other institutia'lS. 'lbe sixth section fcx:::uses on the role of st:amarclized t.estin; in the l."b»llt wave of assessment initiatives. 'lbe seventh sect:ial c::cnnects natiatal. and state CXX'lCeUlS about educational qJality to Onivarsi:ty-based reports focusirq em the quality of undergraduate education. 'lbe fiml sec:t:i.al presents a set of guidelines in developin;J cpU.ity assessarent iniicatcrs.

Olrrently asses.,....•It is viewed as the pradaDinant means t:hrough which ~ in teac"h.i.rg and leamin;J can occur, although in recent decades other instituti.a1al efforts (e.q., faculty davalqmeut, c:urricul.um :revision) W83:8 viewed as the priJDaxy approach. If asseerzwut activities inprove the qJality of undergraduata education, an i:ap:tant criteria in evaluatin;J them is the degree to which their results can be l.inksd to il1st:itutional initiatives and policies which StJRXlrt quality in ~te education. Quality assessnmt can occur at many levels-at the institutional level, at the department level, at the course level, and at the student level. 'Ihe mere rellDt:e the asse=sment is fran the actual experiences of the students, the mere difficult it is to use assessment for PJZPCSes of inst%uctional blprovement. 'Ibis principle is most evident when we try to detel:mine what studart:s have leamad in oollege. A psyt:hol.ogy professor can c:letermine ~ or net studart:s ~ and can apply the principle of rain!orca2B'l1: and, if students naitbar ~ ncr can apply the principle, can taka appropriate steps to clarify the ~. cress (1988) c::cntiJullly ra:i.ms us of the need to insure that assessment matters: "Mcst people think of assesment as a la%:ge-scal.e test:in;J pxogzam c::cn:mct:ad at instituticnll. or state leval.s to c:letermine what students have leamad in college. I believe that we sholld be qivfn1 mcre attention to small-scale assessments c::ax1ucted c::antmuousl.y in college classrixDs by discipline-base teachers to c:letermine what students are leaJ:Tlin;J in that class" (p.l) •

Assessin; the quality of Ul'X)ergl."aduata education will, in the 1llims of many, de little to inpJ:ove ~te education unless the University's ir¥::entive system Yi§ Am urx3ergraduate education~· OVer the past

Prcpcsed Quality Indicators for un:lel:graduate Fducation Page 5

five years, this ccncern has been voiced repeatedly in ccmnittee meetings ani is explicit in the reports of several task forces (e.g., "Ccmnittee on the ()lality of tJ'lD!rg'raduate '1'ea.dl:in;J ani I.eamirx3', " "Final Report of the !:q)lement:ation Task Force on tJndergraduate Fducation on the Twin Cities O!!!pJS. ") '!here is ocnsiderable skepticism on camrus al:xJut expen:ting :resourcas to assess 1.1l'1dergradllte education if there is no institutional CXII'IIIi.tment to allocate :resourcas to impraYa specific aspects of ~education.

Olr.d.c:ular mtam is aw of the pass:ible results of quality assessment efforts focused on student leam:in;J. In order for this ref om to occur, stu::Jents' alrrlcular c:boices (i.e., the ccurses they c::arpleted) llllSt be desc:riJ::Ied ani related to 1~ out:a:mes. '!be Differential Col.ll:'seNork Patterns Project (IXPP; Ratcliff, 1988) is a project furx1ed by the u.s. Department of Education to dete1:mi.ne the associated effects of different college coursewcrk patter.ns on general learned abilities of students in six institut:ialS (i.e., Cl.ayta1 state CCl.lege, Evergreen state College, Georgia state University, Ithaca CCl.lege, Mills College, ani stanford University) • 'Ibis project ani its results can provide us with a useful lliXlel for usin;J assessment activities to revise 1.1l'1dergradllta alrrlcula ..

As wa mcve forward in dsvel.opin;;J a set of quality in:iicators we llllSt be certain to pay equal attention to the ptocesses in place that: (a) reward quality in ~ education: ani (b) offer assistance ani financial SIJR)Ort to impraYa quality. No aw wants to dsvel.op an intensive, costly set of in::1ic::atars that exist separately trail other dscision llli!lkirY; processes al.rei!ldy in placa in the institution. We 11llSt not lese sight of the ultimate goal of illlprcvin; un:Jergraduata education.

2. Definitions of Qwl ity in !!TergxMnaW Education

Rebert Persiq, in his book zen m:l the Art of Motorcycle Maintenance (197 4) , speaks to us al:xJut the dilemma we face when we t.zy to define quality in unde%graduate education:

()lality ••• ya~knew what it is, yet ya1 don't knew what it is. But that's self-cont:radictory. But scma 1:hi.n;s are better than others, that is, they have mere quality. But when yru t.zy to say what the quality is, apart trail the 1:hi.n;s that have it, it all goes pcof. 'there's not:h..iD] to talk al:xJut. But it yru can't say what ()lality is, hew do yru knew what it is, or de yru knew that it even exists? (p. 184).

'!his elusive view of the meanin;r of quality in 1.1l'1dergradllte education has been noted by others as well. Tan (1986), in an article that diSOJSses the problems asscx:iated with measuril'x; quality, noted that "quality, like beauty, is in the eye of the beholder; it has a different meaning for different people" (p.224) • Astin (1986) summarizes five approaches to measurin;;J quality: nihilist approaches, reputational measures, resources i.n:ticators, outcanes measures, m:l value added approach. 'lhe nihilist

Pl:op:seci Quality Imicatcrs for 'O'n::3ergradllte Education Page 6

approach is simi Jar to Pirsiq' s view that datinin; and measuring ~ty is an inpwdble task. Accol:'clin;J to I.svine (1986), the attitude that we carmot define quality but that we knew it when we see it, is destructive to l:lui.lclin;J c:cnfidence in American higher education.

Fortunately, there is 1lllCb to leam frail the ot:har four approaches ncted by Astin that,. c:an use in devel.cpin;J a set of quality in:licators for the '!Win Cities ClPIPlS of the univarsity of Mi.masota. Astin, cn:a a sb:onq advocate of valua aa.i appx:• rtws (i.e. , 'tGlt dces a college ecbJcatiat do to c::han;e S'tudmts1 lcnowl.a3ga, sldJJs, and val\JaS) 1 is IDr pte ipOS:in;J that institutions use ind1.catars of hew rescuz:cas are used, net ~y the ex:i.stence of avamll rasc::urcea - hplt, in dafi.n.in; edur:atic::n!ll quality. Tan (1986) 1 in his raview of types of variables used in quality assessment, hypothesizes that "exoll J ent institut:icns 'Wall.d be these Y1ic:b can affect significantly the intellectual and scholarly devel.q:meut of sttxBlts" (p. 261). Pascarella (1987) givas a c:autials en:3crsaDent of value ack!ed approaches, but aU.y if they focus at specific outccma measures. Astin alsO suggests that we use two criteria in evaluatin; a quality imicator: (a) does it reflect what faculty, administrators an:l students mean when they refer to quality? an:l (b) does usi.rq the iniicator have specific bplic:ations for ~ institutia1al quality? 'lbese two criteria serJe as one useful basis for eval.uatin;J the eighteen iniicatars described in this proposal.

'lhe reseaz:t:h literature on the definitiat and lDMSJitement of quality in higher educatiat is focused at the quality of graduate and professional institut:ia'1S, altb:l¥]h sane wcrk (e.g., SolaDCil and Astin, 1981) has focused specifically at quality in undergraduate ptogLW. SolaDCil and Astin studied sevan dapart:mants (i.e. , biology, bJsinass, c::hemist:J:y, ec:onani.cs I ED;llsh, history, and sociology) and fourd that these institutions with high graduate ratirr;s ware also jl.D;ed to be the best undergraduate institutions. Tan's (1986) exol1 J ent review of the three approaches to measurirg .institutiaal quality (i.e. reputat:iaJal. sbrli es, objective in:li.cator sbnies) provides the basis for sane general o ••"ents about the three approaches to measuring instituticn\l quality.

'!he reputational approach, initially sponsored by the American Council on Fducaticl'l and developed to help baoca1 aureate graduates c::hcose graduate or professia1al institut:iaw, was first used by carttar (1966) • cartter' s approach used the pear eval.uati.a1S of a panel of judges to make evaluative jw;;gments at two dimansicms: quality of graduate faculty, and effectiveness of the dcct:aml. pz:cqxa. SUbaequalt sbrlies by Rcose ani An:3erson (1970) increased the nmtwr of p.rcqxams sbrli ed ani addressed saae of the mathcdoloqical llmi.taticms of cartter•s wcrk, but c::ontinled to focus on perceived quality. Chl of the problE!II& with a reputatiaBJ. approach to defirlin; quality urDal:gradua.te educatiat is that it fails to suggest steps to iDp:'cM! institutiaal quality.

Ci'Jjective in:li.cator sbnies (Tan, 1986) use five categories of variables: faculty variables, st:lx1ent variables (e.g., nDi"ber of students ani selectivity) , measures of institutia12ll or departmental. resources 1 outcome measures for graduate ani professional sc:hcol students, ani multiple

•

Pl:oposed Quality Iniicators for t1rCeJ:graduate Education Page 7

criteria. 'Ibe stxerx;t:h of this approach lies in its objective measurement of quality. 'Ibe probl• with this approach is that mst studies of this type hava 1l1i!lda the assun¢ion that faculty research productivity is the major in:li.catcr of quality. Clearly, other types of variables ani definitiaw of quality are necessaxy to develop a meanin:;Jful set of indicators to reflect quality in un:lergraduate education.

'Ibe quantitative oorrelates sbXIies (e.q., conrad ani Blackbul:n, 1986) are, in a sense, a c:ali:d.natial of the first two types of sbXIies. These studies identify variables that are correlated with high institutional reputations for graduate ani professialal education. Althcugh studies of this sort help us batter Ul'1darstan:1 the meanin; of results of reputational studies, the lack of a tbearetical basis for definirr;J educational quality weakens the itrplicatiaw ani usefulness of this approach.

Of all of the recent xeports a1 the quality of urx!ergraduate education, Inyplvemer¢ in Ienrhv (1984) is the DCSt prescriptiVe about what <X'I'1b:ib.1tes to exaaJJ ence in undergraduate educatia1. 'lbree critical oax:liticns of exraJ 1 ence are noted as bein;J essential to D;n:ovin; undergraduate educat:ia1: involvin;J students, CXIT!!!Imicatin; high expect:at:ials, ani increas.in; student feedback ani assessment. Chickering ani Gamscn (1987) aci:led four additialal criteria ani proposed them in an article "Seven Principles of Good Practice in UlX1ergraduate Education."

Insights about the quality of undergraduate ~tion cane, too, fran an analysis of variables that gg DQt relate consistently or loqically to observed or perceived quality. Isvine (1986), in a presentation about mj san:::eptioos about quality, listed six myths about quality: (a) the more J1SJfW!lY an :institutia1 has, the better its quality; (b) selectivity, in the fcmn of high admissioos st:amal:ds, is synonynDJS with excellence; (c) quality resides in ally certain types of insti.tutioos; (d) one cxmse of study is higher quality than another; (e) one type of instxuction is of higher quality than others; ani (f) quality is net a measure of everything colleges do that are beneficial to students. He concluded by notin;J, "I think quality is the key issue in higher education today. • • • In this respect, quality is a meaS'l%'8 of the best a particular institution is capable of ac::hievin;J. It is a st:ardard that necessarily varies fran college to college" (p. 29) •

3. A§sf'fl!""t Baclsgrowr;i in the Pevelognent of omJ itv Irxli.cators

Fd;artal (1986), in a cleverly written article entitled, "An assessment of assessment" in the 1986 El'S Invitational Ccxlference Proc::eec1i.n;s, talks about "assessment as a play that has lmfolded in four acts" (p. 93). In Act I, assessment began in the selection of officers in the military ani concluded with the selectia1 of .in::tividnals for manaqeria.l. ca:reers. In Act II, assessment came alto college c:alli["JSes to help faculty ani c:ounselors make difficult decisions about admittin;J experienced adult leaz:ners. In Act III in the 1970S ani 1980s, legislators ani governors became c:onc:emed about the quality of urx!ergraduate education, often linked with conc:::el:T'IS about the

PI~ Quality Indicators for ~te Fducation Page 8

states' ecax:my. In the final Act, the scene new 1:ald.n;J place on college and university campJSes acrcss the ccunt:xy, administrators, faculty, and students are wcr.1c:in;J together to define and assess the cpallty of urxlergraduate education.

CCtx:erJJS about detinin; and measuriD.l quality in American higher education ara cma of tba "mal:Xat-forc:es" that ~ our system fran those in. other count:z:ies ('l'rai1 1988). AltbcJ¥j:l the fedaral gcvemment supplies only aw-falrth of the f:inancial suw rt far American higher education, its intlua'lC8 c:an be dr2matic as evidancad by the bpJct of pralOtll'lOEIDSI'l by sec:ret:axy of !l:b::atial, Wf J 1 i am Beluatt. ~ to T%cw 1 American higher education expanti.tura in 198S-86 accamta:i far 2.5 pmceut of the nations' grass naticmal. prcdllct ($102 bill ial). Hartle (1986), in his overview of the an1eut interest in assessrent1 cxn:lu::3ad that "assessment is not likely to be a fad" (p. 8) and that "the issue will not quickly fade away" (p. 9).

AltbC'tgh thara is cx:nli.derable 1:ecent interest in quality assessment in Amarican hjgber education, wa have relatively few reliable and valid methods for assessing iq:lortant variables that affect collage ~ learn:in]. Bok (1986) nat:ed that colleges and universities spend little tima in efforts directed at research aimed at bp:cvin:J student learn:in; 1 but c:cnsiderable time deba't:in:J what students sballd leam. Bok cxn:luded.:. "At p1:esent, universities have no adequate way of measurinq the effects of urXIsrgraduate education or assessin;J tha methods of instructiat they eqll.oy. '1his is· a serious defect" (p. 23) •

If cma ac:capts tba need for increased •sse=mant of educational quality, sana (e.q. 1 Bcwan, 1986) ugue that tba arMament sb::W.d be based on the instituticn's successes in~ sbldent:s1 a "value added" cq:proac::h to quality assesS"'PJ1t. '1be first step, then, is to define goal areas and assess chan;Jes in students as a result of Ul"'dargraduate education. Bowen su;gests five goal areas: CX'3l'litive davalq::maut, aesthetic sensibility, Em:lti.cnal. and mral davalq::ueut, practic:al c:x::up!terx::e, and direct satisfactia1 and enjoyment fran college educatiat. One of the problems with this cq:proac::h at our caq,:tJS is the lack of pr<X."M'Ses in place to develop a CCtlSefJSUS about the goals for urx:Jargraduate education.

In his paper "Asn srent with open eyes: Pitfalls in S'tl.l:iyin; student art:.caDes" 1 Terenzini (1988) su;gests that •ssrs•rent requires oonsideration of these questicns: "Mlat shgll.d a student get cut of college?, What should a student get cut of att:en:lin; ~ college?, and What ~ a student get fran a.tt:em:in;J this college?" '1bese three questiaw serve as the startin;J place for develop1nq a c::x::uprehensive out:ccma assesS"'SSIt plan for the university of Mimesota.

4. Aqpmtahility and Qmlitv Assessment Initiatives

Although the previous sectiat cutlined sc::me of the research on quality in:lic:at:ors in higher educatiat, it did not pit recent developnents in the c::cntext of in::reas1nq requests to collegeS and universities for new

Pl:oposed ()lality In::iicators for undergraduate Education Paqe 9

assessment initiatives that dcx:nment the educational outcomes of higher education. one might make a CCitp!rison with the research on the quality of ~te education, which has shifted fran an emphasis on input am processes to a cxlnceJ:Il for outcc:me measures (Nettles, 1987). 'Ihe results of a survey of the 50 states, COI'lducted by the Education Carmi ssion of the states (Beyer, Ewell, Finney, an:i Min;Jle, 1987) iniicated that ttNo-t:hi:rds of the states bad fcmaal. assessuent initiatives in place. Althalgh there is a cc:nsi.derable role for the state governments an:i leqisl.atures to play, the respa15ibillty for pl.annin; an:i b;llement::in; quality assessment initiatives is an institutialal. responsibility.

Wil.sal (1984) defines acc::a.mt:.ability as "a persistent demani levied on universities by ext:eJ:nal sponsarin; authcrities an:i agencies," an:i as "the fraternal twin of institutional autax:my" (p. 11). 'Ihis view suggests that it is aucial. for institutions to be quickly responsive an:i responsible in terms of prcvidin; a:pp;opdate data al:xJUt the quality of uniergraduate edllcati.on, as well as data a1x:1ut an institutia1' s ather missia'lS am activities. '!he question then beoanes a'18 of identifyi.n;J the set of quality indicators that an institutiat decides is a.ppt'Opriate to provide to its ext:emal. cx:I'1Slituencies.

External c::cn;tituencies have, in a sense, an "information gap" relative to what they lcr1ai al:xJUt the quality of ~te education on the 'IWin Cities C811"pJS. 'Ihis may be due, in part, to the perceived lCM priority that ~te education has had c:hJrirJ1 the past three decades. Some of the increasin; dammds for new quality assessment initiatives cane frali our failure to (X'IIIIImicate with our axt:ema1 cx:I'1Slituencies what is already kncwn al:xJUt the quality of ~ta education on the 'lWin Cities caupJS. By way of. c:cnt:rast, there are very st::t:aq perceptia'lS about features of the university that are bein; correlated with poci)r quality ( e.q. , larqe class sizes an:i infrequent c:ontact between students am faculty) • '!he ~lication of this di soJSSion is that dsvel.opin; am i:q:)lement:in:J a quality iniicators proqxam mquires the expertise of public relatia'lS consultants who can help us develop a strategy for CCil1lllllli.cating quality assessment iniicators to our ccnstituencies.

'l'rc:M (1988) connects the increased acx:ountabllity requirements by states to increased financial support for oolleges an:i universities: "'Ihe fifty states have i.rx:reased their support for the plblic sector of higher education, they have dema!x1ed qreater acx:ountability fran the colleges an:i universities for the use of these fl.lrm" (p. 22). Hartle (1986) suggests that colleges take a leadership role in developin; quality assessment efforts that "meet the plblics' interest while possessin; institutional autonany'' (p. 9). Acccl:l:iin;J to Tan, "our cbsession with qJality is speculated to be inherent in our demcx::tatic culture, in which CDUpetition an:i exo?llence are central values" (p. 224).

Pl:q)osed Qlla].ity Irxiicators for un3ergraduate Education Page 10

5. O]i!lity Assessment Efforts at other Institutions

As previously noted, assessu:ent of urx1£graduate education is a focus for new initiatives in many other states am institutions. Although it is beytn:i the sc:xp of this pr:qJOSa]. to de an exhaustive review of other quality ••essment activities, brief SUIIIII!lrles of efforts elsewhere help put cur efforts into an apprcpriate nat:i.aal CXI1taxt.

First, w in Mil'Jr...,ta need to a:nti.rua to learn fran sucoesses am failures in othar states. Seccn:i, the ctu:aDlcgies of events in five case sbXly states (i.e., CClarado, Mi.SSC"lrl, New Jersey, south Dakct:a, am Vil:qinia) descril::led by Boyer arx1 Ewell (1988b) m:r;est a start-up period of at least three yams. 'lhil:d, state-based quality assessment initiatives m.lSt resporxi to the part:icul.ar state IS CXli1C81:nS al:lout quality arxl, llm'8 i:apn1:antly 1 must be cxnsistent with the ~ policies arx1 ~xres within the state. Finally, st:ate-1::lased ~ to assessment have been na;t successful when institut:ional. arx1 system initiatives have been erx:curaqed am supported, rather than when states have predat.emined part.:ic:ular ~ to quality ass'!SS""P"lt.

Six different types of activities c:baract:erize the ranje of quality a•es•ent initiativas at other institut.i.aw: (a) tac:ul.ty initiated quality a•esmeaJt projects directed at part:icul.ar quality assessment dimensions; (b) the devel.q:maut of centers for resaaxch. at urx1£graduata education; (c) c:x:qzehensiva assessment systems that are closely linked· with institutional goals arxl, in sc:ma casas, legislative tllniiD;J; (d) applicatia1S of educatialal. evaluatiat to develop ampJS-based assessment initiatives: (e) the devel.opoeut of urxJergraduate quality in:ti.cators as part of a cxq:u:ehensive set of institutia1al quality in:lica~: arx1 (f) attempts to include quality assessment activities in institutional initiatives designed to ilip:cve urx3ergraduate education.

Faculty-initiated quality assessment projects ccnsti.tute the first type of activity. 'lhe Harvard Seminar on Assessment, :nt:lfli at the end. of its secorxi year, is an a'J30in:J forum for faculty arx1 administrators to develop concrete assessment projects. 'lha san:lnar, led by Professor Ric::hal:d Light of the John F. I<'a1uedy Sc:bcol of GaYen1ment at Hal:vard, cxndsts of al:xlut 90 regular part:ic:ipants, incl.u:iin;J 60 faculty arx1 administrators fran Harvard and 30 fran other colleges and ~ agen:ies in the area. '!he goal of the activity is to enccurage faculty research that is directed at ~the college experience, rather than in pJ:OVidirr; institutional quality assessment data. As such, it resM .Jes the pr:• p:&!d activities on our canpJS SlJ3geSted in the :z:eport, "Practical steps 'l'cward Enharcln; our ~ of and I'Dp!ct an the o:rx1ergraduate Experience at the university of Minnesota" (Peterson am Hemel, 1987).

'1he secon:i type of activity is ale that assigns quality assessment initiatives as ale of the responsibilities to be assnned by new research centers an canplS. Alla'q this type of activity are those at the University of Arizona am Irdiam University. 'lhe Ariza1a plan, the results of a CCIIIIdttee c:haired by Clift:at Conrad, called for the establishment of a

Proposed Quality In:iicators for TJmergra.duate Education Page ll

center for Research on UrX!ergraduate Education, effective January 1, 1988, to beqin developin;J a framework for institutional quality assessment. 'Ihe four areas to be addressed were: ~ive assessment of the quality of the institutional environment, student lea.mi.n; and developnent outcomes, value added student lea.mirg, and the relationship between the envirorAnent and student lea.mirg. 'lba c::ar:mittee further l:'80 1141E:ided that the first two years of the center's wcec be an ~ period durin;J which various assnsnnt ~ be used at a pilot basis. Irxiiana university's three­part ptooass is simi 1 ar to Ariza1a 's: Ai:POint an assessment pl.a.rlniD;J CXIIIDittee, c::reate an assessment rescurt:e center, and .iq;)lement pilot assessment measures. tl'1'xk:llDtedly, given the pressures many inStitutions are urXIer to in:::reasa assess• :d: activities, centers have been created in numerous other institut:i.a'lS. '!his ~has been discussed as a possible strategy for the university of Mi.mesota. '!he structure and objectives for such a center are desc:ril::led in the :teport, "P:toposal to create a Centralized Resea:tch center on the Twin Cities QmplS for Research on undergraduate F4lcation" (Me:t:Win, 1987) •

'lba thi:td type of activity includes. those institutions that developed, prior to the recent surqe of interest in quality assess"'P'lt, ccmprehensive systems to define and exzmdne c::han;Jes in the quality of un:lergraduate education. Amcrl1 these are systems at Northeast Missrnri state University, the university of Tennessee, alio university, and Alverno COllege. '!he university of Tennessee's c:arplex system, stim1l ated by link.in; state tl.1rxlin; to dala'strated ill:prcvement:s in undergraduate education, is used across their ampJSes. Northeast Mi.sealrl state university's approach is a priJDary example of a value added ~. Chi.o university's approach (Mx3en, 1987), in place for seven years, includes six~ including value, added testin;J, measu::res of social and academic integration, and alumni outc:cme measu::res. Alverno' s ~ (Mertt:kaNski and Ioacker, 1985) focuses on student outccmes -as the centerpiece of assessmPnt activities. What is unique atx:ut their ~ is that assessment of student outcomes in eight ability dimensions (e.g., prci)lem solvin;J, aesthetic response, takin;J environmental responsibility) is intimately~ to curricular design issnes.

A fourth type of activity adapts educational evaluation IOOdels for use as a tool in quality assessment. '!his ~, exenplified by work urXler way at st. Olaf COllege ('lhaDas, 1987), begins by disolSSin;J the relevant inplt, process, and outc:cme dimensions to be used. '!he secorxi step specifies i.nterx5ed educational outccmes, develops :inii.cators for each of the dimensions, and articulates starm:tds to be used in maJd.n;J judgments about quality. '!his approach is especially useful in workin; with groups of faculty and administrators to develop an agreed-upon set of i.n:iicato:tS of the quality of undergraduate education, al't:hc:lugq it is problematic to .iq;)lement in large, diverse faculties characteristic of institutions such as the University of Minnesota.

A fifth type of activity is the attempt to collect, analyze and di.ssetninate quality i.n:iicator data al:xlut un:lergraduate education within the context of a total set of institutional quality iniicators. In 1982, the State Eoard of

..

Pr:qxx;ed Quality Iniicators for ·~te Fducation Page l2

Education in Florida required that postsecon::my systems develop specific in:li.cators of quality to be used in settin; quality jmprovement goals, in justityin; l:Judget requests, arx1 in evaluatin; the results of efforts to illp:'ove quality. In their cutl:ine of the set of inticators fran the Florida university system, Coles, Q1llar arx1 Mitdlell (1987) made the point that each of the quality :iniicators shcW.d a::kb::ess a partic:ul.ar policy issue. Al:tllcu;h Florida • s rmicators P1:og1am is quite different fran the one being ptopcaad for the Oniwrsity ot Minnesota, it is described below as an exJ!!!Ill• ot a set of indicatcz:s.

Florida's I:ndic:atcrs P:tog1am c::attains 21 in:li.catars of quality, nine of whi.d1 are J:atarred to as ~·" indicatcrs arx1 twelve of which are mtarred to as "f~" in:li.catars. '1he upper-quartile in:ticators are these tor wbid1 natia1al CXIIp!risal data are available. ·'lha foundational indicators l:8tl.ect .institut.ia1al. in:iicas which are more closely related to educatia1al. quality, 1::ut camct be evaluated relative to national nonns. '1be nine upper-quartile indicators are: mean tac:ulty salaries, mean faculty salaries by rank, li'U"ber of enrolled National Merit SCholars, state financial aid per student heedc:amt, state ag%cpriatia1S per student headc'omt, nl!l'b!r of scientists arx1 en;rinaers enployed, rank of research libraries, nl!l'b!r of BU. Beta Fappa chapters, arx1 national repltation of rank. 'lha twelve foun:latiCI'lal. measures are pm-post c::cq:erisons on the Colleqe Level Jcadenic Sld.lls 'rest, finiin;s of follaM'-Up placement, lic:::ensure exam:i natial, success of graduates, GRE scores of graduates receivin; baccalaureate degrees, percant of tac:ulty by highest degree, adrn:ission test scores, r11ft!ber of en:k:lwad c:bairs, spedalized or professional accreditatial, st:udent-tac:ulty ratio, 8qUoyDatt gcaJ.s for equal access arx1 q:port1lnity, st:udent m:lnority rec:z:uit3ent, arx1 a::l1tract arx1 grant furWn;r per tac:ulty. Of the nine~· measures, six are inpit inticators, two are ptocess :iniicators, arx1 one is a reputaticnU :iniicator. Of the twelve foun:1atia1al. intic:atoDs, three are inpit measure, five are precess measures, arx1 four are output inticators. Coles, 011Jar arx1 Mitchell (1987) ccncluded: "Regamless of the sbcrtc:cmin;Js, the inticators reports are prcvidin;J infODii!ltial which is potentially helpful in policy analysis arx:1 decision liiBldnf' (p. 16) •

'1be final type of activity is student assessment that is closely tied to institut.ia1al. efforts directed at ~ undel:graduate edlx:ation. sane ot these efforts are closely tied to major c:urriall.um revision projects, two ~es of which are at the university of Minnesota (i.e., Project Prosper at H:n'ris arx1 Project sunrise in the COllege of Agriculture) • others are focused en partic:ul.ar aspects of undergraduate education in need of bpravement. For exanple, at the university of Pitt:sl:lurgh (Wein;Jartner, 1987) four semi-autoncmcus sftxxmni:ttees have been charged with developirq a plan for inprcvement in these areas: instituticn-wide bacx:a.laureate requirements, ~ undexqraduate 'leal:l'liD;, broadeni.rq curricular perspectives, arx1 capitalizin;J on institutional stren;Jths •

Proposed Quality In:iicators for Utx3ergraduate Fducation Page l3

6. '1he Role of §tazilantized Testing

Many of the su;gested refa::ms in umergradtJate education focus on concerns about what students are leaminq in the nation • s colleges and universities. Salle of these c:cncmns are related to hypothesized decreases in academic rigor as higher educatial ac:o l!iirdated to the needs of less well-prepared students who entered colleges and universities in the 1970s. Salle concerns are a ractial to sbldies o:mpsriD;J the perfcmaarx:e of our qraduates with tbcae of other c:cuntries. other CCI'JC8D'lS ware the result of a "spill-over" to pcatsec::amry educatial frail dala'Jst:z:atad pear leam:in; c:utccraes in alE!II8l'Itary and sec:cn:my educaticn. Not suxpris:inqly, ea.c:h of the above ocnc:ems was stil!l'1 ated, in part, J,:,y data collected fran st:an:W:dized tests developed. to ]IIMS!Jre student learn:in;.

Althcugh assessment activities can focus on .i.nputs, processes or outcanes, outcc:mes assessment has been E!ml;ilasized in the recent wave of concem about quality in higher education. Salle instituti.als, such as the University of Texas at Austin (Bansal, 1988) , have focused on students' assessments of educational goal attainment as a means for assessilq college outccmes. 'nli.s ~was first used at the University of Minnesota decades ago when Rebert Pace c::cnducted pioneer:in; research in the General COllege. Although student assessment of goals attained is an inp:lrtant c:::c:mq;xment of outccmes assessment, mcst of the recent efforts have focused on the use of tests to directly_ assess student leam:in;.

Many of the new institutional assessment efforts (e.q. Northeast Missouri state University, Alvaxno COllege, University of Tennessee) focus heavily on data collected fran st:amardized tests. A test-based quality assessment ~ is likely to be viewed positively 1:¥ state legislators, since they have had considerable fami 1 iarity test:in; students in the college arena. As Marchese (1987) has nctad, a test-based system can p:lSe serious problems as a sole criterion meaSJJ:re in quality assessment in higher education: "Applied to a different ente%prise -a college-that tradition can be dan;erous" (p. 5) , sin:e "Assessmpnt per se quarant:ees nothi.n; by way of imprcvement, no mere than a thel:m::meter cures a fever' .(p. 7). Articles by Bok (1986), ~ (1986), and Charxil.er (1987) have voiced similar concerns about test-based quality assessment programs.

steele (1988), in a review of the validity and reliability of measures of general educatial outcc:mes, di sc::usses hew the result of standardized tests, such as the COllege Olb:xme Measures Ptop:am (CXMP), can be used to illlprcve p1:03:tams and curricula at particular instituticns. He notes that "over the last blel.ve years almost five hunll::ed colleges and universities have uti 1 ized the tests and sernces of CXMP" (p. 3) • '1he four case studies he presents (i.e., Bethany I11theran COllege, our I.ady of the Lake University, the University of Pllget sourxt and Austin Peay state University) illustrate hcw st:aroaz'dized test results can influence curricular decisions at particular institutions that place a high etPlasis on qeneral education.

'!he etPlasis on test:in; leaminq outccmes is less problematic in those systems that include tests other than nationally standardized ones. Banta

Pl:qa;ed Quality :rmicators for undergraduate Education Page 14

ani Sdmeider (1986) , in tlleir paper on the experiences of 14 departments at the University of Tennessee which use lccally developed exit examinations for majors, su;gest that lccally developed examinations can be useful in evaluatin; curriculum ani instJ:uction in particular units. One of the problEIIIS, of cxmse, is that developiD; valid ani reliable e»minations is an expensive, t:ime-c:a1sum:i process that few imtitutions are williD; to follow. '1ba QU.vm:sity of Tennessee also uses other tests, such as the ACr ·CXMP Exam ani the kademic Profile, ani is able to ccnt:rast the usefulness of natia1ally st:ard!l2::diz8 varsus lccally davalopa:! examirations (Banta an:i Pika, 1988) •

'1ba pressures em the QU.versity of Minnesota to develop ani use tests that ·document learl'li.n;J cut:ccmes of a baccalaureate deqree are likely to increase in the futm:a. We are mvinq fot'Wa%tl in several small ways, includ.i.n; a Sprin;J 1988 research sta1y usin;J the Acadmic Profile, developed by EIS, an:i the COllege outcaDe Measures Project, developed by ACr. Before w can adopt, as a CJ'llt'US, either a test or an approacb to testin:J baccalaureate graduates, we llllSt first develop a process to brin;J faculty together to disolSS CU%%'iculum ani related learnin;i cut:ccmes. until we catpile that di solSSial, any selectiat of learl'li.n;J outcc:ma measures for all bacca.laureate graduates is praDature. When we are at that point, we DJSt use the scholarly resources available to us to help assess college student learnirq. Q'18 of the best rescurces is .Melman's (1988) Pertonnance ani Judgment:

· Dgre on Principles ani Pn'Ct1r- in th9 hises5!!19Jrt of College student I•rnim· It is app:opriate at this time, ha.v.7ar, to use certain test data for Dp:u:tant sub;rcups of students (e.q., GRE sc:oms for bacx:alaureate graduates .intarested in atten:iin;J graduate sc::hocl) • If we are to proceed as a c:aD'plS to ass ass cut:ccmes in particular areas, we l%llSt be prepared to devote extensive rescurces to develop learnirq outcc.me measures that are reliable, valid ani cx::mpatible with our instituticnU. goals ani climate~ One possible area for us to develop leam.in;J outcaDes is in writin; assessments for our baccalaureate graduates.

Wallace (1986) , in a paper entitled "What ~tes leam: '!he role of assessment in laJ:qe research universities," ~ized the importan:::e of the assessneit of quality of UI'D!rgraduate educaticm "because these institutions set st.andards, pJblic :p£Ceptia1S, an:i expectations for l'lllCh of the education system ani becaJ:JSe they provide diJ:ectl.y the undergraduate educatiat for large n1111be.rs of the ablest an:i 1lCSt fawrecl students in the scciety, p8q)l.e 'Who as graduates will be leaders in all of the institutions of scciety " (p. 15) • He suggastad that the insufficient purposiveness Y1:2 § m l.1l'X3ergradll educaticm in institutions such as ours requires the following' seven spec; al considerations in the design of assessment activities:

• Assessment of learnirq outcaDes should be part of a framework of del jheration, inquiry, ani action that the faculty trusts.

• Assessment of leamin;J outcanes should be part of a framework that students an:i erplcyers tJ:ust.

Proposed Quality Indicators for Urxie.rqraduate Fducation Page 15

• Seek assessment strategies that emphasize continuities with learning in the schcols ani l::.uild on pzo;1:ess in assessment that have been made in the K-12 system.

• Seek cpporbmities to use fOl:lDal assessmP.nt of leaJ:nin;J outcomes to generate a sense of cx:mramity ani shared enteJ:prise in large, c:cmplex l:eS8i!J:'Ch universities.

Beware of sin;Ue distinctions between leamin:; pzocess ani learning product where <:XI!Plex leamirq is c:cnc:m:ned •

• Look for cpporbmities to inct'ease our use of intensively evaluated pilot progl:dDIS •

• Be pragmatic atoit isSIJeS of scope ani scale; we should dare to think small as well as lal:g'e. (pp. 1Q-12).

7. Ccnnections arxi :tp;al Concerns Abgit Un:lergra.c:luate F&luca,tion

'Ihe need to develop a set of quality in:licators for umergraduate education on the Twin Cities camrus has been the subject of disolSSion in several task forces ani ccmnittees dl.lrin; the past five years. 'Ihe foreword to '"'he Final Report of the Task Force on the student Experience" (July 1984) included the folle7td.n; statement: "Un:1ergraduate education has been virtually eveeyaw's respcn;ibility, ani in ma.lity that has meant that it is no a18 1 s 1:eSpCI'lS:ibility. We bal.ieve that it is imperative to focus that l:eSpOl"lS:ibility and to aax:mpmy it with c:cmoensurate authority" (p. 1). 'Ihe 1:epcu:t went a1 to address seven "i561JeS rega.rdj.n:J the quality of teachin; and learning at the university" (i.e., student preparation for lower division, allocation of~ l:eSOU%'CeS to the lorNer division, course acx:ess:ibility for urx3el:graduates, st:x:ager sense of CXJIT!!J!Lmity ancng un:lergraduates, active learning strategies, problems associated with large classes, and one-to-one interaction between-students and faculty), but did not call for the develq:ouent of a set of quality in:licators to mnitor imprcNements in urXIergraduate education.

Mol:e recently, other reports have been 11X)l:9 specific in makin;J rec::c:mmen:!ations alxlut needed quality data and the processes necessazy to insure that the data be used to impJ:cve the quality of un:lerqraduate education. '"nle Final Report of the Special camnittee on Minority Programs in SlJRX%"t of camnitment To Focus" (May 1987) ~''that the university allocate l:eSOU%'CeS to develop a c::auprehensive, centralized, CCilplterized data system to facilitate mnitarin; the pzog1:ess of minority students ••• " (p. 10) • "'lbe Final Report of the Il!plementation Task Force on UlXIel:graduate Education at the Twin Cities QmpJs of the University of Minnesota" (J\me 1987) reo "iiemed ttNo mecbanisms l:el.ated to the university's goals for un:lerqraduate education: '"'he first is priJnarily administrative-to mnitor overall perfonnanc:e ani to ensure that policies and administrative practices are c:han;ed as needed to control enrollments and to impJ:cve un:lerqraduate education" (p. 19). That report also located

Pl:qJOSed Quality Irr:iicators for urx1a'graduate Fducation Page 16

this tunction in the Office of the PrcM:lst and specified three necessary activities: (a) to gather infcmoation, (b) to distril:ute that infonnation to these whc DJSt act to c:han;e perfomanc:e, an::l (c) to design mechanisms to maka sure the infcmaation is acted on and that the University's perfonnance i1lprcMas as a result. Finally, the report suqgested that four tren::!s of in:format.i.al--ca1 variables, input variables, process variables an::l output variables-shcul.d be in:ludsd in a cx:.q;nhensive system to m:mitor tba ettectivaness of undargradllata education.

Another report, "PrepariD; Far tba 'l.\alty'-First Cenb.u:y: Bac1<g:roun:i Paper Far a Discussion at the University ot Mil1nasota Twin Cities Cmpls" (septali:ler 1987) 1 in:luded a sectiCI\ on unclai:gl:ac1uate educatiat and noted several problems (e.q. 1 la.r division advisirq and the need for active learnin;J opportunities) that need to be 21Xb:assed. 'lbat repent suggested that data alxJut inprovements in undergl:aduata education is a critical o "'Lalel'lt of University pl.annirg: "success in pl.ann:in; can be judqed only by ot:servabl.e c:haD;es in cutcane of our programs. Plannin; succeeds only to the extant that we can point to ~, research an::l serrl.ce of higher quality 1 an::l mre responsive to the needs of the public that we serve" (p. 1).

8. Guidelines in Devel.c:pim Qtml ity Asse=rmt Irdicators

'Ihe experiences of many states, inst:itutiaJS1 and assessment experts have yielded a rich set of guidaJ.ines for us to follow in think:i.n; about how to set up a quality indicatcrs pro:Jram for the University of Minnesota. 'Ihe fact that we are a scmewhat late entrant to the quality assessment arena means that we can benefit frail the mistakes and successes of others.

Nettles (1987) 1 in a publication sponsored by the New Jersey state College GaYemin; Boards Association, listed five pri:nciples in develop.in;J quality assessment in:iicators: (a) use DllltidtmensiCI'lal iniicators rather than a sillpl.e iniicator; (b) develop iniicators that e.xcee:i min:imJm st:a.rx:lards; (c) develop ilxli.cators with realistic expect:aticr1s to fulfill goals; (d) provide for analysis of data to ex;mi ne the effects of assesemwmt on ~ ~ an::l leam.in;J; and (e) use assessment ~vities that yield divi.derm wcrt:h the investment. Nettles highlighted the important role of aress.in;J learnin;J in his stat:aDent: "Bec2wse the primary mission of higher education institutiaw is providinq inst:%'uction, it is quite appropriate that sbx1ent leam.in;J and develq::ueut be the centerpiece of assessment policies" (p. 14).

!.Wl.l (1987a, 1987b) has artiauated several principles about hew institutiam shculd deal with the in:reas.in;J requests for assessment initiatives and acx::amtability data. He views quality assessment as be.in;J based on two key ca:cepts. 'Ihe first ca:cept is that of choice, suggesting that no institution can i.nvest:iga.te all of the quality assessment dimensions that might be of value to the institution. '!be secx::ni caM:ept is that of quality assessment as a ''pxogz:am, n a visible, integrated, QD30.in;J effort of an institution to CX1l1!1.11'1icate to c:cnstituencies relevant data alxJut the

Proposed Quality Irxlicators for undergraduate Education Paqe 17

quality of ~te education. His three principles for institutions initiat:i.n; quality assessment efforts are: respon:l visibly, build or streD;Jthen (in::l\.ldizq us.in;J existin;J data), an::1 show action by maki.n; quality assessment activities part of inp:ntant administrative processes. Ewell (1987b) suggests that the dilemma na5t institutions rst::M face concerns "haa to meet growinq demands for acocuntability an::1 at the same tilDe urX!art:aks seri.c:ul internal investigations of instructional effectiveness" (p. 1) • '!he c:halle19! for the university is to develop a set of quality in1icatcrs that answal:S inpntant questions posed by the institution's ccnstituencies an::l, at the same time, ~ areas in need of llrprovement, 1:he!:eCy lead.in;J to the in;D:'ovement of quality in urx3ergraduate education.

Devel.op.in;J an apptoprlate set of quality indicators an::1 assessment processes depends, as TNel.l, on the cx::up.tibillty :between those indicators an::1 processes an:l an institution's value structure. Irxlicators should be developed that reflect o , .. ,. nly held beliefs about institutional st.ren:Jtbs an::1 weaknesses. What that suggests for the '!Win Cities canp.lS of the university of M:imesota is that the indicators reflect concerns abut urx1eJ:'gl:aduate educatiat expressed in recent task force z:eports (e·.q., "Final Rspol:t of the Task FoJ:t:e on the student Experience" (July 1984), '"lbe ccmaittee on Quality Un::1ergl:adllate Teac:h:in::J an::1 Isa.min;'' (August 1985), "'1be Speci aJ camnittee on unified an:l IIx:reased Preparation Requirements" (March 1986) , "'Ihe Special CCIImi.ttee on Ccordinat:i.n; r.,a..,er-oivision Education on the '!Win Cities campJS" (May 1986), an::1 '"lhe Ilrplementation Task Force on ~te Education on the '!Win Cities campus" (J\lne 1987).

A z:elated issue concerns an institution's motivation an::1 energy available to develop a set of quality inii.cators. Rossman (1987) makes the point that an institution's attention to other issues may be so intense that faculty an::1 administrators rray net have sufficient energy to launch major quality assessment iniicators. 'lhe attention given to A ccmnitment to Focus may explain wey develop.in;J a set of quality inii.cators has been difficult to ac::o::uplish in the last few years. Ironically, develop.in;J a set of quality inii.cators for ~te education is entiz:ely consistent with the goal articulated in A camrl.tment to Focus, "to focus on~ the quality of em: un::1ergraduate pl:O:Jl:ans." Ewell (1987a) speaks to these concerns as follaNS: "Det:el:mi.nirxJ an app:opriate assessment appJ:CaCh is an art that depems upon ••• an::l upon an accurate di.agncsis of the local organizational an::1 political.climate" (p. 25).

Finally, in these times of fi.naD:ial. z:estricti..ans an::1 expectations about the decreas.in;J availability of flm:li.n;J, develop.in;J the initial set of quality inii.cators should z:equire limited additional fi..nancial :resources. We should expect to incur scme costs while dj sc·nss.in;J this draft an::1 develop.in;J a set of ~tionally defined inii.cators. OVerlooki.nq such expen:litures is one reason wey assessment costs "are frequently an::1 seriously un::lerestimated" (Lewis an::1 wasecha, 1987) • Ewell (1987b) an::1 Nettles (1987) also have noted the importance of develop.in;J assessment approaches that are cost effective. 'lhe set of quality inii.cators be.in;J proposed for the '!Win Cities cattpJs involves c:ollect.in;J z:elatively little additional info:cnation, since mst of

Pl:qa;ed Quality Irxlica.tors for ~te Education Page 18

the iniicators will be based on data and pz:ocess• already in place on amplS.

'Ihe Task Force on Post-seccndary Quality ~essment was est.ablished in 1987 by the Minnesota Leqislature to study and to make recc •'l'etmtions concerning quality assessment. An inp:lrtant cc:n:::l.usicm in that groups' preliminary report (1988) is that •every state dsvel.ops its quality assessment programs urDir diffennt ci.rcumstances. n Within Minnesct:a, the fact that postsec:c:n.:1a instit:utials traditia1ally assam- a:nd.darable autcnany ~ that assas!!!Mnt activities be daYel.cped to meet the needs of particular institutic:11s. '!be preliminary report o::ntained a set of ten guidin; priD::ipl.es far deYel.cpi,n; ass ISpMilt policies and practices. '!bose principles served as a'18 of the fourxiatials used in pz:oposing the set of ilxiic:ators CXI1tained in this prel.imina%y propcsal., of which the m:JSt relevant are the followin;J:

Principle 2: Hll.tiple and varied measures are mre desirable than a sin;le, starmrdized exam,

Principle 3: Keep the ra'lll!t)er of assessment dimensions to a manageable l?""'ber,

Principle 6: Data collected shcul.d bnild upon exist:in; data and shalld reflect the amp.lS master plan, ani

Principle 8: out:ccmas assessment shcul.d yield infonnation to decisiat makers atxut the quality of the educatiaal experience.

RATiamE roB PB:>mm) INI2ICA'rol§

'lhe iniic::ators that folla« rray be considered as key questions which students, faculty, administrators and citizens of Minnesota pose about the quality of unjergraduate education on the '!Win Cities c;mpJS. We need to think about the appl;'Oprlateness of these iniica.tors for an international, research, land grant instituticm. saue iniicators rray be in conflict with others (e.q., in:iicators of student access versus iniic::ators of student selectivity.) Many of the questialS relate to processes ani inputs, since, rightly or wraqly, there is CXI1Sidarabl.y less ooncem in Minnesota about what students are leamirq in our colleges and universities. Part of why this is true rray be the percaived ani actual high quality educational cg;xn:tunities, fran ki.ndergarten to qraduate sc:hcol, in the state of Minnesota.

In prepari.rg this list of 18 iniicators for di so1SSicm, many others were considered. Astin's (1985) cx:u::ept of student involvement is an .ilrportant umbrella far many institutiooal. activities, bit is too broad to be defined clearly. Irxli.ca.tors of ocurse availability speak to hew well the university is doing in delivering the courses students want ani need, but no systems are in place to describe ocurse demani, except far class closure dates. other iniic::ators reflecting faculty perceptia'lS about quality (e.g., the

Prcposed Quality Il'xlicators for Urxiel:graduate Education Paqe l9

number of children of faculty who enroll at the university) were also c:cnsidered as poss:ible indicators. 'lbese arxi other poss:ible indicators certainly my replace sane of the 18 proposed indicators in the process of di smssin; this dcoment.

0 •!ll•lts ab::ut the p:topc:aad indicators inclu:1e sane di snJSSion of the need tar strategies tar ammarizirq m:l disseninatirq the quality indicators. All activities within the university 1lllSt CXIIJnect these ~te educatial in:1icators to dspart::mantal m:l col.l.eqiate data, or else the p:tq:ased quality in:1icators will be perceived as ~ m:l withcut tnplicaticns tar ac:camt.ability m:l i:mprcwment. Sane of the in:licators may have primaJ:y value within the university, whereas others may be more valuable far external ocnstituencies. certain cxmnittees on canp.1S (e.g., Senate camdttee at Educational Policy m:l the Assenhly camn:ittee on tl'nd£graduate Educatiat) have a critical role in fonrnl atirq guidelines ab::ut the use m:l di sseni natiat of the quality indicators.

Detinin; quality in ~te educatiat historically has concentrated on three different a;:proac:hes. '!bey tOOlS at different parts of the educational experience. '!be first approach concentrates on relevant resam:es or "inputs." '!be second approach fOOlSes at educational ''p:tOC"eSses," includin; the ccntent m:l nature of student experiences as well as institutional practices m:l pxocednres. Process variables are especially hp:Jrtant to ccnstituencies interested in whether or net the university shews the int:entiat to achieve certain desireable cutcanes o~ undergraduate educatiat. 'lhe t:hi.rd approach ~es the results or "outc::cmes" of educational experiences. All three variables shculd be included in a cxmprehensive set of quality indicators, al'thcugh recent disrnssions about assessment in higher educatiat have focused at the need for mere outcome assessment. I.ooJd.n;J at outcaDe measures has value only if you can silllll.taneously describe the inputs ani processes necessary to achieve particular outccmes.

'!be 18 indicators d.escribed on the followin;J pages, classified by type, are as folle7t.IS:

. Il'xlicator 1: . Il'xlicator 2: . Il'xlicator 3: . Il'xlicator 4: . Irdicator 5: . Irdicator 6: . Il'xlicator 7: . Indicator 8: . Indicator 9:

Preparation Requirements (Inplt) Acadenic Potential of Erite:rin:J students (Input) Students of Color (Inplt) Advisirq Resc:urces (Inplt) Cl.assroan Facilities ani study Space (Inplt) Retential Rates far Enterirq Freshmen (Process) Instruct.i.al of I..a.<er Divisiat Students (Process) UrD!rgraduate Olrriculum (Process) Use of Solni Educational Practices ani Principles (Process)

• Indicator 10: Class Size Experiences of Students (Process) • In:ticator 11: Grades Received (Process) • In:ticator 12: 'l'rain:in; of Teachin; Assistants (Process) • In:ticator 13: Mon.itorin; of Student Experiences (Process)

Pl:oposed Quality Inticators for uniergraduate Education Page 20

•

Iniicator 14: student Course Evaluations for Large Enrollment CCUrses (Process)

Iniicator l5: students' Participation in J!:sy university Activities (P.tcx::ess)

Indicator 16: Gr!lduates' PerfODIIU1C8 a'l Graduate :Record Ex2lminatial (Oltc:cme)

Indicator 17: Eq)loyment Experien:es of Gr!lduates (outcane) Indicator 18: Post:baccalaurete EdllcatiCDll Experiences (outcane)

one of the most widely 5URlOtted dsval.opoeut:s fran A rmmitment to Focus was the university's rcle in increasin; the college readiness of students seeld.D:J admi ssia1 to the university of M:innesot:a. 'lhe rec:c:mnen:2tions of the Speed al CCmnittee on unified ani Increased Preparation Requirements, effective for new .freshman ente.rin:] the university beqinrlirq Fall 1991, specify the followin;J ~ requirements:

Four years of l!D;;rlish1

'1bree years of llla't:haaatic::s ( incll.Kii.n:J geCmeb:y ani intel:mediate algebra) 1 .

'Ibree years of science ( incl'IJdin; cme biological ani one physical science) 1 ani

'lW years of a foreign lan;uage

AlthcAlgh it can be aJ:gUed that the university ought not be evaluated on the basis of students' o:urse-t:ak:inj behavior prior to ent:erin;J the University, the university's success in infl1.1el'1Cir¥;J seccn:1al:y education in the state is a clearly articulated institutional goal. 'Ibis in:licator, when juxtaposed with other in:licators that reflect educational quality and leaminq outcomes in the freshman and sqilalme years, sb:W.d be useful in assessin;J the university's responsiveness to the c:harl;in;J c:haract:er of its enterin;J ~tes. If students are ent:erin.J mre ~ prepared, \Ia have an obligaticn to provide improved cxn::liticms for their leam:in; on caD'plS.

Obt:a:in:in; data for this in:licator shculd requize little in tenus of additional c:csts, since i:qllemBl'ltin; the new preparation requirements will require that the admissions p:ocess rec:::oJ:Cs infODDation about the students' high school~ in each of the five preparation requirement areas. one possible operational definition of the in:licator is the percentage of ent:erinq freshmen students 'Who have met one, blc; three, four ani all five of the preparation requirements. Calculations could be done separately for students fran Minnesota ani for students fran other states.

Proposed Quality Irdicators for Ul'Xiergraduate Education Page 21

In:licator 2; Aca.df"M Potential of Entering Students

Although the acadeni c talent of students enterin:;J a particular institution may net affect the quality of what students experience at that institution, an il'lstitutial' s "selectivity" is perceived to be an important :in:iicator of il'lstituti.aBJ. quality. Recent written statements (e.g., A Ccmnitment to ~ (1985) , ani "Preparin; Students For the 'IW1ty-First Centm:y'' (1987) ) , disalSSians on omp.JS, and initiatives to rec::J:Uit ncre high-ability students all ~ the ilzpntance of an ilxiicator that reflects the university's SllCOeSS in rec::J:Uitin; academtcally talented, recent high school graduates.

'Dle twa bases upon which an academic potential :in:iicator may be developed am entrance test scores (e.g., SAT} or high school performance (either overall academic grade point average or high school rank) • Although the two bases CXIDbined yield the best predictor of college performance, high school · perfomarr::e tends to be m:n:e predictive for typical groups of ent:erin; college students. An irr.ti.catcr based at high school performance is mre easily intet:preted, sirre it does net require knowledge about test developuent prcxejllre&, especial Jy the noJ:l!lative meanin; of a particular test score. Virtually aveeyone urXIerstands what it means to have a grade point average of 3. 60 or to have graduated in the top 10 percent of one's high school graduatin; class.

other :in:iicators of talent (e.g., special a.ccc:uplishments in high school) may be worth ccnsi.detin;, althal;h they are less reliably and validly assessed. Sane of these measures are routinely collected through the Post High School Pl.annin;;J Pl:cqLam, l::Jut are net transferred to the admissions data file. At this point in time, we shcW.d begin to corXluct LeSe.arch to identify ~ particular talents ani experiences that add to the prediction of student SllOCeSS.

At;reein; that an :in:iicator based a1 high school performance is mre appropriate than a test-based :in:iicator still requires extensive dj scussion about which 21» of many possible :in:iicators is the IOOSt appropriate for the university of Minnesota given its mission ani location in a large, meb:qoolitan area. Sane of the possibilities are the pm:centage of entering freshmen in the upper 5 percent, 10 percent or 20 percent of their high school g:raduat:in; classes: the nnmber of valedictorians; or the percentage of ent:erin; students with high school gLade point averages of 3. 6 ani above, 3.8 ani above, or 4.0. other indicators of special academic potential (e.g., exrellence in science or CLeative writin;) are possible too, but are expensive to develop ani difficult to intel:pret.

Irx;licator 3; students of Color

Access ani a sense of welccme to higher education for students of color is critical to the university, the Twin Cities area, the state, ani the nation as a whole. one of the ccnsistent messages in diSC'lSSions of A Commitment

Pl:uposed Quality In:iicators for U'n:iergraduate Education Page 22

to Fccys has been to maintain (ani increase) access to the university for those minority students who can be 'Well-serJed by the institution. Although, in theol:y, it nay be possible to achieve increased selectivity am a smaller undergraduate em:ollment while mamtainin;J (am increasin;) access for students of color, ccnsi.derable sent:iJDent exists on c:anp.lS that ac:hievirq beth goals nay be an llr;'ns6ible task.

'lba University is net ala. aDK::n:J pcst:sa:x:nmy institutions in tryin; to ma:.nci.la c:x:mt:l.ictin: ret:* ""•Jdatiaw ragardin;J selectivity an::i access. In saua states, inc:raasiD; enrol.llDants of students net typically sezved by higher educatial h8ve resulted in in::reased state ccn::em for the educaticnll quality at tncse ampJSes. Naticnlll.y, many colleges am univm:sities are struggl.in; with plans for incrauin; the enrollment of students of color, rat:.a:inin;J them beyald tba first year or two, an::i inc:reasinq the rud:lers who receive ba.c:calaureate degr :es. Clearly, an .in:licator that focuses at cur success in att:ractiD; students of celor should be part of a plan to SUIIII8rize relevant data about un:lergraduate education.

'1bere is dan;er in basin; educational p:ogl:ess indicators on the ultimate criteria\ of recaivin; a hacralaureate degree, since it implies that no value has been added to the lives of those students who stayed for one, two or several quarters, 1:ut who did net graduate within a specified time period. At the same time, there are dan;ars in basin; an indicator on the university's success in att:ractin; particul.ar students, since their tenure at <'2"'pJS may be short-lived ani their experien::es may be negative, thereby havin; a detrimental effect at encouragin; students of color to seek higher education.· 'lhere is merit in basin; this .in:licator at the rnnnbar of students of color who enroll, since a1e can azgue that this is an important first step in enc:curaqiD; the University to evaluate its prog1:ess in enhar1ci.n; the un:lergraduate education of students of color.

We DllSt be careful, hcwever, to avoid givin:;J the hp:ession that we operate a "revol.vi.r¥;-door policy" for students of color. A oarprehensive system for helpin; the University in regard to its success with students of color nust include indicators such as 1'Ulb!rs of inquiries, rumbers of applicants, ani nlll'bers of students ac:cept:ed in additiat to n"'*'ers of students of color who subsequently enroll. 'lhcse data will be inprtant pieces of infcmnation for the new Associate PrcNcst of the '!Win Cities Ompls am Associate Vice President for k:ademic Affairs for the university system to Dalitor, :art they are teo tentative to be useful as an indicator of the university • s SI'V:C'SSS in enrollin; minority students. For cur purposes, the :best irxticator is the annual nnnher of .D!if!l students of color who enroll on the '!Win Cities canplS.

Irrl:icator 4: Mvisim Resources

until the last two decades, DCSt of the research to define quality of higher education has been based on either i.npit measures or institutional rep.rt:ation as the basis for definin;J institutional quality. Although the

Pl:oposed Quality Imicators for Un:lergraduate Fducation Page 23

cuu:eut tren:i is to use measures of processes or outcomes, there are some areas in which an input measure is an inpJrtant place to begin in assessing institutional quality. Un:lergraduate advising is one area in which an input meaSJre, the financial resources devoted to advising, seems especially appzoprlate. Faculty time spent en advising should, if ·feasible, be included.

We are all fam1J jar with several amp.lS reports of the last five years cal 1 in; tor substantial bprcveDents in the quality of ~te . educaticn a1 the Twin Cities ampJS. Advisin;J services, especially lower divisial advisin;, have been adversely affected by the decline in academic SIJRX)rt resources. We should not be smprised at students' c::c:uplaints about insufficient time for advising \!hen the adviser-student ratios in some colleges are seriously rut of balance.

Cbt:ainin; data cn resources devoted to ~te advising could become quite~, depen::1:in; upal whether or not an estimate of faculty time spent (3l advisin; is included. '1bese data should be generated sequentially 1

starting with those data that are readily available. Another unresolved issue is whether to focus en resources devoted to la.ver division advising, or to focus cn resources devoted to all urD!rgraduate advising. 'lhe imicatcr could be resources devoted to advising, or resources on a per stlDmt basis.

other aspects of the quality of advising services could include descriptions of the infor:matial and resources available to advisers and students which facilitate high quality advising. 'lhe c::atpiterized degree audit progz;am and the university Cou'rse Infor:maticn Project (UCIP) are CUt tw of the recent developoeuts en amp.lS that should c:::ont.rlblte to improved advising services to students.

In:ticator s; Classroom facilities ani stt.W Space

In the last five years, several reports have addressed conc:erns about the quality ot life en the Twin Cities carrplS, especjaJly as it affects undergraduates. Several significant and visible cha.n;Jes en canplS have made the envircnDent a m:n:e pleasant place, al'thcugh 1lllCh remains to make the canpJS a m:n:e attractive place for OJT students to be and to leam. unfortunately, indicators for saue of these dimensions (e.g., aesthetic quality of the ca.DplS enviranuent) are difficult to define clearly ani nearly i:aplss:ible to put into operaticn. Alt.hcu;h there are neither eapirical nor logical connectia'1S be1:TNeen classrcc:m facilities/study space and stlDmt leaming, the c:ansensus C1JJ1Jn:J students and faculty is that the J;ilysical environment for leaming needs significant ilrprovement.

'lW aspects of the C81lplS environment that are easier to define ani which significantly affect urx1ergraduates' leaming experiences are classroom facilities ani study space. 8oth are ilrp:lrtant aspects because they are

P.tuposed Quality In:U.cators for Un:lergraduate Education Peqe 24

very visible ani are frequently noted in c::attJla.ints about the student experience an the '!Win Cities c:mp.lS.

'lha il'xU.cators for classroclll facilities ani study space are simi Jar to the advisin; in:licatal:', in that they are resource indicators. Althcu;h one might axgua that neither has been diella'&trated to c::on-elate positively with st:uda1t leamirx:J, it is loqical to expect that they might. Mald.n; signitiamt i:q:zoYaDants in each area c:culd be a very visible indicator of the univarsity1 s CXIIIIIitmnt to ~ Ul'ldal:graduate educati~.

NUmaJ:als opticnl exist for ~y definin; an indicator for classroom :faci 1 ities arxl study space. Options al..ra!ldy rray exist in the Minnesota Space Facilities Medal. ClasstOCiii facilities indicators ccW.d be based on additional resan:ces dedicated to upgxade classroans, the :resam:es devoted to classroc:m eqnipnent (especially in large enrollJDent ccurses), or in:iices of student satisfactial with classroc:m facilities. An indicator for study space might llW'sur& tctal square footage designated as study space or irxiic:es of student satisfaction with caDpJS study space.

Devel.op.in:J indicators for classiOCiii facilities arxl study space DllSt be part of a mre broadly based p.lblic rel.aticns effort that identifies effective strategies for ~ the university's image Yi:i A m lll'Xiel::gradute education. c:twicusly, this particular o "i'ent applies to each of the indicators be.in; pz:qa;ed for C:cnsideration.

rmicatnr §; Rst:ention :Bat• for mtez:jm Freshmen

Retainin; ani graduat.in:J the student they admit is a serious problem for llCSt pcstsec:cn:1ary insti:tutians. A review of retention studies (Ramist, 1981) ccn:hlcted at a variety of baccalaureate grantj.n:J institutions indicated graduation rates that rarged fran 25 percent to 70 percent. '!he llCSt recent retention study c:x:nducted at the University of Minnesota (1986) found that 49.5 percent of a sa:aple of students admitted as new tresbmen in Fall 1977 had received baccalaureate degl: a as by 1985, eight years later. ~ with other Biq Ten sc:hcol.s indicated that oor students take lcn;er to graduate ani that the fin:iin;s are related to the high percentage of part-time students at the University.

'Ihe hpntance of reta.in:irq ani graduat.in:J the students it accepts varies as a function of the selectivity of tbe institutiat arxl whether or not it has mre students than it can easily aco , .. ,. date. One might argue that, in the past, since the University enrolled nm:a students than it oculd instruct well ani was not highly selective in admi:tt.in; students: reta.in:irq ani graduat.in:J a high percentage of students was not an institutional priority. As enrollments decline ani the university pays more attention to selecting students llCSt likely to benefit frail the educational experiences the University is prepared to provide, retain:in;J ani graduat:in:;J higher percentages of accepted students wUl becane increasingly inp%tant.

Prcposed Quality In:iic:ators for un:iergraduate Education Page 25

'lhe research literature on retention has chan;ed dramatic:ally over the past several decades. z.tJch of the early literature focused on delto;raphic cmrela:tes of retention (e.q., qenjer, socioeconcmic status). 'lhe problem with this ~ is that it does not explain why students drop out ani it i.dsntifies cmrela.ted variables aJ:x:ut which mt:hin; can be done. More recently, the wcrk of Pascarella (1982) ani Tinto (1987) have provided t:heoratical "X"dels that tty to explain what occurs in the ptocess of ~ out. Will.iD;ham (1988) has p:cvided a taxcncmy of drop-out types that has i:qW.c:a1::ia1S for .insti.tut:ia1a.l policy mak:in;. 'lhe eleven types he i.dentitied are as folla..:

• Graduate

. • caupl.ete other goal

• Plamed transfer

• Elect new altemative (e.q., job, marriage)

• Revise cost-benefit (e.q., family finances)

• Involunt:al:y withdrawal (e.q., health discipline)

• Volunt:al:y withdrawal (weak goal ·cxmnitment)

• Disaffected transfer elsewhere (sb:ag goal)

• Positive :redixecti.on (c:hanqe plans to scmet:hirg incc:ltpitible with university

• Academic di smj ssa.1

• Academj c withdrawal

Certain aspects of our stlxlent population need to be considered in disc1S9ions of a stlxlent p:tog:tess irxtlc:ator: (a) significant numbers of our students do not enroll for all quarters (i.e., they "stop out"); (b) the rumber of years to g:taduatial is high (i.e., an average of over five years) ; (c) we have many part-eme students; ani (d) aJ:x:ut half of our baccalaureate qraduates transferred into the university.

Sane arguments in favor of a retentiat in:iic:ator are that it is easier to calculate, is 1IX)J:8 Ukely to reflect quickly arry attEmpts to improve the freshman year ~ence, reflects year-to-year (e.q. , freshman-to­sq;ilalx)re) retention rates, ani fOOJSes at the aspect of the ~te experience (i.e. , the freshman-sophcm years) that is perceived to be problematic. Finally, research on the prediction of college grades irxtlc:ates it is easier to predict first year grade point average than subsequent year or overall grade point averaqes; this same phenomenon is likely to be true for preclictin;J retention ani graduation. Seemingly small

Piqased Quality In:ii.cators for U'mergraduate Education Page 26

d1an;es in retention mtes will have ilrpntant lltlpllcations for the university's overall enrollment manaqement process.

Pemaps we shalld consider that mea.su:rin; retention as a quality in:licator with the sq:ilaiDra year instead of the freshman year. Olrrently, our official p:Uicy remains to give st1x1ents relatively free a035s to the onivarsity, but to enforce a relatively sttaq wed:irq-out process in the Aestn•n year. If the O'nivarsity CU1times to DlCY8 in the direction of i.nr:aased selectivity, then a year retentiat irxticator '4IICUl.d be mara c::cn;ruent with insti'bltia1al gcal.s ani objectives.

Daci.d.in; which cma or twa of a myriad possible ratentiat iniicators should be used llllSt begin by listin;J these that can be c:alc:ul.ated withcut extensive prograiiiiLi.rq c:csts. Nl.1marCUs reports ani CXIIplter ~o;p:aus already exist that Lepatt ratentiat percentages ani enrollment patteLns ( e.q. , the percentage of students enrolled for all three quarters) for each of the university's undergraduate colleges. '1be c;hallen]e will be to identify the specific iniicators that will be the liCSt ~ to the university's constituencies.

· Belectin; a1e or twa ret:ential :in:ticatcrs shcuJ.d net preclude the deval.opla1t of a mara extensive set of indic:::a1xms that ww:W.d be useful to particular ~o;p:w, departments, offices ani colleges .at <:a.l'plS. For exanple, colleges· need to knew net just year retention mtes, since often persistence-~at iniicators have bplications for advisin; ani instructial. In additiat, ratential in:iicators 1!L1St be reported separately for identifiable sutx;roups of students ( e.q. , transfer students, oJ.der-t:han-av students, students of color, athletes, women, hand:icaR*i students, men, ani high ability studerits) •

Iaiicator 1; Instruction of tq...w Division students

Who teadles our students, what they are "t:au3ht, ani hew 'Well they are taught are three ciNi.CAlS questia1s to ask about the quality of underg:raduate educatial at the '!Win Cities caDpJS. As with the case for sane of the other indicators, infcmaatiat about who teaches our students J1'14Y be useful in portrayj..n; the reality of un3ergraduate educatiat at this C'2ID'pJS arx1 in gradually ~ both tbe perc::epticDI and the reality. '!be anrent reality is prcXIably less positive than we might wish it to be if the data ware to be widely plbl.icized outside the university. We are net alone, hcwever, in that many l.al:ge, research universities rely heavily on graduate t=eac:tti.n;J assistants to prcvide lower divisiat .instJ:uctiat. We llllSt be careful in arfi discJSSions of data for this :in:ticatcr to_place the results in an appx:op:tiate context. Mcreover, we llllSt avoid assnmj n;t that a course tau;rht by a faculty 1llf'llh!r is necessarily better than the same course taught by a teac::hin:J assistant.

OJrrentl.y, instruct:iaal. data are readily available thrcu:3h the Office of Management Plannin;J arx1 Infozmatioo SeNices. 'lhese data should be canbined

Proposed Quality In::licators for un::iergraduate Education Page 27

with student data fran the active student file in an in:ticator that reflects actual student experiences on the Twin Cities canpJS.

'lhere are at least three ways to think about this iniicator in the university's ~te colleges. 'Ihe first is fran a faculty instJ:ucti.cmal. respcnsibilities perspective, 'Which wculd suggest an in:ticator such as the percentaga of faculty JIW'!Ibers who teach at least one undatgraduate cxm:se ead1 year or every quarter. 'Ihe sec:on:i is fran the pexspective of cur.r:ic:ulum, 'Which wcul.d yield an iniicator such as the percentage of 1-xxx, 3-xxx ani 5-xxx level courses that are taught by faculty at 'Which ranks. 'Ihe tllim is fran the perspective of students, in which an iniicatcr such as the percentage of students, by year in school, who have had at least one regular faculty ~..r dllrirg each of the quarters in a qiven year, wcul.d be calculated. 'Ihe mst directly ani easily inter;pretable iniicatcr wcul.d be aw that looks at i.nst:ructional responsibilities fran the pexspective of students.

DisclSSialS about the variCllS in:licators might be especially sensitive an:i jnpntant for this iniicatcr, s:in:::e tzyin; to hprove on the iniicator has il11!1B'tiate ani obvious implications for faculty. TJnless actions are taken to gradually c:::han;a the reward systan as it relates to tea.chirg urxlergraduates, the iniicatcr 'IlBY fail to slDI1 ant significant ilrprcvements .__

What is taught to students is at least as jnpntant as how it is taught in definin; quality in underqraduate education. Professor Allan Bloom's (1986) Cook '!be Cl.osim of the American Min;i, Education secretary William Sennett's frequent proclamatialS about what ocnstitutes valuable college-level ~' ani stanfcm:l university's recent widely publicized revision of its cur.r:icul.ar :requirements, all attest to the :iJrpJrtance of curriculum in definin; quality in underqraduate education. 'Ihere is considerable disagreement today about what should ocnstitute the ·required curriculum in colleges ani universities, although mst agree that curriculum is in'po:rtant in provic:1:inq quality experiences to students.

Given the diversity of opinion about what ocnstitutes a desirable cur.r:iculum, it is ili'P"SSible to develop a quality iniicator that focuses on quantifyin;J the "goodness" of the content of cur.r:iculum in urx1ergraduate educatiat. 'lhe i.Dp:Jrtance of cur.r:iculum in p:reparin;J our graduates for the next centmy suggests that we wcul.d be remiss if we did not include an in:iicator that :retl.ected at cu:rric:ulum. 'lhat suggests we develop an indicator that SUIIIIBrlzes the level of cuu:ent disciSSion on the '!Win Cities canpJS about curricular issues in ~te education.

Although there is dan;er in assumin; that certain processes relative to cu:rric:ulum (e.g., number of meetings of the curriculum ccmnittee) will result necessarily in curricular ;refonn, the processes ltL1St occur if change is to result. one way to obtain relevant process data is through

P:tcposed Quality In:ticators for ~te Education Page 28

departments' responses to the Departmental survey of Practices in ~te Education. 'lhis 22-page survey, an outgrowth of the work of the Ccmaittee on the Quality of t1rD!rgraduate Teac::hin; ani I..eal:'nirq, is intEn:1ed to be used as a means for the Office of the Prcvcst to collect intOI:DIIltial alxlut deparbDents' att:.ention to ~te education. 'lhe questia1s that pertain to curriculum ccuJ.d :be aggzegated to fcmn an imicatar that reflects cm:1:ent activities in retOl:Dlin; curriculum in ur.detgz:cdilata edllcatia1.

WiS¢9t 9; Use of SgDi Edng¢iona], Pmctices arrl Priorities

cna of the .in:U.cators of the quality of ~te education that is often ovarlooksd is the extent to 'Which instituticns en;age in practices that theoey 1 research an;i experience su;)g8St c:ont:ril:A1te to high quality in undel:gr8duate education. Irxlicatcrs of the pz:IX'eSses in place in particular instit:ut.i.als are often ovarlooksd because they are mre difficult to define than input measures, and sean less objective than certain easily quantifiable ClltcaDe measures. Process measures of educational practices have the aa:tit:ia1al advantage of l:leiD; useful in expl.ajnin;J what contrib.rtes positively to ach:i.eviD;J specific educat.:i.a1al CAlt:caDes for students.

If a1e accepts that sc:me of the quality iniicatcrs shalld reflect an institution's educaticn!l. practices, it is by no means obvious haN to define relevant educatia1al. policies and practices regard:iD; collectin;J ~opriate data to use in cxnst:ruct:in; an irdicator. Virtually unl.jJuj,ted possibilities exist, so an initial task is to decide which of a rumber of reasonable strategies is most likely to be fruitfUl. '1he nature and influence of the Oniversity's rewm:d system affects all of the Oniversity's missions. A measure of whether departments have in place, or perceive an incentive system to inp:ave un:3ergraduate education, cx:W.d tell us nu::h about this bp:D:tant mXi.vational. variable. In particular, we might a1:telrpt to develop an .in:U.cator that reflects the institution's c:c:mnitment to improvjn; the quality of teac:hin;J on ampJS. As a ampJS, it is important for faculty ani staff to cxme toqetber and discuss hew to proceed in defini.n; the relevant assesS"'Pt1t dimensions.

One possible strategy is to use a set of seven ''Principles of good practice in~ education" (Clli.c:karinq and Gil!!l!sm, 1987) that has received ocnliderable recent att:ent:ia1 natia1al.ly and at the University of Minnesota. 'lhese seven principles are base::~ on fiva decades of applied research on good t:eachin;J and leaminq in colleges ani lmi.versities. 'lhey present a pez:spectiva, llllCh in vogue today, alxlut what c:cnstitutes good teac:hin;J. It is in sb:aq cart:rast to classical high education in which Wi.vidual students operate as isolated, :irdeperx1ent, reflective learners. 'lhe synthesis of the research was c::ospcnsored lJy the American Association for Higher Education ani the Education C)'mnission of the states (Chickering ani Gamson, 1987). '1he principles ~ that good practice in ~te education has seven characteristics:

Proposed Quality Indicators for U'ndergraduate Education Paqe 29

• Erx:curages contacts between students ani faculty

• Develops reciprocity ani cooperation anrn;r students

• Uses active leamilxj techniques

• Gives PLUitJt feedback

• EIIPlasizes ti:me on task

• CCmlllnicates high expectations

• Respects diverse talents ani ways of leazn:in;

'lbese seven principles are .already beirxJ used in two new assessment initiatives on canp.lS. First, questions pe.rtain:in;J to several of the principles have been inclu:3ed in the Baccalaureate Degl:ee camidates SUrvey that is beirxJ ccn:hlcted for all Sprin; quarter 1988 graduates. Secon:l, the Depart:mental SU%vey on undergraduate Fducation Practices, to be sent to all academic departments that teach ~tes, includes a section on the departments • efforts to i:mplement each of the prj.n:iples. Data fran one or :both of the above data collection processes might serve as the basis for calcul.atiD; an imicator that reflects the university's status in i:mplementin;J good practices in ~te education. One .interestin;­poss:ibility is to look at the disc:repan:ies that exist betWeen graduates' ani departments' perspectives aaass the set of seven principles.

In addition, ali.ckerirq ani Gamsm are developin; an ~tutional Self­Assessment SU%vey that focuses on faculty perceptions of activities in SURXnt of each of the seven principles, '!hey expect to have a fonn available for institutional pilot t:estinq begimin;J Fall 1988. We have been invited to be one of the institutions to participate in pilot testin; their :instrument. We expect to administer their survey to approximately 1000 faculty on the '!Win Cities caDpJS.

Iaticator 10; Class Size Experiences of students

'!here are advantages ani disadvantages to includin;J an iniicator of class size in the set of imicators to assess the quality of ~te education. Historically, legislators ani the general public often have focused on class size at the elementary ani sec::ornary levels in ma.kin; jud;ments abc:ut educational quality. As the University's un::ie%graduate population decreases, it is reasonable to hope that average class size will decrease. 'lhe reality is that unless sanet:h.in;J is done to force that to occur, average class sizes are unlikely to decrease ani, in fact, might increase. Another problem is that the research literature on the effects of class size in higher education are inconclusive. Slnaller classes do not guarantee a higher level of stOOent achievement: larger classes do not mean :i.nparsonality ani lack of stOOent-faculty contact.

PLq:ased Q.lal.ity Indicators for ~te Fducation Page 30

A cxup!ll ;n; reason for irx:luclin; a class size indicator is to reflect another of the plblic's perceptions of urXIergraduate education, especially in the first an:1 secon:i years at the university. '1he perception is tllat nx:st new students erx:omtter only courses that enroll hl.1rXIreds of students. OUr current class size reportjn; procedures fOCllS on averaqe class size by cxm::se laval, department, and colleqe, rather than trail the perspective of tha expari&'1C8 of a student across a specified period of time.

A mare 1DBI!Ilin;tuJ. class size indicator for our pnpcses would require analyzin;J students • CIOU%S8 reqistratials as nctec1 at their transcripts tcget:bar with CIOU%S8 data t%all the ()lartsrly course Invent:oey. '1he result wcul.d be an iniicator such as one of the folla.dn;: percentaqe of freshmen that had at least one c:aD:'Se each quarter that enrolled 50 or fewer students; percentage of treshmen that had all camses in their first year enrol..l.iD; 100 or .fewer students; or percentage of freshmen who had at least one quarter in which 75 or fewer students enrolled in each of the ccurses.

One of the first tasks in davel.opin;J this indicator is to detemine the class-size categcries that slxW.d be used for particular types an:l levels of courses. For EMI!Jit"le, 25 or 30 might be used for many freshman ccurses in C1"{Qiitiat, foreign J.ar¥;uages, an:l laboratoey and :teeitation sections for science and mathaDatics. SClmd pedagoqical. principles should be used as one of the bases for c::cnstruct:in; appropriate class-size categori~.

Inlicator 11; Grades Pe;aive;i

'1be problEIIl with grades indicators is that grades can be influenced easily if these assignin;J grades believe that judgments about quality will be :made on the basis of grade point averages. A secon:i prd:)lem is that grade fluctuaticns, grade infl.atiat in the min:3s of scme, occur without necessarily reflect:i.n; c:han;es in the rigor of instruction or c:haracteristic:s of students. Altha.lgh an overall grade point average indicator is rejected as ~less, particul.ar grade ocdes may yield useful infcmuatiat.

'1be w (withdrawal) grade code is~ as one indicator, since high rates S1.¥}38St problems in: (a) rx:rt givin; students good infcmuation about courses prior to reqistratiat; an:l {b) insb:uctors rx:rt livin; up to elCpE!Ctaticns about i.nstructicral. quality. Red1.¥::in; the rate of ccnrse withdrawals is illpntant in another way: to increase the availability of the course for other students who were unable to register for the ccnrse. '1be in:licator would be useful to the university in assess:i.n; the impact of jnplement.:in;J a DCre sophisticated system of course descriptions an:l program requirements for use by academic advisers.

calculatin; the in:iicator should be relatively straightfor«ard, since it requires no additional data beyaxi what is Olr%'el'ltl.y available. In addition to the Sl1UIIIaZY indicator for the '!Win Cities canpJS; collegiate,

Proposed Quality In:llcators for tJnjergraduate Fducation Paqe 31

departmental, ani course level in:licators could be provided as a mechanism to identify ani influence rates that deviate greatly from the no:cns. 'IWo s±mpl.e operational definitions of the in:licator W'OU.ld be the pe.rcenta.ge of total oourse registrations that resulted in grades of w, or the percentage of individual students with at least one grade of W.

rmicatm= l2i Trainim of Teadlim Assistants

Irxlicator 7 focuses directly en statistics c:xn::e.min; who teaches our un:3erg:raduates, so there is no need to define a specific in:licator that descri:bes teac:hin;J assistants • responsibilities for teachirY;J lawer division students. We Wt:llll.d be remiss, however, if w did not include a process indicator that describes the level, nature, ani quality of attention given by our departments to t:rainin;J future generations of c:clleqe teachers. 'Ihe lack of attenticn given to devel.opin; the tea.c::hin;J talents of teachirY;J assistants in large research lmiversities is often cited, by faculty in our natiat•s c:clleges ani lmiversities, as a factor in the la.r value given to t:eac::hin;, ani urD!rqraduate t:eac::hin; in particular.

'lhe prcposed "Departmental survey an ~te Fducatian Practices" includes a section at departmental efforts in t:rainin;J their teac::hin] assistants, especially new tea.c::hin;J assistants. It Wt:llll.d be possible to use these data .to c::cnst.zuct an indicator that reflects departmental attention given to teac:hin;J assistants. 'lhe sbplest of the indicators wcul.d be the percent of departments that offer scme type of t:rainin;J activities for new teac:hi.n; assistants.

Wicator 13; Monitoring of student Experiences

'lhere is considerable value to arrt organization in havin; feedback fran its c:ansumers c::oncemin;J the se%Vic:es it provides to them. Usually, those feedback systems are DCSt effective in inprcvin; se%Vices, processes ani pxoce~nres when the student provides consumer reaction data close-in-tilne to when the se%Vice was provided. In a parallel fashion, the results should be quickly ani :rcutinel.y available to affected offices so that they can take actiat to inprcve se%Vices. Several exauples exist of feedback systems that have been used recently or are bein; used currently.

DlriD:J the 1985-86 year the stment Experierx:es Project, t't.1rx2:l by the Office for the Vice President for Finance, developed a system for new students to provide feedback regardinq 13 camp.1S se%Vices; reqist:J:ation, financial aid, advisin; I hcusin; 1 libraries 1 bookstores 1 bursar 1 parkin; 1

athletics ani recreation, focxi se%Vices, health se%Vices, information on the University, ani miscelJaneous se%Vices • 'Ihat project resulted in data that was especially valuable in getti.n; a picture of new students' evaluations of diverse se%Vices across a two-quarter period of time.

•

Pl:opcsed Quality Irxlicators for t.1rx!ergraduate Education Page 32

Several other exan'p].es of c:cnsumer services surveys are those developed ani ilrplemented by particular offices on ampJS. 'lbese include registration surveys ocndllct:ed by StlD!nt SU(:pxt services, advi.sin;J surveys ccniucted by the P.remajar Office in Lil:leral Arts StlD!nt services, ani user surveys of variolls sexvices in COffman Memcrial unicm. All of these are examples of feedbN:k systa111 that are closely c:aaa:tad to the c:pm1ticm of particular otficas at ampJS an:J., as such, hava the pot:s1tial. far bein; used quickly to ~services.

~ to c::awt:ruct a maaniD;Jtul CXI'ltant-basad sttDint experiences :in:ticator would be an i"'tX'!S"ibl• task, qivan the nliDI:ers of st1DB'lts serve::1. ani the divarsity ot services prcvidad.. 8aq)l.iD; prQC'8d!U"E wtW.d be necessaey to insure that a :z:eprasart:ativa groJp of students was used to qenerate student experiences data. Mcrecver, overall data aba.1t specific consumer reactions (e.q., time spent in line or consumer satis:factial) would have few clear ivplicatians far i:ap:cvements. What is more relevant, then, is an :in:ticator that focuses at the extent to which offices at canpJS do, in fact, collect c:x:n;umer reacticn data. 'lhe s~est of the :in:ticators wtW.d be the total rumi?er of CXX1SUZDer reacticn pr<X."eSses in place ac:rcss the student services offices a'l the '!Win Cities camrus.

Indicator 14: SP"ent epurse Eva1uatim for ra!!J!t Enr9llpmt;. Courses

Debate c:x:l1t::il'lle aba.1t the value am validity of st1Dmt evaluation of instructial. 5aDe view its value ally as feEd'ack to inst:ructors for the PJ%PCS8 of ccurse ~· others see it as valuable in maki.rq administrative decisions aba.1t faculty hirin;, praDOt.ial am tenure decisions, am sa1a:z:y increases. others think it is helpful in advisin;J students as to COJrSe selection. In tAmDs of perspectives on the validity of COJrSe evaluation data, saue argue that such info:z::ma.tion is "not:hi.n; but a pc:pllarity contest," whereas others su;JgeSt that students are able to make meanin;ful j,D;ment:s aba.1t the quality of their classrocm experiences. 'nle role of COJrSe evaluation is related to department am individual faculty l!'EI!I1II ers 1 views aba.1t the nature of the educa:timal p:z:cx:ess.

Inclu:tinq an .iJxlicatar that reflects students 1 opinions a1::1out their classtGUD experiences is essential, since 1ll:lSt of the other :in:ticators do net relate directly to quality as students experience it in their ooursework at the university. Olrialsl.y and untart::unataly, mcst of the recent di SOlSSiat al::lout assessment in colleges am universities has nX focused on quality assessment dimensions students perceive to be illplrtant.

'Ihe p:z:d:)lem in usin;J student evaluation of instruction data as an :in:ticator of quality are many, rxrt: the least of which is definin;J the pc:pllation of ccurses to be evaluated. Even if the university~ to require that fiNery .instructor evaluate every cxurse each time it is offered (usin;J the same c:curse evaluation fcmn), the resultin;J oormative data wtW.d be difficult to inte:z:pret. Moreover, usin;J the data to improve instruction wtW.d be virtually iDpJSSible without identifyiJ¥] several types ani levels of

..

Proposed Quality In::U.catcrs for undergraduate Education Page 33

courses. An alteJ:native approach is to develop a student ccurse evaluation proce1nre that focuses a1 a particular subset of university courses.

'1be questim then becxares "HeM can we identify a subset of courses that would serve as tha basis for an iniicator of student course evaluatim?" It seems reascmbl.e, altha.z;h pemaps risky politically, to focus a1 the c::curse laval that is most widely thcl¥;Jht to be problematic, namely, 1-xxx level c:curses that enroll 1arga nmters of students. If, in1eecl, such an .imicatcr su;nasts problans with these c:curses, the potential is great for usin;J the results· to j:q;Jrave the educational experiences for large numbers of un:lel:graduate students.

Developin;J an appz:opLiate student eval.uatiat questionnaire ani work:in;J with affected colleges, departments, ani courses wculd be a delicate, time­CIOI'1SUIDi.rx] effort. once the instrument was developed, the precess of admini.sterirr: ani sc::orin; the instrument cculd be han:lled through the Office of MeaS'Jreu:aetlt services. COUrse eval.uatiat in:iices for each of a specified I'JUilb!r of high enroll:ment ocurseS cculd be aggregated across courses to fom a o aup~ite student course evaluatim in:iicator.

'Ihe follcwin;J list of courses is pzoposed as the st:.artirq point in develcpin;J the in:iicator. 'lbey represent the fifty 1-xxx level courses that enrolled the largest number of students durin; the three quarters of the 1986-87 ac:adem]c year. ·

B H

Psy 1001 36U Cl.as 1042 977 CCIIIp 1001 3525 CPsy 1301 971 Sc:lc 1001 2823 Rlil. 1002 963 Ast lOU 2749 Cllem 1005 943 Math 12U 2719 Geog 1301 914 Math uu 2529 Spch UOl 900 Math 1221 2152 Spch U02 897 Biol UOl 2141 Fhys 1271 885 Eccn 1002 2121 Pol 1025 880 Biol 1009 2094 Geog 1401 852 Eccn 1001 2014 CCIIIp 1027 846 '1h UOl 1906 Math U42 825 Acct 1024 lSU Biol 1008 799 !tlS 1001 1632 Anth UOl 778 Rlil. 1001 1622 GC U31 717 Pol 1001 1528 Hist 1301 717 Math 1231 1405 Fhysics 1281 701 Math U31 1236 Fhysics 1291 685 Cllem 1004 U97 GC 1281 673 Geol 1001 U97 GC 1435 650 Acct 1025 U69 Stat 1051 648 Anth U02 U34 Span U02 593 GC 1421 1074 Fren 1101 592

..

P.to::p:lSed Quality Indicators f6r U'lD!rgraduate Education Page 34

mm. 1001 GC 1422

1020 979

Htysics 1311 581 Span 1103 578

'D1e al:xM! list rray iD:l.ude c::cmses that are tc:xl varied in bmDs of size and inst:%uct:ia1al strateqies to yield int:aJ:pretable data. SaDe c:o.u:ses are small Dlll.t.isecti.aw:i o::urses with disnadat O"tawlts, whereas others are l.a%ge lect:u:r:a clares. A small nmi:ler of coura categcries should be daval.c:.ped as a Acwu•G:'k for repcrt:iD; tbase data.

Wiertor 15: Stndmtta • Participaticn in Key uniyersity Activities

ltbat dist.in;uisbes a Joaccalaureate graduata of the university of Minnesota? Few faculty, admini.st:rator, c::un'ellt stlDent:s an:1 citizens of Minnesota could raedil.y ani ccn:isely articulate the differences between our graduates ani graduates frail other institut.iaw in Minnesota. We are not alone in this failure to put a. unique "S"taDp" at graduates that reflects the special experierx::es they have had in their ~ experience. 'lhe need to detine what di.stin;uisbes our baccalaureate graduates has ilrplications for the develcpuent of an .irxlicatcr that reflects hew well we are doing at ~ that em' graduates have had certain critical educational experiences. Oeval.oping relevant in:ll.cators is an i:qxlrtant rut extremely diffiall.t task.

'D1e pl.aca to begin this dismsaiat is within the ocntaxt of the university's characteristics that shculd have bplicatiaw for urxlergraduate education. Am:n; the relevant charact:aristics are the followin;J: a research institutiat, a 1ani grant university, an intemational institution, and an institutiat situated in a l.az'ga wtz:opol.itan area with mmercus businesses, social serlices, gcvemment offices, ani arts ~tians. 'lhese features provide qplrtl.mities that are not avai J able to stlDmts in many other institutians. 'lhe question arises, "Hati good a job are we doing in c:onnec:tin;J students with these qplrtl.mities?" At this point, we have an abysmal data base to use in answerin;J the question. We need to be able to identify these key university activities necessuy for em' baccalaureate graduates to lead productive ani satisfyin;J lives in the twenty-first c:entul:y.

'lhis .irxlicatcr wcul.d, of necessity, need to be an exper.i.erx::e- in:ticator that reflects the percent.age of em' l::acca.l.aurea graduates who have had particular types of experlerx:es. 'lbe data c:cu1d be obtained in two ways: (a) by asJdn;J students at graduatiat to in:ll.cate whether or not they have had particular experien::es: or (b) by developing data systems in the offices ani pz:ogzams at campJS :respc:.11S:ible for arran;in;J certain types of experiences. Bein; able to z:epcnt the experiences data is the first part of a m:D:'t! extensive system that should collect data abalt the quality of the experien=es ani the extent to which the experiences lead to desired outcxmes •

Proposed Quality Indicators for un:Ie.rgraduate Education Page 35

Am::n;J the experiences that might ser.;e as the :basis for this irxii.cator are the followin;J:

• Participation in specified extrac:u:rricular activities (e.g. , student gcvarnment) that research suggests are related to success after graduatial:

• An opportunity to wcrk with a faculty 1IIBIIber on a research project -(e.g., ~research credit, undergraduate Research Opportunities Pl:o;)l:am (UROP):

• An experience that reflects a significant atterrpt to relate to a lani grant activity:

• An intensive advis:in;J or ccunseJ.iD;J experience that gives the student an ac:curate appraisal of intellectual am personal stren;t:hs am weaknesses:

• A significant leamin; experience that is related to one of the trn.iversity•s int:el:natia'lal. activities; am

• An Ultel:nship or ammmity service experience to apply classroom leamin; in the real world.

Once the data base was established, calculat:in;J the irxti.cator would· be relatively simple. one of the opticn$ for operatia'lal.ly defin:in;J the ini:icator 'Wall.d be the percentage of baocal.aureate graduates who have had at least one, at least two, at least three, or all four of the experiences noted above.

Irxiicator 16; Ciraduat" • Perfonnanc:e on Grad\late Record Examination

Considerable recent natia'lal. attention has fcx::used on concerns about student leamin; in baa-alaureate degree pn:gzans am the need for increased testing of baa-a laureate degree canlldates. Unfortunately, test developnent experts ani testin;J o:xporations have devoted relatively little attention to develop:in;J the cartplex am c:::cmprehensive testin;J procedures to adequately assess bpxtant qeneral leami.n;J out:ccmes that result fran a ba.cx::alaureate education. '!he few tests that do exist (e.g., the Acadenic Profile by the Educatia'lal. Test.iD; Serrlc:e ani the COllege out:caDe Measures Project by the American COllege Testin;J Pl:o;x:am) do net yet have the perceived importance and relevance ascribed to tests students take prior to applying to qx:aduate school (e.g., the Graduate Record Examination (GRE)).

Even thcu;h the prjJDary pm:pose in developin;J the GRE was to predict perfonuanc::e in graduate school, it does have sane utility as an irxii.cator of college and university education, but on1v for those students interested in adyanced dt::g:t§#§i. other states (e.g., South Dakota). and institutions have made the mistake of admin.isterin; the GRE to all of their ba.cx::alaureate

PJ:qa;ed Quality ID:ticators for Urx1e.rgraduate Fducation Page 36

graduates. 'Ihe relevance of the test ani. the motivation of students not pl.annin:J to apply to graduate school yielded unintel:pretable results, results that were obtained with considerable ccst ani investment of S'b.D!nts' time.

~ perta:marx:a data a1 the GRE far our 'baccalaureate graduates would raquire little in the W1J:f of addit:ialal c:cst:s or ptucednres, sin::e ETS already rcutinaly sam to the Graduate Sc::hcol. %epcnts of the Scores for university st:ucJants who haw taken tba GRE.

'lbese test data ~ nw maanin;tul. ani can lead to cmricular inprovement when they a%8 analyzed in 1IDr8 specific ways. First, CCIIPirisons am:xq majors allows scma C%t1erin;J based a1 c::urricuJ.ar paths c:hcsen by st\Xlents. seccn:i, ~ amr::n; majors allows saDB C%t1erin;J based on curricular paths c:hcsen by students. Seccnd, cxmpsri.scnl between our graduates' parfODIII!mCe ani the parfODIII!mCe of students in other sets of institutions (e.q., the Biq Ten, the Al.l.iance for ~te Fducation) yields useful OCIIplrativa data. 'lhird, CCIIp!ri.sans with natia1al GRE noma.tive data p%CNides the mcst general basis for mak:iD; cxn:l.usia1s alxlut the academic readiness of our baccalaureate graduates hc.pin; to atten:l graduate school.

We wculd be remiss if we did not izx:lude an imic:atar that reflects on the value of a baccal.auraata education as it affects elll'loyability and eaployment after graduatia'l. Natia'1all.y and within M:innesct:a there is ocnsiderable di•cnssi.al alxlut the ec:CI'lCIDic value, to the irxiividual ani to the reqion, of higher educatia'l. 'Ihe W1J:f in which this econc:mic value is realized is t:hrcu;h higher level, mre productive, and mre satisfyirq eaployment aperiences after graduatia'l. Bel.ievin;J in the need for an eaployment-ral.ated irxiication shalld not diminish, in a.rq way, the value of the college experience as a libaral.izirq, intrinsically rewardin; experience.

Althalgh a ampJS-wide SU'r'l"f of baccalaureate graduates has not been c:x:n:b:ted si%x:e the ·Fcmter student SUrvey Project in 1979, rnmerous collegiate ani depart:DEntal. studies of eaploymant experiences of graduates have :been c:x:n:b:ted si%x:e then. Undcubtedly, results of these studies have proven to be useful to units cxn:b:t:f.n; them to examine hew curricula and related undergraduate activities might be ~ in view of E!!ll;)loyment experiences after graduatia1. Perhaps these unit specific graduate surveys are the ally plausll:U.e way for the university to collect these data, qiven the diversity of the university's bacralaureate degree prog%ans.

Developirq enployment irxiicato:rs wculd require extensive dj srnssion. Different irxiicato:rs wculd be needed for graduates fran libaral. arts programs ani professional p:teyl:ans. 'lbe dj soJSSion probably wculd conclude with the need to have a c::atplex, lllll.tidimp-nsianal set of irxiicators. Collect:i.nq the data periodically fran all baccalaureate graduates five or

·•

Proposed Quality Irxticators for tJnjergraduate Education Paqe 37

ten years after graduation W'OU.ld be an ~ive, tiJne-c:onsum.i effort that 'WOUld require considerable financial support within the University. 'lhese data are bDpractical to collect for large numbers of students, although scmpli.tq strategies nay be used. Prcviclin;J these data in .a tilt1el.y fashion wcul.d be an ilrp=ss:ible task.

It there is consensus that an euployment inticator is critical, the next stap is to identity a group of faculty a1 canpJS with ~ in the measu:tBIEllt of work-related d.imensialS. several departments in at least three of our collages have such upert!se.

Wicator 1a; P9st Bamal aureate Fduc::ational Experiences

one of the gcals of a baccalaureate education is to instill in students the values of cont.imed learnin; and ~ in OJ:der to c:cntinue to be educated citizens. Cevel.opi.tq in students a passion for lifelor¥3' leamin;r is a laniahle goal, b.lt a'1IB that is difficult to assess. What our graduates "knew" at the point of graduatiat rray no lon;er be relevant 15 or 25 years later, althcl.J;h our graduates will ccntinue to be wl.l.-educated citizens if they have developed the skills and nctivation to ccntinue leamin;J len; after graduatj.al. Althcu;h an inticator of this sort nay be desirable, none are available currently that 'WOUld meet our needs. QuestialS on surveys of graduates five to ten years after graduation sanetiines include questions al:x:ut educational experiences sirx:e graduation. other scales and attitude surveys are ava; J able that a1:'tenpt to measure nctivation for continued leamin;J.

In the abstract, irxi:icators of this sort are desirable. 'lhe 101'¥3' payoff period, the ccst of collecti.n; these data, alxi the c::onnectialS to what students experienced a1 canpJS all ~ that this is an bDpractical iniicator. AlteJ:natively, this series of questialS may be one of the aRllied research projects that might be ~te for the university to fun:l a1 a cme-time basis.

Instead of igncrin; this question CXI'l'pletely, a related indicator, the percent of baccalaureate graduates who are admitted to graduate and professiooal ~o;%ans within a specified, short time period after graduatiat, nay be useful as an in::1ex of what our graduates do after they leave the '!Win Cities campJS of the University. QuestialS about atterxlance at graduate am professiooal sc:hcols o:W.d be CXIDbined with enployment­rel.ated questicns in a graduates' survey sent to a rarXlan saq>le of baccalaureate graduates. 'lhe survey might be con:iucted at sane regular time int:erJal., pemaps every five years, so that chan;Jes over time could be observed. 'lhe ccsts involved in bplementin;J a canplS-wide follow-up survey 'WOUld be extensive, due, in part, to the ccsts :incurred in locating students several years after graduation.

Pl:oposed Quality Irxlicators for Undergraduate Education Page 38

.Melman, C. (1988) • (Ed.) Performance mi judgment; FssaVS QD principles mi practices in the asSessment of college §tpgppt Jearnjm. Office of Bmcati.cmal Research ani Impcovement, u.s. Department of Education.

Assn=•nt at Im1ana tl'nivarsi.ty, Bloaai.D;Jt:cll. (1987). Presentation at the 1987 Biq Ten Parsau.al Maet.:in:J, Novadoer 11.

Astin, A. (1985) • k;h,ieyirp rntional. ffl'P',l ence. san Francisco: Jessey-Bass.

Astin, A. w. (1986). Measurin; the quality of un:3el:graduate education: Are trad.iticmal ~ adequate? Presentation qiven to the Texas system.

Astin, A. w., & SOlcm:m, L. c. (1981}. Are rep.ztational ra:tin;s needed to m&aSIJre quality? <llan:ler lJ, Oct:ci:ler, 14-19.

Baker, E. L. (1987} • critical validity issnes in the met:hcdology of higher educatia1 assessment. Ass'§'irp the mtaFe5 of higher education: Pt' C'f'dill§ of the 1986 E1'S Inyi1;ationa1 CODference Prirx:etal, N.J. : FdiJcaticnU TestiD;J service, 39-45.

Banta, T. W., & Pika, G.R. (1988). selected <::Cilpl%'isa1S of the ACr CXM? Exam ani the El'S Academic Profile at the tJ'niversity of Tennessee, ~e, Asseserent Resource Researc:h Report.

Banta, T. W., & SChneider, J. A. (1986). Usin; locally developed c::arprehensive exams for majors .to assess mi · bprcve prog10am quantity. Paper presented at the annual meetin;r of the American Educational Research Association, san Francisc:o, April 19, 1986.

Bloem, A. (1986) • '!he closirp -pf the American mini· <llicaqo: University of <llicago Press.

Bok, o. (1986). Tcwal:d higher leamin;J. ~· November/I)e:ember, 18-27.

:ea..n, H. R. (1986). '!he 'baccalaureate degree: What does it mean? What shcul.d it mean? Pl:esentatiat qiven to the Texas system.

Boyer, C. M., Ewell, P. T., Finney, J. E., Min:fle, J. R. (1987). Assessment mi outccmes mMSillOemettt: A view fran the states. MHE Bulletin. March, 8-12.

Boyer, C. M., & Ewell, P. T. (1988a). state-based approaches to assessment in un::iergraduate education: A glossary mi selected, PS-88-2, Education Ccmnission of the States.

Proposed Quality Izxiicato:r:s for Un::iergraduate Education Paqe 39

Boyer, C. M., & EWell, P.T. (1988b). State-based case studies of Assessment indicators in Ul'X!ergraduate education: Clronology of critical points. PS-88-3, Education CQnmi ssion of the States.

cartter, A. M. (1966). An asSessment of gnality in araduate e:lucation. washin;ton, o.c.: 'lhe American council on Education.

Olandler, J. w. (1987). 'Ihe wt¥, what, and who of assessment: 'lhe college pcspective: Assessim the outcgoes of higher education: Proceedings of the 1986 m Invitatignal Conference· Princeton, N.J. : Educational Testin;J Service, ll-18.

Chickerinq, A. W., & Gamsm, z. F. (1987, Mardl). seven principles for goo:i practice in un:lm'graduate education: MHE Eulletin, March 1, 3-7.

Coles, c., OJl.lar, T., & Mitdlel.l., B. o. (1987). Evaluation of state l.miversity systan of Florida's indicators of excellence program. Presentation at the 1987 Amual. Meeti.n; of the Association for Institutional Research.

camdttee on Quality ~te Teach.in; and I.ea.mirq. (1985) university of M:i.nnesota.

conrad, C. F., & BlackJ::m:n, R. T. (1986). Col::relates of departmental quality in regional colleges am l.miversities. American F.ducational JcmTlA1, ~, 279-295.

cross, K. P. (1987). Usin;J assessment to inprcve instruction. A9sessing the outcgnes of higher education: Pro;eedings of the 1986 EI'S Inyitatiopal Conference (pp. 63-69). Princeton, N.J.: Educational Testin;J Service, 63-69.

cross, K. P. (1988) • Feedback in the classroan: Makin; assessment matter. Prepared for the AAHE Assessment Forum, 'lhird National Conference on Assessment in Higher Education, Chicago, Ulinois.

Edgerton, R. (1987) • An assessment of assessment. AsSessim the outcomes of higher er1nmtion: Pror;perU 00§ of the 1986 El'S Invitational

Conference Princeton, N.J.: Educational 'l'estin;J service, 93-110.

El-Rhwas, E. (1986). Ompys Trends· 1986. Higher Fducation Panel Report No. 73, washin;Jt:a'l, D.C.: American Council on Education.

Ewell, P. T. (1987a). Establi.sh.in; a campJS-based assessment pro;t:am: A framework for choice. National Center for Higher Fdlcation Management systems.

Pl:op:Bed ()Jality Irxlicators for umergraduate Education Page 40

Ewell, P. T. (1987b). AssessmP.nt, ac::c:o.mtal:lility ani ill1prcvement: Managing the ocntradiction: National center for Higher Education Management Systems, Prepared for the American Association for Higher Education.

F:lnal. Rapcn:t of the Dlpl.ement:atiat Task Force at ~te Edl:JCation on the TWin Cities QmplS. · (1987) • university of Minnesota.

Final Rspott of the Task Force at the student Experience. (1984) university of Minnesota.

Gradllatial Rates a1 the TWin Cities QmplS. (1986) • StlDmt. SUpport services, university of Minnesota.

Hansen, G. R. (1988) • Educational Goal Attainment: A IJ::n;itudinal. Study I Pl: aentatiat at the 1988 Natiaal. Forum of the Association for Institutional Research, Bloenix, Arizala.

Hartle, T. w. (1986). '!he growjn; interest in measllrin; the educational adlievwment of Olllege students, Asses"P'Dt in higher educ;ation Office of Educatialal Research an:l Improvement:, U. S. OeparbDent of Education, 1-11.

RUh, G. D. (1981) • Ia:lices of qnal ity in the uniergraduate experience, · AAHE-ERIC, Higher Educatial Research Report No. 4, wa.sb.in:Jton, D.C.:

American Associatial for Higher Educaticm.

Levine, A.E. (1986). Quality--in baccalaureate pro;p:ams: What to look for when David Riesman can •t visit. Presentatiat given to the Texas system.

Lewis, D. R., wasescba, A. M. (1987). Costs an:l benefits of assessment in post-sec:orx2ry educatiat. Paper presented at the 1987 ASHE Annual Meetin;J, san Diego, califomia.

Marc:hese, T. J. (1987, Februal:y). -nlil:d down, ten years to go, MJ;m allletin, Februal:y, t:-P• 3-8.

Mentkcwski, M., & IDacker, G. (1985). Assessin;J an:l validatin; the outcanes of Olllege. In P. T. Ewell (Ed.) Ass=•im Frnntiona.1 outg:;nes. New DirectialS for Institutional Res"rr;h, san Francisco: Jessey-Bass, 47-64.

Moden, G. o. (1987) • A Dlll.ti"iJI!Diiona.1 amroach to mue'7t rutcgnes aressment· Paper presented at the 27th Alnlal. Forum of the Association for Institutiaml Research, Kansas City, Missouri.

Nettles, M. (1987), 'lhe E!!mel:gence of college grtcgne assessments: Prospects for enbancim state colleges am universities. New Jersey state College Govel:nin;J Boards Association, Inc.

Pl:oposed Quality Indicators for urxiel:graduate Education Page 41

NIE Report: Imo1vement in learping; Realizing the potential of American higher education (1984). Washin;ton, D.C.: National Institute of Fdlcation.

Pascarella, E. T. (1982). st1.J:lyin; student retention. New Directions tor Institutional Fes'e!"S'b, ~.

Pasc::arel.la, E. T. (1987). Are value-added analyses valuable? ,Msessing the out:.cgnes ot higher ft'l!Jprtign; Pt;;ooeerU n;m ot the 1986 EI'S Inyitational. Con:Cerence. Prin:et:al, N.J.: Fducational Testin;J Servrice, 71-91.

Peterson, G., & HerDal., D. D. (1987). J?ractical steps toWcm:l enhancin;r our ~ ot am il1p.ct on the un::!ergraduate experience at the university of Minnesota.

Pirsiq, R. M. (1974) • Zen am the art of motorcycle maintenance. New York: William Morrow.

Prel.iminaxy Report of the Task Force on Post-sec::orx:lary Quality Assessment. (1988). Minnescta Higher Education COor:tllnatin;r Board.

Preparin; tor the Twenty-First cent:my: Backgroun:i paper for a dj soJSsion at the university of Minnesota 1Win Cities cmnr;us. (1987). university of Minnesota.

Proposal to Create a Centralized Researc:h Center for Un:iergraduate Education on the 1Win Cities QmplS (1987) • oniversity of Minnesota.

Ramist, L. (1981). College student attrition. (College Board Report No. 81-1). New York: College Entrance Examination Board.

Ratcliff, J. L. (1988). I..ink:in;J assessment am curricular refonn. 'nle differential c:ourser.vork patterns project, ICMa state university.

Roose, K. D., & An:Ierson, c. J. (1970). A ratim of graduate programs. wash:in;Jton, D.C.: 'Ihe American Council on Fducation.

Rossman, J. E., & El-I<hwas, E. (1987) • '"lh.inkin; al:xllit assessment: Pe:t:spectives for presidents am chief acadpmj C Officers • II Prepared for the American Association on Higher Fducation.

Special Ccmnittee on COor:tllnatin;r IJJwer-Division Education on the '!Win Cities campus. (1986) oniversity of Minnesota.

Special Ccmnittee on Unified am Increased Preparation Requirements. (1986). oniversity of Minnesota.

P.t:cposed Quality Inii.cators for un::3erqraduate Fduca:tion Page 42

SOlalal, L. C., Astin, A. W. (1981). Departments without distinguished graduate piXX!JLdllS. Owpe, September, 23-98.

steele, J. M. (1988) • Usin;J measures of student outa:mes to improve college prog:tams. Paper presented at the 1988 Nati.aal Forum of the Association for Instituti.alal Reseal::dl, Bloenix, Arlzam.

Tan, o. L. (1986) • 'Dle assessment of quality in higher education: A critical review of the literature and research. :Res'al"'ffi in Higher rnrnttw, ~, 3, 223-265.

Task force at Assessment of the Quality· and out:ccmes of ~te Edllcatial. (1987). P1an for assM'im wpft'1'Tl1"?1!itpte ed\lcation at the university of Arizcm.

Terenzini, P. T. (1988). Assessment with open eyes: Pitfalls in studying student outccmes. '!he Jom:pal of Hiaher F.cbption. (In press).

'lhallas, A. (1987). Usin; an evaluatiw D:ldal. to guide developnent of a quality assessment plXX!JLa&n. Presentat:iat at the Association of Institut:iaJal. Researc::h-ot;::per MidtM&t Fall catference, october 1-2, Eau ~I Wisccnsin.

Tinto, v. (1987). rryjm college. Chicago: university of Chicago Press.

'I%ow, M. (1988). American higher education: Past, present, and future, mgtjgnal Res'e!!'tlBr, April, 13-23.

wa1 J ace, J. (1986) • What urx3eJ::graduates leam: '!he role of assessment in large research universities. Paper presented on behalf of the Alliance Education at the 1986 AAHE Assessment Forum.

~, R. H. (1987) , ~te Education at the university of Pittsl:m'gh, Office of the Prevost, university of Pittsburgh.

Wilson, D. A. (1984). Acccultabllity and aut:cncmy: A wcrJdn;J paper for NASCmC Project 2000, university of california at Ias An;Jeles.

. . -- DEPARTMENI'AL SURVEY ON THE QUALITY OF UNDERGRADUATE EDUCATION --

This survey begins the implementation of a new procedure for monitoring the quality of undergraduate education on the 'IWin Cities campus.

Such procedures are important for at least two reasons. First, we need to develop additional internal mechanisms to describe the quality of undergraduate education to guide our efforts to improve quality. Second, we need to respond to increasing requests from non-University constituencies interested in the quality of undergraduate education.

The departmental survey is one of several initiatives that have been designed in response to recommendations in the "Final Report of the Implementation Task Force on Undergraduate Education on the 'IWin Cities campus of the University of Minnesota." Recomnendation 14 stated: "The Office of the Provost should establish an administrative structure that is responsible for collecting, analyzing, and reporting the four types of infornation noted above. The Provost should also monitor the results of changes in the quality of undergraduate education and should provide annual reports to the campus Assembly and the Board of Regents."

The four types of information called for include context variables, input variables, process variables, and output variables. The enclosed "Departmental Survey on the Quality of Undergraduate Education" is a set of process variables that describes aspects of the effort departments devote to undergraduate education.

Your department 1 s completed survey should be returned to the Dean 1 s Office by early June. Surveys then will be forwarded to the Office of the Provost. A summary of the results will be available during fall quarter 1989. If you have any questions about the survey, please call Darwin Hendel at 625-0129.

. ~

2

This questionnaire contains questions about diverse aspects of your department 1 s approach to undergraduate education. The content of the sw:vey is organized around several themes that characterize a deparbnent 1 s approach to undergraduate education.

SECI'ION I: undergraduate curriculum

This series of questions asks you about curricular goals and course offerings in your department.

1. Please describe in a few sentences the goals for your undergraduate curriculum for majors and for non-majors. Subsequent questions ask how these goals are translated into the content and methods of undergraduate courses.

a. CUrriculum goals for majors: __________________________________ ___

b. CUrriculum goals for non-majors: ________________________________ _

.·

3

2. Does your department have a corrrrni ttee to address issues concerni.r1g the undergraduate curriculum? (Ched: on_g.)

(1) No. If "No," skip to question 3 on page 4. (2) Yes, a corrrrnittee that is specific to undergraduate st.1:iies. (3) Yes, but the corrrrnittee deals with both undergraduate a1d

graduate curriculum. ( 4) other. Please describe the corrnni ttee' s structure and focus:

a. If "Yes," approximately how many times did the corrnnittee meet during the 1988-89 academic year?

times

b. If "Yes," approximately how many individuals are on the COI:!:'.ittee? Please Wicate the number of individuals in each of the following categories:

Urrlergraduate students

Graduate students

Teaching assistants

Instructors

Assistant professors

Associate professors

Professors

other, Please specify:

Number

c. Please identify the position/rank of the person who typically chairs the carnmittee: ________________ __

4

d. If "Yes," please indicate which of the following topics were discussed at one or more of the undergraduate curriculum connnittee meetings during the 1988-89 academic year. (Cleek all that apply.)

__ (1)

__ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ (1)

__ (1) __ (1) __ (1) __ (1) __ (1) __ (1)

MUltidisciplinary curricular initiatives with other departments. Ways to encourage student-faculty contact. Ways to encourage cooperation among students. Ways to promote active learning. How to give prompt feedback to students. How to teach effective time management and time on task. How to communicate high expectations to undergraduates. Ways to accommodate different v.rays of learning. How to improve departmental advising. Scheduling of classes. International education (e.g. , study abroad opportunities) . CUrriculum changes. Honors programs. Remedial programs. Requirements for a minor in the department. Requirements for a major in the department. Other. Please specify:

3. Please indicate the most recent year in which each of the following occurred in your department:

a. Departmental review of undergraduate courses.

b. Departmental review of required courses in the major.

c. SUbstantial change in the undergraduate curriculum (where "substantial" means not just changing courses, but includes defining curricular goals and objectives and planning how to achieve those goals) .

d. Inventory of courses "on the books" that are open to undergraduates.

4. Which of the following statements best describes your department's approach to reviewing its undergraduate curriculum? (Check one.)

__ (1) __ (2)

__ (3)

We review our curriculum on a regular schedule. We periodically review our curriculum, but not on a predetennined time inteJ:val. We review our curriculum on an irregular basis.

5

5. Does your deparbnent have a special review process just for its 1-xxx level introductory course ( s) ? (Check the one response that best describes your situation.)

__ (1)

__ (2) __ (3)

__ (4)

No, we review the 1-xxx courses as part of our general review of the urrlergraduate curriculum. Yes, we regularly review these courses as a special process. Yes, we have a special process but conduct it on an irregular basis. other. Please describe:

6. Please describe how classroom instru.ctional time is spent in your deparbnent's introductory 1-xxx level course. If your deparbnent has several such courses, describe the course that enrolls the largest number of students. Estimate the approximate percentage of time spent in each of the following activities over the course of the quarter:

~ __ o

__ % __ %

~ __ o

__ % __ % __ % __ % __ % __ % __ %

~ __ o

__ %

100%

In-person lecture(s) In-person demonstration(s) lectures/demonstrations via television or film Recitations; discussions laboratory activities Student presentations Ilrlividual student conferences Field learning experiences Small group activities In-class writing exercises Individual differences alternatives (e.g., mastery learning and computer assisted instnlction) Tests and quizzes other. Please specify: _____________ _

6

7. Please estimate how much emphasis is placed on each of the following learning objectives in your department's 1-xxx level introductory course(s). If your department has several such courses, describe the course that enrolls the largest number of students.

~ __ o

~ __ o

~ __ o

~ __ o

~ __ o

~ __ o

100%

Basic knowledge and principles Application of knowledge/principles Analysis, synthesis, and evaluation of knowledge and principle Skill, manipulation, practice Attitudes, values, self-awareness other. Please specify: ________________________________ ___

8. Does your department have detailed undergraduate course descriptions (e.g., course syllabi or other materials that go beyond the course descriptions contained in college bulletins) available to students in the department's office? (Check one.)

__ (1) __ (2) __ (3)

Yes, for all courses Yes, but only for some courses Answer 8a and 8b No --------------------------~•Skip to Question 9

a. If such descriptions are available, what do these descriptions include?

i. Course content ii. Instructional methods

iii. Course learning objectives i v. List of required readings v. Basis for course grading

Sometimes

b. If descriptions are available, when wasjwere the description(s) of the introductory course(s) revised? Indicate year: __ _

9. Not all of a departments' courses may be offered during a particular year. Please indicate m.nnbers of unique courses (i.e. , count rnul tiple sections of the same course as one course) in each of the following categories:

Number

a. Courses in the curriculum which undergraduates can enroll.

b. Number of courses in which undergraduates enrolled during the 1988-89 academic year (fall, winter, spring quarter).

7

10. Please describe the processes used to detennine which courses are offered, when during the year they are offered, and who teaches the courses:

a. Which courses? ---------------------

b. When offered? _..;.._ ___________________ _

c. Who teaches? ----------------------

11. Please describe your department's efforts to offer special sections of undergraduate courses to meet the needs of certain types of students (e.g. , those who need remedial work and those who need honors opportunities). AnsWer the following questions relative to the period fall quarter 1988 through spring quarter 1989. (Please count rnu1 tiple sections of the same course as one course. )

a. Total number of courses offered

b. Number of courses with one or more honors sections offered

c. Number of honors courses (not sections of regular courses offered)

d. Number of courses with one or more remedial sections offered

1-xxx level

3-xxx level

5-:xxx level

8

12. Are any of your faculty involved in activities that involve discussions with faculty in the Minnesota Community College System about course content and pedagogy in your discipline? (Check one.)

(1) Yes __ (2) No __ (3) Uncertain

If "Yes," please describe: ________________ _

SECTION II: Responsibilities for 'Undergraduate Education

This series of questions concerns responsibilities for different aspects of undergraduate studies in your department.

13. Does your department specify someone to oversee undergraduate studies in your department? (Check one.)

__ (1) __ (2)

Yes If "Yes," please answer questions 13a through 13d. No If "No," please skip to question 14.

a. If "Yes," who is typically assigned the position of Director of Undergraduate Studies? (Check one. )

__ (1) __ (2) __ (3) __ (4) __ (5) __ (6) __ (7) __ (8)

Rotates airOI'XJ ALL faculty members. Rotates airOI'XJ IDS'!' faculty members. Assigned prbnarily to new, nontenured faculty. Assigned prbnarily to experienced faculty. Generally assigned to whoever has the most time available. Generally assigned to whoever has the most interest. Assigned to assistant department head. other. Please describe:

b. If "Yes," approximately what percentage of time is the person expected to devote to the position?

__ % of herjhis appointment.

9

c. Briefly describe the particular tasks and responsibilities included in the position: ______________________________ __

d. If "Yes," how important is the position of Director of Undergraduate Studies relative to other departmental responsibilities (e.g., Director of Graduate Studies) faculty may asstnne? (Check one. )

___ (1) __ (2) __ (3) __ (4) __ (5)

Much less important less important Equally important More important Much more important

14. Please describe the role of your department chair in determining your department's approach to undergraduate education:

15. In general, how much attention to undergraduate education in your department (e.g. , outlining curricular goals, reviewing of the quality of undergraduate courses, improving the educational envirornnent for majors) have department heads given during the~ three years? (Check one.)

__ (1)

- _(2) __ (3) __ (4) __ (5)

Little or no Some Much Very much Extensive

10

16. Does your department have a separate, identifiable budget item (or items) designated for expenses (e.g., instructional equipment, supplies, services) related to your department's undergraduate course offerings? (Check one.)

__ (1) Yes __ (2) No

a. If "Yes," what was the total dollar amount budgeted for the 1988-89 academic year?

$. ___ _

SECTION III: Faculty Involvement in undergraduate Education

'!his section contains a few questions about your departments' hiring and promotion and tenure practices as they relate to undergraduate education.

17. How often does your department explicitly indicate in position descriptions the undergraduate teaching responsibilities for prospective faculty members? (Check one. )

__ (1) __ (2) __ (3) __ (4) __ (5)

Almost never Sometimes About half of the time Usually Almost always

18. What changes, if any, have you seen in the importance of various factors in hiring new faculty during the past three years? (Circle one response for each of the following factors. )

Much less Important

a. Publication record, research productivity scholarship, and potential for obtaining outside funding 1

b. Demonstrated high quality graduate teaching and advising 1

c. Demonstrated high quality undergraduate teaching and advising · 1

d. Savice accomplishments 1 e. other. Please specify:

1

less Important No Change

2

2

2 2

2

3

3

3 3

3

More Much More Important Important

4

4

4 4

4

5

5

5 5

5

11

19. Consider the relative contribution of each of the following factors in making promotion, tenure, and salary increase decisions for faculty in your department. Use the three-year period 1985-86 through 1987-88 in resporxting to this question.

Promotion Tenure Salary Decisions Decisions Decisions

a. High quality undergraduate teaching andjor advising ~ ~ ~ -- 0 -- 0 -- 0

b. High quality graduate teaching andjor advising % ~ ~ -- -- 0 0

c. Publication record, research productivity, scholarship, and/or success in attracting outside funding ~ ~ ~ -- 0 -- 0 -- 0

SeJ:Vice activities ~ ~ ~ -- 0 -- 0 -- 0 d. e. other. Please specify:

~ 0

~ 0 % --

'IOI'AL: 100% 100% 100%

20. What is your department's approach on who teaches undergraduate courses in your department? For each of the four types of courses noted in the colt.nnnS below, in:ticate who typically teaches the particular type of course. (Check as many of the seven options as necessary. )

a.

b.

c.

By faculty who have the most time

By faculty with the most expertise in the area

Shared equally amongst all faculty

1-xxx level courses

for non-majors

d. By whoever is willing to do the teaching

e. By new faculty

f. By teaching assistants

g. By faculty who are the best teachers of undergraduates

1-xxx level Advanced Advanced courses level level

for courses for courses for our majors non-majors our majors

12

21. Please indicate the number of faculty in your department who received each of the following awards or grants during the three­year period (July 1 1 1986 through June 30 1 1989).

a.

b.

c.

d.

22.

Bush Sabbatical Awards

Educational Development 1 Small Grants or SEED Program

Morse-Amoco (now Morse Alt.nnni Association) Awards

for OUtstanding Contributions to Undergraduate Education

Educational Development Program Grants

Number

Have faculty in your department received any external grants to enhance undergraduate education during the three-year period (July 1 1 1986 through June 30 1 1989).

__ (1) Yes __ (2} No

If "Yes" 1 please indicate the following for each grant

Title Source of FUnding Amount of Funding

13

23. Please describe the duties that teaching assistants asst.nned in your deparbnent during the past year. Circle the response that best describes approximately how many of your teaching assistants had each of the following responsibilities.

Almost None Few Some Most All

a. Irrlependently, or under limited supeJ:Vision, designed courses in their entirety (objectives, content, evaluation procedures, etc.) 1 2 3 4 5

b. Provided the principal classroom instnlction for a course 1 2 3 4 5

c. Gave occasional lectures in a course that is the principal responsibility of a regular faculty meJnber 1 2 3 4 5

d. Had complete responsibility for constructing and administering tests 1 2 3 4 5

e. Assisted regular faculty meJnbers in const:ructing and administering tests 1 2 3 4 5

f. Had complete responsibility for detennining students 1 grades 1 2 3 4 5

g. Assisted regular faculty meJnbers in detennining grades 1 2 3 4 5

h. Developed instructional materials for use in courses 1 2 3 4 5

i. Assisted students in developing independent study projects for extra credit 1 2 3 4 5

j. Talked with students about their progress in a particular course 1 2 3 4 5

k. Organized and/or conducted group discussion sessions 1 2 3 4 5

1. Designed laborato:cy assignments andjor experiments or other types of practical exercises 1 2 3 4 5

m. Advised students on course selection and major requirements 1 2 3 4 5

n. Provided tutorial assistance for particular students 1 2 3 4 5

o. Read, commented on, and graded students 1 papers and other written assigrnnents 1 2 3 4 5

p. Provided feedback to regular faculty meJnbers about how well students are responding to the class 1 2 3 4 5

14

24. What is your department 1 s approach to advising undergraduate majors in your department? (Oleck as many of the options as necessacy to describe your department 1 s approach. )

__ (1) __ (1) __ (1} __ (1} __ (1) __ (1) __ (1)

__ (1)

By those faculty who have the most time By those with the most expertise in advising Shared equally by all levels of faculty By whoever is willing Primarily by new faculty By teaching assistants By those faculty who are the best advisors of undergraduates By professional support staff

SECTION IV: Principles of Good Practice in Undergraduate Education

'!his section concenlS your departments 1 views about seven principles that have been advanced as indicative of good practices in undergraduate educationl.

For each of the seven principles:

First, indicate how much attention your department has given to this practice during the past three years.

Second, describe any plans your department has for promoting the practice in the future.

25. Principle 1: Good practice encourages student-faculty contact. Frequent student-faculty contact in and out of classes is an ilnportant factor in student motivation and involvement.

a. In the past three years, how much attention has your department given to increasing student-faculty contact? (Circle one response.)

Little 1

Much 3

Very Much 4

Extensive 5

b. Please describe any current departmental plans for promoting the practice in the future:

!Principles adapted from an article "Seven Principles for Good Practice in Undergraduate Education" by Arthur W. Chickering and Zelda F. Gamson that appeared in a special edition of the Wingspread Journal.

15

26. Principle 2: Good practice encourages cooperation among students. Learning is enhanced when it is a team effort -- collaborative and social.

a. In the past three years 1 how much attention has your department given to developing cooperative learning experiences? (Circle one response. )

Little 1

Very much 4

Extensive 5

b. Please describe any departmental plans for promoting the practice in the future:

27. Principle 3: Good practice encourages active involvement. students learn by talking about what they are learning 1 writing about it, relating it to past experiences 1 and applying it to their daily lives.

a. In the past three years, how much attention has your department given to fostering active involvement of students? (Circle one response.)

Little 1

Very much 4

Extensive 5

b. Please describe any current departmental plans for promoting the practice in the future:

16

28. Principle 4: Gocx:i practice gives prompt feedback. Students need appropriate feedback on perfonnance to benefit from courses, including assessing existing knowledge and giving them frequent opportunities to perform and receive suggestions for improvement.

a. In the past three years, how much attention has your deparbnent given to increasing the feedback about their learning given to students? (Circle one response.)

Little 1

Much 3

Very much 4

Extensive 5

b. Please describe any current deparbnental plans for promoting the practice in the future:

29. Principle 5: Gocx:i practice emphasizes time on task. I.eaming to use one's time well is critical for students. Some students need help in learning effective time management.

a. In the past three years, how much attention has your department given to helping students use their time well? (Circle one response.)

Little 1

Much 3

Very much 4

Extensive 5

b. Please describe any current deparbnental plans for promoting the practice in the future:

17

30. Principle 6: Good practice cormnunicates high expectations. High expectations are inlportant for all students--for the poorly prepared, for those unwilling to exert themselves, and for the bright and well motivated.

a. In the past three years, how much attention has your department given to developing high expectations of students? (Circle one response.)

Little 1

Much 3

Very much 4

Extensive 5

b. Please describe any current departmental plans for promoting the practice in the future:

31. Principle 7: Good practice respects diverse talents and ways of learning. Students bring different talents and styles of learning to college, and need the opportunity to show their talents and learn in ways that work for them.

a. In the past three years, how much attention has your department given to helping students use their different talents and styles of leaming? (Circle one response.)

Little 1

Much 3

Very much 4

Extensive 5

b. Please describe any current departmental plans for promoting the practice in the future:

18

SECTION V: Evaluation of Departmental Quality

32. How would you rate your department's quality in tenns of the educational experience provided to undergraduate students roajoring in your department? Please circle the response that corresponds to your evaluation of your department on each of the following dimensions.

Excellent Very good

Good

COOmes Very ~~~rr a. The overall quality of the instruction? 1 2 3 4 5 6 b. The extent to which the courses appropri-

ately challenge students' abilities? 1 2 3 4 5 6 c. The opportunity for students to leazn about

the research methods in the discipline? 1 2 3 4 5 6 d. The amount of discussion and team projects,

rather than lecture? 1 2 3 4 5 6 e. The variety of courses available in the

roajor? 1 2 3 4 5 6 f. The extent to which roajor requirements fo:nn

a well-integrated program? 1 2 3 4 5 6

Instructors g. The amount of instruction done by regular

faculty, rather than teaching assistants? 1 2 3 4 5 6 h. The accessibility of faculty to meet with

students outside of class? 1 2 3 4 5 6 i. Faculty members' ability to cormmmicate

their knowledge to students? 1 2 3 4 5 6 j. The feedback students are given on their

perfonnance? 1 2 3 4 5 6 k. The overall quality of academic and career-

related advising seiVices provided to undergraduates? 1 2 3 4 5 6

..

19

SECTION VI: Evaluation and Improvement Activities

'!his section contains questions about your department's procedures for evaluating and improving the quality of urxiergraduate education.

33. ])Jes your department have any special procedures or awards to recognize excellence in urxiergraduate teaching? (Check one. )

__ {1} __ {2} __ {3} __ {4)

No Yes, we have an award for teaching assistants only Yes, we have an award for faculty only Yes, we have awards for ooth teaching assistants and faculty

34. ])Jes your department have any special procedures or awards to recognize excellence in urxiergraduate advising? (Check one. )

__ {1} __ {2) __ {3) __ (4)

No Yes, we have an award for teaching assistants only Yes, we have an award for faculty only Yes, we have awards for ooth teaching assistants and faculty

35. ])Jes your unit have an organized training program for classroom teaching assistants? (Check one.)

__ {1} Yes--- Answer Question 35a below. __ (2} No Go to Question 35b below.

a. If your unit has a training program, what is the content of the training? Please indicate which one of the following best describes the content of the training program:

__ (1) __ {2) __ {9) __ (4)

Program is basically administrative Mostly administrative, some pedagogy Mostly pedagogy, some administrative Program is basically pedagogical

b. If your department does not have a training program, what would you need to develop a program: __________ _

20

36. Does your department offer training sessions, specifically designed to assist teaching assistants to perform their duties, for individuals with no prior experience as teaching assistants at the University of Minnesota? (Cleek one. )

__ (1) Yes __ (2) No

If "Yes," please indicate which of the following topics are covered in those meetings by checking those that apply.

__ (1)

__ (1) __ (1)

__ (1) __ (1)

__ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ {1) __ {1) __ {1) __ {1)

__ {1)

__ {1) __ {1)

OVerview of the role and irrp:>rtance of teaching assistants at the University. Procedures for constructing good essay test questions. Procedures for writing good multiple-choice test questions. Grading practices and procedures. Specific discipline or course-related issues (e.g. , use of texts and other materials) . Resporxling to individual differences among students. Creating opportunities for collaborative learning. Developing effective strategies for active learning. Providing feedback to students. Setting and connnunicating high expectations. Classroom presentation skills. I.aborato:ry skills and resources. Use of instructional equipment, including computers. Guidelines for organizing course content. Instructional goals and objectives. One-to-one intel:personal skills. Audio-visual resources. OVerview of libra:ry resources and how to help students use those resources. Handling special student needs (e.g. , disabled and foreign students). Directing students to special resources. other. Please list topics: ____________ _

21

37. What does your department do to help tenure-track faculty plan courses and improve their teaching skills? (Check all that apply.)

__ (1)

__ (1) __ (1)

__ (1)

__ (1)

__ (1)

OUr department has no established mechanisms to help new faculty with urxlergraduate teaching. New faculty can ask for help as needed. Fach probationary faculty is assigned to a senior faculty member. Undergraduate teaching is an item on a regular review with the department chair. We have an infonnal. system to help new faculty with teaching urxlergraduates. We have a fonnal. system to help new faculty with teaching urxlergraduates. Please describe:

38. Does your deparbnent use any type of student evaluation fo:nns to evaluate courses and instruction? (01eck one.)

__ (1) __ (2) __ (3)

__ (4) __ (5)

Yes, we use the standard university fonns Yes, we use fonns we developed ourselves Yes, we use some of both (standard and unit-developed fonns) No, we do not use student evaluation fonns Other. Please describe:

39. Please indicate the approximate PERCENTAGE of courses in which your department used student evaluation fonns during the 1988-89 academic year.

__ % __ % __ % __ %

1-xxx level courses 3-xxx level courses 5-xxx level courses 8-xxx level courses

22

40. Does your department use procedures other than student evaluation fonns (e.g., classroom visits by colleagues) to provide feedback to faculty about the quality of their undergraduate teaching? (Check one.)

__ (1) Yes __ (2) No

If "Yes," please describe:

41. If, through student evaluation fonns or other methods, your unit finds that a faculty member needs assistance in improving hisjher teaching skills, what steps are taken? (Check all that apply.)

__ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ (1) __ (1)

Have faculty member meet with department chair Reassign faculty member Closer supervision Counseling am advice Senior faculty appointed as mentor Referred to canpus specialist to improve instruction Referred to teaching seminars and colloquia other. Please specify:

a. Please describe what additional sel:Vices you would like to see offered for this purpose:

23

42. Does your department have a written procedure available for students to follow if they have complaints or grievances about the quality of teaching or courses in your department? (Check one.)

__ (1) Yes __ (2) No

If "Yes," briefly describe the procedure:

SECTION VII: Basic Departmental Data

This section contains basic questions about faculty size and composition and departmental student credit hours. You may wish to provide the data for questions 44-46 or, alternatively, you may prefer that we obtain the data from central records.

43. Department Name: ________________________________________ ___

44. How many tenured and tenure-track faculty are there in your department for the 1988-89 academic year? (Please consult academic personnel records in responding to this question. )

Tenured Tenure-track

Number

45. How many new tenured and tenure-track faculty did your department hire during the three-year period July 1, 1985, through June 30, 1988? (Please check academic personnel records in responding to this question. )

Tenured Tenure-track

Ntnnber

46. For the current academic year (September 16, 1988, through June 15, 1989), indicate the total number of individuals who held positions as teaching assistants in your departrnent. (Please consult your academic personnel records in responding to this question. )

______________ Total number of teaching assistants appointed.

., . 24

4 7. For each level of instruction in your department, indicate the approximate percent of your department's total student credit hours at that level of instruction for the period fall quarter 1988 through sprin:J quarter 1989.

1-xxx 3-xxx 5-xxx 8-xxx

Student Credit Hours

__ % __ % __ % __ %

100%

- '!hank you for your time an::l cooperation --

Please return your completed departmental questionnaire to the Dean's Office.