document.resume ed 113)395 aleamoni, …document.resume ed 113)395 author aleamoni, lawrence m....
TRANSCRIPT
DOCUMENT.RESUME
ED 113)395
AUTHOR Aleamoni, Lawrence M.TITLE Typical Faculty Cpncerns About StudentJByaluation of
Instruction.. _4
PUB DA7E 24 Mar 74NOTE 23P.; Prepared for presettation,at the ',Symposium"
Methods of Improving'University,Teaching at theTee Inion, srael Institute of Technology, (Haika,0`Idrael March 24i 1974)
/
HE 006 776..
4
EDRS PRICE 11-$0.76 HC-$1.58 Plus PostageDESCRIPTORS College_Students; Conference Reports; *Course
Evaluation; Faculty Evaluation;.*.Higher Education;Participant. Satisfaction; .Questionnaires; Student,Attitudes; Student Evaluation; *Student Opinion;Summative Evaluation; *Teacher Rating; *teachingQuality; Teaching Techniques -
IDENTIFIERS *Illinois Course Evaluation Questionnaire
ABSTRACTSeven typical facUlt concerns about the
'appropriateness of using ratings of ins uctor and instruction are'examined. These are summarized in terms of common observationgfrequently expressed by faculty: (1) Students cannot make, consistent
1 judgments because of their immaturity, lack of experience, andcapriCiousness; and a widely held belief is that only Colleagues withexcellent publication' records and qxperienbe are qualified toevaruate their peer's instruction. (2) Most. student rating schemesare nothing more than a popularity contest. /3) Students are 'not ableto make accurate judgments until they have,been away from the courseand possibly from the university for several years. (4) The studentrating forms are unreliable and invalid. 5) There are extraneousvariables or conditions that can affect student ratings, includingclass size, Schedule, term, student sex, major, and level. (6) Thegrade a student receives in the course is highly correlated with hisrating of the course and the instructor. (7) A, question fiequentlyraised is how student evaluations can be used to improve instruction.Research addressing these problems is cited that makes use of the-Illinois Course ,Evaluation Questionnaire (CEQ) . (LBH)
4
*.***************************************e*****************************Docubents acquired by ERIC include many informal unpublished *
* materials not available from other sources. ERIC makes every efforst ** to obtain the best copy available. Nevertheless, items of marginal .**'reproducibility are often encountered and this affects the quality *
* of the microfiche and hardcopy reproductions'EPIC makes available *
* via the ERIC Document Reproduction Service (EDRS). EDRS is not* responsible for the quality of the original document. ,Reproductions ** supplied by EDRS are the best that can be made from the original. *
**********************************************************************
4
4l
U S DEPARTMENT OF NEALTFIEDUCATION a wE0A/tENATIONAL INSTITUTE OF
EDUCATIONTHIS DOCUMENT HAS REIN REPROOuCED ExAcTCy AS RET EREO .ROMTHE PERSON OtTOR(.NiZATONATING IT POINT% e)T vt vY OR OW,. ONSSTATED DO NOT NE( ESSAR itE pkESENT OFT ( A. NAT:ONAt TT OTEDUCATON POST OT,, OP POL.Ck
.
-I
TYPICAL FACULTY CONCERNS ABOUT
1STUDENT_EVAlUATION OF INSTRUCTION
,LawTence M. Aleamoni2
1-`In the past few years there have been many propOS,Is for evaluating
instruction and afew of them were aso concurned with trying to
evalUafton to the improvement of -instruction. Most proposals suggested
the use of similar elements in ihe.evaluation procedure; These inclu e.
(a) judgment by student, peer, self, and supervisor (department- hea
and (b)-judgments;of(course material, coarse content, courde'obj ctives,
and quality of student learning: ',If, however one looks for aetual,
o
working models of instructional evalliation,0iCis iMmediatelyapparent
that schemes involving systematic ratings by peer, supervisor, or self,
oy of material, tontent, etc., are rarely actualized. More often than
not, the student ratings of instructor and instruction appear as.the
only ele ments in any of the "working models" and .there are many reasons
.
.
one could-cite for this. This paper, however, wil focus specifically1
'4 ,--,k on seven typical faculty concerns aboUtth .appro riateness of using. ,.
-stings of instructar'and instruction, at all: These are summarized
below in terms of common observations f equently expressed by faculty.
1. (a) Students cannot make consistent judgments concerning
the instructor And'nstruction because of their immatur-
ity,ity, lack ot experience, -and-capriciousnesa.
1
Ns
Conversely,
- 1An invited address- Xlrepered for presentation in the Symposium on
Methods of Improvi.Pg Liniversity Teaching at the Technion IsraelInstitute of Technology, Haifa4 Israel on March 24, 1974.
2The author is indebted, to Robert H. Davis and Joseph S. Marcusfor their constructive suggestios,s 1
5,
2.
(b) A wigely.heldbelief.ia that ,only colleagues. with exgell-
ent 'publication records and' experience are qualified to,'
evaluate their peer's instruction.
2. Most student rating schemes are netkiAg more than a popularityf
contest, with The warm, friendly, humorous, easy grading..
Akinstruc.tor emerging as the-winner.,
', :
r.
3. Students are dot able to make 'ccurate judgmenEs until they
fiAI.
have ben .amay from the course and possibly away from'tha uni-
veraity for several years.
4. The student rating forms are both unreliable and invalid.
5. There. are extraneous4Var.iablesor° conditions that can affect
* ° .
student ratings. Some of the more common
(a) t he .size of the class
(b) .the-sex of the student
P(c) the timeof. day the course was offered -
was taking the course ha a re irement
N,.
(d) Whether the stude.$11
...
, .
,. ,
. .
or on an elective basis %-.C? . N
(a) whether ,tte student was a major or .a riion-major'
,-7 (f) the term (of'semester) the coursewas offered
tgi the level of the course (freshmen, sophomore, junior,
senior, gradUate)
(h) 'the tank of the instructor (instructor, assistant
Ones are:
/
.professor, associate prOfessor, full profeseor)
6. The grade or mark a student receives ifi the course is highly
correlated with his rating the course and the instructor;
7. Finally, a question (hat is frequently raised is "How can
student ,evaluations possibly be used to improve instruction?"
. \k/
a
Noe
s
fr
A)
I
-_Surprising as it may be, answers.to'these problems and questions4
cam-be found, in a 4ethon4a. Of research' that spans' at lease 50 years.v- .
Most m; this research hailoleen conducted .using student evaluation \(rating) questionnaires similar to tpe one presently Used at the UM'-
4 varsity of Illinois at Urbana-Champaiiii-X0IUC)- and entitled the
Illinois Comxse Evaluation Questionnaire (CEQ). A copy of the-CEQ
is presented inAppendix A.
Befpre beginning to cite and summarize,the reseadfch addressing
each of the ptoblems and questions above,
) .-
CEQ will be presented ins. order tp provide
ence forth various studies.
a brief description of the7='w
a Meaningful frame. of refet-
The CEQ is an'instxjnment used to collect studenx attitudes and
opinions toward a course (Aleamoni, 1972a;Aleamoni and Spencer, 1973).:
Its purpolse is to enable faculty members to collect evaluative infor-
mation about their teach The data is collected and proceased by
course-
section but may'a so be processed by course, department, c liege,
etc. ,Extensive normativ4 data has been gathered on the CEQ over the
past 10-years and this p ovides the instructor with valuable comparisons
on the\inseructional dimensions of (a) Generil Course Attitude, (b) Course
stroction, (d) Interest and Attention of the
tructor.\ These instructional dimensions repre-
EQ. The 23 items thia:make up the aubscales
some diagnostic feedback to the..instructor
cbmparisons. For example, each instructor's
content, (c) Method of I
Students; and (e) the In
sent thesubscales of th
are used to help provide
through appropriate norm
results are compared with those of pther instructors of his own academic
C.
1
,rank, teaching at .the..dame course level and in his own department,
-college o university, as'well as'with, co44es thot have used the' CEQa,
throughout the United Stdres.
The student-responses to the CEQ ore anonyrdbusi and two copies
4.
of the quesFloPnaire6,esults (see Appendix B) along with interpretative
information are returned` only the instructor. The instructor Petty.e
decide, to submit ene copy to his department chairman for, rank, pay, an&...
tenure consideration, but the CEQ is primarily 'used to prpvide feedbek4
to instructors as to where potential problema may exist in the classroom.ra _.
. .._ . /*,. .
.
.
-'There is ample space on the CEQ form for instructors to utilize
..
).
specifically diagnostic items in identified,problem areas. Co9ferences-
J
relatille to-the interpretation ano(utilieatio° of results may be
arranged at the instructor's convenience with.consultants from the ..
Measurement-and Research Division of the Offisce of InStructional Resource's
at,UIUC. U
With that brief description of the CEQ as backgrounds responses to
the (seven concern are now presented in the order of their presentation.
above 2'
'1. (a) Students cannot vonsistentl 112181 instructor andiinstruc-.
tion. There is amp/e-evidence on thisSpoin; dating .back-to 1924,
according to 'Guthrie (1954), in which reliabilities of student ratings.
-remain in the .80 to .90 range. More reCent'literature on the subject
by Costin, Greenough add Menges (1971) and Aleamonti (1972b) pas shown
that well developed Instruments and administration procedures can
silently yield'high reliabilities in the .90's).
.J %
(b) Onl ollaa ues with excellent.publibation records and eikieri-'
5;
1.ences are qualified .to evaluate their peer's instruction'. Wherever a
diAtia;ion about or a proposal for student ratings emergrs, this tate-
6ment can usually be heard. Recently, a well known statistician presenteci
just such an agrument in The American.Statistician (Deming, 1972).
Fortunately for un fortunately for those who believe such acontention), e
a bout the time. that Professor Deming's article appeared a study had been'
campleted'addressink that very, topic. Aleamoni and Yimer (1973) found
that colleague and student ratings were not significantly relAted to the
ainstructors research productivity (the correlations were .07 and -.04,
respec;ively). In addition, there was ample evidence previously (Guthrie,. ,
1954; Stallings and Spencer, .1967; Swanson and Sisson, 1971) to show that
:colleague and student ratings were very highly correlated (from-.63 to
. .765. Theresponse:to Professor Deming's arttcle can be found' in a
eater issue'of The American Statistician (Aleamoni, 1972),.
2. MOst student rating schem p.. are nothing more thana popularity
contest. Answers to this problem are presented from published and lib-.
published studies, on the CEQ and The Advisor (a student- sonsored fotm)-
(Feldman, 1970) at UIUC. The studies conducted, in developing and uti-,
.t'
lining the CEQ subscalca, (Aleamoni. and Spencer, 1973) indicated that no
single subscale (t.e., Method Of Instruction)' completely overlapped the
other subscales. Basical this means that an instructor vho received
a high decile rating on the Instructor subscale (made up of items like
"The insfitie-terseemed to be interebted'in students as persons.") wohld
not be guaranteed high decile ratingolon the other four subscales4
qe \,
itt
a
6.
V
(General Course Attitude, Method of Instruction, Course Content, and
Interest and Attention). 'In order to more fully explaxe this problem,
the written comments made by students pn both -the CEQ aridThe Advisor
were revUwed and comOared to their objective responses. The results
indicated that students would frankly praise an instructor lor his
warm, friendly, humorous, etc., manner in the classroom, but if his
,
course was not well organized or his method of Stimulating students
to' learn was poor, the students would equally frankly criticize him,
in those areas. ,When these comments were compared to the objective
measures in the same trees, a high degree of relationship was observed.
This'evtdence, in additjon to.that presented b ostin, et al., (1911),
indicates that students are discriminating udges and not easily fooled
'by the gobd Vshowl'utm" who is lacking in the otlier.ilistructional quali-
ties. #
3. 'Students cannot make accurate judgments until they are out of
the course an away from the university fOr several years. This point
is repeatedly raised by faipulty and was recently presented by McKeachie
(1969). This problem is a very difficult one with which to deal becausef . P
longitudinal;Collow-up studies need a great deal of attention to the
question of whether samples are comparative and representative. The
sampling problem is further compounded by the fact that almost all stu-
dent attitudinal data relating to a course or instruttor is-gathered
anonymously. Most studies in this area, therefore, have relied on sur-
veys of alumni and/or grfilduating seniors.
One of the earliest studies by Drucker and Remmers (1950, 1951)
showed that alumni who were out of school five te, ten years rated the
a
SI
a.
4 order to answer this problem it Rust be divided into two portions, one
concerting the reliability of student rating forms and the other; the
a
instructors much the same as students currently enrolled. More recent
evidence by Aleamoni and Yimer (1974) ,further substantiates the earlier.
findings. This evidence seems to indicate, contrary to popular belief..,..
d speculation, that students are veit-TerCeptiVe in,thekrjudgments
and,are in'substantial agreement with peers who have been out of thea
course and away from the university for several years. ,Avery care-
fully controlled follow-up study, however, needs to be conduotCd in. ;
order to fully answer this problem.I,
4. The student rating forms are .oth -unreliable and invalid. In
1-5
validity of student rating forms.
Thereseerch ,literature it replete with studies that answer the
question of.the reliability or sudent rating forms. Almost all of the
instruments which have been carefully constructed andtesteddby pro-
fessionals yield reliabilixies at.the level of .80 ,and .90 on thea
subecales as well as the total instrument (Costin, et al., 1971).
COmpue6d on'the items and subacales makinm up the CEQ
(Aloamoni, 1972b), for example, have yielded item reliabilities rang-a
ing from .73 to .94 andubscale reliabilities ranging from .80 to
.98. It should' be'moted, however, that wherever student rating forms
are not carefully constructed with the aid of professionals, as in the
case of mast - student generated forms (Everly and Aleamoni, 1972), the
reliabilities may be no low as to-completely negate the evaluation
effect and its results.
Trying to answer the mblem of'th validity of student rating
forms is much more difficult than adtcessingthe reliability problem.
/In order to something one must have a "criterion measure for
/
comiarison. One of the criterion measures that can be-used to vali-
date student rating forms is to determine how well the items and
subscales measure what is intended (called conternt validity). This
is usually accomplished by carefully constructing the instrument so.
that it contains'items and subscales that will yield measures in the
areas that are considered necessary by an individual or group of
experts in the Field under consideration. Most of the. student rating41
forms generated,were validated by'using just this sort of approach
(C tin, et al., 1971). The use of statistical tools like Factor
Analysis have also been used to verify sub ectrvely determined dim-
ensions of the instructional setting und process. The C*() used both4
statistical (Factor Analysis) and 'subjective expert judgments in
.gener61Cing the items and sOscales that make up the form (Aleamoni
..ad Spencer, 1973),...
Many other criterion measures have been suggested by which to
vuidate st,..dent ratings. Some of those are pde (or colleague rat-
;lags), expect judb.S' ratings, student learning, etc. You will
notice that many of the problems and questions that faculty pose,
such as the seven suated above can also be interpreted as validity4
concerns. In order to avoid any redundancy of answers to the faculty
concerns, let me simply indicate that studies in which student rat-
ings were c(4p0led to (a) cglleague ratig (Guthrie, '954; Swanson
and.Sisson, 1911; Aleamoni and Yimer,,1973), (b) expert/judges'
9
ratings (Sta
..
lingo and,
Spencer, 1967), and(
9.
) student learning measures
. )roiten and erger, 1970) all indicated the existence of high .positive.
correlations which can be considered as providing additional validity
N .
evidence.
5. What extraneous variables or conditions affect student ratingd?
Studies conducted on the (a) size of the classv(Guthrie, 1954; Costin,
et al., 1971; Aleamoni and Graham, 1970, (b)" sex of the student (Costin,
et al., 1971; Aleamoni, 1972b),' (c) time of day the course was Offered
4(Aleamoni, 1972b), and (d) term (9 semester)' the course was offered
(Costin, et al., 1971; Aleamoni, 1972b), indicate that'these variables
had little or no relationship to the student ratings. The,xank of the
instructor (Guthrie, 1954; Costin, et al., 1971; Aleamoni and
1974) seems to have some effect but it isNusuallY not quite of
significant.
raham,I
tistically
On the other Dand, ra).whether the student was taking'the course ad
a requirement or od' an elective basis (Costin, 'et al., 1971; Gillmore
and Brandenburg, 1974), and (b) the level of the ourse (Costin, 1971;
Aleamoni and. Graham, 1974) yielded significant effiects on student ratings.
Such effectsk however, can be controlled through the use of appropriate,
normative data, which is an:important feature of he results reported for
the CEQ.
The one study cited by Costin, et al., -(1971) indicated that 'whether ,
the student was a major or a non7major.did not affect his-faculty rating.
This concern is presently ieins-Investigated using CEQ data
6. . The grade or mark a student receives in the course is highly
correlhted to his rating of the 'course and the instructor. There is ample
10
10.
/
evidence in'the research literature to provide a definite answer to/
this concern. In almost all of the' studies cited in ,Cotin t al.
0
(1971) and by investigators such as Guthrie (1954), Remmers c1-900)
and Weever (1960) little or no relationship has been-found/between
a student's grade plid'faculty rating. In'face, the positive corre-
lations seldom exceed .30. The evider, th7tfore, indicatep that
students ao not necessarily rate an instr4,46tor or course based upon
the grade they have or are a out to receive.e
7. How ckn student' eval tions, sibl be usedlto improve
.instruction? This cpuld well be the most important question to be .
asked concerning student evaluation of instruction. There has been
a great deal Of discussion in the research literatUre about how, when,
and where such evaluations should be used, but no clear-cut evidence
has been offered to show that it does have an effect on instruction.
The studieS by Miller (1971), Braunstein, Kle and Pa'hla (1973), /
and Centra (3,973) were inconclusive'with espec to the effect of
feedback at midterm to instructors w instr aion was again eval-
uated at the end of the term. How er, such evidence it found in a
recently completed study by Ale 'oni (1974), where student ratings, .
r- I.
gathered near the end of the erm, on thp CEQ, were presented to Uni-, /I" .
versity instructors along Nliti the opportunity of discussing their
results with a nipasurement and evaluation expert. The CEQ ratings
were again gathered at the end of the same tourees (one semester to%
.-. a year later) taught by the name instructors. The results revealed
that there was a significant increasein the student'ratings of0
these faculty on the two lowest. rated CEQ subscales that were discussed
/ in the meetings.(with the measurement and evaluation expert. Op the-
- 1 I
11.
4
r
F.
other hand, the group of faculty who was, not able to emil themigevss,
0,f expert consUliatAn but did receive ,the' CEQ results, remained un-
changed in their subscale ratings. Even thoilgh this study needs to
be replicated, it represi.nts the first tangible evidence that,student
-ratings can be used to improve instructidli.
It should be obvious,by now that the problems and questions that
faculty typically raise about the appropriateness of using student
ratings of instructor and instruction have-very definite answers which.
.can be interpreted as eing highly supportive for using such ratingd.. .
...
- ...
.
Interestingly enough,oany faculty will atill"disregard the evidence.
. 4(
and will' maintain ei.at there still is notenotighevidence or that_there
are still too any unaoswered6qUestions and pioblens to take student .--
ratings seriously. However, these same faculty would stoutly tend
their own methods of evaluating Students even though they might not be
-able t6 present any evidence to substantiate theft. claims.
In conclusion, therefore, it might be= useful to reverse the situa-
tion and ask faculty members to consider the seven concerns posed.above
from the point of view of faculty 'evaluating students! Hoy" much evi-0
<fence would' or could be pro -tided to convince_ the students that:
4: (a) Faculty can consistently judge student learning?, and4
0)- the performance of studpts wi h excellent ability is not usedcit '
to set the standard for the rest of the students in the course?
2.- Most faculty grading schemes are not affected by the at;entive,
polite, conforming, and non-creative student?
3. What was taught in the course is useful in other courses or
outside that university?
I
-41
4.. The course examinations are-reliable and valid?
5. An iinstructor'S markll or grades are not affecfed,by:
(a), the size of the class?'A'
CO. the sex of the student .
thetime of day the course was offered?
ghether.the student was taking the coUrge.as a required
r an elective?
(e) 'whether the student was a\majorOr a-on-majorT,'
(f) the term (or seme r).thg course was offered?
(a the level of the course?
(h) his professional rank?
6. Irlstriactors who had a partic rlyrOugh time when they were.
in college 0 :not tend to be just ag rough\On their s.tudentsl
7.. The courseaexaminations are useful. in .improving their learning?
ct.
13.
Reference
Aleamoni, L. M. Illinois course evaluatio questionnaire (CEQ) results
interpretation manual form 66 and form 32. Restarch'Report No. 331.
Urbana, Illinois: Measurement and Resea ch Division, Office ofi7
InstrUctional Resources; University of Illi
Aleamoni, L. M. A review of recent reliability
the Illinois course evaluation questionnaire
ois, 1972d.
validity studies on
CEQ). Research Memotan,
dum No. 127. Urbana, Illinois: Measurement a d Re,carc4 Divisfon,
Office of Instructional Resources, University of Illinois, 1972b.
Aleamoni, L. M. 'Response to Professor W. Edward De
teaching." The American Statistician, 1972
Aleamoni, I. M. The usefulness of student e.ral tions in improving
collSge teaching: Research Report No. 356. Urbana, Illinois:
g's "memorandum on
.4
Measurement and Research Division, Office of Instructional Resources,.
Uniyersity-of Illinois, 1974.
Aleamoni, L. M. & Grajam, M. H. The, relationship between CEQ ratings ando
instructor's rank, class size and course level. Journal of Educationalo
Measurement, 1974 (accepted for publication Jule,26 ,1973).
Aleamoni, L. M. & Spencer, R. E. The Illinois course evaluation questionnaire:4
A description of its development and a report of some of.its results.
Educational and Psychological Measurement, 1973,-33, 669-684,..
AleamonWL. M. & Yimer M. An investigation of the relationship between
colleague rating, student ratting, research productivity, and academl.c
rank in rating instructional effectiveness. Journal of Educational
Psychology, 1973, 64, 274-277.
14.
Aleamoni, L. M. & Yimer, M. Graduating senior ratings lelationship to
colleague rating, student rating, research productivity and academic
rank in rating instructional effectiveness. Research. Report No. 352.
Urbana Illinois: Measurement'and Research Division, Office of
InstruCtional Resources, University'of Illinois, 1974.
Braunstein, D.N., Klein, Se'Pachla, M. Feedback expectancy and0
shifts in. student ratings of college faculty. Journal of Applied
Psicholpgy, 1573, 58, 254-258.
Centre, J. A. Effectiveness of student feedback in miodifying college
instruction. Journal. of Educational Psychology, 1973, 65, 395-401.
::Cohen, S. H. & Berger, W. G. DiMensions of student's wings of college,
instructors underlying subsequent achievemenf on course examinations.
Proceedings;-78th Annual Convention, American Psychological.Associa-.
tion,:1970, 605-606.
"Cdstin,' F.,'Greedaughi W.T., & Menges, k.Y. Student ratings of college
teaching: Reliability, validity, and usefulness. Review of Edur.
dational Research, 1571, 41, 5.11-5A35,-
1"
Deming, W. E. Memorandum on:teaching. The'American Statistician, 1972,
26(1) 47.
Drucker, A. J. & Remmers, H. H. Do alumni and students differ in their
attitudes toward ihstructors? Purdue University Studies in Higher
Education, 1950, 70, 62-64. T
Drucker, A. J. & Remmers, H. H. ,D4."2.Suaihi and students differ in tair
artituded toward instructors? 'Journal of Educational Psychology,
1951, 42, 129-143.C
15.
ji\.
,SVerly, J. -C..& Aleamoni, The rise and fall of, the advisor . .
students atteMpttb evaluate their i?tstructors. Journal of the
National As.soeia ion of Colleges and Te ers of Agriculture,
1972, 16,, 43-A5i.
Feldman, S. The AdviSor. Research Report No. 325. Urbana, Illinois:. .
teasurement-andrteseareh Division, Office of Instructional ResourCes,.
University of. inois, 1970,
Gillmore G. M. & Bra denburg, D. C.. Wouldthe proportion of students- ,
taking a class a a requirement effect the stutlent 1`' ting of the
ittcourse? Research Report No. 347. Urbana, Illinois:r Measurement
ti
end Redear DitSion, Office of Instructional Resources, Univ sity
oc Illinois, .194.
Guthrie, E. R. 'The evaluation .of A progress report. Seattle,
4
Washington: UniVersity of Washington, 1954.
MoKedchie, W. J. Student ratings of fadulty. AAUP Bulletin, .1969,
'439-444.
Miller, M. T. 'Instrpctok attitudes tCwwYd, and their use of, student
ratings of teachers.' J.ournal of Educational Psychology, 1971, 62,
235-239.
mmers, Manual,.,of instruction -for the Purdue rating scale for
instruction. Westafayettee Indiana: University Book Store,
360 State, Street, 1960.
Stallings, W. M. & Spencer, R. E. Ratings of instructors in Accountancy.
.roi from video-tape clips. Redearch*ReporeNo. 265. Urbana,
1
Illinois: Measur4ment and Research Division, Office of 'Instructional
,esources, University of Illinois., 1967.
It 4
16.
Swanst"-R. A. & Sisson, D. J. The deV ,tlopment evaluation, and utiliza-.
tion of a deartmental faculty appraisal system. Journal 8Y
'Industrial Teacher EdUcation, 1971, 9, 64-79.
Weaver, C. H. Instructor rating by college students. Journal of
Educational Psychalogy,.1960, 51, 1-25.
WM.
0+;
. ,
O
/
1
4
V
S
4
et 0 .
4.1
I
Appendix:4
18
t
4
v.
17.
C L
PLEA ;F l'SE NE F' THE FIRM PSIJ youR PERSONAL COMMENTS ON TEACHER urrEcTrvENE''s AND GENERAL COURSE VALU.
THEN TJRN T '-=, ANL A'.';%,ER T,g ,,,JrcT,vr .),,Etilow:, ON THE r THER ',IDE, HS,NE:.PENCIL .oNty. YOUR INSTRUCTORIs. . .'"---eir-
WILL NOT ',FE CoMcLFTII, EVA:NATION )N1'4*, AFTER F INALi ADEN ARE %N FOR YDNR CONRSE. .
, %__ dikr J
cpuRsE CONTENT . t.
PLEASE GIVE ,,,,- 17,NIMENT', ON THE coNR r (FNTENT, .,H.JECT MATTERSANP ANY PARTICULAR RELEVANCE TH1', COURSE HAS
(
HAD TO YOUR AREA OF STUDY.
/fs.. . /.
A
IPi
.
INSTRUCTORS
WRITE. THE NAME OF YOUR PRINCIPAL INSTRUCTOR T.A.
E WHAT ARE V +1R GENERAL COMMENTS AHOUT THE iN',TRUCTOR N THIS COURSE?
I
OSTRUCTIONALornJECT!VE7,$fERE THE iNsTRvCTIONAL OHJECTtVU, tLEARLY
ASTATED FOR TH1', COURSE? Y
PAPERS AND HOMEWORK
CommCNT oN'THE VALUE OF HOORI4W HOMEWORK,#ANO PAPERS (IF ANY) IN THIS COuRSE,
EXAMS
Cot ENT ON THE EXAMS (oNipES, PRACJICALS AS TO DIFFICULTY, rk-,RNESS, ETC.;
GENERAL
1, WHAT IMPROVEMENTS IN THIS COURSE WOULD YOU SUGGEST?
2, PLEA1E GIVE YOUR THOHGHTFNL EVACUATION OF THiS,COURSE WITH COMMENTS. ARF YOU SATISFIED WITH WHAT YOU ,
GOT OUT. OF THIS COURSE? D YOU CONSIDER it A VALUAULE
A'REOHIREMEtaT?. C1R A DliApPCINTMCNT? PLEASE COMMENT.
U P 31-1173
EDUCATIONAL EXPERIENC?
ft
PLEA$E'FILL OUT THE OTHER SIDE
iLMPLY A. MEANS OF PASSING
est.t OPTICAL SCANPIIPM COCPOCATION
FORM 73 ILLINOIS COURSE EVALUATION QUESTIONNAIRE CE6tiniverslt y of Illinois, 1972Mednurrment Arvi senr. h Div 181,1. lit t 1,11.41 Rrs ,ur, in
0W W NW
IF YOU STRIl.,10' AGRE 7,0"ICCWITH THE EM ''0131-D
>41,`"z0 fF;'-19-"r
MARK.SA
A
D
SD
',0U4V2PLL MODENA Tt L IWITH THE 1TE m
IF YOU DISAGREE MODERATELY',./Ii.g*H THE ITEM
IF YOu 5TRONC,4Y D'SACIREIE,WITH THE ,TEM
WJrv'Cr 0
>-
Lu
SAMPLE 04,1. MARK ONLY O(j) MAR KS 0
' uRESPONSE PER ITEMUSING PENCIL ONLY a) I
t
tn..- tu
2. ERASE CHANGED, rc
ANSWERS CLEANLY 0AND COMPLETELY. b) I U
.
c
i"gn w
O oN IA
O 0cc ,xO wIft CC
U3
Z
Z zcc a ..../0 .L
:4It was dude billing,
..c z r .ALIA LTA .
Es
Overall the course was Rood.4....
E
'
UQz0YU'<Li
D*0 0>-u
in a.-CC
9 4a
Booed ut
D(C O
I would takemother course that was taught this
The instructor see 'led to be
0.
U
4440
.4
aim
N
derested in students as per saps
it
WCO U.' Ut
LL,
LL, z
,!:,
- u .
r, 41 r
I oi. Id h. Preferred another method of teacii7ng ul this Course.
It was easy to remain attentive.
The instrti NOT syntheslie,Itegrale of 4t.,7.1rIze effectIvelv.\
NOT much ''..)s gained by taking this course.
:The instructor ericour,gfd the develop:iment oi new viewpoints and appreciations.
I leaf n more when other teaching methods are'ClVed.
4, The curse material seemed worthwhile.
The tryctor was excelient.
The ins ructor demonstrated a thorough knowledge of the subject matterI,
1"Nould r)ther NOT lake another course from this instructor.
It was a v¢iy fVorthwhnle course.
SOIne things were NOT explained very .veil.
The course material was too difficult.
This was One of my poorest courses'. C.
The Instrtictor seemed to considergeaching. as a (horn ot mutine activity.
It was quip:cresting,
V TO
I think that the course' was taught quite well.
TPte course conteri was excelledt.
Some days I eras Ne very interested in this course.
4J2114-nzooUL./
, 4 1
24 A .s4 A 0 f,,E1
1-11
,t
B r
G
N
A
L A
T ' A A LA TAO
',A A !D (3!
T,A1 A C., T;(11
- -
A D SO
*,,A, A 0 ;0
T sn A C) SE)]
3 ',A A C3 SU
SA A .0 SDI
I SA A !, SC I SA AC
JAS -4-'
5.A, A CI SO,EM"0.,2/ A
SS. 4
E sr, A
C
T ! ',A A
;-.1.1
,AL A I, SA A ci r4r10N 4
A A I ' I I. SA A U *,0 I.
*IA A 0 ct)
"44, A L3 fl);
'3,A A U 's !
_
`.A A C) **IC)!
,0 , d SA A D $D
. I
5A A D -SD
I,I ..A A 1.) t,0
.._
. D
'A A D
a. 4A A 0 50
OPTIONAL ITEM SECTION II i! . 01) A -13 (: 0 E ,1 A 11 C L), I ',', A, ii C D L O t 4 A 0 f. 0 r i- A a C 0 at ji.,ABCDf 71A I3CCIE
4
-
;1 A B C b i iv A R is D t 80 A 11 L. f) F '' 1 A o (') o i i A ft C 6 E 83 A tIl C. 0 E 64 A ro c p i pr, A II f: 5 E'I. 0.., .. .. .
; PLEASE FILL OUT THE OTHER SIDEC *iiipO .**1.4.., OPTICAL SCANNING CORPORATION 0 9-. .4.49
40,
er
It
"5-
r
r
Appendix B
r
A
4
A
19.
""
4
I. -I 4171" -'
._-
* * * MERMAC 1-- TEST ANALY IS Al 0 tJUE S I N AI E
,d1
.. .{FEESULTS FOR, THE OBJECTIVE ITEMS ()N )'HE CEO FORM 73 Q0,EST,113NrIAIRE -\
4
'
4 tnoiN os v fti 4 4 . e J : I f I'l t.71 `1 f.
STATUS.
.FR tbSli Steli Ji, SR URAL) U 1 HER Uilli0.87 . 0.13 / 0.00 0.00_ 0.00 . 0 .00 0.00 - -
.f _ -7 P EU ,I EX EL-1 G R A DI
o' ' . . A 8 C 0 E OMIT0.40 0:53 0.07 o.uo . 0.00 0.00,,t ,
- PASS-FAILi YES NO OMIT
-
\o-.01:-) C). ell 0.1.3/. t
COURSE OPTION
,....,..--......,-
RV(,) tLECT OMIT"2" . 0.40 0.47 0.13
' MAJUk-NNUR..0,MAJOR MINOid. OTHER OMIT.... 0.13 1).00 J.60 0.07
.
.b. -SEX*
- FEMALE MALE OMIT ,
. .
0,..,. _ 0.0 / 0.31(0.07
/I -,
.. .
CC NT E HT IIAT I NGMEL V Glr G004 FA1a POUR V PR OMIT
a 0.27, 0.47 0.20 0.00 0.00 0 ...GO 0.07,
i \ I Hs I i*, RATING ' - .
EXCEL V GU 6,6Ut) F-Al R . POOR V PR OMIT,, . 0.80 0.07 0.07, 0.00 0.00 0 ... 0 0 0 .07
,. ,it
- ,CUURS -4 TI NG
EXCEL V GO 60.6U7-FA IR _POOR V PR OMIT.. 0.27 0.60 0.b.17 0.00 0.00 0.00 0.O1va
w.SEMESTLP.
,FALLSPRlivoSUALR atilt
, .o.ou 0.60 0.00 0.20.
.
ITEM SA A O' SD UMIT AST MEAN ID. JECL 01234567891. 0,67 0 33 0.00 0.00 0.00 SA 3.67 0.49 9.. 2. 0.93 J.07 0.00 0.00 0.00 . SA 3.93 0.16 9
'" .3. 0 J.27 0.27 0.47 0.00 SD 3.20 0.80 6 *4. .67 0.33. 0.00 0:00 0.00 .SA 3.67 0.49 9 *
4
_---v.5'. 0.00 J. Ot1 0:33 04.0 r 0.00 Sil 3.61 0.49 4-; IL
6. 0.00 0. OU 0C33 0.67 U. U0 SO 3.:37 0.49 87. 0.33 0.53 0.13 0.03 ilipt) SA 3.21) 0.68 6.. . 8. 0.00 0.21 .47 a. .I I isKio d0.47 -I M: 3.00 0. to 6
.
t
* * * MEA4AC -- TEST AcIALYSIS. ANL) wJESTILo!4AIRL PACNAt;'7:
9. 0.60 U.40 0.0U 0.30 0.u010. 0.93 0.0! 0.00 0.0, 0..jJ
11. 0.t7 J.13 0.0u 0.0..E u.JJ12. .0.0U. J. JU 3.20 .0.JJ O.J.j
-13. 0.47 J.)3 0.0J O.U. ,J.UU
14. p.ou J.13 0.4u 0.-t/ 0.JO15. 0.00 0.00 0.53 0.47 0.j016. 0.00 0.07 J.)3 6.ou i.0)17. 0.00 J.JO 0.27 .0.0 0.0018. 0.53 3.47 0.0U U.00 J.JJ19. 0,53 .U.471`,40.00 U.Ou 0.Uu'20. 0.33 3.00-}0.07 0.00 0.0021. 0.00 3.20 0.40 0.4u u.Uu22. 0.00 J.00 U.20 0:(,0 u.JU23. 0.73 0.27 0.0u '0.00 U.OU
sA 3.60 0.51SA 3.93 0.26
3.87. 0.35.sO 3.t30 0.411SA 3.47sO 3.33 0.7.
3.47 .523.53 C.04'
sD 3.73 0.46SA 3.53 0.52JA 3.53 0.52SA 3.27. 0.59SO 3.20 0.77s0 3.80 0.41SA 3.73 0.46
9
9.6
rs
9.
9
0
8
9
9
9
--SUBSCORE-- ITEMS RESP MEAN S.D. REL RANK/
GtNcRAL ATTITUDE 4 1.00.1 3.6U .)..) 0.35METHOD. 4 1.00 3...li 0-33- 0.93 /8CONTENT 4 1.0C 3.42 J.45 0.86 . 9
INTEREST 4. 1.00 3.3D u.4u 0.77 5'
INSTR:GENERAL 2 1.00 3.07 0.30 0.70 9
INSTR:SPECIFIC 5 1.00 3.06 .i.27 0.49 9
ro rAL 23 1.up i.JJ .:,.3.) 0.90 9
SV4PLE1 SI l' = 15
LEVEL
99
:9.
5
9
'9
9
INSTI COL-r,-GF
OVER-ALL
/ .0.1NE
//'d
/ 8' JCNE ,8
-- 9 -MCNE 9
9 r,C,E
9 Nillii 9
9 NIA9 i:UNt
qty